Contents Menu Expand Light mode Dark mode Auto light/dark mode
Flowman documentation
Logo

Quick Start Guide

  • Flowman Quick Start Guide

Concepts

  • Core Concepts
  • Main Entities
  • Lifecycles and Phases
  • Execution and Data Metrics
  • Schema Migrations

Tutorial

  • Flowman Tutorial
  • Lesson 1 — Flowman Basics
  • Lesson 2 — Schema Definitions
  • Lesson 3 — Transformations
  • Lesson 4 — Parameters
  • Lesson 5 — Multiple Build Targets
  • Lesson 6 — Flowshell
  • Lesson 7 — Execution Phases and Lifecycles
  • Lesson 12 — Deployment

Command Line Tools

  • Flowman CLI Tools
  • Flowman Executor (flowexec)
    • Project Commands
    • Job Commands
    • Target Commands
    • Miscellaneous Commands
  • Flowman Interactive Shell (flowshell)
  • Flowman Kernel Server (flowman-kernel)
  • Flowman Interactive Remote Shell (flowrshell)
  • Flowman History Server (flowman-server)
  • Flowman Schema Generator

Workflow

  • Development & Deployment Workflow
  • Using Standard Maven Plugins
  • Using Flowman Maven Plugin

Reference Documentation

  • Reference Documentation
  • Projects
  • Modules
  • Mappings
    • Aggregate Mapping
    • Alias Mapping
    • Assemble Mapping
    • Case Mapping
    • Cast Mapping
    • Coalesce Mapping
    • Conform Mapping
    • Deduplicate Mapping
    • Distinct Mapping
    • Drop Mapping
    • Earliest Mapping
    • Empty Mapping
    • Explode Mapping
    • Extend Mapping
    • Filter Mapping
    • Flatten Mapping
    • Grouped Aggregate Mapping
    • Historize Mapping
    • Iterative SQL Mapping
    • Join Mapping
    • JSON Extract Mapping
    • JSON Unpack Mapping
    • Latest Mapping
    • Mock Mapping
    • Observe Mapping
    • Project Mapping
    • Read Hive Mapping
    • Rebalance Mapping
    • Recursive SQL Mapping
    • Relation Mapping
    • Repartition Mapping
    • Schema Mapping
    • Select Mapping
    • Sort Mapping
    • SQL Mapping
    • Stack Mapping
    • Stream Mapping
    • Template Mapping
    • Union Mapping
    • Unit Mapping
    • Upsert Mapping
    • Values Mapping
  • Relations
    • Delta File Relations
    • Delta Table Relations
    • Empty Relation
    • File Relations
    • Generic Relation
    • Hive Query Relations
    • Hive Table Relations
    • Hive Union Table
    • Hive View Relations
    • JDBC Query Relations
    • JDBC Table Relations
    • JDBC View Relations
    • Kafka Relations
    • Local Relations
    • Mock Relation
    • SQL Server Relations
    • Template Relation
    • Values Relation
  • Targets
    • Blackhole Target
    • Compare Target
    • Console Target
    • Copy Target
    • CopyFile Target
    • Count Target
    • Delete File Target
    • Delta Vacuum Target
    • Document Target
    • Drop Relation Target
    • Empty Target
    • File Target
    • GetFile Target
    • Hive Database Target
    • JDBC Command Target
    • Local Target
    • Measure Target
    • Merge Target
    • Merge Files Target
    • Put File Target
    • Relation Target
    • SFTP Upload Target
    • Stream Target
    • Template Target
    • Truncate Target
    • Validate Target
    • Verify Target
  • Jobs
    • Flowman Job Execution
  • Datasets
    • File Dataset
    • Mapping Dataset
    • Relation Dataset
    • Values Dataset
  • Schemas
    • Avro Schema
    • Inline Schema
    • JSON Schema
    • Mapping Schema
    • OpenAPI Schema
    • Spark Schema
    • Swagger Schema
  • Connections
    • JDBC Connection
    • SSH Connection
  • Tests
    • Flowman Test Execution
  • Assertions
    • Columns Assertion
    • Expression Assertion
    • Schema Assertion
    • SQL Assertion
    • Unique Key Assertion
  • Measures
    • SQL Measure
  • Templates
    • Assertion Template
    • Connection Template
    • Dataset Template
    • Mapping Template
    • Measure Template
    • Relation Template
    • Schema Template
    • Target Template
  • Execution Metrics
    • Console Metric Sink
    • JDBC Metric Sink
    • Prometheus Metric Sink
  • Execution Hooks
    • JDBC Hook
    • Report Hook
    • REST Hook
    • Web Hook
  • Namespace
  • Fields, Data Types & Values
  • Script Expressions
  • Profiles

Testing

  • Testing with Flowman

Documenting

  • Documenting your Project
  • Checking Model Properties
  • Configuring the Documentation
  • Documenting Mappings
  • Documenting Relations
  • Documenting Targets

Setup & Configuration

  • Setup & Configuration
  • Installation Guide
  • Running Flowman in Docker
  • Running Flowman on Windows
  • Deploying Flowman to AWS EMR
  • Deploying Flowman to Azure Synapse
  • Configuration Properties
  • Building Flowman from Sources

Cookbooks

  • Advanced JDBC Database Features
  • Data Quality Checks
  • Data Validation before Build
  • Execution Metrics
  • Execution Phases
  • Hadoop Dependencies Installation
  • Impala Metadata
  • Kerberos Authentication
  • Override jar versions
  • Parallel Execution
  • Performance Tuning
  • Securely providing Secrets
  • Sharing Entities between Projects
  • Syntax Highlighting
  • Target Execution Ordering

Connectors & Plugins

  • Connectors
  • Plugins
    • Avro Plugin
    • AWS Plugin
    • Azure Plugin
    • Delta Lake Plugin
    • Impala Plugin
    • JSON Plugin
    • Kafka Plugin
    • MariaDB Plugin
    • MS SQL Server Plugin
    • MySQL Plugin
    • OpenAPI Plugin
    • Oracle Plugin
    • PostgreSQL Plugin
    • SFTP Plugin
    • Swagger Plugin
    • Trino Plugin

Misc

  • FAQ
  • Releases
  v: latest
Versions
latest
stable
build-docs-with-poetry
branch-1.0
branch-0.30
Downloads
On Read the Docs
Project Home
Builds
Back to top
Edit this page

Avro Plugin#

Provided Entities#

  • Implements avro schema

  • Adds support for Avro file format in file relation

Activation#

The plugin can be easily activated by adding the following section to the default-namespace.yml

plugins:
  - flowman-avro 
Next
AWS Plugin
Previous
Plugins
Copyright © 2023, The Flowman Authors
Made with Sphinx and @pradyunsg's Furo
On this page
  • Avro Plugin
    • Provided Entities
    • Activation