Reference Documentation#

Flowman uses so-called flow specifications which contain all details of data transformations and additional processing steps. The user provides flow specifications as (potentially multiple) YAML files using a purely declarative approach.

In addition to data flows, these specifications also include information about data sources and sinks, schema information, credentials if required and complete workflows using jobs.

The top level object of a Flowman specification is always a Project. Each project contains the full specification of all data transformation and processing steps and can be executed independently.

All specification files are written as declarations in YAML syntax, which provides easy readability. These YAML files are read in by Flowman executables and data flows are executed as specified on the command line (more on that in Flowman CLI)