Generic Relation#
A generic
relation gives you access to Spark data relations otherwise not directly
supported by Flowman.
Example#
relations:
advertiser_setting:
kind: generic
format: "csv"
schema:
kind: inline
fields:
- name: id
type: Integer
- name: advertiser_setting_id
type: Integer
Fields#
kind
(mandatory) (string):generic
schema
(optional) (schema) (default: empty): Explicitly specifies the schema of the relation.description
(optional) (string) (default: empty): A description of the relation. This is purely for informational purpose.options
(optional) (map:string) (default: empty): All options are passed directly to the reader/writer backend and are specific to each supported format.format
(optional) (string) (default: empty): Specifies the name of the Spark data source format to use.
Automatic Migrations#
The generic
relation does not support any automatic migration like adding/removing columns.