Script Expressions#
Flowman allows using expressions at many places in the YAML specification files. For example, you can reference
variables defined in the environment
section or also access a set of predefined objects inside values.
Flowman uses Apache Velocity as the template engine. Please also read its documentation for advanced feature like conditional expression or recursive evaluation.
Simple Expressions#
You can access the contents of a variable via a dollar sign ($
), like for example:
mappings:
weather:
kind: read
source: $weather_source
This example uses a variable weather_source
to define which relation to read from. This way,
different environments can be supported by simply setting different values for that variable.
Variables can be set globally in a project’s environment
section or in a profile of a
project (which then has to be activated) or on the command line.
Predefined Objects#
Flowman also provides some predefined objects, which are available in the expression language. These objects either simply hold some data (like the location of the project), or they also provide some functions (for working with date and time).
project
#
project.basedir
project.filename
project.name
project.version
Integer
#
Integer.parse(string) -> int
Integer.valueOf(string) -> int
Float
#
Float.parse(string) -> float
Float.valueOf(string) -> float
Boolean
#
Boolean.parse(string) -> boolean
Boolean.valueOf(string) -> boolean
LocalDate
#
LocalDate.now() -> LocalDate
LocalDate.today() -> LocalDate
LocalDate.parse(date:string) -> LocalDate
LocalDateTime.valueOf(date:string) -> LocalDate
LocalDate.format(date:string, format:string) -> string
LocalDate.addDays(date:string, days:int) -> LocalDate
LocalDate.addWeeks(date:string, weeks:int) -> LocalDate
LocalDate.addMonths(date:string, months:int) -> LocalDate
LocalDate.addYears(date:string, years:int) -> LocalDate
LocalDateTime
#
LocalDateTime.now() -> LocalDateTime
LocalDateTime.parse(datetime:string) -> LocalDateTime
LocalDateTime.valueOf(datetime:string) -> LocalDateTime
LocalDateTime.ofEpochSeconds(epoch:int) -> LocalDateTime
LocalDateTime.format(datetime:string, format:string) -> string
LocalDateTime.add(datetime:string, duration:string) -> LocalDateTime
LocalDateTime.subtract(datetime:string, duration:string) -> LocalDateTime
LocalDateTime.addSeconds(datetime:string, seconds:int) -> LocalDateTime
LocalDateTime.addMinutes(datetime:string, minutes:int) -> LocalDateTime
LocalDateTime.addHours(datetime:string, hours:int) -> LocalDateTime
LocalDateTime.addDays(datetime:string, days:int) -> LocalDateTime
LocalDateTime.addWeeks(datetime:string, weeks:int) -> LocalDateTime
LocalDateTime.addMonths(datetime:string, months:int) -> LocalDateTime
LocalDateTime.addYears(datetime:string, years:int) -> LocalDateTime
Timestamp
#
Timestamp.parse(ts:string) -> Timestamp
Timestamp.valueOf(ts:string) -> Timestamp
Timestamp.ofEpochSeconds(epoch:int) -> Timestamp
Timestamp.add(ts:string, duration:string) -> Timestamp
Timestamp.subtract(ts:string, duration:string) -> Timestamp
Duration
#
Duration.ofDays(days:int) -> Duration
Duration.ofHours(hours:int) -> Duration
Duration.ofMinutes(minutes:int) -> Duration
Duration.ofSeconds(seconds:int) -> Duration
Duration.ofMillis(milliseconds:int) -> Duration
Duration.between(datetime:string, datetime:string) -> Duration
Duration.parse(duration:string) -> Duration
Duration.valueOf(duration:string) -> Duration
Period
#
Period.ofYears(years:int) -> Period
Period.ofMonths(months:int) -> Period
Period.ofWeeks(weeks:int) -> Period
Period.ofDays(days:int) -> Period
Period.parse(period:string) -> Period
Period.valueOf(period:string) -> Period
System
#
System.getenv(env:string) -> string
System.getenv(env:string, default:string) -> string
System.getProperty(property:string) -> string
System.getProperty(property:string, default:string) -> string
String
#
String.concat(left:string, right:string) -> string
String.concat(s1:string,s2:string, s3:string) -> string
String.concat(s1:string,s2:string, s3:string, s4:string) -> string
String.concat(s1:string,s2:string, s3:string, s4:string, s5:string) -> string
String.urlEncode(url:string) -> string
String.escapePathName(url:string) -> string
String.partitionEncode(url:string) -> string
URL
#
URL.encode(url:string) -> string
URL.decode(url:string) -> string
File
#
File.read(filename:string) -> string
JSON
#
JSON.path(json:string, path:string) -> string
AzureKeyVault
(since Flowman 1.1.0)#
The following functions are provided by the Azure Plugin
AzureKeyVault.getSecret(vaultName:string, secretName:string) -> string
AzureKeyVault.getSecret(vaultName:string, secretName:string, linkedService:string) -> string
AwsSecretsManager
(since Flowman 1.1.0)#
The following functions are provided by the AWS Plugin
AwsSecretsManager.getSecret(secretName:string, region:string) -> string
AwsSecretsManager.getSecret(secretName:string, keyName:string, region:string) -> string