Job Variables

This article is part of a series on Variables.


Overview

Variables can be defined within the scope of a single job. These variables can still be Copied and Shared with regard to their behaviour in iterator and flow components.

  • Job variables will override any environment variables of the same name within that specific job. 
  • Job variables are always included in jobs that are imported or exported and are not available for optional inclusion like environment variables are.

 

Creating Job Variables

To create a job-scoped variable, right click on the desired job within the explorer panel and select Manage Variables. This will bring up the Manage Job Variables dialog.

The Manage Job Variables dialog gives a complete list of all Job Variables with their properties:

Name: Name of the variable
Type: Matillion ETL Data Type of the variable. For more information on Variable types, please see here.
Behaviour: Determines its 'branch behaviour' inside of a job. That is, how the variable is updated when more than a single job branch is making use of it. For more information on Variable behaviour, please see here.
Visibility: If Private, this variable cannot be discovered and overwritten when this job is called from a Run Orchestration or Run Transformation component.
Value: The default value for this variable in this job.

Clicking the + button will create a new variable. Clicking the pencil icon will edit the details of an existing variable. In either case, a new dialog will appear to enter details for the variable.



 This description has no consequence or functionality beyond being present in the dialog for editing the variable and when the job is being called from a Run Orchestration or Run Transformation component.
 

Setting Job Variable Values

When a job begins, all variables are initialised with their default value that was set in the Manage Job Variables dialog. The real power of a variable is that its value can be updated, for example, through the following methods:

 

Iteration Components

Iteration components work by setting variables to a new value for each iteration. So you must define variables you wish to iterate in advance, and then use them when configuring iteration components. Iterations can be run in parallel too - in that case the variable scope must be set to Copied to ensure each parallel components sees its own Copied copy of the variable.

 
Python Scripts

Warning: Variables become first class variables in Python and Bash scripts and great care should be taken to avoid naming them in a manner that clashes with key words in either language. We recommend using a prefix (for example, v_) to ensure no such conflicts occur.

Python scripts can push new values into variable using their built-in context object. See the python component documentation for an example of this.