Advanced

Triggering ETL from an S3 Event via AWS Lambda

Overview This example links the arrival of a new object in S3 and automatically triggers a Matillion ETL job to load it, transform it and append the transformed data to a fact...

Using Variables

Overview Variables are name-value pairs stored within each environment. Variables can be used in all sorts of parameters and expressions to allow the user to pass and centralise environment specific configuration. Variables...

Expression Editor

Overview The Expression Editor in Matillion ETL allows the user create expressions in the Calculate and Join components. From the editor, users can browse from lists of available functions (dependent on platform)...

Securing Matillion ETL

Overview By default Matillion ETL users are authenticated against an internal user file however it is possible to authenticate users against an Active Directory or other LDAP directory server, turn on and...

Type 6 Slowly Changing Dimensions Example

Overview Here we outline Matillion's approach to maintaining slowly changing dimension data using Matillion ETL by way of an extended example. Often when using Matillion ETL, a customer requires the ability to...

Integration with Amazon SQS

Setting up When using a Matillion ETL as part of a larger process the best practice way to initiate an orchestration job is to use Simple Queue Service or SQS. To do...

Schema Support

Overview Matilion ETL is aware that you may have multiple schemas. Each Matillion ETL environment specifies a default schema. The default schema is used by Matillion ELT in the following ways: It...

Using incron to automatically copy data to S3

Overview This is not strictly related to Matillion ETL however it is a common task to get data easily into S3 from a premise or system. This approach is especially used in...

Twitter Data Model

Connection String Options Account Id Auto Cache Cache Connection Cache Driver Cache Location Cache Metadata Callback URL Connect On Open Firewall Password Firewall Port Firewall Server Firewall Type Firewall User Initiate OAuth...

Google Analytics Data Model

Connection String Options Auto Cache Cache Connection Cache Driver Cache Location Cache Metadata Connect On Open Default Filter Firewall Password Firewall Port Firewall Server Firewall Type Firewall User Initiate OAuth Location Logfile...