Orchestration Components

Check if Job is running

Hello, we have to ETL jobs: "main" runs at night and "realtime" runs every 15 minutes. To avoid waste data, both jobs must not run at the same time. Therefore...

Loop until condition is met

For example, I want to make an API call every one minute, until the data in the API response meets a certain condition. How can I set up an iterator...

Stopping Database Query against Postgrs

I want to control length of a transaction against a Postgres server (a slave), to make sure not to disturb this production server. I try to set socketTimeout to 2...

Google Spreadsheets Metadata

Is there a way to force Redshift to refresh the meta data associated with a Google Spreadsheet? I've changed the column structure of the spreadsheet and Matillion does not recognize...

Iterate through file in S3 Load component using specific pattern

Hi, I want to load all files with following naming sitting in a root S3 bucket. There are other files there too but I need to selecively only want the...

S3 Put Object

I'm trying to run an S3 Put Object to transfer data from an SFTP server to AWS. I'm getting the error: "socket is not established" Thanks,

"Wait" for a specified time period?

In an orchestration job, is there a way to have Matillion just wait for some specified time period, between the completion of one component and starting the next component? I'm...

context.UpdateVariable

I've read numerous places about how a variable isn't updated until a Python script is done, but it seems I'm confused at how the update happens. I have a job...

INTEGER Data Type Validation in Matillion

Hi All, I am trying to do data type validation on my source data, NULL/NOT NULL , DATE and INTEGER I am able to check the data types for NULL/NOT...

loading STL_LOAD_ERRORS data in a redshift table

Hi Experts, I am new to matillion and trying to capture the detail of of STL_LOAD_ERRORS table which is shown in the "task" tab whenever Orchestration job fails . Is...