Orchestration Components

checking if the file exists

Gents, I wonder how can i implement the following logic: If the file does NOT exists download to S3 from URL else move processing to some other component. Thank you...

How can I use a variable for Google Sheets Spreadsheet name in Basic Mode?

We want to load a bunch of Google Sheets from the same Drive account. The names may change occasionally, so I want the job to be fairly easy to modify....

S3 manifest builder not respecting Manifest Path

For the S3 manifest builder component there appears to be a bug in the manifest loading or I'm not understanding what manifest path is supposed to do. Here is my...

unzip a zip file in s3 and than load to redshift

Current S3 load component doesn't support Zip format. Any idea how can I load data to redshift table from zip file present in s3 using Matillion.

Useful Python Examples

You may have some python snippets that could be useful to others - please post them here!

Accessing .zip file from website, unzipping, then gzipping to s3 bucket, but it never makes it.

I have many orchestrations that pull .zip files from a website via the "S3 Put Object". I select the option to "Unpack ZIP file" and also "Gzip s3 Data". This...

Is there a general exception handler?

Hi, My question relates to errors and exceptions in an orchestration job. I have a job with eight RDS query components that feed into eight respective transformation jobs. I also...

Google Analytics component question

I'm having trouble with the Google Analytics component. An important process of our ETL is joining GA TransactionId to another table. TransactionId is normally found under the Ecommerce list of...

Unload

Hi Would it be possible to allow a SQL statement to be added to the Unload component? We need to access a very large table and only pull a small...

Database Query component fails when querying PostgreSQL boolean data type

When I create a Database Query component with a query against a Postgres database, and the query contains a column with a Boolean datatype, the query returns an error. An...