Orchestration Components

"Bash script killed; timeout exceeded."

I am running a python application using "Bash Script component". The "Bash Script component" will stop automatically right after 6 minutes. My application takes more than 20 minutes to finish....

SNS/SQS

I am trying to create alerts for failed jobs and I am using SNS alerts. I want to know if SNS is prefered or we prefer SQS? If SQS -...

S3 - Kick off Job

Hi, I want to kick off a matillion job as soon as the file has put in S3 bucket. Is there a file watcher component in Matillion that does this?...

Processing large XML field out of Oracle DB

Hi, I have a situation where we have a source oracle table that contains a CLOB whose XML data I need to extract and parse out a few fields that...

Insert data to Redshift

My job is going to receive an SQS message then assign to an "Environment Variables" in Matilion project. Does any way that I can insert the value of "Environment Variables"...

Pass "Environment Variables" to Bash Script Component ?

Does any way I can pass "Environment Variables" to Bash Script Component ?

install python module in bash script component

I tried to install python smar-open by the following command in Bash Script component, but it is not working. would you please double check for me ? pip install smart-open...

Question for json layout setting of S3 load component

I have a json file in S3 and I want to load it to redshift. Below is my json file. { "customer_id":452435, "products":[ { "item_id":222, "item_type":"Qcommerce::Pack", "price":78, "quantity":1 }, {...

How to create csv file with double double quote texts ?

Hi, How to setup Text output (or other component) to create CSV file with double quote all texts ? Can I set Output type to something like CSV quote as...

Escaping Iteration / Handling large result sets from API Queries

I am looking to set up an API query which pages the results without knowing how many total pages there will be. Does anyone have an example of an orchestration...