Transformation Components

Remove duplicates before storing into staging tables

Hi, Is it possible to, remove duplicates based on few columns( ie: when the column has a null or empty value should be removed) before storing it into the staging...

Task Log in e-mail in Mtaillion for Bigquery

I need to set up e-mail notifications for job status and message in Matillion for Bigquery. Is there any example or tutorial for this?

Segment Data Component

Is there a component which can split/segment a data set into two more datasets. e..g I have a list of cars and want to split it into petrol and diesel...

Why INSERT not working in SQL?

I'm trying to insert an SQL result which I got, from a join operation into a Redshift table. But the SQL editor isn't picking the INSERT keyword? Is it possible...

Converting Redshift table data to Parquet

Hi, I'm already having some Redshift tables, and I"m in need to unload those data back to S3 once I have done all the ETL transformations. So I wanted to...

Stage variable

Hi, Is there any component to create the stage variable and then use it for multiple output olumns. We have option in IBM Datastage ETL tool where we can create...

Row count with table iterator

I am loading files from S3 into a temp table on redshift, after which using a table iterator to load data into my fact tables. I want to do a...

Microsoft OneDrive via Matillion S3 Put Object

Can Matillion pull data from Microsoft OneDrive? I've setup an S3 Put Object using the following settings and it's not connecting, assuming it's due to the requirement for login credentials?...

Creating task from transformation job via Web API

Why is it not possible to create tasks out of transformation jobs by using the Web API? Is there any plan to support it in the future? It does not...

File iterator duplicating data

Hi, I had to generate a filename from the files uploaded to S3, hence I found a very good article here by adding a filename to table which helped me...