Transformation Components

Transformation job takes way longer when run on its own

I've noticed I have one job, TransformationJobX, where when I run it as part of a scheduled job that also runs many other jobs, it consistently takes about 3-4 minutes...

SQL component

Is it possible to pass a column as a variable within a query such as: select row_number() over (partition by ${columnvariable} order by 1,* from ${tablevariable}? Regards,

Remove duplicates before storing into staging tables

Hi, Is it possible to, remove duplicates based on few columns( ie: when the column has a null or empty value should be removed) before storing it into the staging...

Task Log in e-mail in Mtaillion for Bigquery

I need to set up e-mail notifications for job status and message in Matillion for Bigquery. Is there any example or tutorial for this?

Segment Data Component

Is there a component which can split/segment a data set into two more datasets. e..g I have a list of cars and want to split it into petrol and diesel...

Why INSERT not working in SQL?

I'm trying to insert an SQL result which I got, from a join operation into a Redshift table. But the SQL editor isn't picking the INSERT keyword? Is it possible...

Converting Redshift table data to Parquet

Hi, I'm already having some Redshift tables, and I"m in need to unload those data back to S3 once I have done all the ETL transformations. So I wanted to...

Stage variable

Hi, Is there any component to create the stage variable and then use it for multiple output olumns. We have option in IBM Datastage ETL tool where we can create...

Row count with table iterator

I am loading files from S3 into a temp table on redshift, after which using a table iterator to load data into my fact tables. I want to do a...

Microsoft OneDrive via Matillion S3 Put Object

Can Matillion pull data from Microsoft OneDrive? I've setup an S3 Put Object using the following settings and it's not connecting, assuming it's due to the requirement for login credentials?...