Getting Started

Doing Data Quality Checks

We're going to be loading a large number of files from different data sources and would like to do some checks on S3 files before they're loaded into Redshift. Among...

cursor.execute creates Phython exception

I try to realize the following idea: https://redshiftsupport.matillion.com/customer/portal/questions/16653780-using-value-from-redshift-as-variable-to-a-bash-script?b_id=8915 The script cursor = context.cursor() cursor.execute(‘SELECT count(*) FROM tdc.dim_test’) result = cursor.fetchone() print result runs into "Exception: Cannot create PyString with non-byte...

Where can I change Google Analytics sub-account?

I have multiple Google Analytics sub-accounts (-1,-2,-3). How can I switch between them?

Email from Matillion using python

I am trying to send an email after job execution. I am using python script to send email. When I run this from console on the server, it works fine....

move data between redshift instances

Is it possible to move data directly between two redshift instances (NOT schemas) or do I need to create 2 tasks; 1 to read from the source cluster and write...

Get root parent of child in Hierarchical table

I am using Matillion and Redshift, looking for a solution to get root parent of child from a Hierarchical table. Scenario explained as below: TABLE 1: (input) ChildId ParentId -------...

S3 Validation failure with memory error

Hi There, I have S3 validation failure with memory error. I am using 50MB file in m3 medium Matillion and DC1 large redshift to do an S3 copy. It has...

Redshift Weird Views

I see some weird views still remaining in the Redshift. I am thinking these views might have got created by Matillion process. I still see them hanging in there. I...

hourly ETL and getting date components as variables

Hi, We have files in S3 in the format: s3://bucket/{year}/{month}/{day}/{hour}.gz We need to load this data into matillion hourly and I'm struggling to figure out how to do this. Is...

Importing rules & mappings from csv files

Has anyone any experience of importing csv data (via JSON etc. if necessary) into Matillion rather than having to type them in from scratch? Thanks Jo