Transformation Components

How to update variables from db values

We use one table as a parameter table, whose values affect subsequent processing. At present, I read the values in a Python script component and update the variables via the...

What's the table key when I append rows?

We're using the Table Output component to write rows to TargetTable. The TargetTable has been defined with PRIMARY KEY ("rowid"). RowId gets its values from our source system. The Table...

Date format change

Hi all , I have one isue with date format , In my source file we are getting date format like below 20170610000000 20170611000000 20170612000000 In my staging table we...

Cassandra json extraction

I would like to know how do we get cassandra's column output like <attached> into RS. { "id": "0001", "type": "donut", "name": "Cake", "ppu": 0.55, "batters": { "batter": [ {...

Stack overflow error in 'SQL Component'

Hi, I have a singe SQL query which left joins 14 tables together. It outputs result in 26 columns. When I run the job, SQL Component which contains this query...

Filter IN / NOT IN

Hi, would be great to add IN / NOT IN support for filter component. Better yet, please allow free-style filter expressions without being limited to a predefined list of comparison...

Transpose rows component

Hi, I saw the tutorial for pivoting tables that uses Transpose rows followed by Split. The issue with this approach is that it is highly inefficient and does not scale...

How to see SQL ran by components?

Hi, I am exploring the Matillion functionality. One thing that is not obvious to me – how do I debug the actual SQL ran by the Matillion components? I know I...

Matillion not being able to access UDF

Hi everyone, I have created a number of UDF on Redshift but it seems like they cannot be called in Matillion. Error being the typical one: ERROR: function FUNCTION_NAME(bigint) does...

Database Query Concurrency max value

Hi, how I can increase the "Database Query" component's property "Concurrency"? Currently it only allows a maximum number of 16 and if I input more than that, it shows the...