Orchestration Components

Oracle CLOB field to Snowflake VARCHAR

I'm working on data transfer between two databases, one Oracle 12c and one Snowflake Data Warehouse. In the Oracle DB, there is a CLOB field which needs to be loaded...

Delete All Redshift Tables / Access System Tables?

I'd like to set up a job to detect and then delete all my tables (and views) that I've created on my Redshift cluster. For when I want to re-initialize...

JSON/XML from API

Hi, I am using Matillion for Snowflake. We would like to use "API Query" component to fetch data from an API and store the XML or JSON that the API...

Python Access to Bing Ads API OAuth refresh token

Hello, I'm creating a Python script that will interact with Bing Ads via their SDK python library (https://github.com/BingAds/BingAds-Python-SDK) I've already set up Bing Ads authentication as described on https://redshiftsupport.matillion.com/customer/portal/articles/2312291-oauth ,...

Python script

How do I set a delimiter within a variable in a python script: ( this script is under an iterator with many (jobsname) and (rowcount) I want every jobname and...

Copy Parquet data to Redshift

Is copying from parquet format data from S3 to Redshift not yet supported in latest version of Matillion ? The feature was made available by Amazon back in Jun-2018, however...

Load Excel file, only if recently modified?

I am using the "Excel Query" component, to load data from and Excel file I have stored on S3. When I go into S3 in the AWS console, I see...

Analyze after a Load, or a Rewrite, or a Truncate+Append?

From what I have read, I think is it case that when a Redshift table is created (or re-created) with any of the Load components—such as Database Query, RDS Query,...

Reading custom headers

I know how to write to custom header, but how do I read one? For example, if in curl if I provide -i, for the API I am using (Harvest),...

Excel Query Component - Adding Filename variable in Data Selection

I have a File Iterator attached to a Excel Query Component that is able to read and process multiple spreadsheet files into a staging table (thank you to Community for...