Name is required.
Email address is required.
Invalid email address
Answer is required.
Exceeding max length of 5KB

AWS Billing Data

Has anyone worked out a solution using Matillion to import their AWS billing data into Redshift? Here is AWS's documentation on how to get billing and usage data into Redshift:

http://docs.aws.amazon.com/awsaccountbilling/latest/aboutv2/billing-reports-costusage-upload.html

The problem that we see is there is no way from Matillion to execute the .SQL file that they produce in order to create the proper schema in Redshift. We have already seen differences month-to-month in the schema, so it's important to be able create the monthly tables in Redshift according to the Create statement they produce.

Any ideas/suggestions are welcome.

2 Community Answers

Matillion Agent  

Ian Funnell —

Hi Ed,

You should be able to execute the .SQL file from an SQL Script component. The generated SQL contains a CREATE TABLE statement and a COPY (i.e. a bulk load).

Other alternatives might be to use the Matillion S3 Load Generator on the file, or to access the files through Athena/Spectrum. But if the schema keeps changing like you’ve observed then it will be best to use their generated .SQL file.

Best regards,
Ian


Ed Sesek —

Thanks, I missed the SQL Script component. That should do the trick.

Post Your Community Answer

To add an answer please login