My team is in the process of evaluating ETL tools to load data from 30+ data source into our data lake. I have two questions:
- Does Matillion Support loading data INTO AWS S3 (SO S3 as a destination). I see from the docs that Matillion has an S3 load component but that just supports loading data from S3. This is important to us as we want to be able to access some data using tools like AWS Athena?
- I am not seeing a link showing the monitoring features of the actual ETL is there a way to monitor the state ETL programmatically (via an api, ...)
Thank you Ramzi
1 Community Answers
Dan D'Orazio —
Hi Ramzi -
Welcome to Matillion!
Assuming you are evaluating our product on AWS (for Snowflake or Redshift)
1. Yes, Matillion does support loading data into S3. There are a number of ways that this can be accomplished. All, or nearly all, of the Components in our Orchestration > Load/Unload category will stage the associated data in S3 prior to copying that data into your warehouse. The S3 Unload Component will specifically pull data out of your warehouse into an S3 location of your choosing.
2. I think this support article shows how to accomplish this via the API, specifically the /task endpoint, if I understand the question.