Name is required.
Email address is required.
Invalid email address
Answer is required.
Exceeding max length of 5KB

streaming ETL

Hi....is anyone using Matillion in a streaming fashion where it is reading directly from Kinesis or Kafka?

1 Community Answers

Matillion Agent  

David Lipowitz —

Hi Gordon,

Thanks for the question. Most of our customers will use Firehose for Kinesis (or the Kafka equivalent) to write the stream contents to S3. And often they will trigger an SQS message to be sent when a new file arrives from these streams into the targeted S3 Bucket.

Users then configure Matillion to listen the SQS queue and trigger jobs based on those messages. More information is available on this feature here: Manage SQS Configuration

Hope that helps and please let us know if you have any additional questions.

Best Regards,
Dave

Post Your Community Answer

To add an answer please login