Name is required.
Email address is required.
Invalid email address
Answer is required.
Exceeding max length of 5KB

Clustered/Highly Available

Do you have further details on the Clustered/Highly Available configuration of Matillion? I'm particularly interested in what additional AWS components are required (e.g. RDS instances) and whether there is an AWS architecture diagram showing the setup.

2 Community Answers

Matillion Agent  

Ian Funnell —

Hi Peter,

A Clustered/Highly Available mode is available for our Redshift product. It fundamentally comprises multiple peer instances of Matillion which share the workload.

You’re right about extra AWS components. These include:

  • An RDS database to hold metadata (replacing the on-instance MongoDB which performs the same function in a single-instance deployment)
  • An ALB load balancer, which distributes the HTTP sessions between the two instances. You aim your browser at the ALB and not at one of the instances.

There’s more about considerations for designing a Job for a High Availability cluster in this document

When launching a HA cluster, we provide various CloudFormation templates according to what infrastructure you already have in place.

It’s possible to migrate from a single-instance deployment to an HA cluster using the Server Migration Tool

Best regards,
Ian


Peter McLean —

Hi Ian,

Thanks for your reply. Do you have an architectural diagram which shows all the AWS components, and where they are located within a VPC (e.g. private/public subnets etc)?

Many thanks,
Peter

Post Your Community Answer

To add an answer please login