Launching: Detailed Instructions

Overview

This support page describes how to launch and connect to Matillion ETL from the AWS Marketplace.

Matillion ETL is an AMI-based ETL/ELT tool built specifically for Amazon Redshift. Modern, browser-based UI. Powerful, push-down ETL/ELT. To learn more about the product, watch this video .



Important: For best results, DO NOT USE the Launch from Website option and follow the instructions below. Following this will allow AWS to manage what services Matillion ETL can interact with via IAM Roles which is the most secure approach.
If you do use Launch from Website, you will need to enter your credentials manually. 


Step 1: Finding the correct product to set up.

To launch and configure Matillion ETL, you should select it from the AWS Marketplace and start the launch/setup procedure. To do this, use the following steps:
  • Locate Matillion ETL for Redshift on the AWS Marketplace.
  • From the Matillion ETL product page on the AWS Marketplace, press the Continue to Subscribe button.
  • From the Subscription page press the Continue to Configuration button.

 

Choosing Version and Location


On the Configure this software page, select Amazon Machine Image with a 64-bit Amazon Machine Image (AMI) as your Fulfillment Option.


It is highly recommended that users always select the most up-to-date product version available and in a region which they (or their current AWS services) reside in.

Click Continue to Launch to continue the setup. Note that in spite of the name, clicking this will not yet launch your instance or charge your account.

 

Launch this software


On the Launch from software screen, simply select Launch through EC2 on the Choose Action dropdown and then click Launch.


Again note that this simply continues with the launch setup and will not yet launch the instance or charge your account.


Step 2: Choose an Instance Type

Important: The instance type you select affects how many users can use Matillion ETL concurrently. For more information see our article on instance sizes

Choose one of the supported Instance Types. Each Instance Type is appropriately sized to support a given number of users and the software itself recognises the Instance Type it is running on and restricts maximum concurrent users on this basis.
  • For teams of 1-2 data professionals using Matillion ETL concurrently, choose t2.medium
  • For teams of 3-5 data professionals using Matillion ETL concurrently, choose m4.large
  • For teams of 6-12 data professionals using Matillion ETL concurrently, choose m4.xlarge


You can read more about Instance Types and pricing at here.

Choose the instance size you require, then click Next: Configure Instance Details.

Important: DO NOT click the blue “Review and Launch” button yet, as there are options you will want to configure on later screens.
 

Step 3: Configure Instance Details

This screen is why we’ve used Manual Setup instead of 1-click setup. There are some settings you can specify on this screen, unavailable through 1-click, that improve the functionality of your Matillion Instance.
We advise you configure this screen as follows:
  • Number of instances – leave as default (unless you want multiple instances)
  • Purchasing option – leave as default i.e. unchecked
  • Network – Choose a VPC in the same Availability Zone as your redshift cluster. An Availability Zone is Amazon’s terminology for a physical data centre, and you want Matillion ETL running in the same data centre as your Redshift cluster.
  • Auto-assign public IP –  This depends very much on the setup, by default a new VPC will not have VPN connections or NAT Gateways available, so in order for Matillion ETL to connect to the internet and for you to access Matillion this will normally need to be set to Enable.
  • IAM Role: See the section below.
  • Shutdown behaviour: Select Stop (default).
  • Enable termination protection: We suggest setting to, ‘Enabled’.
  • Monitoring: We suggest setting to, ‘Enabled’.
  • Tenancy: Run a shared hardware instance.

Warning: Do not continue with the launch until a new IAM Role is created and selected for the instance. See the below section on completing this.

 

Setting up a new IAM Role.

An IAM Role is used to allow your Instances to use the Amazon API's securely without manual management of security keys. To use all the features of Matillion ETL, you should configure an IAM Role for your instance to use. This procedure assumes you do not already have an appropriate IAM role setup (if you do, simply select it).
  • Click Create New IAM Role
  • Click the blue Create New Role button
  • Choose AWS Service and then EC2 as the role type.
  • This should bring up additional options below. Select EC2 as the Use Case.
  • Click Next: Permissions to continue. From this new page we will need to add policies to the role. For more information on the policies see Managing Credentials.
  • From the list select
    • AmazonRedshiftReadOnlyAccess
    • AmazonSNSFullAccess
    • AmazonSQSFullAccess
    • CloudwatchFullAccess
    • AmazonRDSReadOnlyAccess
    • AmazonS3FullAccess
 
  • Click Next: Review.
  • Give your new role a name and, optionally, a description.
  • Finally click the Create Role to complete.
Back on the Step 3 screen of the launch process  (Configure Instance Details), select your newly-made role from the IAM Role dropdown menu (may require a page refresh to display your new role). This should complete your Instance Details. Click Next: Add Storage.
 


Step 4: Add Storage

  • You can choose the default root volume size here however for production implementations we would recommend increasing this by a factor of 5.
  • Click Next: Add Tags.


Step 5 Add Tags

  • Add any instance tags your require.
  • Click Next: Configure Security Group.


Step 6: Configure Security Groups

  • A security group should be created with the minimum set of ports. If required, name and adjust as to your security requirements.
  • The default recommended security group uses SSH (port 22) and HTTP(S) (port 80 and 443) access to the instance. The range of allowed IPs should be tailored to your needs.
  • Click Review and Launch.

Hint: you can make connectivity to Redshift easier if you use the same security group for Matillion ETL. 
 


Step 7: Review Instance Launch


You are now ready to launch your Matillion ETL for Redshift AMI. Hit the blue Launch button at the bottom-right.

Once the AMI has initialised, which normally takes a few minutes, you can access Matillion ETL by entering the hostname or IP of the instance into a web browser.  

Log in to your copy of Matillion with ec2-user and the instance ID i-xxxxxxxx (e.g. i-88ed92c6)


Next Steps