Launching: Detailed Instructions

Overview

Matillion ETL is an AMI-based ETL/ELT tool built specifically for Amazon Redshift. Modern, browser-based UI. Powerful, push-down ETL/ELT. To learn more about the product, watch this video .
This support page describes how to launch and connect to Matillion ETL from the AWS Marketplace.


Important, for best results, DO NOT USE the 1-Click launch and follow the instructions below. Following this will allow AWS to manage what services Matillion ETL can interact with via IAM Roles which is the most secure approach.

If you do use 1-click set-up, you will need to enter your credentials manually. Please follow the 1-Click launch instructions.


Selecting Matillion ETL from the AWS Marketplace

To launch and configure Matillion ETL, you should select it from the AWS Marketplace and start the launch/setup procedure. To do this, use the following steps:
  • Locate Matillion ETL for Redshift on the AWS Marketplace.
  • From the Matillion ETL product page on the AWS Marketplace, press the Continue to Subscribe button.


Choosing Pricing and Location

Important: Use the “Manual Launch” tab and follow the instructions below. If you use 1-click setup, Matillion ETL will still work, but credentials will have to be managed manually. Begin by selecting the Manual Launch tab.


Choose a pricing mode by selecting Hourly or Annual on the software pricing tab. If this is the first time you have used Matillion ETL and if you choose Hourly, the first 14 days will be a free trial (Note: AWS infrastructure fees will still apply). You will be able to choose an Instance Type on the next screen


Select a Version of Matillion ETL you wish to install. Generally this will be the highest number (and thus most recent) version. Then select a Region using the dropdown selector beneath. For an explanation of AWS regions, visit here. Matillion ETL is available in all AWS regions.


When choosing a Deployment type, select Single AMI. All other options lead to Cloud Formation launches.

Selecting one of these should present a list of launch regions below. Click Launch with EC2 Console on the appropriate region to continue.


Step 2: Choose an Instance Type

Important: The instance type you select affects how many users can use Matillion ETL concurrently. For more information see Choosing an instance size

Choose one of the supported Instance Types. Each Instance Type is appropriately sized to support a given number of users and the software itself recognises the Instance Type it is running on and restricts maximum concurrent users on this basis.
  • For teams of 1-2 data professionals using Matillion ETL concurrently, choose t2.medium
  • For teams of 3-5 data professionals using Matillion ETL concurrently, choose m4.large
  • For teams of 6-12 data professionals using Matillion ETL concurrently, choose m4.xlarge


You can read more about Instance Types and pricing at here.

Choose the instance size you require, then click Next: Configure Instance Details.

Important: DO NOT click the blue “Review and Launch” button yet, as there are options you will want to configure on later screens.
 

Step 3: Configure Instance Details

This screen is why we’ve used Manual Setup instead of 1-click setup. There are some settings you can specify on this screen, unavailable through 1-click, that improve the functionality of your Matillion Instance.
We advise you configure this screen as follows:
  • Number of instances – leave as default (unless you want multiple instances)
  • Purchasing option – leave as default i.e. unchecked
  • Network – Choose a VPC in the same Availability Zone as your redshift cluster. An Availability Zone is Amazon’s terminology for a physical data centre, and you want Matillion ETL running in the same data centre as your Redshift cluster.
  • Auto-assign public IP –  This depends very much on the setup, by default a new VPC will not have VPN connections or NAT Gateways available, so in order for Matillion ETL to connect to the internet and for you to access Matillion this will normally need to be set to Enable.
  • IAM Role: See the section below.
  • Shutdown behaviour: Select Stop (default).
  • Enable termination protection: We suggest setting to, ‘Enabled’.
  • Monitoring: We suggest setting to, ‘Enabled’.
  • Tenancy: Run a shared hardware instance.

Warning: Do not continue with the launch until a new IAM Role is created and selected for the instance. See the below section on completing this.

 

Setting up a new IAM Role.

An IAM Role is used to allow your Instances to use the Amazon API's securely without manual management of security keys. To use all the features of Matillion ETL, you should configure an IAM Role for your instance to use. This procedure assumes you do not already have an appropriate IAM role setup (if you do, simply select it).
  • Click Create New IAM Role
  • Click the blue Create New Role button
  • Choose AWS Service and then EC2 as the role type.
  • This should bring up additional options below. Select EC2 as the Use Case.
  • Click Next: Permissions to continue. From this new page we will need to add policies to the role. For more information on the policies see Managing Credentials.
  • From the list select
    • AmazonRedshiftReadOnlyAccess
    • AmazonSNSFullAccess
    • AmazonSQSFullAccess
    • CloudwatchFullAccess
    • AmazonRDSReadOnlyAccess
    • AmazonS3FullAccess
 
  • Click Next: Review.
  • Give your new role a name and, optionally, a description.
  • Finally click the Create Role to complete.
Back on the Step 3 screen of the launch process  (Configure Instance Details), select your newly-made role from the IAM Role dropdown menu (may require a page refresh to display your new role). This should complete your Instance Details. Click Next: Add Storage.
 


Step 4: Add Storage

  • You can choose the default root volume size here however for production implementations we would recommend increasing this by a factor of 5.
  • Click Next: Add Tags.


Step 5 Add Tags

  • Add any instance tags your require.
  • Click Next: Configure Security Group.


Step 6: Configure Security Groups

  • A default security group will be created with the minimum set of ports. If required, name and adjust as to your security requirements.
  • Click Review and Launch.

Hint: you can make connectivity to Redshift easier if you use the same security group for Matillion ETL. 
 


Step 7: Review Instance Launch


You are now ready to launch your Matillion ETL for Redshift AMI. Hit the blue Launch button at the bottom-right.

Once the AMI has initialised, which normally takes a few minutes, you can access Matillion ETL by entering the hostname or IP of the instance into a web browser.  

Log in to your copy of Matillion with ec2-user and the instance ID i-xxxxxxxx (e.g. i-88ed92c6)


Next Steps