Launching: Detailed Instructions
OverviewMatillion ETL is an AMI-based ETL/ELT tool built specifically for Amazon Redshift. Modern, browser-based UI. Powerful, push-down ETL/ELT. To learn more about the product, watch this video .
This support page describes how to launch and connect to Matillion ETL from the AWS Marketplace.
Important, for best results, DO NOT USE the 1-Click launch and following the instructions below. Following this will allow AWS to manage what services Matillion ETL can interact with via IAM Roles which is the most secure approach.
If you do use 1-click set-up, you will need to enter your credentials manually. Please follow the 1-Click launch instructions.
Selecting Matillion ETL from the AWS MarketplaceTo launch and configure Matillion ETL, you should select it from the AWS Marketplace and start the launch/setup procedure. To do this, use the following steps:
- Locate 'Matillion ETL for Redshift' on the AWS Marketplace
- From the Matillion ETL product page on the AWS Marketplace, press the yellow ‘Continue’ button
Choosing Pricing and LocationImportant: Use the “Manual Launch” tab and follow the instructions below. If you use 1-click setup, Matillion ETL will still work, but credentials will have to be managed manually.
From the Manual Launch screen, choose pricing and select your region. Matillion ETL is available in all AWS regions.
To do this, use the following steps:
- To choose pricing, select Hourly or Annual on the software pricing tab. If this is the first time you have used Matillion ETL and if you choose Hourly, the first 14 days will be a free trial (Note: AWS infrastructure fees will still apply). You will be able to choose an Instance Type on the next screen
- Select a Region by clicking on Launch with EC2 Console next to the region of your choice. For an explanation of AWS regions, visit here. (Note: You will have opportunity to review your setup before launching your instance and starting your trial/paid subscription)
Choosing the Instance TypeImportant: The instance type you select affects how many users can use Matillion ETL concurrently. For more information see Choosing an instance size.
Choose one of the supported Instance Types. Each Instance Type is appropriately sized to support a given number of users and the software itself recognises the Instance Type it is running on and restricts maximum concurrent users on this basis.
- For teams of 1-2 data professionals using Matillion ETL concurrently, choose t2.medium
- For teams of 3-5 data professionals using Matillion ETL concurrently, choose m4.large
- For teams of 6-12 data professionals using Matillion ETL concurrently, choose m4.xlarge
You can read more about Instance Types and pricing at here.
Choose the instance size you require, then click Next: Configure Instance Details.
Important: DO NOT click the blue “Review and Launch” button yet, as there are options you will want to configure on later screens.
Configure Instance DetailsThis screen is why we’ve used Manual Setup instead of 1-click setup. There are some settings you can specify on this screen, unavailable through 1-click, that improve the functionality of your Matillion Instance.
We advise you configure this screen as follows:
- Number of instances – leave as default (unless you want multiple instances)
- Purchasing option – leave as default i.e. unchecked
- Network – Choose a VPC in the same Availability Zone as your redshift cluster. An Availability Zone is Amazon’s terminology for a physical data centre, and you want Matillion ETL running in the same data centre as your Redshift cluster.
- Auto-assign public IP – This depends very much on the setup, by default a new VPC will not have VPN connections or NAT Gateways available, so in order for Matillion ETL to connect to the internet and for you to access Matillion this will normally need to be set to Enable.
- IAM Role – Follow instructions below:
IAM Roles are used to allow your Instances to use the Amazon API's securely without manual management of security keys. To use all the features of Matillion ETL, you should configure an IAM Role for your instance to use. This procedure assumes you do not already have an appropriate IAM role setup (if you do, simply select it)
- Click Create New IAM Role
- Click the blue Create New Role button
- Enter a suitable name into the Role Name field e.g. Matillion_ETL_IAM_Role
- Choose Amazon EC2 as the role type, by pressing the select button on this row
- From the list select
- Click Next Step. For more information on the policies see Managing Credentials.
- Finally click the Create Role button.
Returning to the Configure Instance Details screen, you can now select the IAM role you have just created.
To finish configuring the Configure Instance Details, choose settings as below for Shutdown behaviour, Enable termination protection and Monitoring:
- Shutdown behaviour – leave as default i.e. Stop
- Enable termination protection – we suggest setting to, ‘Enabled’
- Monitoring – we suggest setting to, ‘Enabled’
- Tenancy - Run a shared hardware instance
- Click, Next – Add Storage
- You can choose the default root volume size here however for production implementations we would recommend increasing this by a factor of 5
- Click, Next - Tag Instance
- Add any instance tags your require
- Click, Next – Configure Security Group
- A default security group will be created with the minimum set of ports. If required, name and adjust as to your security requirements.
Hint: you can make connectivity to Redshift easier if you use the same security group for Matillion ETL.
You are now ready to Review and Launch your Matillion ETL for Redshift AMI.
Once the AMI has initialised, which normally takes a few minutes, you can access Matillion ETL by entering the hostname or IP of the instance into a web browser.
Log in to your copy of Matillion with ec2-user and the instance ID i-xxxxxxxx (e.g. i-88ed92c6)