Administering Matillion ETL
Note: From version 1.28.7 onward, it is recommended users use the Admin Menu for all Matillion ETL administering.
From version 1.21.5 onwards, Matillion ETL offers an administration page which automates many server administration tasks that used to require editing configuration files by hand.
The /admin url will present a login page (similar to the login for Matillion) - the default user is ec2-user and the default password is set to the instance-id on first launch.
It is possible to perform basic administration tasks on Matillion ETL instances if you have admin permissions to that instance. GCP offers a simple SSH command line tied to each particular instance that can be found through the main GCP menu under 'Compute Engine'. Clicking the SSH button by the desired instance will launch the SSH command line window.
Matillion can listen for HTTP and/or HTTPS requests, depending on the configuration. By default, Matillion listens only for HTTP requests (port 80).
This can be changed to HTTPS (port 443) or both by selecting the appropriate button. If SSL is enabled, you may optionally upload SSL certificates to allow clients to validate the identity of the server.
Save Configuration saves any changes in this section but does not immediately apply it.
You may provide a certificate and key file using the Upload Certificates button once you have chosen the appropriate files. This is optional, but clients using HTTPS will get a certificate validation error until you provide valid certificates.
Finally, for the changes to take effect, you must restart the server - this will disconnect any connected client sessions and abort any currently running jobs.
In order to Enable and Disable SSL, please refer to the documentation on Securing Matillion ETL. Note that SSL is enabled by default with Matillion ETL.
There are 3 different login schemes.
NONE disables all security and lets anyone use the system. (This is not an option for the admin page itself, that always requires authentication).
INTERNAL uses an internal database of username and passwords.
You can add remove and modify users in the Security Configuration section.
To add a user:
- In the Security Configuration -> Add User section enter a Username and click Add User
- Enter a password for the user. If you wish the user to also be able to access the admin page tick the admin checkbox.
To change the user's password click the change password link
To Remove a user click the X
EXTERNAL is used for linking to an existing directory server (e.g. OpenLDAP or Microsoft Active Directory).
The sample values below are for an Active Directory server running for the realm EXAMPLE.COM.
|Connection Name||The name of a user to make the initial bind to the directory.
For active directory, that will include a realm using the form "user@REALM"
|Connection Password||The password for the user to make the initial bind to the directory.|
The location of the directory server, using one of the forms below:
For non SSL - ldap://<hostname>
|User Base||The part of the directory tree to begin searching for users.
|User Search||The attribute to search for user names.
|Role Base||The part of the directory tree to begin searching for groups/roles - often the same place as users.
|Role Name||The name of the attribute containing the role name.
How to find all the roles for a user.
|METL Role Name||The role a user must be a member of to gain access to the Matillion ETL application.|
|METL Admin Role Name||The role a user must be a member of to gain access to the Matillion ETL administration page - this can be different to the METL Role Name but is not required to be.|
Once the configuration is Saved, you will need to restart the server to take effect - use the Restart Server button on the top-right of the screen.
Inside the SSH command prompt, browse to the tomcat directory using a command such as:
Then open the file containing the user information using a command such as:
sudo vi tomcat-users.xml
Using your text editor of choice, browse to the bottom of the file and, between the admin's credentials and the closing tag, add a line for the new user, completing the username and password fields to your own liking.
<user username="<username>" password="<password"> roles="Emerald,API,Admin" />
Save and close the file.
Note: This describes the Matillion ETL update process for versions prior to 1.28.7. For more recent versions of Matillion ETL, please refer to the 'Updates' section of the Admin Menu documentation.
You may check for software updates using the Check for Updates button.
If any updates are found, the Update button will become available:
This will download any updated packages and apply them - once applied the server will be restarted, which will disconnect any users and abort any running tasks.
Using the SSH command line, enter the below command to instigate an update:
sudo yum upgrade
This will attempt to update both the OS and Matillion ETL instance. Alternatively, to upgrade only Matillion ETL, use the below command:
sudo yum upgrade matillion-*
Either command will prompt the user for confirmation (y/n) for the update before continuing.
Updating to a specific version
In some cases it may be desirable to update to a specific release of Matillion ELT. This is possible in cases that you are using a relatively recent Matillion ETL version but is strictly not recommended. It is advisable that customers always update to the latest version or contact support.
In all cases it is vital that you backup your instance before continuing.
1. Check your version by issuing the following command from an SSH session as root:
yum list installed matillion*
Assuming you're on a Matillion version from the last 6 months or so, you should see two packages:
2. Check package availability with these commands:
yum list available matillion-emerald --showduplicates yum list available matillion-emerald-cdata --showduplicates
3. To upgrade to a specific version, by default choose the latest point release inside the 1.x version.
You must keep the 1.x the same for both packages. For example you might choose:
matillion-emerald.noarch 1.31.8-1 matillion-emerald-cdata.noarch 1.31.8-3
4. Using the above example, you could then stop the tomcat service and install your selected version thusly:
service tomcat8 stop yum install matillion-emerald-1.31.8-1 yum install matillion-emerald-cdata-1.31.8-3
5. Finally, restart the tomcat server:
service tomcat8 start
Access Server Logs
Tomcat Catalina server logs of your Matillion ETL instance are available through the 'More' menu at the upper-right of the admin page.
This will download a 'catalina.out' file that contains the servlet's logs in plain text. The download link for a given instance is always:http://<ServerIP>/admin/FileDownload
Server logs record only for a finite time and size so it is likely your download will contain only recent history. For this reason, if an error is encountered, it is recommended that users attempt to reproduce the error then immediately download the log file.
Using the SSH command line, browse to the tomcat server log directory using a command such as below:
sudo ls -al /var/log/tomcat
Then use your preferred text editor to open the file for viewing:
sudo less /var/log/tomcat/catalina.2017-05-02.log
Use tail to view last few logged incidents. This will autorefresh the output can be useful to monitor a currently-running instance.
sudo tail /var/log/tomcat/catalina.2017-05-02.log