Google 3rd Party OAuth Setup
For other options on authenticating Google 3rd Party services using Service Account, see here.
Note: It may be that some of these services incur costs to your GCP account. Please consult the GCP documentation for pricing.
Matillion ETL enables users to perform orchestration and transformation jobs using their data from the following Google 3rd party apps:
- Gmail Query
- Google Adwords Query
- Google Analytics Query
- Google BigQuery
- Google Custom Search Query
- Google Sheets Query
- YouTube Analytics Query
- YouTube Query
Creating a Project
In order to connect to Google data from various services, Matillion ETL requires that OAuth is configured. Before you can configure OAuth in Matillion ETL you first need to enable it in your Google account. To do this:
- Navigate to https://console.developers.google.com
- Log in with your normal Google credentials.
- Click Select a project.
- Click NEW PROJECT (top right) or select one of your pre-existing projects to use if you have any.
- Enter a Project Name, and ensure that your Organisation and Location are correct, then click Create.
When your project has been created (which can take a few minutes), you will be redirected to the Dashboard.
- Once you return to the Dashboard, click Credentials on the left, as shown in the below image.
- Now click OAuth consent screen and provide an Application Name. You also need to decide whether to make the Application type Public or Internal. You can also add a logo for your application, if you so wish.
- Next, you'll need to add an Authorised Domain.
The following user assistance is taken from Google Cloud Platform Console Help:
To protect you and your users, Google restricts your OAuth 2.0 application to using Authorized Domains. If you have verified the domain with Google, you can use any Top Private Domain as an Authorized Domain.
After you add an Authorized Domain, you can use any of its subdomains or pages, and any other associated country codes.
For example, if you have added https://google.com you can use https://flights.google.com, https://google.com/flights, and https://google.co.uk/flights.
For additional help with Authorised Domains, we advise referring to Google's Documentation.
- Once you click Save, you will be taken to the Credentials page. From here, click on the Create credentials button, and then click OAuth client ID.
- Select Web Application from the Application Type list, and you will then be prompted to give a Name for this OAuth client ID. Then, add an Authorised redirect URI in the second Restrictions field. The Authorised redirect URI must be an IP address. It should be in the following format:
http(s)://<matillion instance hostname>/oauth_redirect.html
If you access Matillion ETL via an IP you will need to get your system administrators to provide a DNS entry. Please consult the documentation on Changing the host file.
- Click Create and make a record of your client ID and client secret. You will need to use these credentials in the Matillion ETL OAuth setup.
- Finally we need to authorise the application to use the correct google API's. Select APIs & services → Library from the left-hand menu and find the API you want to use.
- For Google BigQuery choose BigQuery API.
- For Google Sheets choose the Google Sheets API.
- For YouTube Analytics choose YouTube Analytics API.
- For Google Drive Table choose Google Drive API.
- For Google Custom Search choose Custom Search API.
- For Adwords the API is automatically enabled; however, you will need to apply for a developer token.
- For Google Analytics choose Analytics API.
- In the image below, we've chosen the Google Big Query API. Our next step is to click the Enable button.
- Next, see Setting up OAuth in Matillion ETL to finish the configuration.
Note: Google OAuth from Matillion ETL may request permission to access multiple Google services such as YouTube, Adwords, Analytics and Sheets, even though you might not use such services yourself or indeed have them enabled in the API. This is because the Google 3rd Party OAuth is responsible for authorising potentially many different components in Matillion ETL at once.
Enabling Credentials Warning
When inspecting a particular API (accessed through APIs & services), some services (such as Custom Search) might show a warning indicating the need for additional credentials.
Clicking on the Create Credentials button will guide you through a short process to create the credentials required. For example, in the case of the Custom Search API, an API key must be generated.
For credentials such as API Keys, it is usually a good idea to make a note of them once created, because they will likely be required to configure the corresponding Matillion ETL component.