RSD Generator

The RSD generator allows users to automatically create their own RSD for a table of their choice such that this RSD can be used with the API
Query component to load data from a 3rd party into a table. This is all reliant on the 3rd party allowing database access via a RESTful API.

RSD Generation and its configuration is based on the CDATA JDBC REST driver. Column definitions and RepeatElements can also been considered in this same manner.

The RSD generator can be reached through the Matillion ETL client through Project → Manage API Profiles and creating or editing a profile. From there, click 'Generate' to bring up the Generator dialog.

Table Name Any name you want to give to the table
Description Any description you wish
XPath to Repeat Element Endpoint that exposes repeating elements to be taken
Data Format The data format that the repeating elements exist in.
URI URI of the endpoint with file extension
Properties API parameters that might be needed: e.g. username and password, separated by semi colons.
 

Note on using static files.

The RSD generator can be pointed at static files and does not necessarily have to go directly through the service. The URI property can be pointed at an XML or JSON sitting on S3 (this file must be public!), for example, instead of expecting one to be returned from an API call.

This method is recommended as it has the advantage of allowing users to use their own preferred API tool to construct their API calls. This can be particularly useful when your API call has many parameters that would be too cumbersome to place together in the 'properties' field or if using a POST method instead of GET.

After using a static file in the generator, the RSD can be altered by hand to change necessary parts such as the uri or to create additional structures.


Example

Table Name: TicketData
Description: Ticket data taken from zendesk
XPath to Repeat Element: /tickets
Data Format: Choose between data being collected in JSON or XML format.
URI: https://matilliondevelopment.zendesk.com/api/v2/tickets.json
Properties: AuthScheme=BASIC;User =user@matillion.com;Password =mypassword;OAuthVersion=;




Clicking 'OK' will save the RSD for use. Now, editing the 'Profile' property of an API Query component will yield this option available to use.