Hi. We are using matillion in three different instances (prod, qa, dev). We are exploring the possiblity to to store variables in a centraly managed database to be used by all environments. Do you have any suggestions in how this could be achieved. We have been thinking about using an rds instance but i cant figure out a way to assigning the varibles in the different instances in a smooth way.
Regards Johan Forssell
1 Community Answers
Ian Funnell —
Interesting idea there! Matillion is designed so the master default values of the Environment Variables are held inside its own metadata. You can set defaults per environment, but in your case it’s not going to help because you have separate instances.
Your idea of using RDS would work. You’d need a table with 3 columns: environment, variable and value. You can access the Environment name at runtime via the builtin “environment_name” variable, which would enable you to filter to the correct set. You could loop through the records inside a Python script, and set the values of all the Matillion variables at the start of every Orchestration job run.
You’d need to:
Make sure the actual variables are present in every instance
Remember to run the Python script at the start of every job
We do have some plans for extending Matillion’s API, and perhaps including some functionality to manipulate environment variable defaults, although there are no definite designs in place yet.