Name is required.
Email address is required.
Invalid email address
Answer is required.
Exceeding max length of 5KB

Password option for Environmental variables

Hi

We have a large number of tables to connect to and are using environmental variables as much as possible to reduce the amount of work needed for config changes, however for the password to a database if we use an environmental variable the value is in plain text.

Would it be possible to add a new type to the environmental variables that doesn't show the
value ie password field type?

Thanks
Hamish

12 Community Answers

Matillion Agent  

Ed Thompson —

Hi Hamish,

We hope to introduce a password manager which will allow you to manage passwords across a project, and then reference the password by name in the components. At that point, using a variable for the password’s name will be fine, and its value will be retrieved when the job runs.

Because of this plan, it is unlikely we will introduce a password-style variable type.

I’ll post again once we know when it will be available.

Regards


David Murphy —

Hi Ed

Has there been any update on this functionality of using environmental variables from the Password Store?

Regards,
David


Matillion Agent  

Ian Funnell —

Hi David,

Matillion now does have the Password Manager which Ed was referring to last year.

If you’re not taking advantage of this functionality already, it could be what you are looking for?

Best regards,
Ian


Johan Forssell —

Is it possible to acces the passwords and usernames from batch- and pythonscripts?


Johan Forssell —

.. and i mean bash not batch


Matillion Agent  

Ian Funnell —

Hi Johan,

Currently this isn’t possible unfortunately, although we do have a change request (internal reference EMD-3076) to add this functionality.

Many thanks for the suggestion.

Best regards,
Ian


Johan Forssell —

To bad its not possible but its good news to hear that you are working on it. So my only way going forward is to enter my credentials in plain text either in the script or save them as a variable?


Matillion Agent  

Ian Funnell —

Hi Johan,

For Bash and Python scripts that’s the case, yes: you will need to embed them in the script, either as a string literal or using a Matillion Environment Variable. It’s better to use an environment variable because then it can automatically have different values in dev/test/production etc.

I’ll update this thread once additional options have become available.

Best regards,
Ian


Sam Johnson —

Hi Ian,

We have a similar problem to this, we spoke on a demo last week so you may have answered us already although now when implementing it we are coming across a blocker.

Basically we have a number of users that will have a job run for each of them where we can pass in a variable and run a job but then need to dynamically extract a password for sftp upload via the password manager. We can do this all the way up to the last stage but we have to set the password hardcoded in the options for the sftp put object. The sftp is for each user therefore don't want to have a separate job for each user? We also want to be careful about where we store the passwords for these users.

Cheers,
Sam


Matillion Agent  

Ian Funnell —

Hi Sam,

Thanks for raising this question.

The Matillion development work EMD-3076 on password management has not yet been completed, but it won’t include the possibility to have passwords vary by “external” (i.e. non-Matillion) user. The reason being that your list of users is actual data rather than metadata.

To avoid hardcoding, and keep the passwords secure, the solution is to implement some kind of data-driven password management protocol yourself. Examples would be:

  • Include the password in a data table containing your list of users, and use the Table Iterator
  • Same but encrypt or obfuscate the password
  • Write a “getPassword” operating system utility that behaves differently in the Live and Development environments, and which requires a “username” parameter
  • Similar but implemented as a REST web service, and maybe invoke via Python or an API Query

You would need to choose a technique that matches your needs both in terms of convenience and security.

Best regards,
Ian


Johan Forssell —

Hi Ian.
Any ETA on including the use of passwords contained in the password manager in scripted components such as python and bash?

Regards
Johan Forssellk


Matillion Agent  

Ian Funnell —

Hi Johan,

Sorry, I still don’t have an ETA on this functionality. It’s still in the development backlog, but has not yet been allocated to a planned Matillion release.

Best regards,
Ian

Post Your Community Answer

To add an answer please login