Name is required.
Email address is required.
Invalid email address
Answer is required.
Exceeding max length of 5KB

Task Log in e-mail in Mtaillion for Bigquery

I need to set up e-mail notifications for job status and message in Matillion for Bigquery. Is there any example or tutorial for this?

9 Community Answers

Matillion Agent  

Kalyan Arangam —

Hi Manan,

There are no direct options for sending emails from the tool. You could use a python script component and write some code to send an email.

If you are after sending notification emails (without attachments), you may write to a pub-sub topic which is then forwarded to a service that can process that message and send an email. See the following link for sending an email notification (using Mailgun) by writing to a pub-sub topic.

https://cloud.google.com/container-builder/docs/configure-third-party-notifications

Hope that helps.

Best
Kalyan


Manan Kshatriya —

I made a text variable and assigned it to store status of the job after the completion, but in the mail, the variable is blank.

I want to set up an email to get status and message and other details of a failed job in an email.

Please help


Matillion Agent  

Kalyan Arangam —

Hi,

Its difficult to comment without looking at your job and how you have configured your components. Are you sending email via Python?

I would recommend printing the variable just before you send via email to see if the appropriate value is being captured.

If task-history shows the appropriate message, this may mean you need to review the component/code that is sending email.

Hope that makes sense.

Best
Kalyan


Manan Kshatriya —

Hey, I just realized that my python component is also starting simultaneously along with the 'database component' (which i am failing deliberately to check whether variables are getting set or not).

How do architect my job so that python component waits for all the database components to finish and python scripts executes even some database components have failed?

If i use OR function, python component starts simultaneously and hence i can't capture the variables which are being set after the database components gets completed.

AND function will not work if one database components fails.

Please help.


Matillion Agent  

Laura Malins —

Hi

That’s a good question. In this circumstance I’d usually recommend an OR component but as you say it won’t wait for everything to finish. So I think what I’d do in this case is write call the python from a different Orchestration job. In this Orchestration job, call a sub-job which runs your database components. Then you can connect the python to both the success and failure links on this. You might need 2 copies of the python or go via a couple of OR gates.

Thanks
Laura


Manan Kshatriya —

Hey,

Can you elaborate ? i am having a hard time understanding this.
Also , i am not clear how to set a variable which will hold the status of the job as a whole and not the status of a single component.


Matillion Agent  

Laura Malins —

Hi

Apologies I wasn’t clear. I’d do the below steps:

1. Put all of your database query components in one Orchestration job, let’s call it run_database.
2. Create a new Orchestration job. In this job add a Run Orchestration Component and link to your run_database job
3. Now add a Python script component to send your email. Connect this to the “On Success” link of the Run Orchestration component
4. Connect another, identical, Python component to the failure link
5. Right click on the canvass of your Orchestration job with the python components in and create a new variable with the Manage Variables option
6. Click on your Run Orchestration component and select the Export tab at the bottom.
7. Click on Edit to associate the Status with your variable.

Please note steps 3 and 4 could be replaced with 2 OR gates connecting to one Python component rather than 2 Python components.

Thanks
Laura


Manan Kshatriya —

Hey Laura,

Thanks a lot. You cleared a lot of things.

I am able to print the status but can I print the message description of the error?

Thanks


Matillion Agent  

Laura Malins —

Hi

Yes you can grab the message too – this is the message option in the Export.

Thanks
Laura

Post Your Community Answer

To add an answer please login