Name is required.
Email address is required.
Invalid email address
Answer is required.
Exceeding max length of 5KB

Monitor long running processes

We did not see any out of the box options to monitor and notify if a job is running for more than certain time period. We have some processes which normally finish in 5-10 minutes range but every so often we see them running for a long time (> 30 minutes) depending or workload on the cluster. We are trying to find a way to monitor these jobs outside Matillion and send a notification. Can you let us know if this is something on the road map or an alternative way to achieve the same.

3 Community Answers

Matillion Agent  

Dan D'Orazio —

Hello -

I believe you can accomplish this by sending requests to the Matillion API

You may want to take a look at the rest/v1/group/name/{}/project/name/{}/task/running endpoint as a starting point. The response for this endpoint contains information like ‘enqueuedTime’, ‘startTime’, and ‘endTime’. You should be able to interrogate that response, for each job that’s in the request, and determine how long it’s been running.

Best -
Dan


Devang Patel —

Thanks Dan. Is that something that need to be setup for each job that we want to monitor ? Or can there be one API request / response for all of the currently running jobs.

Devang.


Matillion Agent  

Dan D'Orazio —

Hi Devang -

That call should return all the running jobs for the project specified. If you want all running jobs, across all projects, you would just send a get request for each project of concern.

For example:
rest/v1/group/name/Development/project/name/Accounting/task/running
rest/v1/group/name/Development/project/name/Supply%Chain/task/running

Where the Project names are ‘Accounting’ and ‘Supply Chain’, both in the ‘Development’ Project Group.

Best -
Dan

Post Your Community Answer

To add an answer please login