we are on Matillion version 1.38. Our question is around scheduling capability / behavior of Matillion. When we schedule a job to run every 5 minutes and if the job runtime in some instance is 15 minutes then we see that next two instances of schedule are queued to be run after the first one finishes. Is there a way to setup a schedule where it does not run the (skip) the next schedule if the current instance of the same schedule runs for more than recurrence interval of the schedule ? (in this case 5 minutes).
2 Community Answers
Dan D'Orazio —
Hi Devang -
This isn’t achievable via the schedule, but you may be able to handle this functionality in the job itself.
One way to accomplish this would be to schedule the job once per day and then invoke the job again, from inside the job, using an SQS Component. You could put a Python Component in between the last component and the SQS Component as well, to provide some sleep capability if you want to wait before restarting the job. You would want to provide some conditional logic as well, to ensure that the last time it runs, before the next schedule, allows the job to complete without restarting itself.
Let us know how that works out, or if a quick screen share might help explain the pattern better.