Name is required.
Email address is required.
Invalid email address
Answer is required.
Exceeding max length of 5KB

Validate component

When Matillion runs any jobs it always validates the component and then executes it. I understand, it needs to validate when developing and running for the first time but not sure whey it needs to validate every time it runs.

Even when a version is locked (no changes allowed without unlocking), it seems to validate the component before running it. Can we override this behavior? It seems like an overhead for a job with 100 different components, when it has to validate all 100 components every time (considerable overhead for NRT extracts) before running them.

4 Community Answers

Matillion Agent  

Ian Funnell —

Hi Devang,

Many thanks for raising this question.

At the moment, Matillion always re-validates components before running them, in order to regenerate accurate SQL in case of changes to variables and/or database structures. There is no option to override this, even if you are confident that the database objects have not changed, and you’re not using variables.

We are, however, considering how best to possibly optimize this approach in the future, without losing the integrity of the generated code.

Best regards,
Ian


Devang Patel —

Thank you very much. Makes sense.


Benj Fayle —

This is quite problematic if your query uses a limit on a subquery as the validator logic is flawed and removes LIMIT clauses from subqueries and replaces them with LIMIT 1 which can cause issues too.


Matillion Agent  

Laura Malins —

Hi Benj

As discussed on your ticket, I have raised this to the development team to investigate and fix if required.

Thanks
Laura

Post Your Community Answer

To add an answer please login