I am testing a new job that uses a table iterator, iterating over a large table (> 5,000 rows).
When I started testing, I noticed in the Task History that it said "5000 iterations generated" ... when I was expecting more iterations. Because there are more than 5,000 rows in the table.
My iterator turned out to be quite slow, and I canceled it before it finished, so I haven't seen what it does after those 5,000 iterations.
But what behavior should I expect?
(I don't see mention of this "5,000 rows" thing in the documentation of the Table Iterator component. Does the Table Iterator component have a strict limit of 5,000 rows...then it just stops? Or, is it just breaking things up into chunks of 5,000 iterations...such that after finishing the first 5,000, it would start the next 5,000?)
3 Community Answers
Paul Johnson —
All Iterator components support a maximum of 5000 iterations.
The design choice behind this is because all “bulk” operations should be pushed down to the database rather than be processed on the instance. Can you provide some detail around the use case? I might be able to offer an alternative solution.