Name is required.
Email address is required.
Invalid email address
Answer is required.
Exceeding max length of 5KB

Out of Memory

Hello,

My Matillion has restarted several times due to Out Of Memory errors.
I am doing initial load of AdWords data for about 200 accounts for the past 2 years. So I have a loop over the accounts and then another one over monthly intervals. This results in about 5000 iterations in total. The job usually makes about 1000 iterations before failing. Data volume is not the issue, because when I start from where the job failed, it runs fine for another 50 accounts or so.
I have tried to monitor the memory usage - it usually starts at 37-38% and slowly crawls up to 42-43% at which point it fails, regardless of how much time it took to get there. Also the log doesn't have any indication - the only relevant message is "# java.lang.OutOfMemoryError: Java heap space".
Are there any limits, timeouts or other considerations I should be aware of when I design the jobs to avoid this? Ideally, I would like to let the job run for a day and catch up with the data.

1 Community Answers

Matillion Agent  

Kalyan Arangam —

Hi Danail,

Thanks for contacting support.

It might help to review some files from your environment before commenting on this.

Unfortunately, one cannot add attachments to a a public ticket like this. I will create a private ticket and request you for further information. Please look out for another email shortly.

Best
Kalyan

Post Your Community Answer

To add an answer please login