If I back the number of columns down to 202 from 300, the validation succeeds. The union statement being generated is pretty big, about 31K in size. Is this error caused by some kind of limit in the component or in Redshift? Is there some parameter I can set to fix it or do I have to split the transformation into two with one doing the first 150 and the second doing the remainder.
3 Community Answers
Chris Tuerpe —
Thanks for contacting support. The error that you are getting is not generated from Matillion, it is generated from the database. I would like to recreate the error.
Would you be willing to forward along the export of the matillion job and the create statement of the source table.
I agree, it is definitely generated by the database. I was more wondering if you'd seen it before and if there was a parameter that redshift has that can be set i.e. size of SQL buffer etc. I'll open a support ticket and attach the code so it is easier to communicate. I'll reference this thread.