Also as an extension to above question, will all of the spectrum transformation components (external table rewrite, external table output, copy to external table) support option to add header row since I am guessing behind the scene they are all using unload to S3 command.
We looked at the text output component and seems like it has option to output with header. This would be a good option for smaller workloads. In our case we are planning to regularly unload cold data to S3 in specific spectrum structured tables for multiple tables. Ability to do above in parallel will be key as data volume grows. If we get some insight around timing of this feature, we would rather wait for a release than to develop something with text output component and rewrite the process.
I appreciate your request for timing on this feature, but as we use an agile development methodology we cannot say with precision when this feature will be available. Improvements such as this one are often added (and sometimes dropped) at the last minute before a release.
The best we can say is that the soonest it might be included is in our next release, which is likely in 6 to 8 weeks. However, it’s quite possible that this feature would fall into a subsequent release.
Hope that makes sense and apologies for not being able to offer a more precise timeline there. Please keep an eye on our release notes, which is the best way to stay up-to-date on what features have been incorporated into which release: Matillion ETL for Redshift Release Notes