Name is required.
Email address is required.
Invalid email address
Answer is required.
Exceeding max length of 5KB

Loading data from Redshift into RDS (SQL Server)

We have a scenario where some data needs to be transferred out of Redshift and loaded into a SQL Server RDS instance. Is there a Transformation to do this?

5 Community Answers

Matillion Agent  

Laura Malins —

Hi Tejas

You can do this in an Orchestration job using the RDS Bulk Output component.

Thanks
Laura


Tejas Baxi —

Hi Laura,

I don't see SQL Server as an option in RDS Bulk Output (Only Aurora, MySQL, Mariadb,Postgres.Can you attach an image or provide step by step instructions on how to do this using RDS Bulk Output


Matillion Agent  

Laura Malins —

Hi Tejas

Apologies I misread your initial message. Yes you can only export to Aurora, Mariadb, MySQL and PostgreSQL.

You can bulk export to MS SQL Server, but to do this you’ll need to install the bcp command on the server

Another option is to use the S3 Unload component to send the data to files in S3. You can then import it to MS SQL server from there

Thanks
Laura


Tejas Baxi —

Thanks , Laura. That is a disappointment. Is this feature on the product roadmap or likely to be included soon where Matillion can write to SQL Server?


Matillion Agent  

Kalyan Arangam —

Hi Tejas,

We are able to perform bulk output to certain RDS databases as they support Java bulk-import interfaces and the respective utilities are either open source or allow the bulk-import drivers to be bundled with Matillion.

Unfortunately bcp utility which (I think) is the only bulk-load option for SQL Server cannot be run from linux. Also, microsoft’s licensing restrictions do not allow us to bundle their software with matillion.

Other customers have requested this feature in the past. Unfortunately, we are unable to support this option due to the reasons listed above. Our suggestion for SQL Server is to export data via S3 Unload then perform bulk import from a windows machine (optionally) hosted in AWS. You may use the S3 Get component to copy files from S3 to a Windows Share on this windows machine.

I think the latest version of SQL Server does run on Linux so I suspect its possible to run BCP on Linux in future.

Hope that gives some context.

Best
Kalyan

Post Your Community Answer

To add an answer please login