Talend Redshift performances

Talend Redshift performances

In this quick post, i wanted to show you how we can load data into Redshift without going throw AWS S3.

In this example we are using the smallest Amazon Redshift cluster size with only one node.

in this job we pull the data from MS server, we load the data directly to Redshift , 15k rows per second.

Talend has a native component that allows direct loading to Redshift.

Talend redshift

Talend redshift

 

by using parallel executions you can go up to x3 times quicker 45 000 rows / second

 

if you choose to commit by batch of 1000 rows

the performance increase to 75 000 rows per second which means you can load 4.5 millions per minutes.

remember you are not using S3 here , so there is no cost of S3 for this upload.

If you need any help on this, please contact us using the form below :

Talend Expert is the specialist of the implementation of Data Integration, big data and master data management projects using Talend. We are a Talend consulting firm trusted by the world’s leading companies. An IT consulting firm responsible for leading and supporting integration and big data projects..

Talend Expert can assist your organization with the adoption of Talend’s suite of products, Talend consulting for Open Studio, Integration Suite, the MDM suite, ESB and Big data suite, and in any stage of your project including Architecture and Planning, Design, Development and Deployment, Optimization of data integration jobs, Audit .. etc.

 

Leave a Reply