Split Data into Multiple output CSV Files in Talend

 Scenario: Split the incoming 5 million data into multiple files of 1 million records in each

Components used for demonstration:

TRowGenerator: used for creating sample records

TFileOutputDelimited: used for writing a CSV file

Flow:

Configuration:

TRowGenerator:

Add the sample columns and set the "Number of rows for RowGenerator" to 5 million records

TFileOutputDelimited: Basic Settings

TFileOutputDelimited: Advanced Settings

"Rows in each output file" - mention the number of records that should be in each output file. In our case, it is 1 million records

Generated Output Files:

Posted in: ETL, File, Talend