Skip to main content
Docs
Feature Requests Onboarding Trust Center Status Page

Community

[Fivetran Copy Command: Split Files]

Answered
Pamela Antoinette Rivas User

 

Hello Team,

Would like to ask if we can do the splitting of the files prior to executing the Fivetran Copy Command?

This is to maximize the massively parallel processing (MPP) architecture to read and load data in parallel from files in an Amazon S3 bucket as recommended as well by the AWS Redshift Advisor. Please refer to the links/ticket ID below for reference.

 

Fivetran Support Ticket ID: 19661

Reference: https://docs.aws.amazon.com/redshift/latest/dg/t_Loading-data-from-S3.html

 

Thank you!

Please sign in to leave a comment.

Comments

1 comment

Date Votes
  • Official comment
    Kevin Kim Fivetranner

    Hi Pamela,

    Thanks for your question! We do split data into multiple files for Redshift automatically as part of our service. This happens behinds the scenes, so users do not have to worry about setting this up! 

    Fivetran will create multiple files to upload into Redshift for your syncs. Of course, our team is always looking to optimize performance and actively investigates possible improvements like concurrency on the destination side. Hope this answers your question, please let me know if you have any additional feedback.