Skip to main content
Docs
Feature Requests Onboarding Trust Center Status Page

Community

Connector Improvement: CSV file format flexibility improvements

Answered
Achyuth Pramod User

Hi team,

I wanted to ask if there was a way to improve the way fivetran handles csv files that arent very conformant to RFC 4180 standards. This means dealing with files that have records with escape characters within them (enclosed with " ") or backslashes/special characters breaking records. We have a connector to an SFTP source that is unusable because of this.

A solution like that or the ability to add additional parameterization so customers are able to handle this would be great.

Let me know if you have questions.

 

 

Please sign in to leave a comment.

Comments

2 comments

Date Votes
  • Official comment
    Parmeet Kohli Fivetranner

    Hi Achyuth,

    Thanks for flagging this. With our current configuration options, Fivetran should be able to ingest CSV files containing records with escape characters. 

    I’ll reach out via email to review your connector setup and file sample in more detail so we can identify the best way to resolve this.

    Thank you,
    Parmeet

  • Shivaprasad Mogili Fivetranner

    Hi Achyuth,

    We recognize the challenges in handling CSV files that do not fully conform to certain patterns. We are currently working on enhancements to the file connector parsing options, which should provide greater flexibility for these scenarios.

    If you are open to it, one of our product managers can reach out to discuss your specific requirements and use case further. This will help us ensure that the improvements we develop meet your needs.

    Let us know your preference.

    Thanks,
    Shiva