Skip to main content
Docs
Feature Requests Onboarding Trust Center Status Page

Community

Connector Improvement: Google Cloud Storage: Include a string replacement part

Completed
Christoph Gmeiner User

Hi there,

it'd be very helpful if we could (in case of csv files) replace a certain string with another (also using regex).

thanks and best regards

Christoph

 

Please sign in to leave a comment.

Comments

6 comments

Date Votes
  • Official comment
    Alison Fivetranner

    Hi Christoph,

    Thanks for submitting this request! What an intriguing suggestion. Can you tell me more about the use case you have for this?
    Where does this data come from?
    What strings would you be looking to replace and with what replacement?
    Have you considered running a script on the files either before they are picked up by our connector or afterwards when they are in your Data Warehouse?
     
    Listening to our customers is at the center of building Fivetran so I really appreciate you sharing your experience.

    Best regards,
    Alison
  • Christoph Gmeiner User

    Hi Alison,

    thanks for your message.

    The reason is when we have malformatted CSV files provided by 3rd parties. Currently e.g. we wrote a default python module which escapes " properly.

    thanks and best regards

    Christoph

     

  • Alison Fivetranner

    Hi Christoph,

    Thank you for the additional information.
    Do you find it is just incorrectly escaped " that your pre-processing script has to handle?
    Have you found you need to run the script for multiple 3rd parties or have you had to write more than one of these pre-processing scripts?

    I really appreciate the detailed insight.

    Best regards

    Alison

  • Christoph Gmeiner User

    Currently only the one. thanks and best regards

    Christoph

     

  • Alison Fivetranner

    Hi Christoph,

    Thanks again, We don't currently have this enhancement planned into our roadmap but it does seem like handling a few more variants of escaping for " 's in our csv processor could be very helpful, I will add it to the things we explore.

    Best regards

    Alison

  • Alison Fivetranner

    Hi Christoph,

    We are adding some additional handing for double quotes to out CSV parsing logic. I'm hopeful your special script wont be need after 23 May.

    Please give it a try and let us know if we've solved the issue.

    Best regards

    Alison