Destination Improvement: # in column name is replaced with _ while $ in column name is replaced with _ as well creating duplicates
Completed
We’ve identified an issue with column normalization during the sync process from our source table column name having #'s and $'s
The source table contains the following columns:
EMLTD#, EMLYR#, EMYTD#, EMLTD$, EMLYR$, EMYTD$
Fivetran is normalizing these column names by replacing special characters (# and $) with underscores (_), which is resulting in name collisions like:
EMLTD# → emltd_
EMLTD$ → emltd_
EMLYR# → emlyr_
EMLYR$ → emlyr_
EMYTD# → emytd_
EMYTD$ → emytd_
This leads to:
Duplicate columns in the destination schema
Incomplete or incorrect data syncing for these fields
Can you advise on the best way to handle these conflicts? Ideally, we would like to retain all original columns with their data intact.
Please let us know how to proceed to ensure full data fidelity from the source.
-
Official comment
Hi Sateesh,
We're excited to announce that the improvement for handling # and $ characters in column names is now live and generally available! This update addresses the replacement issues and prevents unintended duplicates in your destination tables.
You can review all the details in our official documentation here: https://fivetran.com/docs/connectors/databases#sourcenaming
Thank you all for your valuable feedback and patience as we worked on this enhancement.
Best regards,
Egidio Terra
Please sign in to leave a comment.
Comments
1 comment