Skip to main content

Community

Connector Improvement: Ablity to replicate multibyte characters

Planned

Please sign in to leave a comment.

Comments

2 comments

    Hi Ankur,

    Thank you for the detailed explanation and for highlighting the challenges with multibyte character replication from MSSQL to Sybase.

    We’re actively looking at how we can enhance HVR’s capabilities in this area to solve this issue. Specifically, we’re exploring the option to override the default database encoding with a user-specified encoding to ensure accurate replication of multibyte characters, such as Japanese and German.

    This improvement is currently under feasibility assessment by our engineering team, and while we don’t yet have an estimated timeline, we’ll keep you updated as we make progress.

    Best regards,
    Edwin

     

    Hi Edwin/FT engineering, 

    Post the active ongoing discussion between all parties including FiveTran and new findings here is a high level summary ask : 

    Support HVR Replication of non-english/international characters in varchar fields from :

    1. Sybase ASE - ISO 8859-1 (Latin-1)  to Microsoft SQL Server -Latin1_General_100_BIN2_UTF8

    2. Microsoft SQL Server -Latin1_General_100_BIN2_UTF8 to Sybase ASE - ISO 8859-1 (Latin-1)

    3. Sybase ASE - ISO 8859-1 (Latin-1)  to Snowflake (UTF8 default collation)

    Based on our in-house tests/repro, none of the above scenarios work.