Summary:
Enable HVR to automatically map source UTF-8/CESU-8 encoded character columns to NVARCHAR (or NCHAR) on the target system to preserve Unicode characters and prevent encoding issues.

Current Behavior

• Source: Oracle Database (UTF-8 → CESU-8)
• Target: Azure SQL Database (SQL_Latin1_General_CP1_CI_AS → WINDOWS-1252)
• When replicating data, columns defined as VARCHAR on the target cannot store UTF-8 characters.
• Example:

-- Source
  select substr(groes,5,1), ascii(substr(groes,5,1)) from mara where matnr='000000000001253780';
  Result: ⅛ (ASCII: 14845339)

  -- Target
  select substring(groes,5,1), ascii(substring(groes,5,1)) from mara where matnr='000000000001253780';
  Result: ? (ASCII: 63)

• Data integrity is lost because special characters are replaced with ?.

Problem

• Current workaround requires manual configuration:
  • Adding ColumnProperties action with Datatype=nvarchar and specifying column length.
  • This is impractical for large schemas and dynamic environments.
• With table property with the coercion type 'ENCODING'
• End users see incorrect text in reports and analytics, see above.

Expected Behavior

• HVR should automatically detect source encoding (UTF-8/CESU-8) and map all CHAR/VARCHAR columns to NCHAR/NVARCHAR on the target, if target does not support UTF-8.
• Preserve all Unicode characters during replication.

Business Impact

• Ensures accurate data representation in data warehouses.
• Eliminates coercion errors
• Reduces manual intervention and risk of human error.