PostGreSQL Target types mapping with source system types
Which destination?:
PostGreSQL
Additional details:
Source in my case is SAP.
Voici la version corrigée et renforcée :
Which destination?: PostgreSQL
Additional details:
My source system is SAP.
SAP provides full data type definitions through its DDIC (Data Dictionary), which contains precise field types, lengths, and domain definitions for every table and column in the system. It is therefore surprising that Fivetran does not leverage this metadata during the initial connector setup to create the target table with the correct column types and lengths in PostgreSQL.
The current behavior causes a recurring issue: Fivetran frequently issues ALTER TABLE statements against PostgreSQL when it encounters a value whose length exceeds what was observed in the initial dataset. This is not sustainable for the following reasons:
- Any
ALTER TABLEon a table that has dependent views will cause the Fivetran sync to fail, requiring manual intervention - This forces us to monitor the connector every morning, which is not acceptable in a production environment
- The root cause is avoidable — SAP DDIC already contains the exact field lengths and types
Proposed enhancement: When initializing a new SAP connector, Fivetran should query the SAP DDIC metadata to generate the target PostgreSQL table with the correct column types and maximum lengths from the start, rather than inferring them from the first batch of data.
This would eliminate the need for ALTER TABLE statements during incremental syncs and significantly improve connector reliability.
Thank you for considering this enhancement.
Please sign in to leave a comment.
Comments
0 comments