Connector Improvement: Feature Request: Record-Level Sync Comparison Between Source and Destination
AnsweredHi Fivetran Support Team,
We recently encountered a data integrity issue where certain records from our Oracle source were not inserted into our Snowflake destination, leading to data discrepancies. While investigating, we noticed that Fivetran currently does not provide a native feature to compare or validate record-level sync completeness between source and destination systems.
We’d like to request a feature enhancement that allows users to view or export a comparative sync summary at the record level — highlighting any missing or mismatched data between source and destination for a given sync period. This would help proactively identify and address data integrity issues, especially in production pipelines handling large or critical datasets.
Please let us know if this enhancement is on your roadmap or if there are any workarounds available for monitoring record-level data consistency.
Thank you for your continued support and for considering this request.
Regards,
Shyamsunderreddy.P
-
Official comment
Hi Shyamsunderreddy,
We are currently rolling out a feature to validate customer data automatically as part of the connection's sync process. For Postgres, Oracle, SQL Server, MySQL, and SAP ERP on HANA connectors, we run periodic checks of data that sample data between the source and destination to check for data integrity. We are currently working on the customer-facing UX for this feature, so stay tuned for that!Could you tell me which connector type you're looking for? Please let us know if this capability would satisfy your need or if there is a more detailed report you would need.
-
Hi Kevin,
We are specifically using the Oracle connector. While the periodic checks on sample data are a good start, they would not fully meet our requirement. Our primary need is to ensure complete data accuracy verifying that every record from the Oracle source is correctly inserted or updated in Snowflake.
A comprehensive validation report comparing the source and destination at both the record count and column data levels would be essential. This would help us detect missing or mismatched records proactively and maintain data integrity across critical production pipelines.
Please sign in to leave a comment.
Comments
2 comments