Connector Improvement: Data Reconciliation
Hi, my team ran into an issue replicating from a Postgres source into a Snowflake destination. A handful of records were not successfully brought into Snowflake, and even when the records were updated in the Postgres source they were not brought into Snowflake destination. My team struggled to figure out which records in which tables were affected, and we are not confident we identified all cases of this. It would be great if Fivetran had a feature to detect when records are missing or different in the source and destination. This could trigger a notification to users to take some action, or the missing records could be automatically synced. This data reconciliation feature would increase Fivetran's reliability and give users a lot more confidence in the product.
-
Official comment
Jacob,
The Fivetran HVR technology has a data compare feature that validates the data and create a report on differences. See https://fivetran.com/docs/hvr6/advanced-operations/analyzing-diff-file for more information.
We are building data validation capabilities into Fivetran so more users can benefit from such capabilities going forward.
Mark.
Please sign in to leave a comment.
Comments
1 comment