Connector Improvement: Oracle and HVA oracle connector improvements around data inconsistency and duplicates
Answered
We've observed that Oracle and HVA connectors fail to sync correctly when there are key or constraint changes in the source, resulting in duplicate data at the target (e.g., Snowflake). Fivetran recommends manual backup and resync, which isn't ideal since we expect automated, accurate syncing with proper reconciliation.
The connectors should handle such changes automatically by marking outdated records with _fivetran_deleted='TRUE' during sync and reloading the data from source to target, ensuring consistency without manual intervention.
There are many support tickets that we have raised for these duplicate or data inconsistency issues. For example you can refer to our latest one: 271407
-
Team, Could you please update on this request if it is being taken into consideration?
-
Rueben,
Apologies for the radio silence on your request.
Do you have a scenario in which you remove columns from the key (resulting in duplicate rows for what didn't use to be duplicates)? Whilst I agree that we should be able to handle such scenario cleanly, I would also expect that this is quite an uncommon scenario.
If you add a column to a key then I cannot see that you would end up in the situation you describe.
Or if you drop the constraint and nominate a new key then of course you could see duplicates.
Anyway I will look at the ticket you referenced and follow up internally.
Mark.
Please sign in to leave a comment.
Comments
2 comments