Question
I have partitioned your database and database tables. Periodically, I will perform a ‘DROP’ operation on those tables. Will Fivetran automatically resync if I drop a database partition?
Answer
Perform a resync against the table or connector to ensure data integrity.
Context
Dropping a partition won't incur a Fivetran resync, but neither will this mark a record in the dropped partition as deleted.
What will happen is:
- Fivetran will report that a ‘DROP’ event has occurred in the connector logs:
- "properties" : { "REMOVAL" … }
- The record will still exist in the destination with '_fivetran_deleted = false'
A resync will identify that those records no longer exist in your database and will change the ‘_fivetran_deleted’ flag to ‘true’.
Considerations
Fivetran will not automatically resync this data, so it is your responsibility to leverage the Fivetran Log Connector , those same logs surfaced in the Fivetran UI or an external logging service to manually or programmatically trigger a resync