Use Case/Scenario:
You have partitioned your database and database tables. Periodically, you will perform a ‘DROP’ operation on those tables.
Fivetran Recommendation:
- Perform a resync against the table or connector to ensure data integrity
Context:
Dropping a partition won't incur a Fivetran resync, but neither will this mark a record in the dropped partition as deleted.
What will happen is:
- Fivetran will report that a ‘DROP’ event has occurred in the connector logs:
- "properties" : { "REMOVAL" … }
- The record will still exist in the destination with '_fivetran_deleted = false'
A resync will identify that those records no longer exist in your database and will change the ‘_fivetran_deleted’ flag to ‘true’.
Considerations:
-
- Fivetran will not automatically resync this data, so it is your responsibility to leverage the Fivetran Log Connector, those same logs surfaced in the Fivetran UI or an external logging service to manually or programmatically trigger a resync