Question
I would like to cleanse test data that is present at the source (and has synced to the destination). What are the best practices?
Environment
- Data source
- Fivetran dashboard
- Data destination
Answer
Please follow the steps below:
- Remove all the unwanted (test) data from the warehouse first.
- Pause the connector.
- Cleanse the source of all testing data.
- Resume the connector - this will fetch recent data that does not contain testing data.
Consideration
- The assumption here is that you do not want the test data to be written over to your warehouse or to be present at all.
- A historical re-sync will only refresh all the data existing in the warehouse, unless you would like to do this as part of the cleansing process it is not really required.
- I suggest to pause the connector as I believe cleansing the data will take a longer time to wait until the connector is not active between two syncs.