Destination Improvement: Redshift - Remove Deleted Source Data From the Destination
PlannedCurrently, in the Redshift [destination], Fivetran is executing a soft delete operation by marking the _fivetran_deleted field as TRUE. We need to set up certain tables in a way that allows Fivetran to carry out hard delete operations, thereby removing the record in the destination automatically. This setup will ensure that the table remains an exact mirror of the source and avoid steps like DELETE or DROP command to hard-delete rows deleted from the source, or write post-load transformations to delete data where _fivetran_deleted = TRUE.
-
Official comment
Hi Rajeev,
Thanks for writing in. Good news, this is on our roadmap and I expect that we will release support for this in the next few months. We will support hard deleting rows first, and will then look to support DDL as well (i.e. dropping a table/column that has been dropped on the source).
Please sign in to leave a comment.
Comments
1 comment