Transformations: Add option to not filter Quickstart Transforms on Deleted flag
CompletedQuickstart transforms filter on the _FIVETRAN_DELETED column. All Transforms have identical code. Here's a code sample:
FROM "RAW"."SALESFORCE"."USER" AS mt
WHERE NOT mt."_FIVETRAN_DELETED"
Our problem is that we're reading from the Quickstart views and building Slowly Changing Dimensions (Type 2) models. When we see a record added or modified, we make the appropriate change to our models. But when the record is soft-deleted, it just disappears from Quickstart views, so we don't know it was deleted. Right now we have records in our models that look like they're active, but have been deleted, the _FIVETRAN_DELETED is True.
We'd like to have the Transformation filter on _FIVETRAN_DELETED to be optional. When we choose to not filter, the records with _FIVETRAN_DELETED set to True would be visible in Quickstart tables/views.
For our purposes, one Y/N option for all Transforms would be perfect. But a separate option for each table would also be fine.
-
Official comment
Hi John,
Update: This is now available. We've added a way to include deleted records in Salesforce formula field translation models (previously these were filtered out by default).
Please reach out to Fivetran Support and ask them to enable it for your connection. Note it applies to the whole connection (not individual tables), and formula fields that depend on other Salesforce objects still won't populate for deleted records.
Best,
Sadie
Please sign in to leave a comment.
Comments
1 comment