For tables that have a schema, it would be useful if there was a way for custom connectors (AWS Lambda, Azure Functions, Google Cloud, etc.) to specify that any rows from that table should be deleted if their primary key does not appear in the latest batch of data.
Example scenario: An API offers an endpoint to get a full list of users (with an ID for each user) but does not offer an endpoint to track user deletions. Historically, this endpoint would return users [1, 2, 3, 4, 5]. Since then, users 2 and 4 have been deleted, so now the API returns users [1, 3, 5].
Current results: Although the API (and therefore, custom cloud connector too) only returns users 1, 3, and 5, the table in our data warehouse will still have 5 users. There isn't a way for the custom cloud connector to know that our warehouse still has users 2 and 4 (in order to send a deletion request).
Desired results: Since Fivetran manages the rows in the warehouse and already knows about users 2 and 4, we should be able to set a flag value to specify that Fivetran should only keep rows in the table if they still appear in the latest set of data. In this case, since Fivetran wouldn't see users 2 and 4 in the result set of [1, 3, 5], it should mark users 2 and 4 as deleted.
TL;DR: a "deleteRowsNotPresentInLatestResponse" setting (but with a better name)