Other: Information on incremental sync results via API
AnsweredPlease add functionality to the API so the information on whether any data was replicated in an incremental sync is available to the API user.
This info helps to determine whether it is necessary to run subsequent transformations - on not - in case no data was replicated.
-
Official comment
Hey Victor,
We're thinking about doing this both in the API and via a trigger in the user interface for transformation jobs. Are you executing transformations outside of Fivetran?
Jimmy
-
Yes, Jimmy. We run transformations in DBT on Snowflake. The are triggered by Fivetran after a sync cycle completes.
If we don't have the info above, DBT always starts, initializes all models (may take up to 1 min), and then replicates nothing if there were no synced data. Our syncs in production run every 5 min or more frequently, so we pay a lot of money for useless Snowflake usage on multiple environments for many clients - to start DBT, and to find out that there were no synced data in the last cycle.
Adding the functionality that you described will save us a significant amt. of money.
Thank you.
Please sign in to leave a comment.
Comments
2 comments