Connector Improvement: reflect in fivetran rest API calls the offset of each table for shopify connectors
Answered
Sometimes the historical sync for connectors (specifically shopify ones) may take several weeks.
This is especially true with mature shopify stores.
In any self hosted ETL you would easily be able to understand what is the offset of the ETL, with fivetran shopify connectors it is simply impossible.
Please reflect via fivetran rest API the offset (how much into history ) did fivetran already complete
for instance, in the following get call
https://api.fivetran.com/v1/connectors/***/schemas
the response includes tables
you could reflect the off set (the updated_at date) per each table
-
Hey Alon, Calvin from the Product team here!
Thanks for this request - I can understand why this information may be useful. To clarify, would it be helpful to get information on the state/cursor values of each endpoint for a corresponding connector? See example below:
"backwardCursor": "2021-05-03",
"backwardSyncLimit": "2020-08-26",
"forwardCursor": "2022-10-31"
Please sign in to leave a comment.
Comments
1 comment