Connector Improvement: Shopify Configurable Historic Data Loads
Not plannedFor some connectors (e.g. GA) we are able to set the amount of historic data we load into our database (e.g. load the last 6 months and then schedule loads going forward)
For Shopify we only have the option to pull back data for all time, which is time consuming (first test load took 2 days) and additional cost.
Can a field be added to allow us to configure when we start loading data from (e.g. 12 months, 6 months, 3 months etc)? This date would then be kept so if historic sync's were re-run in the future, there would be the option to load data back from the initial date set (e.g. 6 months from when the connector was first setup)
Thanks
-
Official comment
Hi Sean,
Thanks for the request. Generally, missing historical data is considered by Fivetran to be a violation of data integrity. Is your use case such that there is no need to analyze data older than some set range?
Please sign in to leave a comment.
Comments
1 comment