Connector Improvement: Factbird connector | Monthly re-syncs look back window
AnsweredHi,
We’ve noticed that our Factbird Fivetran connector performs a full historical resync every month to capture deletes. While we understand the need for periodic resyncs to ensure data integrity, especially for deletions, performing a full reload of the entire dataset monthly seems excessive—particularly since our older records rarely change.
This approach results in unnecessary data processing and increased costs, as we essentially pay to reload the entire dataset each month.
Would it be possible to implement a configurable lookback window for the historical resyncs? For example, if we could limit the resync to only the last 2 months of data, we’d still capture relevant changes and deletions without incurring the overhead of reloading years of static data.
This would give users more control over their connector behavior and allow for a more cost-efficient and targeted sync strategy.
Thanks in advance for considering this enhancement.
Best regards,
Ari Johannesson
-
Official comment
Hi Ari,
Thanks for your inputs. We are working on this, to ensure we are able to give the flexibility as well as data integrity. We will keep you posted as we progress on this one.
Thanks,
Unni
Please sign in to leave a comment.
Comments
1 comment