Connector Improvement: Dynamic date range filter
AnsweredWe would like to request support for dynamic, rolling date range filters on connector tables (for example, current_date - 30 or current_date - 7) that apply incrementally rather than as a static filter.
Currently, when we apply or modify a date filter on a table (as shown in the attached example), 
Fivetran triggers a table-level historical re-sync. In a recent instance, this resulted in a re-sync that ran for approximately 6 hours, extracting around 460 GB of data.
Our objective is to configure a dynamic date range filter that:
-
Automatically adjusts over time (rolling window)
-
Syncs only data within the defined range
-
Does not trigger a historical re-sync when the filter definition changes
-
Applies incrementally rather than statically
Business Impact / Justification
-
Cost Optimization: Avoiding unnecessary historical re-syncs would significantly reduce data extraction and compute costs associated with large-volume tables.
-
Improved Data Freshness: Eliminating long-running re-syncs ensures that incremental data continues to be delivered to end users without delays.
-
Operational Efficiency: Dynamic rolling filters would reduce manual intervention, scheduling coordination, and the need for off-hours maintenance windows.
-
Scalability: As data volumes grow, static filters that trigger full table re-syncs become increasingly impractical and disruptive.
-
Official comment
Hi Vamshi,
Thanks for your request in our portal. We have many other customers asking for similar relative functions such as for dates with Row Filtering -- while I don't have an exact timeline on this feature please stay tuned for updates as we work on this
Please sign in to leave a comment.
Comments
1 comment