Connector Improvement: Enhancing Connector Efficiency via Object-Level Sync Schedule Control
Not plannedHi Team,
Problem: In connectors where a high percentage of objects (e.g., 31 out of 33 in the Slack connector) do not support incremental data synchronization, the overall sync process is inefficient and time-consuming. Full table syncs for these objects occur at the global connector frequency, even if they contain static or slow-changing data.
Requested Feature: Introduce a mechanism allowing users to define an independent synchronization frequency (schedule) for specific objects or tables within a connector.
Benefits:
-
Reduced Sync Time: By setting less frequent schedules for non-incremental objects, the overall execution time of the connector sync run will be significantly reduced.
-
Improved Resource Utilization: Decreases load on the integration service and the source application (e.g., Slack API).
-
Enhanced Customer Convenience: Provides customers with greater control and customization over their data refresh rates.
Recommendation: While this requirement is urgent for the Slack connector, it should be implemented as a standardized feature for all connectors that include objects lacking native incremental sync support.
Regards,
Sreejith
-
Official comment
Hi Sreejith,
Thanks for submitting this request. At this time, enabling independent object-level sync schedule control is not planned on our roadmap. We recognize the value and will consider this for future development.
The only workaround currently available is to configure multiple connections for the same source, each with a distinct schedule for specific object groups according to importance.
Thanks,
Shiva
Please sign in to leave a comment.
Comments
1 comment