Connector Improvement: Need option to specify sync behavior and timing for SFDC full sync tables
AnsweredWe've encountered a limitation of Fivetran's SFDC connector when ingesting SFDC tables that do not allow incremental ingestion in fivetran such as the case_milestone table. Based on feedback from fivetran support, it is our understanding that the fivetran connector automatically limits synchronization of such tables to once per week, usually on Saturday, with no control over when the table is synchronized. This behavior does not give us sufficient control over when the sync is performed and we have no obvious way of tracking sync operations for audit purposes.
Fivetran's current workaround requires that we switch the connector's schedule to manual and use an external scheduler to trigger syncs via REST API call. This not a desirable solution nor is it scalable for our customer base.
We request an enhancement to add the ability to configure a refresh schedule for any table that requires a full table fetch natively in the fivetran UI. We should have the ability to separately specify synchronization behavior for each table that needs to be refreshed in this way.
Ideally there should also be an option to perform these refreshes as a separate asynchronous process from refreshing of incrementally maintained tables to limit the performance impact of time sensitive refreshes.
-
Official comment
Hi Matt,
Thanks for the detailed feedback. We have taken note of it and are actively exploring options on how we can help you schedule the refreshes with more control. We will keep you posted as we make progress on it.
Regards,
Unni
Please sign in to leave a comment.
Comments
1 comment