Skip to main content
Docs
Feature Requests Onboarding Trust Center Status Page

Community

Connector Improvement: Implementing Incremental Sync for the Token Table Instead of Full Sync

Answered
Daksh Sharma User

Hi Team,

I recently submitted a ticket to Fivetran support (Ticket link: High usage api by market prod – Fivetran Support) concerning the high API usage of the Marketo API by Fivetran. Upon investigation, it was revealed that the Token table is currently performing a full sync instead of an incremental sync, which has been identified as the primary consumer of our Marketo API quota.

Additionally, we attempted to allow a full day for synchronization. However, the Fivetran sync process lasted for 12 hours without successfully syncing the data, as the job was rescheduled due to the exhaustion of the API quota.

Given these challenges, I kindly request the addition of a feature to the Marketo connector that would enable the Token table to perform incremental syncs rather than full syncs. This change would significantly reduce our API usage and improve the efficiency of our data synchronization processes.

Thank you for considering this request.

Please sign in to leave a comment.

Comments

1 comment

Date Votes
  • Official comment
    Lucas Alexander Fivetranner

    Hi Daksh, 

    Luke from the Product team here! Thanks for submitting this feature.

    Unfortunately, Marketo's API doesn't provide a mechanism for syncing this data incrementally. Unlike other endpoints (e.g. Bulk Lead Extract), the endpoint for getting Tokens data doesn't allow filtering by a field like updatedAt, which would allow us to retrieve only updated or new records. To ensure data integrity we have to do a full sync.

    Please let me know if you think there is another way we could obtain this data. I also encourage you to submit a request to Marketo to update this endpoint. We would be happy to implement this feature once Marketo supports it.

    Thanks,
    Luke