Skip to main content
Docs
Feature Requests Onboarding Trust Center Status Page

Community

Connector Improvement: Add Bulk API Limit to Marketo Connector

Not planned
Andrew Karcher User

The Marketo Connector utilizes both the Bulk API and the Rest API. For the Rest API there is a configuration setting to limit the number of API calls so the connector does not exhaust all the API calls and kill the Marketo environment. However, there is no such limit for the Bulk API. So during initial syncs and possibly after as well, the connector will exhaust all of the Bulk API data limit. 

I understand that the initial sync is going to use a significant amount of the Bulk API limit and that once the initial sync is done it may not be an issue, but it is a hole that I think should be fixed. The goal for limit settings is to ensure that the connector does not become an issue that affects normal operations. Not having a setting on the Bulk API is a big risk that it would cause issues with normal operations.

Please sign in to leave a comment.

Comments

2 comments

Date Votes
  • Official comment
    Lucas Alexander Fivetranner
    • Edited

    Hi Andrew,

    Luke from the Product team here! Thanks for submitting this feature request.

    We investigated this and have concluded it is not something we'll support at this time. 

    Marketo exposes an endpoint that provides the REST quota consumption for the REST API. We use this endpoint to restrict the maximum number of API calls our connector can make.

    Marketo doesn't provide a similar endpoint for the Bulk API. Also, the quota for the Bulk API is in MB, not API calls. Marketo only calculates the data extracted after we extract data from the Bulk API. So, it isn't possible for us to accurately cap the Bulk API consumption, and the implementation would be very complex.

    We'll continue to monitor Marketo's API changes for an update that provides a way to track consumption from the Bulk API as they have for the REST API.

    Luke  

  • Tali Kelley User

    Hello,

    This is also an issue that my team is currently experiencing. We need to sync Marketo data. We have limited the amount of REST API calls that Fivetran can make to Marketo, but are not able to restrict the BULK API to use only a certain amount of MB per day. We need something like this in place in order to not have Fivetran exhaust our Bulk API in Marketo.