Understanding how the Marketo connector affects MAR
Marketo Connector Overview
- Fivetran pulls from both Marketo's REST API and Bulk API. REST API limit is dynamic (docs here), but Bulk API is 500mb a day. We recommend upgrading Bulk API to 2GB to complete historic sync, but this can vary greatly depending on how much data the customer has.
- Sync strategy is to start with the lead and activity tables, and then move to the remaining tables alphabetically.
- Priority-first sync is used on activities data/tables. For initial sync, Fivetran fetches most recent seven days of data. In subsequent syncs, we work our way backwards until we hit the end (after syncing most recent data).
- Most recent version of the connector uses only Bulk API for activity data, but REST API for all other data.
- Must have “Corona” enabled to perform incremental replication of Leads data (must contact Marketo to have it enabled). If enabled, we use the “updatedAt” query parameter to retrieve only the new and updated data from the Leads API endpoint.
- For Program_Membership table, we pull program IDs from activities then use these ids to sync all the leads that belong to a specific program.
- To capture deleted leads, we use Get Deleted Leads API endpoints. Marketo retains the activity of deleted leads for only 14 days. Newest version of our connector (V3) fetches deleted_lead activity before all other activities. If connector is paused for longer than 14 days, we may not be able to identify deleted leads. V2 connector doesn’t prioritize deleted_leads, so if sync takes longer than 14 days we might lose deleted values.
Managing MAR for this Connector
Activities and Leads (and the corresponding LEAD and ACTIVITY_*) are going to typically be the biggest drivers in MAR. Activities happen to leads, and there are a ton of different activities that can happen since Marketo is so broad (emails opened, specific search term used in search engine, link clicked on website, etc.). Per Marketo, there are a very large number of potential activity types, which may vary from subscription to subscription, and have unique definitions for each. While every activity will have its own unique id, leadId and activityDate, the primaryAttributeValueId and primaryAttributeValue will vary in their meaning.
Ensure you do priority-first sync (which takes a while to pull historical) during two-week free trial. Otherwise, historical data during priority-first sync would count towards MAR.
Make sure you evaluate data in the `activity_change_data_value` table (typically 54% of all MAR from Marketo comes from this table) which is allowed to be blocked.