Marketo develops marketing automation software that provides inbound marketing, social marketing, CRM, and other related services.
|Data Blocking||check||Column level, table level, and schema level. We don’t allow blocking for the
|Re-sync||We have disabled re-sync from the Marketo connector UI because it can take months. Contact support if you wish to run a full re-sync.|
|Priority-first sync||check||All activity tables|
|dbt Package||check||Get the package: source / transform|
Follow our step-by-step Marketo setup guide to connect Marketo with your destination using Fivetran connectors.
We use Marketo’s REST API, including the Bulk Activity export. Initially, we sync the lead and activity tables and then sync the remaining tables alphabetically. Our sync strategy for activities data varies depending on which version of our Marketo connector you use. See the Sync strategy for activities section for more information.
Because accidentally activating a historical re-sync can cause lengthy disruptions, we’ve disabled the historical re-sync button in the Fivetran dashboard for this connector. Contact our support to enable a full historical sync.
Sync strategy for activitieslink
We use priority-first sync to pull your activities data. Priority-first syncs fetch your most recent data first so that it’s quickly ready for you to use.
During your initial sync, Fivetran fetches your most recent seven days of data.
In all subsequent syncs, we continue to fetch your most recent data first. We also sync historical data until we reach the end of your activities data retention period. For example, if Change Score activities are retained for 90 days, we will sync 90 days’ worth of Change Score data.
Priority-first syncs allow us to keep your most recent data up to date while syncing your historical data over time.
The latest version of our Marketo connector uses only BULK API to fetch activity data, because BULK API guarantees data integrity. This connector uses REST API to fetch all other data.
If you are using our latest Marketo connector, make sure you purchase sufficient BULK API quota from Marketo.
When Fivetran requests data from Marketo via BULK API, we can provide a start date and an end date for the range we want to sync using the
untilDateTime query parameters. In this way, we fetch activities for a defined time range, save the state, and request the next time range. Because we can save state accurately, we preserve data integrity even if syncs fail or are interrupted.
Combined BULK and REST API
Earlier versions of our Marketo connector use REST API to fetch all data but activity data. These connectors use a combination of BULK API and REST API to fetch activities.
BULK API is more reliable and is the preferred method whenever possible. The BULK API part of the combined strategy works exactly the same as the BULK API-only strategy of the latest version of our connector.
When Fivetran uses REST API to request activity data from Marketo, we can request a start date using the
sinceDatetime query parameter. However, unlike with BULK API, we don’t have the option to specify an end date, so we have to rely on sorted records to save the state. That is, when we see an activity, we store its timestamp in the state. We do this to keep track of the time until we have fetched the activities.
However, Marketo’s activities endpoint does not always provide the records in sorted order. When Marketo activity records are not in sorted order, we can’t save the state correctly. If the connection fails or we have to reschedule because we reach the API quota limit, we might end up saving the wrong state and missing some records.
Why don’t we simply wait to save the state until we have fetched all the activities instead of attempting to store the intermediate state? If we don’t store the intermediate state and the sync fails or is rescheduled, we have to start the entire sync from the beginning. Because Marketo has large amounts of data and its API is already very slow, that could take an extremely long time. Moreover, we could reach the API quota limit repeatedly without finishing the sync, and get stuck in a loop. Saving the intermediate state is a necessary, if imperfect, workaround to syncing through the Marketo REST API.
Semantic activity types
Semantic activity types define how changes propagate in Marketo. We fetch semantic activities from Marketo even if you don’t select them in the Schema tab of your connector’s dashboard. We can’t sync data for other tables (
PROGRAM_MEMBERSHIP) if we don’t fetch these activities.
We use your BULK API to fetch the following semantic activities:
|13||Change Data Value|
|24||Add to List|
|25||Remove from List|
|34||Add to Opportunity|
|35||Remove from Opportunity|
|104||Change Status in Progression|
|113||Add to Nurture|
Activity types recommendation
Some activities trigger an automatic historical re-sync that impacts the sync speed of the connector. We do not recommend including the following email activity types:
|27||Email Bounced Soft|
|111||Sent Forward to Friend Email|
|112||Received Forward to Friend Email|
Leads replication and Marketo Corona
Fivetran uses the
updatedAt query parameter to retrieve only the new and updated data from the Leads API endpoint. However, this feature is only supported for those accounts that have enabled Corona.
To allow us to perform incremental replication of your leads data, contact Marketo to enable the feature for your account.
Sync strategy for PROGRAM_MEMBERSHIP table
When we sync the activity tables we obtain the program IDs from the activities. We then use the program IDs to fetch details about the programs. We individually check all the programs and then sync all the leads that belong to a specific program into the
We use Marketo’s
Get Deleted Leads API endpoint to fetch the deleted leads. Marketo retains the activity details of deleted leads for only 14 days. See Marketo’s REST API documentation for more information.
If you are using the latest version of our connector (v3), we fetch the
deleted_lead activity before all the activities. If you pause your connector for more than 14 days and Marketo removes the
deleted_lead activity, we may not be able to identify some of the leads as deleted.
If you are using the previous version of our connector (v2), we fetch the
deleted_lead activity along with all the activities. If your connector doesn’t complete a successful sync in 14 days, we may not be able to identify some of the leads as deleted because Marketo pruned the activity details.
Marketo’s schema follows Fivetran’s standard API schema rules.
Extracting data from Marketo can leave you with sparse, duplicative, and confusing tables if you’re not careful. Fivetran helps to organize this data into a relational data model, making your queries more efficient, your destination less cluttered, and your data easier to analyze.
Marketo supports creating custom tables. The custom tables that can be in your Marketo account can vary depending on which integrations you have, and how you have configured it. Most of the custom tables are basically custom event tables. For example, “Event X” becomes “activity_event_x” in our system. We will make tables for the activities you’re trying to track.
This schema applies to the latest version of our Marketo connector, released in private preview in November 2019, and applies to all connectors created today Beta
This schema applies to the generally available version of our Marketo connector, and all versions of our Marketo connector created before November 1, 2019.