Marketo develops marketing automation software that provides inbound marketing, social marketing, CRM, and other related services.
Featureslink
| Feature Name | Supported | Notes |
|---|---|---|
| Capture Deletes | check | LEAD, LIST_MEMBERSHIP, OPPORTUNITY_MEMBERSHIP, and PROGRAM_MEMBERSHIP tables. We read the activities stream, and once we see a delete activity, we mark the record as deleted. |
| Custom Data | check | LEAD, COMPANY, and all activity tables. For each activity, we create a table in the format activity_{activity_name} |
| Data Blocking | check | Column level, table level, and schema level. We don’t allow blocking for the LEAD table. |
| Column Hashing | check | |
| Re-sync | We have disabled re-sync from the Marketo connector UI because it can take months. Contact support if you wish to run a full re-sync. | |
| History | check | EMAIL_TEMPLATE_HISTORY table only |
| API Configurable | check | |
| Priority-first sync | check | All activity tables |
| dbt Package | check | Get the package: source / transform |
Setup guidelink
Follow our step-by-step Marketo setup guide to connect Marketo with your destination using Fivetran connectors.
Sync overviewlink
We use Marketo’s REST API, including the Bulk Activity export. Initially, we sync the lead and activity tables and then sync the remaining tables alphabetically. Our sync strategy for activities data varies depending on which version of our Marketo connector you use. See the Sync strategy for activities section for more information.
Because accidentally activating a historical re-sync can cause lengthy disruptions, we’ve disabled the historical re-sync button in the Fivetran dashboard for this connector. Contact our support to enable a full historical sync.
Sync strategy for activitieslink
Priority-first sync
We use priority-first sync to pull your activities data. Priority-first syncs fetch your most recent data first so that it’s quickly ready for you to use.
During your initial sync, Fivetran fetches your most recent seven days of data.
In all subsequent syncs, we continue to fetch your most recent data first. We also sync historical data until we reach the end of your activities data retention period. For example, if Change Score activities are retained for 90 days, we will sync 90 days’ worth of Change Score data.
Priority-first syncs allow us to keep your most recent data up to date while syncing your historical data over time.
BULK API
The latest version of our Marketo connector uses only BULK API to fetch activity data, because BULK API guarantees data integrity. This connector uses REST API to fetch all other data.
If you are using our latest Marketo connector, make sure you purchase sufficient BULK API quota from Marketo.
When Fivetran requests data from Marketo via BULK API, we can provide a start date and an end date for the range we want to sync using the sinceDatetime and untilDateTime query parameters. In this way, we fetch activities for a defined time range, save the state, and request the next time range. Because we can save state accurately, we preserve data integrity even if syncs fail or are interrupted.
Combined BULK and REST API
Earlier versions of our Marketo connector use REST API to fetch all data but activity data. These connectors use a combination of BULK API and REST API to fetch activities.
BULK API is more reliable and is the preferred method whenever possible. The BULK API part of the combined strategy works exactly the same as the BULK API-only strategy of the latest version of our connector.
When Fivetran uses REST API to request activity data from Marketo, we can request a start date using the sinceDatetime query parameter. However, unlike with BULK API, we don’t have the option to specify an end date, so we have to rely on sorted records to save the state. That is, when we see an activity, we store its timestamp in the state. We do this to keep track of the time until we have fetched the activities.
However, Marketo’s activities endpoint does not always provide the records in sorted order. When Marketo activity records are not in sorted order, we can’t save the state correctly. If the connection fails or we have to reschedule because we reach the API quota limit, we might end up saving the wrong state and missing some records.
Why don’t we simply wait to save the state until we have fetched all the activities instead of attempting to store the intermediate state? If we don’t store the intermediate state and the sync fails or is rescheduled, we have to start the entire sync from the beginning. Because Marketo has large amounts of data and its API is already very slow, that could take an extremely long time. Moreover, we could reach the API quota limit repeatedly without finishing the sync, and get stuck in a loop. Saving the intermediate state is a necessary, if imperfect, workaround to syncing through the Marketo REST API.
Leads replication and Marketo Corona
Fivetran uses the updatedAt query parameter to retrieve only the new and updated data from the Leads API endpoint. However, this feature is only supported for those accounts that have enabled Corona.
To allow us to perform incremental replication of your leads data, contact Marketo to enable the feature for your account.
Sync strategy for PROGRAM_MEMBERSHIP table
When we sync the activity tables we obtain the program IDs from the activities. We then use the program IDs to fetch details about the programs. We individually check all the programs and then sync all the leads that belong to a specific program into the PROGRAM_MEMBERSHIP table.
Sync noteslink
We use Marketo’s Get Deleted Leads API endpoint to fetch the deleted leads. Marketo retains the activity details of deleted leads for only 14 days. See Marketo’s REST API documentation for more information.
If you are using the latest version of our connector (v3), we fetch the deleted_lead activity before all the activities. If you pause your connector for more than 14 days and Marketo removes the deleted_lead activity, we may not be able to identify some of the leads as deleted.
If you are using the previous version of our connector (v2), we fetch the deleted_lead activity along with all the activities. If your connector doesn’t complete a successful sync in 14 days, we may not be able to identify some of the leads as deleted because Marketo pruned the activity details.
Schema informationlink
Marketo’s schema follows Fivetran’s standard API schema rules.
Extracting data from Marketo can leave you with sparse, duplicative, and confusing tables if you’re not careful. Fivetran helps to organize this data into a relational data model, making your queries more efficient, your destination less cluttered, and your data easier to analyze.
Marketo supports creating custom tables. The custom tables that can be in your Marketo account can vary depending on which integrations you have, and how you have configured it. Most of the custom tables are basically custom event tables. For example, “Event X” becomes “activity_event_x” in our system. We will make tables for the activities you’re trying to track.
Schemalink
This schema applies to the latest version of our Marketo connector, released in private preview in November 2019, and applies to all connectors created today Beta
To zoom, open ERD in new window
This schema applies to the generally available version of our Marketo connector, and all versions of our Marketo connector created before November 1, 2019.