Understanding how Salesforce affects MAR
Salesforce Connector Overview
- The Salesforce connector syncs data via the Bulk and REST APIs. The initial historical sync leverages the Bulk API and subsequent incremental syncs leverage the rest API.
- Salesforce is notorious for updating their API 3 times a year. Fivetran is currently updating our connector to pull from V50 of the Salesforce REST API. This is important to note since some objects are only available with specific versions of the API.
- Upon setting up the connector, Fivetran will automatically select the tables that can be replicated into the destination. This includes custom tables unique to the customer.
- Fivetran might not be able to sync some tables in the destination. There are 2 main reasons why you cannot sync some tables: 1) the table is not queryable or 2) the user does not have permission to read from the table.
- Some tables need to be re-imported on each sync and are not recommended.
- History Mode - Customers may elect to activate History Mode which captures type 2 slowly changing dimensions.
- Customers are typically concerned with the number of API calls used by Fivetran. Fivetran automatically reschedules syncs for one hour later anytime 90% of the Salesforce API call quota has been consumed. The system table FIVETRAN_API_CALL tracks information about API calls that are made to sync the data into the warehouse.
What Contributes Towards MAR?
There are 5 primary actions that will contribute toward MAR:
- Syncing tables that need to be re-imported each time they are synced.
- Custom Objects are unique to each customer and could significantly impact MAR.
- Enabling History Mode will keep multiple versions of the same record.
- Adding new fields to objects/tables that have multiple records.
- Number of active people updating records in Salesforce.
Managing MAR for this Connector
- Ultimately, MAR is determined by the number of objects selected by the customer. There could be custom tables in Salesforce as result of an integration with Salesforce. Ensure the tables selected in the schema are necessary for analytical purposes.
- Avoid historical re-syncs if at all possible. Table syncs are possible if changes are only made to one object/table. If significant changes are made to the data in salesforce, it might be worth exploring the option of creating a new connector.
- History Mode will impact MAR. If customers do not find History Mode valuable, they should disable.
Reference
Tracking Salesforce API Requests
Consumption Based Pricing Overview