Question

My sync is taking longer than normal. What is causing this?

Environment

All Connectors

Answer

Several factors can cause this:

1. Increase in data volume
   There has been an increase in the data volume to sync. To check for this, you can hover over the sync bar in the dashboard and see a breakdown of the extract, process, and load volumes. You can then compare this to another shorter sync to verify if the volume has increased for the longer sync.
2. Schema Change
   If there has been an additional column or table synced, then this can cause an increase in the sync time. You can filter the dashboard logs by schema change events to check for this.
3. Large change in source
   If there has been a large change in data at the source, such as a mass delete or mass record update, this will result in an increase in changes to sync, increasing the sync time.
   You can check the following to confirm this: 
   1.  the source for any large changes
   2. following sync completion, search the dashboard logs with the value records_modifed to verify if any tables have a large increase in the count parameter for the records_modifed event.

   3. Check the number of updated records by querying your warehouse with a WHERE clause on the updated_at column. Note, that child tables may not have a updated_at column and are usually updated as per their parent record. 

4. Network Latency
   There may be intermittent network latency that is slowing the sync. This may occur on connecting to a source database connector or the destination database. Check with your network team to verify if there are known or potential network latency Issues.
5. API rate limits (Application Connectors Only)
   The source API rate limits may be hit, requiring the connector to wait until it can query the API again. This can slow the sync if it is regularly hit. You can filter the dashboard logs by warnings only to see Warnings for this and verify if this is the cause for the slower sync.