Issue

How does Priority-first Sync work?

Environment

Check our documentation for an up to date list of connectors that support Priority-first sync

Resolution

Priority-first syncs fetch your most recent data first so that it's quickly ready for you to use.  During your initial sync, Fivetran updates your destination with your most recent data. How many days of recent data we fetch and which tables we sync on a priority-first basis vary by the connector.

1. When a connector that uses Priority-first sync starts the initial historical sync, a forwardCursor and backwardCursor will be at the same point (15 days prior to the sync's start date).
2. The sync goes forward until it reaches the current date and then updates the forwardCursor.
3. Once the forward sync is complete and the latest data has been fetched we then go back to get the remaining historical data using the backwardCursor. 
4. We start fetching your historical data in small increments as part of the backward sync.
5. The backward sync will run for a limited duration during each sync. 
   Note: The Backward Sync Duration for each of our connectors can be found in our Priority-first sync documentation.
6. Once the backward sync has been in progress for that duration we process and load the data collected, save state, and the sync ends.
7. The backward sync will keep taking place during each sync until the backwardCursor has reached the backwardSyncLimit. 

Attention: If you do not want the historical data, we can manually change the cursors on our end to stop the backward sync and allow only forward increments on the connection. This way your connection runs frequently based on the data.

During Priority-first sync a new object in the state is introduced, a bidirectionalCursor which consists of the forward and backward cursor for each endpoint in which bidirectional sync is enabled. To track any progress in the sync this particular object should be observed. The range between forwardCursor and backwardCursor gives us the date range for which the data has been fetched.