Skip to main content
Docs
Feature Requests Onboarding Trust Center Status Page

Community

Don't let one table failure prevent other tables from syncing

Answered
Derek Sasthav User
  • Edited

Which connector?: Salesforce

 

Additional details: See support case 378119

Issue: We have a Salesforce connector with 100+ tables being synced. We also have a job running an automated PII tagger every night. That job tagged one of the Salesforce tables as PII. Our Fivetran service account did not have access to read that PII tag so that table failed, which is expected. But then many other tables also stopped syncing. Support confirmed that some tables that were queued when the PII table failed were able to finish. But tables that were not queued never got queued and never synced the latest data. So one table failure caused a bunch of other tables to fail.

I don't know if this issue is specific to Salesforce connector or not.

Request: ensure that individual table failures do not impact other tables from syncing, at least for Salesforce connector

Please sign in to leave a comment.

Comments

3 comments

Date Votes
  • Egidio Terra Fivetranner

    Hi Derek,

    Thanks for raising this—this is a helpful scenario to better understand how we could handle table-level failures and their impact on the overall sync.

    To make sure we capture the right requirements, I’d like to clarify a few points:

    • What conditions should determine whether a table is skipped versus causing the entire sync to fail?
    • Should data from problematic tables be redirected to a quarantine location instead of failing or blocking the sync?
    • For tables that are skipped, should we automatically retry them in the next sync, or disable them until the issue is resolved?

    Understanding your expectations here will help us design the right behavior for failure isolation and recovery.

    Best regards,

  • Derek Sasthav User

    Responding to this: https://support.fivetran.com/hc/en-us/community/posts/40276955373719/comments/40330568290199

     

    1. I think one table should only affect other tables if other tables are dependent on that table via relationships or otherwise. If they are unrelated tables, then the other tables should still be completed
    2. I would be more interested in seeing a list of tables that failed and detailed error descriptions than a quarantined location
    3. For retries, I think it is worth retrying in the next sync if the error is transient. If it's a permissions error, then maybe don't retry. The issue is that the permissions error might be fixed in following sync so it's probably worth retrying regardless
  • Egidio Terra Fivetranner

    Thanks — I’ll add this to our backlog and review how best to address the requirement. I’ll update the ticket once we have a clear plan.