Skip to main content
Docs
Feature Requests Onboarding Trust Center Status Page

Community

Error Handling Mechanism for Multi-Table Database Connectors

Not planned
Kate Grimshaw User

Currently, the Postgres connector (and presumably other similar connectors) will fail to sync all tables in a database when an error is encountered for just one table. We recently had a corrupted parquet file in one table out of 96 in a connector, and the entire connector failed to sync until that table was removed from the sync list. There should be a mechanism to proceed when a table-specific error is encountered to minimize the impact of such issues. I'd obviously recommend still flagging the error, but there's no reason the rest of the tables could not continue to sync. 

Please sign in to leave a comment.

Comments

2 comments

Date Votes
  • Official comment
    Valentin Kulichenko Fivetranner

    Hi Kate,

    Fivetran syncs operate on connection level rather than sync level, so this would be a significant architectural change. We might consider such improvements long-term, but currently this is not planned.

    That said, I think error handling can be improved here, we will take a closer look into that.

    Thanks,
    Val Kulichenko, Fivetran Product team

  • Jaya Kishnani User

    In case of similar failures, preferably we should be getting a list of table name(s) (along with error message). This will help customers debug the issue, current error messages are not that expressive to help us debug the cause of failure.

    In the recent failure, for which we submitted this ticket - https://support.fivetran.com/hc/en-us/requests/369875, the error message was not obvious for us to decide disabling that table.