Connector Improvement: Explicit Failure / Reschedule when a table switches from timetravel to teleport
This is from the fivetran support case - Data mismatch between snowflake and databricks – Fivetran Support
Following an internal review by the engineering team, it has been determined that the system is capable of automatically handling these scenarios. To address the issue, we will transition to teleport-based sync and initiate a resync. This will address the issue without the need to failure or rescheduling the connector.
As this is expected behavior, we recommend implementing an alert mechanism to monitor data loading from the connector sync. This will enable you to proactively trigger a resync when necessary.
We understand you were anticipating a solution for this behavior from our end. However, this is an expected system behaviour.
Our downstream processes rely on the fivetran sync's success condition to run the next jobs. We need the sync to fail or reschedule is any such change happens similar to what happens on the schema changes.
-
Additional information on this behavior observed in this support ticket - Different Behaviors of connectors despite pointing to the same source – Fivetran Support
-
Our snowflake tables has grown substantially, any switch to teleport is a big issue. Current connector behaviour is that it auto switches to teleport. This is a major issue in the making and escalation.
-
Our snowflake tables has grown substantially, any switch to teleport is a big issue. Current connector behaviour is that it auto switches to teleport. This is a major issue in the making and escalation.
Please sign in to leave a comment.
Comments
3 comments