Connector Improvement: SQL Server validations
Answered
Would be good to include counts on source and destination tables when syncing SQL Server data to a database destination so that there would be a quality comfort level that we are likely not missing rows. Today, we occasionally see tables in our destination that have drifted out of sync with tables in our SQL Server source over time and have to manually resync.
-
Official comment
Hi Bob,
Thank you for sharing feedback on the SQL Server connector. To better assess this request, can you tell us how often your tables are out of sync? Is this isolated to a specific connector, or are you experiencing this across multiple SQL Server connections?
On your current setup: I noticed you're using Teleport as your update method. While Teleport is great for certain use cases, it does have some limitations that might be contributing to what you're seeing. Have you had a chance to evaluate Change Tracking or Change Data Capture? These methods typically provide more reliable sync behavior for scenarios where data freshness is critical.Understanding these details will help clarify how this improvement could address the challenges you're facing.
Thanks,
Rean -
Rean,
It happens periodically that a user tells us that the contents of the table in Snowflake does not match what is in SQL Server. I do not have metrics on how often it occurs. Due to system architecture limitations, our database admins have been reluctant to enable CT/CDC in SQL Server so, indeed, we are currently using Teleport.
Bob
Please sign in to leave a comment.
Comments
2 comments