Connector Improvement: Ability to more easily recover from a database failover
Planned
While utilizing a connector to Google Cloud MySQL there was a failover from the primary to the secondary. This required a full re-sync as there is no way to recover.
This not only requires downtime while the sync is running (~4 hours for us) and also costs us a lot because it brings every row into the month.
Submitting a support ticket it appears that there is another option
"For your use-case, a potential workaround would be for you to provide us the new binlog filename and position number, which we would use to manually update the connector "cursor". There would be some downtime in this scenario of about ~30 minutes - 1 hour."
It would nice to have a better way to handle a failover without having to do a full sync. It is just not cost effective.
-
Official comment
Hi Andrew,
Thanks for your post on our feature request portal!
We are currently working to roll out a feature known as Teleport Syncs, which captures states of each table you are syncing and calculates differences between those states quickly and efficiently. Our plan is to automatically run this under-the-hood whenever a re-sync is required, so that rather than running a full re-sync (which is costly and painful), we run an efficient re-sync that only captures the changes between states!
For example, let's say you have a scheduled database failover on Tuesday morning. If your last Teleport efficient re-syncs ran Sunday morning, then we would only have to re-sync the changes between Sunday (last Teleport snapshot) and Tuesday (failover event).
While I don't have a timeline for this at the moment, this is currently in our project plan and a high priority for the team to implement. Please stay tuned for more updates on our progress, and let me know if you have any questions!
Please sign in to leave a comment.
Comments
1 comment