Add fivetran log event about connector save state
Not plannedI recently experienced an issue with the MySQL connector where > 8 hours in I paused a first-time sync, and then the sync had to start from scratch again. According to fivetran support, log info can be added to tell when connector state is saved so we know whether there is a safe restore point to continue from. Details from her below.
---
It looks like the connector did not actually save the progress even though the import_progress event shows that the tables completed.
The connector needs to save state, meaning it needs to save its progress. This happens either after 6 hours or after we process about 100GB of data, whatever happens first. This information is something that is currently accessible via the logs and only visible internally.
I checked with our engineering team and they mentioned that it's possible to get this info logged in our Fivetran Log connector. I would suggest you to raise a feature request for this via our portal so that you can directly communicate with our product team about this and be able to receive alerts/ notifications on any updates.
-
Official comment
Hi Rob - thank you for posting this improvement suggestion here. We have added it to our backlog and it will be prioritized against our other ongoing and planned work.
Please sign in to leave a comment.
Comments
1 comment