Connector Improvement: Add multi-threading for Twilio Connector
Answered
My team relies heavily on a Twilio connector for understanding our calls and messages data with customers. However, our sync times have recently grown to exceed the maximum (24 hour) sync time. Currently, we only sync 5 or so tables, yet the connector has grown from taking ~18 hours to over 24 hours in the past 5 months. This all while we expect to continue to grow as a business, and having these insights in a reasonable time is crucial for us.
We have been in contact with Fivetran support. Here is the feedback we received:
There is currently no alternative and it is not possible to change the sync logic to improve performance, as there is a limitation from the source API for the conversations endpoint.
However, we can improve the sync performance by introducing the multi-threading functionality for this connector ... we suggest you submit a feature request for adding a multi-threading feature for this connector.
-
Official comment
Hi David,
Apologies for the delayed response, and thank you for submitting this feature request.
We’ve added support for multi-threading in the Twilio connector to our backlog. While this isn't currently on our immediate roadmap, we’ll prioritize it based on customer demand and update you on any future developments.
Best,
Sandeep
Technical Product Owner -
For reference, here are our sync times (in minutes) over the past 6 months. They have grown almost linearly over time, and it is not scalable for us to continue to rely on this connector in its current form.
Please sign in to leave a comment.
Comments
2 comments