Proxy Agent capacity to be increased
Answered
Which connector?:
Proxy Agent capacity
Additional details:
Currently, the proxy agent has a limitation of 50 connections per proxy, and we need this capacity to be increased to 100+ connections at the earliest.
The current limitation requires us to provision multiple EC2 instances as the number of connections increases. This results in additional infrastructure costs and operational overhead, along with the need for periodic proxy agent setup and configuration.
Increasing the proxy agent capacity to 100+ connections would allow us to consolidate connections, reduce the number of EC2 instances required, minimize infrastructure costs, and simplify ongoing operational maintenance.
Requesting your support to increase the proxy agent capacity to 100+ connections on priority.
-
Official comment
Hi Kaliyaperumal --
thanks for taking the time to write in. The Limit is currently 30, but that's a bit unprecise - it is really dependent on data sync job size, network throughput, host machine hardware, e.g. too many variables to really articulate fixed / exact number for every scenario. I recommend testing it with your specific configuration to determine what your hard limit is, what we document is an approximation.
Fivetran has some pre-requisite work before we are ready to increase that:
(1) more load testing
(2) improvements to proxy server (on our side) - vertical scaling
(3) improvements to proxy server (on our side) - horizontal scalingthat work is going to come after work we hope to accomplish on auto-upgrade, docker, k8s.
at the moment, this wouldn't be until late 2027 but obviously things can change so that is an estimate and not a commitment.
Please sign in to leave a comment.
Comments
1 comment