Skip to main content

Community

Connector Improvement: Stripe Rate-Limit Configurability

Please sign in to leave a comment.

Comments

2 comments

  • Official comment

    Hi Staty - Thanks for submitting this feature request. We don't currently have this planned, but we will keep on eye on it for demand. 

    Just curious what a maximum request rate would be for your use case? Is this something you would want to change on a frequent basis? 

    Best,

    Erin

    Hi Erin,

    Since Stripe supports 100req/sec in Live Mode (and 25req/sec in Sandbox mode) I think teams would do some calculations like "what is our maximum throughput/sec each day for the last 30/60/90 days?" and then create a bit of cushion just in case in choosing their maximum request rate by Fivetran.  In particular, I suspect few teams would like their ability to interact with Stripe (collecting revenue or otherwise) to be impacted by data replication.

    In terms of tuning frequently there are two primary modes to consider - the initial/historical sync and subsequent incremental syncs. 

    In the case of initial/historical syncs - volume could potentially be very high, so I would err on the side of caution and let the sync take longer to prevent impact on our customers. 

    During incremental syncs, the volume is often lower so the impact could be less severe.  I could imagine two different settings.  In terms of changing the request rate for incremental updates. I think there is a boiling-frog problem that could arise, and the initial choice of rate limit may not suffice over time.  In that case, the ability to tune on a daily/weekly/monthly basis would probably make sense.

    An additional caveat on the incremental update volume is that while those syncs may generally be of lower volume and thus could hypothetically use a higher request rate that is not always the case.  If the sync happens to overlap with a high-throughput period of the day/week/month (for customers with highly periodic request patterns) there could be consequences.  As such either tuning the request rate _or_ tuning the request start-times (and not just period) could be helpful.