Multi-tenant system with separated circuit breakers? #2564
-
|
I have a multi-tenant application with 1000+ tenants and a shared database. I want to avoid situations where one tenant overloads the database (which is shared by all tenants) by spamming expensive queries, which run into timeouts. I thought about adding circuit breakers with a failure threshold of 0.5 for each tenant, but then I read that policy creation is expensive. |
Beta Was this translation helpful? Give feedback.
Replies: 2 comments 3 replies
-
|
Sorry, it's early 🤣 |
Beta Was this translation helpful? Give feedback.
-
|
This sounds like a scenario better served to rate limiting rather than circuit breakers. That can be partitioned per-tenant easily with a single Polly.Core v8 strategy being shared by all the requests. The per-tenant rate limiters will be created as needed by the Rate Limiting implementation. |
Beta Was this translation helpful? Give feedback.
-
|
I looked into the RateLimiter docs and saw that multi-tenancy would be easily achieved via https://github.com/App-vNext/Polly/blob/main/docs/strategies/rate-limiter.md/#partitioned-rate-limiter. Unfortunately, I don't think that RateLimiter suits our scenario, since defining limits for our tenants would certainly lead to less efficient resource consumption (meaning avaible resources are not fully utilized). Our circuit breaker approach with low timeouts, where we lock out customers if they produced too many timeouts in a certain period of time, would not have this disadvantage |
Beta Was this translation helpful? Give feedback.
-
|
@martincostello Have you seen my last response? What are the risks of using 1000 circuit breaker instances? Or ist there a better approach (besides using rate-limiter) |
Beta Was this translation helpful? Give feedback.
-
|
Well it's just going to use more resources. You'd have to test it out and see how it fares to work exactly how many. "Expensive" is a relative term. Throwing exceptions to control program flow is relatively expensive, but how often it's done determines whether doing so would cause performance issues in a web application deployed at scale. As a general rule of thumb, it's best to create as few execution strategies as you can and re-use them. For example, if you wanted a 1 second retry policy for all request, you'd create a single timeout strategy configured to do that and re-use it for every single request - you wouldn't create a new one for every request. |
Beta Was this translation helpful? Give feedback.
Well it's just going to use more resources. You'd have to test it out and see how it fares to work exactly how many.
"Expensive" is a relative term. Throwing exceptions to control program flow is relatively expensive, but how often it's done determines whether doing so would cause performance issues in a web application deployed at scale.
As a general rule of thumb, it's best to create as few execution strategies as you can and re-use them. For example, if you wanted a 1 second retry policy for all request, you'd create a single timeout strategy configured to do that and re-use it for every single request - you wouldn't create a new one for every request.