From 93f8575339a4a3771ff74d5358fdabb1985ff0b9 Mon Sep 17 00:00:00 2001 From: GefMar Date: Tue, 6 May 2025 09:08:07 +0200 Subject: [PATCH 1/2] add: Doc for Smart Retry Middleware --- docs/available-components/middlewares.md | 51 ++++++++++++++++++++++++ 1 file changed, 51 insertions(+) diff --git a/docs/available-components/middlewares.md b/docs/available-components/middlewares.md index 147374e1..cf091de9 100644 --- a/docs/available-components/middlewares.md +++ b/docs/available-components/middlewares.md @@ -34,6 +34,57 @@ async def test(): `retry_on_error` enables retries for a task. `max_retries` is the maximum number of times,. +## Smart Retry Middleware + +The `SmartRetryMiddleware` automatically retries tasks with flexible delay settings and retry strategies when errors occur. This is particularly useful when tasks fail due to temporary issues, such as network errors or temporary unavailability of external services. + +### Key Features: + +* **Retry Limits**: Set the maximum number of retry attempts (`max_retries`). +* **Delay Before Retry**: Define a fixed delay or use additional strategies. +* **Jitter**: Adds random delay variations to distribute retries evenly and prevent simultaneous task execution. +* **Exponential Backoff**: Increases the delay for each subsequent retry, useful for gradually recovering system performance. +* **Custom Schedule Source (`schedule_source`)**: Enables using a custom schedule source, such as an external queue or service, to manage delayed task execution. + +### Middleware Integration + +To use `SmartRetryMiddleware`, add it to the list of middlewares in your broker: + +```python +from taskiq import ZeroMQBroker +from taskiq.middlewares import SmartRetryMiddleware + +broker = ZeroMQBroker().with_middlewares( + SmartRetryMiddleware( + default_retry_count=5, + default_delay=10, + use_jitter=True, + use_delay_exponent=True, + max_delay_exponent=120 + ), +) +``` + +### Using Middleware with Tasks + +To enable retries for a specific task, specify the following parameters: + +```python +@broker.task(retry_on_error=True, max_retries=10, delay=15) +async def my_task(): + raise Exception("Error, retrying!") +``` + +* `retry_on_error`: Enables the retry mechanism for the specific task. +* `max_retries`: Maximum number of retries (overrides middleware default). +* `delay`: Initial delay before retrying the task, in seconds. + +### Usage Recommendations + +Use jitter and exponential backoff to avoid repetitive load peaks, especially in high-load systems. Choose appropriate `max_delay_exponent` values to prevent excessively long intervals between retries if your task execution is time-sensitive. + + + ### Prometheus middleware You can enable prometheus metrics for workers by adding PrometheusMiddleware. From 5fb64aa6ffee9c814ab8cecb8f6beea3d92d541b Mon Sep 17 00:00:00 2001 From: GefMar Date: Wed, 7 May 2025 08:17:12 +0200 Subject: [PATCH 2/2] fix: Doc --- docs/available-components/middlewares.md | 2 +- 1 file changed, 1 insertion(+), 1 deletion(-) diff --git a/docs/available-components/middlewares.md b/docs/available-components/middlewares.md index cf091de9..4bd28e5d 100644 --- a/docs/available-components/middlewares.md +++ b/docs/available-components/middlewares.md @@ -44,7 +44,7 @@ The `SmartRetryMiddleware` automatically retries tasks with flexible delay setti * **Delay Before Retry**: Define a fixed delay or use additional strategies. * **Jitter**: Adds random delay variations to distribute retries evenly and prevent simultaneous task execution. * **Exponential Backoff**: Increases the delay for each subsequent retry, useful for gradually recovering system performance. -* **Custom Schedule Source (`schedule_source`)**: Enables using a custom schedule source, such as an external queue or service, to manage delayed task execution. +* **Custom Schedule Source (`schedule_source`)**: Enables using a custom schedule source to manage delayed task execution. ### Middleware Integration