Skip to content

Add tenacity utilities/integration for improved retry handling #2282

New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Merged
merged 6 commits into from
Jul 25, 2025

Conversation

dmontagu
Copy link
Contributor

This came up in a discussion in public slack with @mpfaffenbergerhttps://pydanticlogfire.slack.com/archives/C083V7PMHHA/p1752430089758299.

Some things to resolve before merging:

  • Are we okay with this module naming? Should we call it pydantic_ai.retries or similar instead of pydantic_ai.tenacity?
  • Are the tests/docs adequate?
  • Do we need to add some integration for Model? I have a tenacity-integrated WrapperModel, but I'm not sure if it's necessary/very useful on top of the async transport stuff currently included in this PR.

@dmontagu
Copy link
Contributor Author

In case it's useful, here is the RetryModel implementation for future reference.

from __future__ import annotations as _annotations

from collections.abc import AsyncIterator
from contextlib import asynccontextmanager
from dataclasses import dataclass
from typing import Literal

from tenacity import AsyncRetrying

from . import KnownModelName, Model, ModelRequestParameters, StreamedResponse
from .wrapper import WrapperModel
from ..messages import ModelMessage, ModelResponse
from ..settings import ModelSettings


@dataclass(init=False)
class RetryModel(WrapperModel):
    def __init__(
        self,
        wrapped: Model | KnownModelName,
        retry: AsyncRetrying | None = None,
        retry_stream: AsyncRetrying | Literal[False] | None = None,
    ):
        super().__init__(wrapped)
        self.controller = retry
        self.stream_controller = retry if retry_stream is None else retry_stream

    async def request(
        self,
        messages: list[ModelMessage],
        model_settings: ModelSettings | None,
        model_request_parameters: ModelRequestParameters,
    ) -> ModelResponse:
        async for attempt in self.controller:
            with attempt:
                return await super().request(messages, model_settings, model_request_parameters)
        raise RuntimeError('The retry controller did not make any attempts')

    @asynccontextmanager
    async def request_stream(
        self,
        messages: list[ModelMessage],
        model_settings: ModelSettings | None,
        model_request_parameters: ModelRequestParameters,
    ) -> AsyncIterator[StreamedResponse]:
        if not self.stream_controller:
            # No special retrying logic for streaming in this case:
            async with super().request_stream(messages, model_settings, model_request_parameters) as stream:
                yield stream
                return

        entered_stream = False
        async for attempt in self.controller:
            attempt.__enter__()
            try:
                async with super().request_stream(messages, model_settings, model_request_parameters) as stream:
                    entered_stream = True
                    attempt.__exit__(None, None, None)
                    yield stream
                    return
            finally:
                if not entered_stream:
                    attempt.__exit__(None, None, None)
        raise RuntimeError('The retry controller did not make any attempts')

Copy link
Contributor

hyperlint-ai bot commented Jul 23, 2025

PR Change Summary

Added tenacity utilities for improved retry handling in HTTP requests, enhancing error resilience and user experience.

  • Introduced the pydantic_ai.tenacity module for retry functionality in HTTP requests.
  • Added detailed documentation on using tenacity for handling transient failures.
  • Implemented transport classes for both asynchronous and synchronous HTTP clients.

Added Files

  • docs/api/tenacity.md
  • docs/retries.md

How can I customize these reviews?

Check out the Hyperlint AI Reviewer docs for more information on how to customize the review.

If you just want to ignore it on this PR, you can add the hyperlint-ignore label to the PR. Future changes won't trigger a Hyperlint review.

Note specifically for link checks, we only check the first 30 links in a file and we cache the results for several hours (for instance, if you just added a page, you might experience this). Our recommendation is to add hyperlint-ignore to the PR to ignore the link check for this PR.

Copy link

github-actions bot commented Jul 23, 2025

Docs Preview

commit: 3cd330d
Preview URL: https://a88e5ec5-pydantic-ai-previews.pydantic.workers.dev

@DouweM
Copy link
Contributor

DouweM commented Jul 23, 2025

  • Are we okay with this module naming? Should we call it pydantic_ai.retries or similar instead of pydantic_ai.tenacity?

I think that'd be better

  • Are the tests/docs adequate?

I think so, save for some comments I left

  • Do we need to add some integration for Model? I have a tenacity-integrated WrapperModel, but I'm not sure if it's necessary/very useful on top of the async transport stuff currently included in this PR.

I don't think we need it, acting directly on the HTTP client level is more powerful

@mpfaffenberger
Copy link
Contributor

Watching you guys work is awesome. Appreciate this! Thank you!

@DouweM DouweM enabled auto-merge (squash) July 25, 2025 17:24
@DouweM DouweM merged commit 4941468 into main Jul 25, 2025
17 checks passed
@DouweM DouweM deleted the dmontagu/retry-handling branch July 25, 2025 17:34
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Projects
None yet
Development

Successfully merging this pull request may close these issues.

3 participants