Automatic retries / Cloudflare 502 bad gateway #339

albinekb · 2025-04-30T09:05:44Z

albinekb
Apr 30, 2025

Hello!
We are using portkey-ai = "^1.12.0"

In this PR #281
I think the intention was to handle gateway errors from cloudflare?

Response:

{
http.fragment: ,
http.method: POST,
http.query: ,
http.response.status_code: 502,
reason: Bad Gateway,
thread.id: 132495671757696,
thread.name: MainThread
}

The error str(e):

<html>
<head><title>502 Bad Gateway</title></head>
<body>
<center><h1>502 Bad Gateway</h1></center>
<hr><center>cloudflare</center>
</body>
</html>

Is there something we have to configure to get this to work? Is it expected that we should handle retries of this case?

Thanks!

csgulati09 · 2025-04-30T09:07:31Z

csgulati09
Apr 30, 2025
Maintainer

Hey!

Can you also tell which method are you trying to call here?

2 replies

albinekb Apr 30, 2025
Author

AsyncPortkey client

client.with_options(
    metadata={
        "function": "analyze",
        "project_id": project_id,
    }
).chat.completions.create(
    model=self.model,
    messages=messages,  # type: ignore
    response_format={"type": "json_object"},
    # temperature=0.2,
)

csgulati09 Apr 30, 2025
Maintainer

Thanks for sharing it.

csgulati09 · 2025-04-30T10:19:21Z

csgulati09
Apr 30, 2025
Maintainer

The base logic is to try a request once.

The criteria to retry the request on the basis of the logic written in the _should_retry() method.
Yes, a Cloudflare error will be retried.

But if the error is encountered again, it will be returned as is.

There is a chance that the request was retried internally before returning the error.

I tried to enforce a cloudflare error and test it, it actually does retry one more time.

Can you confirm this once?

2 replies

albinekb May 2, 2025
Author

Can we make it retry more than once in case of cloudflare error? It looks like the retry is made directly without backoff, so high chance it will fail again?

csgulati09 May 2, 2025
Maintainer

We have the logic for exponential backoff and/or acknowledges retry-after header on the gateway. This means that if the request reaches the gateway and then fails somehow, in that case the retries are handled as mentioned above.

In case of the Cloudflare error, we retry it only once because they usually work with only one retry. The current logic in the SDK, retries only once.

Please let me know if this is becoming a common issue for you, we can look into why is it being a regular occurrence for you.

Provide feedback

Saved searches

Use saved searches to filter your results more quickly

Uh oh!

Automatic retries / Cloudflare 502 bad gateway #339

Uh oh!

{{title}}

Uh oh!

Replies: 2 comments 4 replies

Uh oh!

{{title}}

Uh oh!

Uh oh!

{{title}}

Uh oh!

Uh oh!

{{title}}

Uh oh!

Uh oh!

{{title}}

Uh oh!

Uh oh!

{{title}}

Uh oh!

Uh oh!

{{editor}}'s edit

{{editor}}'s edit

Uh oh!

Uh oh!

{{title}}

Uh oh!

Select a reply

Uh oh!

Automatic retries / Cloudflare 502 bad gateway #339

Uh oh!

albinekb Apr 30, 2025

Replies: 2 comments · 4 replies

Uh oh!

csgulati09 Apr 30, 2025 Maintainer

Uh oh!

albinekb Apr 30, 2025 Author

Uh oh!

csgulati09 Apr 30, 2025 Maintainer

Uh oh!

csgulati09 Apr 30, 2025 Maintainer

Uh oh!

Uh oh!

albinekb May 2, 2025 Author

Uh oh!

csgulati09 May 2, 2025 Maintainer

albinekb
Apr 30, 2025

Replies: 2 comments 4 replies

csgulati09
Apr 30, 2025
Maintainer

albinekb Apr 30, 2025
Author

csgulati09 Apr 30, 2025
Maintainer

csgulati09
Apr 30, 2025
Maintainer

albinekb May 2, 2025
Author

csgulati09 May 2, 2025
Maintainer