Skip to content

Feature Request: Get rate limit headers from request  #385

Open
@aniskand

Description

@aniskand

Feature Description

Provide rate limit headers through API.

Problem it Solves

OpenAI provides rate limit headers in the response,

https://platform.openai.com/docs/guides/rate-limits/rate-limits-in-headers

however there is currently no way to access through the API.

Proposed Solution

I'm thinking of 3 possible solutions:

  1. Add separate functions that return response with headers. I created a pull request with an example of this, [FEATURE]Get response headers from messages api #380 but I'm not really in love with this solution. The only benefit of it is that it doesn't change anything for existing users of the API and is fairly straightforward code wise. It's definitely not an ideal solution though.

  2. Add a parameter to the RequestOptions that indicates if user wants headers and include them (or not) in a parent object that includes the response object + headers. This would be fairly simple and clean code wise, but would essentially break the API for all current users.

  3. Keep a persistent store of the rate limit header values in the OpenAI client object and update with every call. I kind of like this solution the best as it would decouple the rate limit headers from the response and make them always available. It also shouldn't break the API for any existing users. It could be a bit complicated to implement due to concurrency issues but I don't think it's insurmountable.

Please feel free to suggest other solutions. Thanks.

Additional Context

Add any other context, examples, or references that might help illustrate the feature or its benefits.

Metadata

Metadata

Assignees

No one assigned

    Labels

    enhancementNew feature or request

    Projects

    No projects

    Milestone

    No milestone

    Relationships

    None yet

    Development

    No branches or pull requests

    Issue actions