We read every piece of feedback, and take your input very seriously.
To see all available qualifiers, see our documentation.
Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.
By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.
Already on GitHub? Sign in to your account
Stream last block not return token usage info when create_chat_completion_openai_v1 or create_chat_completion but server does.
Python api is better for me, but no token usage returned, is this by disign?
Called with
response = llm.create_chat_completion_openai_v1( messages=[ { "role": "system", "content": "请生成该指令的场景标签和功能标签,并以JSON列表的格式回复。", }, {"role": "user", "content": """<some content>"""}, ], temperature=0, max_tokens=192, stream=True, )
Set stream_options={"include_usage": True}, seems not work, any other ways?
Please provide detailed steps for reproducing the issue. We are not sitting in front of your screen, so the more detail the better.
The text was updated successfully, but these errors were encountered:
I think the #1552 is the one I needed.
Sorry, something went wrong.
No branches or pull requests
Prerequisites
Stream last block not return token usage info when create_chat_completion_openai_v1 or create_chat_completion but server does.
Python api is better for me, but no token usage returned, is this by disign?
Expected Behavior
Called with
Current Behavior
Failure Information (for bugs)
Set stream_options={"include_usage": True}, seems not work, any other ways?
Steps to Reproduce
Please provide detailed steps for reproducing the issue. We are not sitting in front of your screen, so the more detail the better.
The text was updated successfully, but these errors were encountered: