Get metris (input_token, output_token) from arun function #3465
Unanswered
hoanganh2070
asked this question in
Q&A
Replies: 0 comments
Sign up for free
to join this conversation on GitHub.
Already have an account?
Sign in to comment
Uh oh!
There was an error while loading. Please reload this page.
-
I'm using await self.client.arun(message=message, stream=True) to stream chat responses in my chat_stream method. It works fine and I can iterate over the responses, but I would like to extract metrics (e.g., token etc.) from the stream.
How can I access the metrics from the response_stream or each response during or after streaming?
I tried printing dir(response_stream) and dir(response) but couldn't find a clear metrics or usage attribute.
Is there a built-in way to access this metadata, or do I need to enable/configure something in the client
Beta Was this translation helpful? Give feedback.
All reactions