Skip to content

Add Model Name and System Fingerprint to llm_output in _convert_response_to_chat_result #84

New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

Sign up for GitHub

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Merged
merged 7 commits into from
Mar 20, 2025
Merged

Add Model Name and System Fingerprint to llm_output in _convert_response_to_chat_result #84

Changes from 3 commits
Commits
Show all changes
7 commits
Select commit Hold shift + click to select a range
3cfff03
Enhance ChatResult output with model details and token usage
Mar 13, 2025
a902a91
Merge pull request #1 from jmoreno11/langchain/additional-metadata
jmoreno11 Mar 13, 2025
2345c5a
Update chat_models.py
jmoreno11 Mar 13, 2025
bb949fd
Dump metadata in llm_output and create new unit test
Mar 19, 2025
dd38d3b
Fix linting error
Mar 19, 2025
cac1d70
revert changes
Mar 19, 2025
60a1dc2
Fix linting errors (#2)
jmoreno11 Mar 19, 2025
File filter

Filter by extension

Filter by extension
Conversations
Failed to load comments.
Loading
Jump to
Jump to file
Failed to load files.
Loading
Diff view
Diff view
8 changes: 6 additions & 2 deletions integrations/langchain/src/databricks_langchain/chat_models.py
View file
Open in desktop
Original file line number Diff line number Diff line change
Expand Up @@ -307,8 +307,12 @@ def _convert_response_to_chat_result(self, response: Mapping[str, Any]) -> ChatR
)
for choice in response["choices"]
]
usage = response.get("usage", {})
return ChatResult(generations=generations, llm_output=usage)
llm_output = {
"token_usage": response.get("usage", {}),
Copy link
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Can we rename this to usage so it doesn't change the API for existing users?

Suggested change
"token_usage": response.get("usage", {}),
"usage": response.get("usage", {}),

Copy link
Contributor Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

I did as @bbqiu suggested, so usage remains intact

Copy link
Contributor Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

I also created a new unit test, since the existing ones were not altered by this change

"model_name": response.get("model", self.model),
"system_fingerprint": response.get("system_fingerprint", ""),
}
return ChatResult(generations=generations, llm_output=llm_output)

def _stream(
self,
Expand Down