Why supported_props for tool message?

[SeanIsYoung]

Checked other resources

• I added a very descriptive title to this question.
• I searched the LangChain documentation with the integrated search.
• I used the GitHub search to find a similar question and didn't find it.

Commit to Help

• I commit to help with one of those options 👆

Example Code

N/A

Description

I'm currently serving a hermes model using vLLM. vLLM have just added support to their api for function / tool calling. However, with the hermes models I run into an error where the "name" property isn't being passed for the tool responses and so the chat template throws and error.

EDIT: The chat template has since been updated, but I'm still curious as to why this is the case?

In trying to find a fix for this I came across this:

https://github.com/langchain-ai/langchain/blob/76bce4262955d3460e5b6cadd738cf710a4dbebb/libs/partners/openai/langchain_openai/chat_models/base.py#L225C1-L230C87

    elif isinstance(message, ToolMessage):
        message_dict["role"] = "tool"
        message_dict["tool_call_id"] = message.tool_call_id


        supported_props = {"content", "role", "tool_call_id"}
        message_dict = {k: v for k, v in message_dict.items() if k in supported_props}

The reason for my error is that anything not a member of supported_props is removed from the dictionary.

Why is this being done?

And what would be a good fix?

• adding "name" to supported_props
• removing supported_props
• letting the user configure supported_props

I've also noticed that just above this something wierd is going on with tool_calls. There are two different ways of handling tool_calls. But one of them has supported_props while the other doesn't. Any reason for this either?

System Info

System Information

OS: Linux
OS Version: #1 SMP Fri Mar 29 23:14:13 UTC 2024
Python Version: 3.11.9 (main, Apr 19 2024, 16:48:06) [GCC 11.2.0]

Package Information

langchain_core: 0.2.38
langchain: 0.2.16
langchain_community: 0.2.16
langsmith: 0.1.105
langchain_huggingface: 0.0.3
langchain_milvus: 0.1.4
langchain_openai: 0.1.23
langchain_text_splitters: 0.2.2
langchain_unstructured: 0.1.2

Optional packages not installed

langgraph
langserve

Other Dependencies

aiohttp: 3.10.5
async-timeout: Installed. No version info available.
dataclasses-json: 0.6.7
httpx: 0.27.2
huggingface-hub: 0.24.6
jsonpatch: 1.33
numpy: 1.26.4
openai: 1.42.0
orjson: 3.10.7
packaging: 24.1
pydantic: 2.8.2
pymilvus: 2.4.5
PyYAML: 6.0.2
requests: 2.32.3
scipy: 1.14.1
sentence-transformers: 3.0.1
SQLAlchemy: 2.0.32
tenacity: 8.5.0
tiktoken: 0.7.0
tokenizers: 0.19.1
transformers: 4.44.2
typing-extensions: 4.12.2
unstructured-client: 0.24.1
unstructured[all-docs]: Installed. No version info available.

[dosubot[bot]]

The "name" property is not included in supported_props for ToolMessage because it is being added to the additional_kwargs dictionary instead of being directly set as a property of ToolMessage. This filtering is likely causing your issue with the missing "name" property in tool responses.

To fix this issue, you should modify the _convert_dict_to_message function to directly set the name property of ToolMessage instead of adding it to additional_kwargs. Here is the corrected code:

def _convert_dict_to_message(_dict: Mapping[str, Any]) -> BaseMessage:
    """Convert a dictionary to a LangChain message.

    Args:
        _dict: The dictionary.

    Returns:
        The LangChain message.
    """
    role = _dict.get("role")
    if role == "user":
        return HumanMessage(content=_dict.get("content", ""))
    elif role == "assistant":
        content = _dict.get("content", "") or ""
        additional_kwargs: Dict = {}
        if function_call := _dict.get("function_call"):
            additional_kwargs["function_call"] = dict(function_call)
        tool_calls = []
        invalid_tool_calls = []
        if raw_tool_calls := _dict.get("tool_calls"):
            additional_kwargs["tool_calls"] = raw_tool_calls
            for raw_tool_call in raw_tool_calls:
                try:
                    tool_calls.append(parse_tool_call(raw_tool_call, return_id=True))
                except Exception as e:
                    invalid_tool_calls.append(
                        dict(make_invalid_tool_call(raw_tool_call, str(e)))
                    )
        return AIMessage(
            content=content,
            additional_kwargs=additional_kwargs,
            tool_calls=tool_calls,
            invalid_tool_calls=invalid_tool_calls,
        )
    elif role == "system":
        return SystemMessage(content=_dict.get("content", ""))
    elif role == "function":
        return FunctionMessage(
            content=_dict.get("content", ""), name=_dict.get("name", "")
        )
    elif role == "tool":
        return ToolMessage(
            content=_dict.get("content", ""),
            tool_call_id=_dict.get("tool_call_id", ""),
            name=_dict.get("name", "")
        )
    else:
        return ChatMessage(content=_dict.get("content", ""), role=role or "")

This change ensures that the name property is directly set on the ToolMessage object. Additionally, you should update the _convert_message_to_dict function to include the name property in the dictionary representation of ToolMessage:

@staticmethod
def _convert_message_to_dict(message: BaseMessage) -> dict:
    message_dict = {"content": message.content}
    if (name := message.name or message.additional_kwargs.get("name")) is not None:
        message_dict["name"] = name
    if isinstance(message, ChatMessage):
        message_dict["role"] = message.role
    elif isinstance(message, HumanMessage):
        message_dict["role"] = "user"
    elif isinstance(message, AIMessage):
        message_dict["role"] = "assistant"
        if message.tool_calls or message.invalid_tool_calls:
            message_dict["tool_calls"] = [
                _lc_tool_call_to_openai_tool_call(tc) for tc in message.tool_calls
            ] + [
                _lc_invalid_tool_call_to_openai_tool_call(tc)
                for tc in message.invalid_tool_calls
            ]  # type: ignore[assignment]
        elif "tool_calls" in message.additional_kwargs:
            message_dict["tool_calls"] = message.additional_kwargs["tool_calls"]
            tool_call_supported_props = {"id", "type", "function"}
            message_dict["tool_calls"] = [
                {
                    k: v for k, v in tool_call.items()  # type: ignore[union-attr]
                    if k in tool_call_supported_props
                }
                for tool_call in message_dict["tool_calls"]
            ]
        else:
            pass
        if "tool_calls" in message_dict:
            message_dict["content"] = message_dict["content"] or None  # type: ignore[assignment]
    elif isinstance(message, SystemMessage):
        message_dict["role"] = "system"
    elif isinstance(message, ToolMessage):
        message_dict["role"] = "tool"
        message_dict["tool_call_id"] = message.tool_call_id
        supported_props = {"content", "role", "tool_call_id", "name"}
        message_dict = {
            k: v for k, v in message_dict.items() if k in supported_props
        }
    elif isinstance(message, FunctionMessage):
        raise ValueError(
            "Function messages are not supported by Databricks. Please"
            " create a feature request at https://github.com/mlflow/mlflow/issues."
        )
    else:
        raise ValueError(f"Got unknown message type: {message}")
    if "function_call" in message.additional_kwargs:
        ChatMlflow._raise_functions_not_supported()
    return message_dict

This ensures that the name property is included in the dictionary representation of ToolMessage ^[1][2][3].

To continue talking to Dosu, mention @dosu.