ChatLlamaCpp Function Call(s) - Cannot pick from a list of tools

[evan-dayy]

Checked other resources

• I added a very descriptive title to this question.
• I searched the LangChain documentation with the integrated search.
• I used the GitHub search to find a similar question and didn't find it.

Commit to Help

• I commit to help with one of those options 👆

Example Code

llm = ChatLlamaCpp(
    model_path="./Hermes-2-Pro-Llama-3-8B-Q8_0.gguf",
    verbose=False,
    repeat_penalty=1.5,
    top_p=0.5,
    max_tokens=512)

@tool("get_magic_function", args_schema=MagicFunctionInput)
def magic_function(magic_function_input: int):
    """Get the value of magic function for an input."""
    return magic_function_input + 2
@tool("get_magic_function", args_schema=MagicFunctionInput)
def magic_function(magic_function_input: int):
    """Get the value of magic function for an input."""
    return magic_function_input + 2


llm_with_tools = llm.bind_tools(
    tools=[get_weather, magic_function],
    tool_choice={"type": "function", "function": {
        "name": "get_current_weather"}},
)

ai_msg = llm_with_tools.invoke(
    "What is current weather of SF?"
)

Description

It seems that ChatLlamaCpp cannot be binded with a list of tools. I try to set the tool_choice as any, but it give me an error message.

Is this because ChatLLlamaCpp does not support choice between tools yet? Anyone help?

Thanks!

System Info

File "/Users/yichaod/.pyenv/versions/3.10.14/lib/python3.10/site-packages/langchain_community/chat_models/llamacpp.py", line 364, in bind_tools
raise ValueError(
ValueError: Tool choice tool_choice='any' was specified, but the only provided tools were ['get_current_weather', 'get_magic_function'].

[dosubot[bot]]

Hello, @evan-dayy! I'm here to help you with any bugs, questions, or contribution guidance you may need. Let's tackle this issue together!

Yes, the ChatLlamaCpp model currently does not support choosing between tools automatically. The tool_choice parameter does not support "any" or "auto" choices like the OpenAI tool-calling API. You need to specify a specific tool to force its use.

Here's the relevant part of the bind_tools method:

def bind_tools(
    self,
    tools: Sequence[Union[Dict[str, Any], Type[BaseModel], Callable, BaseTool]],
    *,
    tool_choice: Optional[Union[Dict[str, Dict], bool, str]] = None,
    **kwargs: Any,
) -> Runnable[LanguageModelInput, BaseMessage]:
    """Bind tool-like objects to this chat model

    tool_choice: does not currently support "any", "auto" choices like OpenAI
        tool-calling API. should be a dict of the form to force this tool
        {"type": "function", "function": {"name": <<tool_name>>}}.
    """
    ...

This means you need to specify the tool explicitly in the tool_choice parameter ^[1].

For example, if you want to use the get_magic_function tool, you should set the tool_choice parameter like this:

llm_with_tools = llm.bind_tools(
    tools=[get_weather, magic_function],
    tool_choice={"type": "function", "function": {"name": "get_magic_function"}},
)

This will force the model to use the get_magic_function tool.

To continue talking to Dosu, mention @dosu.

[raj-maharajwala]

Any Workaround solution to bind multiple tools to llm and auto detect tools or follow the workflow??

llm = ChatLlamaCpp(.........)

llm_with_tools = llm.bind_tools(
[list of all the tools tool 1, tool 2, tool 3],
tool_choice = ...........custom_work........)