ChatOllama doesn't call Tool

[kurup-007]

Checked other resources

• I added a very descriptive title to this question.
• I searched the LangChain documentation with the integrated search.
• I used the GitHub search to find a similar question and didn't find it.

Commit to Help

• I commit to help with one of those options 👆

Example Code

from typing import List

from langchain_ollama import ChatOllama
from typing_extensions import TypedDict


def validate_user(user_id: int, addresses: List) -> bool:
    """Validate user using historical addresses.

    Args:
        user_id: (int) the user ID.
        addresses: Previous addresses.
    """
    return True


llm = ChatOllama(
    model="llama3-groq-tool-use",
    temperature=0,
).bind_tools([validate_user])

result = llm.invoke(
    "Could you validate user 123? They previously lived at "
    "123 Fake St in Boston MA and 234 Pretend Boulevard in "
    "Houston TX."
)
result.tool_calls

Description

I am trying to migrate my code from OllamaFunction to ChatOllama but when I tried a sample case with a function but model doesn't call the tool and responds from it's knowledge and the result.tool_calls array is empty.

System Info

System Information

OS: Windows
OS Version: 10.0.22631
Python Version: 3.12.4 | packaged by Anaconda, Inc. | (main, Jun 18 2024, 15:03:56) [MSC v.1929 64 bit (AMD64)]

Package Information

langchain_core: 0.2.29
langchain: 0.2.12
langchain_community: 0.2.11
langsmith: 0.1.98
langchain_experimental: 0.0.64
langchain_ollama: 0.1.1
langchain_text_splitters: 0.2.2
langgraph: 0.2.3

Optional packages not installed

langserve

Other Dependencies

aiohttp: 3.10.2
async-timeout: Installed. No version info available.
dataclasses-json: 0.6.7
jsonpatch: 1.33
langgraph-checkpoint: 1.0.2
numpy: 1.26.4
ollama: 0.3.1
orjson: 3.10.7
packaging: 24.1
pydantic: 2.8.2
PyYAML: 6.0.2
requests: 2.32.3
SQLAlchemy: 2.0.32
tenacity: 8.5.0
typing-extensions: 4.12.2

[dosubot[bot]]

Hey @kurup-007! I'm here to help you with your code and answer any questions you have while waiting for a human maintainer. Let's work together to solve this issue!

To ensure that the ChatOllama model calls the validate_user tool instead of responding from its own knowledge base, you need to specify the tool_choice parameter when binding the tool. This parameter forces the model to prioritize the tool call. Here is how you can modify your code:

from typing import List
from langchain_ollama import ChatOllama
from typing_extensions import TypedDict

def validate_user(user_id: int, addresses: List) -> bool:
    """Validate user using historical addresses.

    Args:
        user_id: (int) the user ID.
        addresses: Previous addresses.
    """
    return True

llm = ChatOllama(
    model="llama3-groq-tool-use",
    temperature=0,
).bind_tools(
    tools=[validate_user],
    tool_choice={"type": "function", "function": {"name": "validate_user"}}
)

result = llm.invoke(
    "Could you validate user 123? They previously lived at "
    "123 Fake St in Boston MA and 234 Pretend Boulevard in "
    "Houston TX."
)
print(result.tool_calls)

In this example, the tool_choice parameter is set to {"type": "function", "function": {"name": "validate_user"}}, which ensures that the model prioritizes calling the validate_user tool when invoked ^[1].

To continue talking to Dosu, mention @dosu.

[kurup-007]

@dosu thanks for the suggestion, i updated my code and tried again but still I get the same empty array response.

[dosubot[bot]]

If updating the code with the tool_choice parameter didn't resolve the issue, there might be other factors at play. Here are a few additional steps you can take to debug and ensure the tool invocation works correctly:

1. Verbose and Debug Mode: Enable verbose and debug modes to get more detailed logs about the tool invocation process.

Verbose Mode

from langchain.globals import set_verbose

set_verbose(True)

Debug Mode

from langchain.globals import set_debug

set_debug(True)
set_verbose(False)

2. Check Tool Schema: Ensure that the tool schema is correctly defined and matches the expected format.

Example Code with Verbose and Debug Mode

from typing import List
from langchain_ollama import ChatOllama
from typing_extensions import TypedDict
from langchain.globals import set_verbose, set_debug

set_verbose(True)
set_debug(True)

def validate_user(user_id: int, addresses: List) -> bool:
    """Validate user using historical addresses.

    Args:
        user_id: (int) the user ID.
        addresses: Previous addresses.
    """
    return True

llm = ChatOllama(
    model="llama3-groq-tool-use",
    temperature=0,
).bind_tools(
    tools=[validate_user],
    tool_choice={"type": "function", "function": {"name": "validate_user"}}
)

result = llm.invoke(
    "Could you validate user 123? They previously lived at "
    "123 Fake St in Boston MA and 234 Pretend Boulevard in "
    "Houston TX."
)
print(result.tool_calls)

3. Test with a Simple Tool: Try using a simpler tool to see if the issue is specific to the validate_user tool.

Simple Tool Example

def add(a: int, b: int) -> int:
    """Adds two numbers."""
    return a + b

llm = ChatOllama(
    model="llama3-groq-tool-use",
    temperature=0,
).bind_tools(
    tools=[add],
    tool_choice={"type": "function", "function": {"name": "add"}}
)

result = llm.invoke("What is 3 + 4?")
print(result.tool_calls)

4. Check for Known Issues: Be aware of the known limitations with the ChatOllama model. The OllamaFunctions wrapper is experimental and may lead to unpredictable behavior ^[1].

If these steps do not resolve the issue, there might be a deeper problem with the tool invocation mechanism in the current version of ChatOllama.

To continue talking to Dosu, mention @dosu.