Why does agent invoke the tool multiple times?

[Yueming-Yan]

Checked other resources

• I added a very descriptive title to this question.
• I searched the LangChain documentation with the integrated search.
• I used the GitHub search to find a similar question and didn't find it.

Commit to Help

• I commit to help with one of those options 👆

Example Code

from langchain_core.messages import HumanMessage
from langchain_core.tools import tool
from langchain_experimental.llms.ollama_functions import OllamaFunctions
from langgraph.prebuilt import create_react_agent

count = 0


@tool
def multiply(a: int, b: int) -> int:
    """Multiplies a and b."""
    global count
    count += 1
    print(count)
    return a * b


tools = [multiply]
llm = OllamaFunctions(model="llama3", format="json")

query = "What is 3 * 12?"

agent_executor = create_react_agent(llm, tools)
response = agent_executor.invoke({"messages": [HumanMessage(query)]})
result = response["messages"]
print(result)

Description

I have defined a variable named 'count'
Once the tool named 'multiply' be invoked, variable 'count' will increase then print the current value
I created an agent with this tool named 'agent_executor'

When above code was executed, I found the tool was invoked multiple times
This is the printout I got

1
2
3
4
5
6
7
8
9
10
11
12

My questions:
Why does agent invoke the tool multiple times? And how to invoke only once?

System Info

System Information

OS: Windows
OS Version: 10.0.19045
Python Version: 3.12.4 (tags/v3.12.4:8e8a4ba, Jun 6 2024, 19:30:16) [MSC v.1940 64 bit (AMD64)]

Package Information

langchain_core: 0.2.12
langchain: 0.2.7
langchain_community: 0.2.7
langsmith: 0.1.84
langchain_anthropic: 0.1.19
langchain_experimental: 0.0.62
langchain_openai: 0.1.14
langchain_text_splitters: 0.2.2
langgraph: 0.1.5

Packages not installed (Not Necessarily a Problem)

The following packages were not found:

langserve

[dosubot[bot]]

The agent invokes the tool multiple times because agents are designed to decide how many times to use tools based on the input. This flexibility allows the agent to handle complex tasks that may require multiple tool invocations.

To ensure the tool is invoked only once, you should use a chain instead of an agent. Chains are suitable when you know the specific sequence of tool usage needed for any user input. Here is an example of how to create a chain that ensures the tool is invoked only once:

from operator import itemgetter

chain = llm_with_tools | (lambda x: x.tool_calls[0]["args"]) | multiply
chain.invoke("What's three times 12")

This setup ensures that the tool is invoked exactly once by extracting the arguments of the first tool call and passing them to the tool.

Additionally, if you are using the LangChain framework, you can create an agent that uses self-ask with search prompting, which expects exactly one tool. Here is an example of how to create such an agent:

from langchain import hub
from langchain_community.chat_models import ChatAnthropic
from langchain.agents import (
    AgentExecutor, create_self_ask_with_search_agent
)

prompt = hub.pull("hwchase17/self-ask-with-search")
model = ChatAnthropic(model="claude-3-haiku-20240307")
tools = [...]  # Should just be one tool with name `Intermediate Answer`

agent = create_self_ask_with_search_agent(model, tools, prompt)
agent_executor = AgentExecutor(agent=agent, tools=tools)

agent_executor.invoke({"input": "hi"})

This setup ensures that the agent uses exactly one tool named Intermediate Answer, adhering to the constraints specified in the LangChain framework ^[1][2][3][4].

To continue talking to Dosu, mention @dosu.

[Yueming-Yan]

@dosu
no, although agent has its flexibility to decide how many times to use tools based on the input
according to my input 'What is 3 * 12?', agent should call specific tool only once instead of more than 12 times
I don't think chain is a placement of agent in this case

[dosubot[bot]]

To ensure that the agent invokes the multiply tool only once given the input "What is 3 * 12?", you can use the create_tool_calling_agent method to set up the agent with the specific tool. Here is the relevant code:

from langchain_core.tools import tool
from langchain import hub
from langchain.agents import AgentExecutor, create_tool_calling_agent
from langchain_openai import ChatOpenAI

@tool
def multiply(first_int: int, second_int: int) -> int:
    """Multiply two integers together."""
    return first_int * second_int

tools = [multiply]

# Get the prompt to use - you can modify this!
prompt = hub.pull("hwchase17/openai-tools-agent")
prompt.pretty_print()

# Use a model with tool calling capabilities
llm = ChatOpenAI(model="gpt-3.5-turbo-0125", temperature=0)

# Construct the tool calling agent
agent = create_tool_calling_agent(llm, tools, prompt)

# Create an agent executor by passing in the agent and tools
agent_executor = AgentExecutor(agent=agent, tools=tools, verbose=True)

# Invoke the agent with the query
query = "What is 3 * 12?"
response = agent_executor.invoke({"input": query})

This setup ensures that the multiply tool is specifically chosen and invoked once when the query "What is 3 * 12?" is processed. The AIMessage.tool_calls attribute of the output will show the tool invocation.

Additionally, you can use the JsonOutputParser to parse the model's output and ensure it requests the correct tool invocation. Here is an example of how to set up the prompt and parse the output:

from langchain_core.output_parsers import JsonOutputParser
from langchain_core.prompts import ChatPromptTemplate
from langchain_core.tools import render_text_description

rendered_tools = render_text_description(tools)

system_prompt = f"""\
You are an assistant that has access to the following set of tools. 
Here are the names and descriptions for each tool:

{rendered_tools}

Given the user input, return the name and input of the tool to use. 
Return your response as a JSON blob with 'name' and 'arguments' keys.

The `arguments` should be a dictionary, with keys corresponding 
to the argument names and the values corresponding to the requested values.
"""

prompt = ChatPromptTemplate.from_messages(
    [("system", system_prompt), ("user", "{input}")]
)

chain = prompt | llm | JsonOutputParser()
response = chain.invoke({"input": query})

This ensures that the model correctly outputs the required tool information in a structured format, which can then be used to invoke the tool exactly once ^{[1][2][3][4][5]}.

To continue talking to Dosu, mention @dosu.

[noggynoggy]

related #25546

[bliveinhack]

Hi @dosu
Can you give same solution in JS ?

[gztchan]

I highly recommend that you write your tool schema and append it into the prompt that you want to submit to the LLM. Then, let it just decides which tool it wants to invoke and what params for the tools. The advantage is that it is now a transparent process and you can fully take control.