Dynamically creating tool chains

[ragrawal]

Hi all, I am trying to develop a general purpose chatbot that can have bunch of tools. I would like llm to automatically determine the right sequence of tools to call dependency. My problem is that it seems that llm understands the general strategy but I don't see right tools in tool_calls. Below is an example

from langchain_openai import AzureChatOpenAI
from langchain_core.tools import tool
from langchain_core.tools import InjectedToolArg
from typing_extensions import Annotated
from random import randint
from langchain_core.messages import HumanMessage, SystemMessage

@tool(parse_docstring=True, return_direct=False)
def get_item_id(item_name: str) -> int:
    """Return item id for the given item name

    Args:
        item_name: Name of the item

    Returns:
        str: Item id

    """
    return randint(1, 100)


@tool(parse_docstring=True, return_direct=False)
def get_item_description(item_id: Annotated[int, InjectedToolArg]) -> str:
    """Return the description of an item for the given item id

    Args:
        item_id: Id of the time

    Returns:
        str: Item description

    """
    return str("Description = " + randint(1, 100))



llm_with_tools = AzureChatOpenAI(
    deployment_name="gpt-4o-2024-08-06",
    model_name="gpt-4o-2024-08-06",
    streaming=True
).bind_tools(tools=[get_item_id, get_item_description])

messages = [
    SystemMessage(content="""
You are a general AI Chat that can answer questions within the scope of the tools provided. 
For the give human message, do following actions
1. Explain your strategy to determine the best possible way to respond to the human message
2. Execute the strategy and respond with final answer. 
""")
]

Example of showing the strategy

output = llm_with_tools.invoke(messages + [HumanMessage(content="Explain your strategy to get description of xyz")])
print(output.content)

Outcome

To obtain the description of an item named "xyz," I will follow these steps:

1. **Get the Item ID:** 
   - First, I will use the `functions.get_item_id` tool to get the unique identifier (ID) for the item with the name "xyz." This step is essential because descriptions are usually linked to item IDs rather than item names.

2. **Retrieve the Item Description:** 
   - Once I have the item ID, I will use the `functions.get_item_description` tool to get the detailed description of the item associated with that ID.

By executing these two steps, I will be able to provide the description for the item "xyz." This strategy ensures that I accurately fetch the description based on the correct item ID.

Now, I will execute this strategy to find the description of "xyz."

Executing the strategy

output = llm_with_tools.invoke(messages + [HumanMessage(content="get description for xyz")])
print(output.tool_calls)

Output
[{'name': 'get_item_id', 'args': {'item_name': 'xyz'}, 'id': 'call_tNrdUKQvtDC6C3pdPvWcSYzs', 'type': 'tool_call'}]

Any idea why the strategy explained by the LLM looks correct but tool_calls doesn't contain both the tools