How to implement bind_tools to custom LLM to call custom functions in agent

[chandrahk]

Checked other resources

• I added a very descriptive title to this question.
• I searched the LangChain documentation with the integrated search.
• I used the GitHub search to find a similar question and didn't find it.

Commit to Help

• I commit to help with one of those options 👆

Example Code

How to implement bind_tools to custom LLM to call custom functions in agent?  It appears there is nothing exposed in base LLM to do such a thing now

Description

How to implement bind_tools to custom LLM to call custom functions in agent? It appears there is nothing exposed in base LLM to do such a thing now

System Info

N/A

[dosubot[bot]]

Hey @chandrahk! Great to see you diving into the depths of LangChain again. How's the code wizardry going?

Based on your question, it seems like you're trying to bind custom functions to a custom Language Model (LLM) in LangChain. However, as per the LangChain codebase, there is no direct method available in the base LLM to bind tools or functions.

The bind_tools method is available in the ChatMistralAI class, which is a subclass of BaseChatModel. This method is designed to bind tool-like objects to the chat model, assuming the model is compatible with the OpenAI tool-calling API. You can find more details about this in the ChatMistralAI class definition in the LangChain codebase.

If you're trying to bind functions to the ChatOpenAI model, you might want to use the bind_functions method instead. This method accepts a list of function definitions, which can be a dictionary, a Pydantic model, or a callable. Pydantic models and callables are automatically converted to their schema dictionary representation.

Here's an example of how you can use the bind_functions method:

llm = ChatOpenAI(openai_api_key="xxx",temperature=0.5)
tools = [get_band_packages]
prompt = ChatPromptTemplate.from_messages(
    [
        (
            "system",
            system_msg
        ),
        MessagesPlaceholder(variable_name="chat_history"),
        ("user", "{input}"),
        MessagesPlaceholder(variable_name="agent_scratchpad"),
    ]
)
chat_history = []
functions = [format_tool_to_openai_function(t) for t in tools]
llm_with_tools = llm.bind_functions(functions=functions)
agent = (
        {
            "input": lambda x: x["input"],
            "agent_scratchpad": lambda x: format_to_openai_function_messages(
                x["intermediate_steps"]
            ),
            "chat_history": lambda x: x["chat_history"],
        }
        | prompt
        | llm_with_tools
        | OpenAIFunctionsAgentOutputParser()
)
agent_executor = AgentExecutor(agent=agent, tools=tools, verbose=True)
agent_executor.invoke({"input": "xxxxxxx","chat_history":[]})

In this example, the bind_functions method is used to bind the functions to the llm instance of the ChatOpenAI class. The functions parameter is a list of functions that have been formatted for OpenAI using the format_tool_to_openai_function function.

Please try this approach and let me know if it resolves your issue. If you have any further questions or need additional clarification, feel free to ask.

Sources

• Using tools in non-ChatGPT models
• openai.error.InvalidRequestError: Unrecognized request argument supplied: functions
• raise AttributeError(name) from None AttributeError: OPENAI_FUNCTIONS
• Functions not called when i use langchain_openai.ChatOpenAI as llm
• libs/partners/openai/langchain_openai/chat_models/base.py
• libs/partners/mistralai/langchain_mistralai/chat_models.py

About Dosu

This response is meant to be useful and save you time. It is not meant to be a precise solution, but rather a starting point for your own research.

Help me be more useful! Please leave a 👍 if this is helpful and 👎 if it is irrelevant.

If you want to continue the conversation, start your reply with @dosu-bot.

Sources

• AttributeError: 'ChatOpenAI' object has no attribute 'bind_tools'
• libs/langchain/langchain/agents/structured_chat/base.py
• libs/core/langchain_core/language_models/base.py

About Dosu

This response is meant to be useful and save you time. It is not meant to be a precise solution, but rather a starting point for your own research.

Help me be more useful! Please leave a 👍 if this is helpful and 👎 if it is irrelevant.

If you want to continue the conversation, start your reply with @dosu-bot.

[deepakagrawal]

@dosu-bot you did not answer to the OP's question. You are using ChatOpenAI in your example. However, the question was about using a custom LLM which is just a REST endpoint for LLama2.

[zlienqwe]

@deepakagrawal @chandrahk Did you find a way?

[deepakagrawal]

No

[mohammednuruddin]

So they also have a documentation for using models that do not support tool calling. THis solved my problem.
Here is the link

[javiermontesVN]

@chandrahk did you solve it?

[deepakagrawal]

@dosu-bot you did not answer to the OP's question. You are using ChatOpenAI in your example. However, the question was about using a custom LLM which is just a REST endpoint for LLama2.

[abdulqadircp]

@deepakagrawal have you solved it ?

[sepiatone]

@chandrahk The BaseChatModel exposes a bindtool interface. Your custom model could subclass BaseChatModel and implement bindtool.

Or maybe I'm understanding your requirement incorrectly.

[usersina]

Could you elaborate more on how .bind_tools should be exactly implemented? I did create a custom chat model but I'm a little stuck on how to make it properly return the correct JSON output for tool calling, especially considering the model I'm using does not support it natively. Thanks!

[sepiatone]

@usersina please create a separate discussion with a mre and someone could help you

[usersina]

@sepiatone - sure! Here's the link to the new discussion.

[mohammednuruddin]

Checkout this documentation to call tools without binding them. This works for custom models.

[yaoduo11]

@chandrahk Have you found a solution now?

[travisbikkle]

Here is an example to use LangChain agents with DeepSeek-v3(which is not supported by the ChatOpenAI api).

https://blog.eusoftbank.com/en/2025/04/use-any-llm-with-langchain-agent/

[abdulqadircp]

Any body has solved it ?

[usersina]

It's been a while, but maybe this example helps.