ChatOpenAI AgentExecutor not accessing tools when using LLMs hosted on server

[Sreeni1204]

Checked other resources

• I added a very descriptive title to this question.
• I searched the LangChain documentation with the integrated search.
• I used the GitHub search to find a similar question and didn't find it.

Commit to Help

• I commit to help with one of those options 👆

Example Code

from langchain_openai import ChatOpenAI
from langchain_core.tools import tool
from langchain.agents import AgentExecutor, create_tool_calling_agent

model_name="Llama-3.3-70B-Instruct"
self.llm = ChatOpenAI(
    openai_api_key="******",
    temperature=0,
    base_url="link to private server",
    default_headers={"Authorization":f"Basic {token}"},
    model=model_name,
    seed=11,
    stream=True,
)

@tool("Weather of a City")
def weather(city: str) -> str:
    """Use this tool to get the weather of city"""

    api_key = "******"
    url = f'http://api.openweathermap.org/data/2.5/weather?q={city}&appid={api_key}&lang=de'

    response = requests.get(url)

    if response.status_code == 200:
        data = response.json()
        temp = data['main']['temp']
        desc = data['weather'][0]['description']
        return f'Temperature: {round(temp-273.15)} C\nDescription: {desc}'
        
tools = [weather]

agent = create_tool_calling_agent(
    llm, 
    tools, 
    prompt,
)

agent_executor = AgentExecutor(
    agent=agent,
    tools=tools,
    verbose=True,
    max_iterations=3,
    handle_parsing_errors=True,
)

response = agent.invoke(
        {
            "messages": [HumanMessage(content=current_input)],
            #"messages": current_input,
        },
        config={"configurable": {"session_id": "unused"}},
        )

Description

With the above example code, I Am getting the following issue:

With normal interaction I am able to get the response from the agent, so there is not connectivity issue with the private server. But the AgentExecutor is not interacting with the tools. The same tools are accessed when I change the model with gpt4. I know the model hosted on the private server has the ability to interact with tools. Also, while I enabled the debug option, I can see below response which looks weird for finish_reason and model_name

[llm/end] [chain:RunnableSequence > chain:RunnableWithMessageHistory > chain:check_sync_or_async > chain:AgentExecutor > chain:RunnableSequence > llm:ChatOpenAI] [1.16s] Exiting LLM run with output:
{
  "generations": [
    [
      {
        "text": "",
        "generation_info": {
          "finish_reason": "streamingstreamingstreamingstreamingstreamingstreamingstop",
          "model_name": "Llama-3.3-70B-InstructLlama-3.3-70B-InstructLlama-3.3-70B-InstructLlama-3.3-70B-InstructLlama-3.3-70B-InstructLlama-3.3-70B-InstructLlama-3.3-70B-Instruct"
        },
        "type": "ChatGenerationChunk",
        "message": {
          "lc": 1,
          "type": "constructor",
          "id": [
            "langchain",
            "schema",
            "messages",
            "AIMessageChunk"
          ],
          "kwargs": {
            "content": "",
            "response_metadata": {
              "finish_reason": "streamingstreamingstreamingstreamingstreamingstreamingstop",
              "model_name": "Llama-3.3-70B-InstructLlama-3.3-70B-InstructLlama-3.3-70B-InstructLlama-3.3-70B-InstructLlama-3.3-70B-InstructLlama-3.3-70B-InstructLlama-3.3-70B-Instruct"
            },
            "type": "AIMessageChunk",
            "id": "run-49a0246c-3f78-43ac-b32f-791e81775df7",
            "tool_calls": [],
            "invalid_tool_calls": []
          }
        }
      }
    ]
  ],
  "llm_output": null,
  "run": null,
  "type": "LLMResult"
}

Could you please support.

System Info

python 3.10
langchain==0.3.20
langchain-community==0.3.19
langchain-core==0.3.41
langchain-ollama==0.2.3
langchain-openai==0.3.7
langchain-text-splitters==0.3.6

[PragyanTiwari]

It's a bit confusing for me. You're using ChatOpenAI, which is designed for OpenAI models, but you're passing Llama-3.3-70B-Instruct. This mismatch is likely causing the issue since LLaMA models don’t natively support OpenAI-style tool calling.

To fix this:

• Use an OpenAI model that supports tool calling (e.g., GPT-4).
• If you prefer Llama-3.3-70B-Instruct, you can either use ChatLlamaAPI or ChatGroq (will work best for you).

Also, ensure that you're using llm instead of self.llm, if that's not already the case.