create_tool_calling_agent only ouput tool result in json instead of a straightfoward answer

[renyuantime]

Checked other resources

• I added a very descriptive title to this question.
• I searched the LangChain documentation with the integrated search.
• I used the GitHub search to find a similar question and didn't find it.

Commit to Help

• I commit to help with one of those options 👆

Example Code

from langchain.agents import AgentExecutor, create_tool_calling_agent
prompt = ChatPromptTemplate.from_messages(
    [("system", "You are a helpful Finance/Investment AI assistant."),
        ("placeholder", "{chat_history}"),
        ("human", "{input}"),
        ("placeholder", "{agent_scratchpad}"),]
)

#Construct the Tools agent
agent = create_tool_calling_agent(llm, tools, prompt)

agent_executor = AgentExecutor(agent=agent, tools=tools, verbose=True)
agent_executor.invoke({"input": "what is apple's revenue on 2022? give me a short answer"})

Description

I built a create_tool_calling_agent that has the get_income_statement tool, which takes in company ticker,start_date, end_date and returns corresponding income statement data. when I asked "What is apple's revenue in 2022? ", the agent only returned me the data from the tools in JSON format instead of a text like' Apple's revenue in 2022 is xxxx'.( see picture) what went wrong, why can't the agent observer the tool result and return me a text with the answer of my question in it.

System Info

Linux

[dosubot[bot]]

Hey there, @renyuantime! I'm here to help you with any bugs, questions, or contributions you have. Let's tackle this issue together while we wait for a human maintainer.

The create_tool_calling_agent outputs the tool result in JSON format because tools in LangChain are designed with a JSON schema for their inputs and outputs. This schema ensures that the language model (LLM) can understand and specify the actions to take. The JSON format provides a structured and clear way to represent the data, which is essential for the LLM to process and use the tool effectively.

To get a straightforward text answer, you need to adjust the agent's logic to process the JSON output and format it into a human-readable response. Here are some steps to achieve this:

1. Modify the Conversational Prompt Template: Update the template to explicitly instruct the agent to include the search results in its response. This might involve adding placeholders or specific instructions within the template that tell the agent to format and present the search results.

2. Ensure Proper Formatting with JSON Blob: When invoking your custom tool, make sure the output is formatted in a way that the agent can easily include in its responses. This might mean adjusting the output of your tool to a JSON blob that includes keys like "action" and "action_input", making it straightforward for the agent to parse and include these details in its response.

3. Incorporate Detailed Results in Agent's Response: Adjust the agent's logic to ensure it doesn't just acknowledge the search results but also formats and includes the detailed results in its response. This could involve tweaking the logic that processes the tool's output, ensuring it extracts and formats the relevant details (e.g., revenue) for inclusion in the chatbot's messages.

By following these steps, you can enhance your agent to not only perform searches using your custom tool but also to present those search results directly in its conversations with users, making the bot more informative and helpful ^[1].

Additionally, the .with_structured_output() method in LangChain allows models to return output that matches a specific schema, such as a TypedDict class, JSON Schema, or a Pydantic class. This method ensures that the output is structured and validated, making it easier to use with downstream systems or databases ^[2].

To continue talking to Dosu, mention @dosu.