howtos/How to add few-shot prompt to an agent with memory and tool usage?

[BEbillionaireUSD]

Components

• agent with tools
• agent with memory
• few-shot prompt with ToolMessage

Specifically, I'm wondering how to combine an agent with bonded tools and memory

from langgraph.checkpoint.memory import MemorySaver
from langgraph.prebuilt import create_react_agent
create_react_agent(model=llm, tools, state_modifier="you are an assistant", checkpointer=MemorySaver())

with few-shot prompt like the example in https://python.langchain.com/docs/how_to/tools_few_shot/:

examples = [
    HumanMessage(
        "What's the product of 317253 and 128472 plus four", name="example_user"
    ),
    AIMessage(
        "",
        name="example_assistant",
        tool_calls=[
            {"name": "Multiply", "args": {"x": 317253, "y": 128472}, "id": "1"}
        ],
    ),
    ToolMessage("16505054784", tool_call_id="1"),
    AIMessage(
        "",
        name="example_assistant",
        tool_calls=[{"name": "Add", "args": {"x": 16505054784, "y": 4}, "id": "2"}],
    ),
    ToolMessage("16505054788", tool_call_id="2"),
    AIMessage(
        "The product of 317253 and 128472 plus four is 16505054788",
        name="example_assistant",
    ),
]

system = """You are bad at math but are an expert at using a calculator. 

Use past tool usage as an example of how to correctly use the tools."""
few_shot_prompt = ChatPromptTemplate.from_messages(
    [
        ("system", system),
        *examples,
        ("human", "{query}"),
    ]
)

That means, I would like to either edit state_modifie when calling create_react_agent
Or add memory to a chain chain = {"query": RunnablePassthrough()} | few_shot_prompt | llm_with_tools

[dosubot[bot]]

Hey there, @BEbillionaireUSD! I'm here to help you with any bugs, questions, or contributions you have. Let's tackle this together!

To combine an agent with bonded tools and memory using create_react_agent with a few-shot prompt, you can follow these steps:

1. Define the Tools and Model:
   Initialize the tools and the language model you want to use. For example, using a custom tool and OpenAI's chat model:

   from langchain_core.tools import tool
   from langchain_openai import ChatOpenAI
   
   model = ChatOpenAI(model="gpt-4o")
   
   @tool
   def magic_function(input: int) -> int:
       """Applies a magic function to an input."""
       return input + 2
   
   tools = [magic_function]

2. Create a Few-Shot Prompt:
   Define a few-shot prompt that will guide the agent's behavior:

   from langchain_core.prompts import ChatPromptTemplate
   
   examples = [
       HumanMessage("What's the product of 317253 and 128472 plus four", name="example_user"),
       AIMessage("", name="example_assistant", tool_calls=[{"name": "Multiply", "args": {"x": 317253, "y": 128472}, "id": "1"}]),
       ToolMessage("16505054784", tool_call_id="1"),
       AIMessage("", name="example_assistant", tool_calls=[{"name": "Add", "args": {"x": 16505054784, "y": 4}, "id": "2"}]),
       ToolMessage("16505054788", tool_call_id="2"),
       AIMessage("The product of 317253 and 128472 plus four is 16505054788", name="example_assistant"),
   ]
   
   system = """You are bad at math but are an expert at using a calculator. 
   Use past tool usage as an example of how to correctly use the tools."""
   
   few_shot_prompt = ChatPromptTemplate.from_messages(
       [
           ("system", system),
           *examples,
           ("human", "{query}"),
       ]
   )

3. Create the Agent with create_react_agent:
   Use create_react_agent to create the agent, specifying the state_modifier with your prompt:

   from langgraph.prebuilt import create_react_agent
   
   def _modify_state_messages(state):
       return few_shot_prompt.invoke({"messages": state["messages"]}).to_messages()
   
   agent = create_react_agent(model, tools, state_modifier=_modify_state_messages, checkpointer=MemorySaver())

4. Add Memory to the Agent:
   To add memory to manage the history of messages, use ConversationBufferMemory and ReadOnlySharedMemory to ensure tools can access but not modify the memory:

   from langchain.memory import ConversationBufferMemory, ReadOnlySharedMemory
   
   memory = ConversationBufferMemory(memory_key="chat_history")
   readonlymemory = ReadOnlySharedMemory(memory=memory)
   
   # Example of adding a tool that uses memory
   from langchain.chains import LLMChain
   from langchain.prompts import PromptTemplate
   
   template = """This is a conversation between a human and a bot:
   
   {chat_history}
   
   Write a summary of the conversation for {input}:
   """
   prompt = PromptTemplate(input_variables=["input", "chat_history"], template=template)
   summary_chain = LLMChain(
       llm=OpenAI(),
       prompt=prompt,
       verbose=True,
       memory=readonlymemory,
   )
   
   tools.append(Tool(
       name="Summary",
       func=summary_chain.run,
       description="useful for when you summarize a conversation. The input to this tool should be a string, representing who will read this summary.",
   ))
   
   agent_executor = AgentExecutor(agent=agent, tools=tools, memory=memory)

This setup allows the agent to use tools and maintain a conversational context with memory, while also allowing you to modify the state with a custom function. The use of ReadOnlySharedMemory ensures that tools can access the conversation history without altering it, maintaining the integrity of the memory ^[1][2][3].

To continue talking to Dosu, mention @dosu.

---

^{Help Dosu learn! Give it feedback: Great Response | Irrelevant Answer | Incorrect Sources | Too Verbose | Hallucination | Other}

[BEbillionaireUSD]

Thanks. Can you tell me how to invoke agent_executor? For example, I want to ask it a question like
"What's the product of 11 and 7 minus 10"
@dosu

[dosubot[bot]]

To invoke agent_executor and ask a question like "What's the product of 11 and 7 minus 10", you can use the following code:

agent_executor.invoke(
    {
        "input": "What's the product of 11 and 7 minus 10"
    }
)

This code snippet sends the input question to the agent_executor, which processes it using the tools and logic defined in the agent ^[1].

To continue talking to Dosu, mention @dosu.

---

^{Help Dosu learn! Give it feedback: Great Response | Irrelevant Answer | Incorrect Sources | Too Verbose | Hallucination | Other}

[BEbillionaireUSD]

You've used multiple keywards: query, messages, input.
Do you agree that they should all be the same? So we have:

few_shot_prompt = ChatPromptTemplate.from_messages(
    [
        ("system", system),
        *examples,
        ("human", "{messages}"),
    ]
)

def _modify_state_messages(state):
    return few_shot_prompt.invoke({"messages": state["messages"]}).to_messages()

and finally

agent_executor.invoke(
    {
        "messages": "What's the product of 11 and 7 minus 10"
    }
)

Also, where to set config when invoking? @dosu

[dosubot[bot]]

In LangChain, it's important to use the correct keys for different contexts. The recommended key for input when invoking agents is action_input ^[1]. However, if you're using messages consistently across your setup, ensure that it aligns with how your ChatPromptTemplate and agent are configured.

For your setup, if you decide to use messages as the key, make sure it's consistently applied throughout your code:

few_shot_prompt = ChatPromptTemplate.from_messages(
    [
        ("system", system),
        *examples,
        ("human", "{messages}"),
    ]
)

def _modify_state_messages(state):
    return few_shot_prompt.invoke({"messages": state["messages"]}).to_messages()

agent_executor.invoke(
    {
        "messages": "What's the product of 11 and 7 minus 10"
    }
)

Regarding the config parameter, you can include it when invoking agent_executor to pass additional configuration options, such as thread_id, callbacks, tags, and metadata. Here's how you can set it:

agent_executor.invoke(
    {
        "messages": "What's the product of 11 and 7 minus 10"
    },
    config={
        "configurable": {"thread_id": "abc123"},
        "callbacks": [...],
        "tags": [...],
        "metadata": {...}
    }
)

This allows you to manage the execution flow and logging, and maintain state across interactions ^[2].

To continue talking to Dosu, mention @dosu.

---

^{Help Dosu learn! Give it feedback: Great Response | Irrelevant Answer | Incorrect Sources | Too Verbose | Hallucination | Other}