usage with langchain's trim_messages function

[scotchka]

The size of the chat history grows unbounded despite trim_messages. Using the example, I added two print statements:

async def chatbot(state: State):
    print('starting number of messages:', len(state['messages']))   # <----------------
    memory = await zep.memory.get(state["session_id"])
    system_message = SystemMessage(
        content=f"""You are a compassionate mental health bot and caregiver. Review information about the user and their prior conversation below and respond accordingly.
        Keep responses empathetic and supportive. And remember, always prioritize the user's well-being and mental health.

        {memory.context}"""
    )
    messages = [system_message] + state["messages"]
    response = await llm.ainvoke(messages)
    messages_to_save = [
        Message(
            role_type="user",
            role=state["first_name"] + " " + state["last_name"],
            content=state["messages"][-1].content,
        ),
        Message(role_type="assistant", content=response.content),
    ]
    await zep.memory.add(
        session_id=state["session_id"],
        messages=messages_to_save,
    )
    state["messages"] = trim_messages(
        state["messages"],
        strategy="last",
        token_counter=len,
        max_tokens=3,
        start_on="human",
        end_on=("human", "tool"),
        include_system=True,
    )
    print("trimmed number of messages:", len(state['messages']))  # <----------------
    return {"messages": [response]}

Example of two successive invocations:

starting number of messages: 11
trimmed number of messages: 3
Hello! It seems like you're reaching out again. I'm here to support you. If there's anything you'd like to discuss or if you just want to chat, let me know!

then

starting number of messages: 13
trimmed number of messages: 3
I'm really sorry to hear that you've been feeling stressed. Work can be a significant source of stress, and it sounds like your dog might be adding to that. If you feel comfortable sharing, what specifically has been stressful for you at work? And how is your dog doing? I'm here to listen and help however I can.

It appears that the trim operation does not, at least as is, persist. And LangChain's own documentation on trim_messages suggests that it is to be used as a filter before the messages are sent to the LLM, in contrast to the above example where it is applied after the LLM call.

[glicerico]

I noticed the same problem. I traced with LangSmith and the LLM is actually getting the whole list of messages, not the trimmed version.
Did you find a solution?

[glicerico]

Oh, I found this RemoveMessage function which helped, it's shown in this tutorial

[scotchka]

I just pass the output of trim_messages to the LLM instead of the original messages.

[glicerico]

Great, that worked well @scotchka . I created a PR to fix it #258