CombinedMemory for ConversationEntityMemory and ConversationBufferWindowMemory

[carbbin]

Checked other resources

• I added a very descriptive title to this question.
• I searched the LangChain documentation with the integrated search.
• I used the GitHub search to find a similar question and didn't find it.

Commit to Help

• I commit to help with one of those options 👆

Example Code

from langchain.chains import ConversationChain
from langchain.memory import (
    CombinedMemory,
    ConversationBufferMemory,
    ConversationSummaryMemory,
)
from langchain.prompts import PromptTemplate
from langchain_google_vertexai import VertexAI
from utils.utils_markdown_files import read_markdown_file
from langchain.memory import ConversationBufferWindowMemory
from langchain.memory.entity import ConversationEntityMemory


def model_gemini_pro():

    llm = VertexAI(model_name="gemini-pro", max_output_tokens=8192, temperature=0.0, top_p=1.0, top_k=40, verbose=True,)
    return llm

llm = model_gemini_pro()

conv_entity = ConversationEntityMemory(llm=llm, input_key="input")
conv_window = ConversationBufferWindowMemory(llm=llm, input_key="input")


# Combined
memory = CombinedMemory(memories=[conv_entity, conv_window])
_DEFAULT_TEMPLATE = """The following is a friendly conversation between a human and an AI. The AI is talkative and provides lots of specific details from its context. If the AI does not know the answer to a question, it truthfully says it does not know.


Human: {input}
AI:"""
PROMPT = PromptTemplate(
    input_variables=["input"],
    template=_DEFAULT_TEMPLATE,
)

conversation = ConversationChain(llm=llm, verbose=True, memory=memory, prompt=PROMPT)
conversation.predict(input = 'Please, tell me about large language models')

Description

Hi!

I want to use CombinedMemory with ConversationEntityMemory and ConversationBufferWindowMemory.

I have started using the example of the link bellow:
https://python.langchain.com/docs/modules/memory/multiple_memory

I am using the model Gemini-Pro for this and I have tried a lot of ways, even combining with ConversationBufferMemory and ConversationSummaryMemory, but i do not get through this.

Does anyone tried to combine this two type of memories? The ones in the example of the link works ok.

System Info

The same variables {'history'} are found in multiplememory object, which is not allowed by CombinedMemory. (type=value_error)

[dosubot[bot]]

Hey @carbbin! Great to see you diving into another LangChain adventure. I remember working through our last challenge together. How's everything going?

Based on the error message you're seeing, it seems that the ConversationEntityMemory and ConversationBufferWindowMemory classes are both managing a variable named 'history'. This is causing a conflict when you try to combine these two memory types using CombinedMemory, which doesn't allow overlapping variables.

To resolve this issue, you need to ensure that the variables managed by ConversationEntityMemory and ConversationBufferWindowMemory do not overlap. Here's how you can do this:

1. Identify Overlapping Variables: Check the memory_variables property of each memory class to identify any common variables. In this case, it seems like the 'history' variable is common to both.

2. Resolve Overlaps: Once you've identified the overlapping variables, you have a few options to resolve the conflicts:

   • Rename Variables: If possible, rename the overlapping variables in either ConversationEntityMemory or ConversationBufferWindowMemory so that each memory object manages a unique set of variables.
   • Partition Variables: Alternatively, you could partition the responsibilities of each memory object so that they manage completely separate sets of variables. This might involve rethinking how you're using each memory type in your application.

3. Implement Changes: After deciding on a strategy to resolve variable overlaps, implement the necessary changes in your memory classes. This might involve modifying the classes themselves or how you instantiate them in your application.

4. Instantiate CombinedMemory: With the overlaps resolved, you can now safely instantiate CombinedMemory with both ConversationEntityMemory and ConversationBufferWindowMemory without encountering the error.

Here's an example of how you might modify your code to resolve the overlap:

from langchain.memory import CombinedMemory, ConversationEntityMemory, ConversationBufferWindowMemory

# Instantiate ConversationEntityMemory and ConversationBufferWindowMemory with unique variable names
entity_memory = ConversationEntityMemory(llm=llm, input_key="input_entity")
buffer_window_memory = ConversationBufferWindowMemory(llm=llm, input_key="input_window")

# Combine the memories
combined_memory = CombinedMemory(memories=[entity_memory, buffer_window_memory])

In this example, I've renamed the input_key variable in each memory class to ensure they're unique. You'll need to adjust this based on your specific use case and the actual variables managed by each memory class.

I hope this helps! If you have any further questions or run into any issues, please don't hesitate to ask.

Sources

• libs/langchain/langchain/memory/combined.py
• libs/langchain/langchain/memory/entity.py
• libs/langchain/langchain/memory/buffer_window.py

About Dosu

This response is meant to be useful and save you time. It is not meant to be a precise solution, but rather a starting point for your own research.

Help me be more useful! Please leave a 👍 if this is helpful and 👎 if it is irrelevant.

If you want to continue the conversation, start your reply with @dosu-bot.

[maximeperrindev]

Hi @carbbin ,

I think its because you're not defining different variable names for your template.

conv_window = ConversationBufferMemory(
    memory_key="chat_history_lines", input_key="input"
)
conv_entity = ConversationEntityMemory(llm=llm, input_key="input")
# Combined
memory = CombinedMemory(memories=[conv_entity, conv_window])

_DEFAULT_TEMPLATE = """The following is a friendly conversation between a human and an AI. The AI is talkative and provides lots of specific details from its context. If the AI does not know the answer to a question, it truthfully says it does not know.
Summary of conversation:
{history}
Current conversation:
{chat_history_lines}
Human: {input}
AI:"""

PROMPT = PromptTemplate(
    input_variables=["history", "input", "chat_history_lines"],
    template=_DEFAULT_TEMPLATE,
)

conversation = ConversationChain(llm=llm, verbose=True, memory=memory, prompt=PROMPT)
conversation.predict(input = 'Please, tell me about large language models')

Tell me what you think about this.

[carbbin]

Hi @maximeperrindev
Ty for your quick response.
The first thing is that i am trying 'ConversationBufferWindowMemory' instead as 'ConversationBufferMemory' but it is also a good try to change only one memory library.

I have tried to run your code with the 'ConversationBufferMemory' but it stills gives me error:
Got unexpected prompt input variables. The prompt expects ['chat_history_lines', 'history', 'input'], but got ['entities', 'history', 'chat_history_lines'] as inputs from memory, and input as the normal input key. (type=value_error)

I think it is something related to the 'ConversationEntityMemory' arguments. I have tried to put 'entities' instead of 'history' but it is no good.

[maximeperrindev]

@carbbin My bad didn't read good enough ^^
You need to add the entities input variable in your template. I think that will do the work.

_DEFAULT_TEMPLATE = """The following is a friendly conversation between a human and an AI. The AI is talkative and provides lots of specific details from its context. If the AI does not know the answer to a question, it truthfully says it does not know.
Summary of conversation:
{history}
Entities context:
{entities}
Current conversation:
{chat_history_lines}
Human: {input}
AI:"""

[carbbin]

@maximeperrindev
Yes it was the template.
Ty very much for your response. It works!!

[rohit2609]

`def build_chain(_documents):
    embeddings = OpenAIEmbeddings(openai_api_key=OPENAI_API_KEY)
    vectorstore = FAISS.from_documents(_documents, embeddings)
    retriever = vectorstore.as_retriever(search_kwargs={"k": 5})`

`prompt_template = """
You are an expert SQL query generator for Google BigQuery.

Your task is to generate **only syntactically valid BigQuery SQL** based on the schema provided. Use **only** the columns listed below in the schema context.

{context}

Conversation history:
{chat_history_lines}

Summary of Conversation:
{history}

Extracted entities:
{entities}

Now answer this business question:
Question:
{question}
"""`

prompt = PromptTemplate(template=prompt_template, input_variables=["context", "history", "chat_history_lines", "entities", "question"])
    llm = ChatOpenAI(model="gpt-4", openai_api_key=OPENAI_API_KEY)
    conv_window = ConversationBufferMemory(memory_key="chat_history_lines", input_key="question")
    conv_entity = ConversationEntityMemory(llm=llm, input_key="question")
    memory = CombinedMemory(memories=[conv_entity, conv_window])

    # Retriever setup
    retriever = vectorstore.as_retriever(search_kwargs={"k": 5})
    retrieval_chain = RunnablePassthrough.assign(context=retriever)

    # Chain setup
    chain = ConversationalRetrievalChain.from_llm(
        llm=llm,
        retriever=retriever,
        memory=memory,
        combine_docs_chain_kwargs={"prompt": prompt}
    )
    return chain

Can anyone help to understand what mistake i am making here?
I am getting errors such as missing chat_history, key_error : question etc.

TIA