ConversationChain with customised prompt

[ragesh2000]

I need to create a ConversationChain with the prompt template that contains two input variables input, context also with ConversationBufferMemory as memory. How to do that?

[ccurme]

If you're not tied to ConversationChain specifically, you can add memory to a chat model following the documentation here. The example showcased there includes two input variables. If you wanted to use ConversationBufferMemory or similar memory object, you could tweak the get_session_history function:

from langchain.memory import ConversationBufferMemory

def get_session_history(session_id: str) -> BaseChatMessageHistory:
    if session_id not in store:
        store[session_id] = ConversationBufferMemory()
    return store[session_id].chat_memory

Since you mentioned that one of the inputs is named "context", if you are interested in implementations of question answering or RAG with memory you can follow the documentation here. Note that in this example the user manages the memory by extending the chat_history list; if you want an object to do that for you you can append this code to the example:

from langchain_community.chat_message_histories import ChatMessageHistory
from langchain_core.chat_history import BaseChatMessageHistory
from langchain_core.runnables.history import RunnableWithMessageHistory

store = {}

def get_session_history(session_id: str) -> BaseChatMessageHistory:
    if session_id not in store:
        store[session_id] = ChatMessageHistory()
    return store[session_id]

with_message_history = RunnableWithMessageHistory(
    rag_chain,
    get_session_history,
    input_messages_key="question",
    history_messages_key="chat_history",
)

question = "What is Task Decomposition?"
print(question)
response = with_message_history.invoke(
    {"question": question, "chat_history": chat_history},
    config={"configurable": {"session_id": "abc123"}},
)
print(response.content + "\n\n")

second_question = "What are common ways of doing it?"
print(second_question)
response = with_message_history.invoke(
    {"question": second_question, "chat_history": chat_history},
    config={"configurable": {"session_id": "abc123"}},
)
print(response.content)

What is Task Decomposition?
Task decomposition is a technique used to break down complex tasks into smaller and simpler steps. This process helps agents or models handle intricate tasks by dividing them into more manageable subtasks. Different methods like Chain of Thought and Tree of Thoughts are employed to guide the decomposition process effectively.


What are common ways of doing it?
Task decomposition can be achieved through various methods such as using Language Model (LLM) with simple prompting, providing task-specific instructions, or incorporating human inputs. These approaches help in breaking down complex tasks into smaller components that are easier to manage and execute. Each method offers a unique way to guide the decomposition process based on the nature of the task at hand.

[ragesh2000]

Actually what i need is a rag system with following workflow.
Whenever a user enters a query, system should extract the context from the vector db (faiss). Using this context and input query in the prompt template i have to create a conversation chain with memory in it. @ccurme

[ccurme]

You might find the walkthroughs here or here to be useful. Let me know if I misunderstood what you're trying to do.

Here is something relatively simple:

from operator import itemgetter

from langchain_community.chat_message_histories import ChatMessageHistory
from langchain_community.vectorstores import FAISS
from langchain_core.chat_history import BaseChatMessageHistory
from langchain_core.output_parsers import StrOutputParser
from langchain_core.prompts import ChatPromptTemplate, MessagesPlaceholder
from langchain_core.runnables import RunnablePassthrough
from langchain_core.runnables.history import RunnableWithMessageHistory
from langchain_openai import ChatOpenAI, OpenAIEmbeddings


llm = ChatOpenAI(model_name="gpt-3.5-turbo", temperature=0)

doc_contents = ["Garg likes cats.", "Greg likes dogs."]
vectorstore = FAISS.from_texts(doc_contents, embedding=OpenAIEmbeddings())
retriever = vectorstore.as_retriever()

qa_system_prompt = """Answer the question based only on the following context:

{context}"""
qa_prompt = ChatPromptTemplate.from_messages(
    [
        ("system", qa_system_prompt),
        MessagesPlaceholder(variable_name="chat_history"),
        ("human", "{question}"),
    ]
)

rag_chain = (
    RunnablePassthrough.assign(context=itemgetter("question") | retriever)
    | qa_prompt
    | llm
    | StrOutputParser()
)

store = {}

def get_session_history(session_id: str) -> BaseChatMessageHistory:
    if session_id not in store:
        store[session_id] = ChatMessageHistory()
    return store[session_id]

chain = RunnableWithMessageHistory(
    rag_chain,
    get_session_history,
    input_messages_key="question",
    history_messages_key="chat_history",
)

chain.invoke(
    {"question": "What is garg's favorite animal?"},
    config={"configurable": {"session_id": "abc123"}},
)

"Based on the provided context, Garg's favorite animal is a cat."

chain.invoke(
    {"question": "What did I just ask"},
    config={"configurable": {"session_id": "abc123"}},
)

"You just asked about Garg's favorite animal."

The problem with this is that follow up questions (e.g., "what did I just ask") are fed straight into the retriever and might produce context that confounds the LLM. A solution is to first rephrase the question as a standalone question, and the docs linked above elaborate on that solution.

If you do not want to run the retrieval step on each query, but want to give the LLM discretion on when to run a search, you can also look into using agents in a retrieval context as demonstrated here.