Retreival option in langchian 0.3

[ZainZia0341]

I am creating a simple chatbot to test the mongodbWithChatHistory module with Langchain 0.3. I have used Chroma as the vector database, but I am facing an issue with passing the retrieved documents from Chroma to the chain.

I have tried multiple approaches, but it seems that the knowledge base is not being accessed correctly. Could someone please help me identify where I might be going wrong? Additionally, any relevant resources or links would be greatly appreciated.

Below is my code:

import streamlit as st
from langchain_chroma import Chroma
from langchain_openai import ChatOpenAI, OpenAIEmbeddings
from langchain_core.runnables.history import RunnableWithMessageHistory
from langchain_core.messages import HumanMessage
from langchain_core.prompts import ChatPromptTemplate, MessagesPlaceholder
from langchain_text_splitters import RecursiveCharacterTextSplitter
from langchain_community.document_loaders import WebBaseLoader
from langchain_mongodb.chat_message_histories import MongoDBChatMessageHistory
from dotenv import load_dotenv
import bs4
import os

Load environment variables

load_dotenv()
OPENAI_API_KEY = os.getenv("OPENAI_API_KEY")

Initialize OpenAI LLM

llm = ChatOpenAI(model="gpt-4o-mini", api_key=OPENAI_API_KEY)

Load, chunk and index the contents of the blog.

loader = WebBaseLoader(
web_paths=("https://lilianweng.github.io/posts/2023-06-23-agent/",),
bs_kwargs=dict(
parse_only=bs4.SoupStrainer(
class_=("post-content", "post-title", "post-header")
)
),
)
docs = loader.load()

Initialize Vectorstore (Chroma)

PERSIST_DIR = './chroma_db'
text_splitter = RecursiveCharacterTextSplitter(chunk_size=1000, chunk_overlap=200)
splits = text_splitter.split_documents(docs)
vectorstore = Chroma.from_documents(documents=splits, embedding=OpenAIEmbeddings(), persist_directory=PERSIST_DIR)
retriever = vectorstore.as_retriever()

Prompt Template

SYSTEM_TEMPLATE = """
Answer the user's questions based on the below context.
If the context doesn't contain any relevant information to the question, don't make something up and just say "I don't know":

{context} """

question_answering_prompt = ChatPromptTemplate.from_messages(
[
("system", SYSTEM_TEMPLATE),
MessagesPlaceholder(variable_name="messages")
]
)

from langchain.chains.combine_documents import create_stuff_documents_chain

document_chain = create_stuff_documents_chain(llm, question_answering_prompt)

from langchain_core.messages import HumanMessage

document_chain.invoke(

{

"context": docs,

"messages": [

HumanMessage(content="Can LangSmith help test my LLM applications?")

],

}

)

from typing import Dict

from langchain_core.runnables import RunnablePassthrough

def parse_retriever_input(params: Dict):

return params["messages"][-1].content

retrieval_chain = RunnablePassthrough.assign(

context=parse_retriever_input | retriever,

).assign(

answer=document_chain,

)

MongoDB setup for history (assuming you have MongoDB properly configured)

def get_chat_history(session_id):
return MongoDBChatMessageHistory(
session_id=session_id,
connection_string="",
database_name="chatbot_db",
collection_name="conversations"
)

Streamlit frontend

st.title("Conversational AI with RAG and MongoDB")
st.write("Ask any question and let the AI respond with knowledge from the documents and history.")

Set session ID for user history

session_id = st.text_input("Session ID", "1234")

Get question from user

user_question = st.text_input("Your question:")

if st.button("Submit"):
if user_question:
# Retrieve relevant documents
docs = retriever.invoke(user_question)
chain = question_answering_prompt | llm
# Run the question-answering pipeline
document_chain = RunnableWithMessageHistory(
chain,
get_chat_history,
input_messages_key="messages",
history_messages_key="history"
)

    # response = document_chain.invoke(
    #     {
    #         "context": docs,
    #         "messages": [
    #             HumanMessage(content=user_question)
    #         ]
    #     },
    #     config={"configurable": {"session_id": session_id}}
    # )
    
    response = document_chain.invoke(
        {"messages": "hi im bob!"},
        config={"configurable": {"session_id": "1"}}
    )

    st.write(response)
else:
    st.write("Please enter a question.")

[dosubot[bot]]

Hey @ZainZia0341! 👋 I'm here to help you out with your question.

I found some similar discussions and solutions that might help you with your issue:

1. create_retrieval_chain does not store RedisChatMessageHistory: This discussion suggests manually logging the chat history using add_user_message and add_ai_message in RedisChatMessageHistory every time you invoke the create_retrieval_chain ^[1].

2. Context missing when using Chroma with persist_directory and embedding_function, but not when created from documents: This discussion provides a code snippet to ensure that the context is correctly retrieved and passed to the chain ^[2].

To correctly pass the retrieved documents from Chroma to the chain in Langchain 0.3 when using the mongodbWithChatHistory module, you can follow these steps:

1. Initialize the retriever and create the history-aware retriever chain:

   from langchain_community.chat_models import ChatOpenAI
   from langchain.chains import create_history_aware_retriever
   from langchain import hub
   
   # Pull the rephrase prompt from the LangChain hub
   rephrase_prompt = hub.pull("langchain-ai/chat-langchain-rephrase")
   
   # Initialize the language model
   llm = ChatOpenAI()
   
   # Initialize your retriever (replace with your specific retriever)
   retriever = vectorstore.as_retriever()
   
   # Create the history-aware retriever chain
   chat_retriever_chain = create_history_aware_retriever(
       llm, retriever, rephrase_prompt
   )
   
   # Invoke the chain with input and chat history
   result = chat_retriever_chain.invoke({"input": user_question, "chat_history": get_chat_history(session_id).get_messages()})
   
   # The result will be a list of documents
   print(result)

2. Set up MongoDB and OpenAI API keys:

   import getpass
   
   MONGODB_URI = getpass.getpass("Enter your MongoDB connection string:")
   OPENAI_API_KEY = getpass.getpass("Enter your OpenAI API key:")

3. Initialize MongoDB client and vector store:

   from langchain_mongodb import MongoDBAtlasVectorSearch
   from pymongo import MongoClient
   from langchain_openai import OpenAIEmbeddings
   
   client = MongoClient(MONGODB_URI, appname="devrel.content.python")
   DB_NAME = "langchain_chatbot"
   COLLECTION_NAME = "data"
   ATLAS_VECTOR_SEARCH_INDEX_NAME = "vector_index"
   collection = client[DB_NAME][COLLECTION_NAME]
   
   # Initialize embeddings
   embeddings = OpenAIEmbeddings(
       openai_api_key=OPENAI_API_KEY, model="text-embedding-ada-002"
   )
   
   # Create vector store
   vector_store = MongoDBAtlasVectorSearch.from_connection_string(
       connection_string=MONGODB_URI,
       namespace=DB_NAME + "." + COLLECTION_NAME,
       embedding=embeddings,
       index_name=ATLAS_VECTOR_SEARCH_INDEX_NAME,
       text_key="fullplot",
   )
   
   # Use the MongoDB vector store as a retriever
   retriever = vector_store.as_retriever(search_type="similarity", search_kwargs={"k": 5})

4. Create a RAG chain with chat history:

   from langchain_core.prompts import MessagesPlaceholder
   from langchain_core.runnables.history import RunnableWithMessageHistory
   from langchain_mongodb.chat_message_histories import MongoDBChatMessageHistory
   
   def get_session_history(session_id: str) -> MongoDBChatMessageHistory:
       return MongoDBChatMessageHistory(
           MONGODB_URI, session_id, database_name=DB_NAME, collection_name="history"
       )
   
   # Define the standalone question prompt
   standalone_system_prompt = """
   Given a chat history and a follow-up question, rephrase the follow-up question to be a standalone question. \
   Do NOT answer the question, just reformulate it if needed, otherwise return it as is. \
   Only return the final standalone question. \
   """
   standalone_question_prompt = ChatPromptTemplate.from_messages(
       [
           ("system", standalone_system_prompt),
           MessagesPlaceholder(variable_name="history"),
           ("human", "{question}"),
       ]
   )
   
   question_chain = standalone_question_prompt | model | parse_output
   
   # Generate context by passing output of the question_chain to the retriever
   retriever_chain = RunnablePassthrough.assign(
       context=question_chain
       | retriever
       | (lambda docs: "\n\n".join([d.page_content for d in docs]))
   )
   
   # Create a prompt that includes the context, history, and the follow-up question
   rag_system_prompt = """Answer the question based only on the following context: \
   {context}
   """
   rag_prompt = ChatPromptTemplate.from_messages(
       [
           ("system", rag_system_prompt),
           MessagesPlaceholder(variable_name="history"),
           ("human", "{question}"),
       ]
   )
   
   # RAG chain with history
   rag_chain = retriever_chain | rag_prompt | model | parse_output
   with_message_history = RunnableWithMessageHistory(
       rag_chain,
       get_session_history,
       input_messages_key="question",
       history_messages_key="history",
   )
   
   # Invoke the chain with a question and session ID
   with_message_history.invoke(
       {"question": user_question},
       {"configurable": {"session_id": session_id}},
   )

By following these steps, you should be able to correctly pass the retrieved documents from Chroma to the chain in Langchain 0.3 while using the mongodbWithChatHistory module ^[3][4][5][6].

To continue talking to Dosu, mention @dosu.