how we add chat history in my retrievalQA chain

[shraddhaa26]

Checked other resources

• I added a very descriptive title to this question.
• I searched the LangChain documentation with the integrated search.
• I used the GitHub search to find a similar question and didn't find it.

Commit to Help

• I commit to help with one of those options 👆

Example Code

qa_chain = RetrievalQA.from_chain_type(
    llm=llm,
    chain_type="stuff",
    retriever=db.as_retriever(search_kwargs={"k": 2}),
    return_source_documents=True,
    chain_type_kwargs={"prompt": prompt},
)

Description

i'm trying to add chat_history in retrieval chain but it doesnt addable

System Info

model_name_or_path = "TheBloke/Llama-2-13B-chat-GPTQ"
model_basename = "model"

tokenizer = AutoTokenizer.from_pretrained(model_name_or_path, use_fast=True)

model = AutoGPTQForCausalLM.from_quantized(
model_name_or_path,
revision="gptq-4bit-128g-actorder_True",
model_basename=model_basename,
use_safetensors=True,
trust_remote_code=True,
inject_fused_attention=False,
device=DEVICE,
quantize_config=None,
)
DEFAULT_SYSTEM_PROMPT = """
You are a helpful, respectful and honest assistant. Always answer as helpfully as possible, while being safe. Your answers should not include any harmful, unethical, racist, sexist, toxic, dangerous, or illegal content. Please ensure that your responses are socially unbiased and positive in nature.

If a question does not make any sense, or is not factually coherent, explain why instead of answering something not correct. If you don't know the answer to a question, please don't share false information.
""".strip()

def generate_prompt(prompt: str, system_prompt: str = DEFAULT_SYSTEM_PROMPT) -> str:
return f"""
[INST] <>
{system_prompt}
<>

{prompt} [/INST]
""".strip()

llm = HuggingFacePipeline(pipeline=text_pipeline, model_kwargs={"temperature": 0})
SYSTEM_PROMPT = "Use the following pieces of context to answer the question at the end. If you don't know the answer, just say that you don't know, don't try to make up an answer."

template = generate_prompt(
"""
{context}

Question: {question}
""",
system_prompt=SYSTEM_PROMPT,
)

[dosubot[bot]]

🤖

Hey @shraddhaa26, great to see you back with another interesting question! Hope you've been doing well.

Based on the information you've provided, it seems like you're trying to add chat history to a RetrievalQA chain. To do this, you can use the ConversationalRetrievalChain which allows for passing in a chat history. This chain can be used to allow for follow-up questions.

However, if you want to modify the _generate method in the ChatLiteLLM class, you can do so by passing the chat history as a list of BaseMessage instances to this method. Here's how you can modify this method to include the chat history:

from langchain_core.messages import BaseMessage, HumanMessage, AIMessage, SystemMessage

# Create BaseMessage instances for each message in the chat history
chat_history = [
    HumanMessage(content="Hello"),
    AIMessage(content="Hello, how can I assist you today?"),
    SystemMessage(content="Please provide more information."),
    # Add more messages as needed
]

# Pass the chat history to the _generate method
output = llm._generate(chat_history)

In this code, you need to create BaseMessage instances for each message in the chat history and pass them to the _generate method. You can use the HumanMessage, AIMessage, and SystemMessage classes to create these instances.

For more details, you can refer to the files in the LangChain repository:

• libs/langchain/langchain/memory/chat_message_histories/zep.py
• libs/langchain/langchain/chat_models/ollama.py
• libs/langchain/langchain/chat_models/litellm.py

Please note that this is a simplified example and the actual implementation would depend on the specifics of your application and how you want to process the chat history.

I hope this helps! If you have any further questions, feel free to ask.

Sources

• how to add chat history to runnablesequence instance
• Issue: <Please write a comprehensive title after the 'Issue: ' prefix>
• getting error while adding memory to LCEL chain
• libs/core/tests/unit_tests/chat_history/test_chat_history.py
• libs/langchain/tests/integration_tests/chains/test_retrieval_qa.py

About Dosu

This response is meant to be useful and save you time. It is not meant to be a precise solution, but rather a starting point for your own research.

Help me be more useful! Please leave a 👍 if this is helpful and 👎 if it is irrelevant.

If you want to continue the conversation, start your reply with @dosu-bot.

[shraddhaa26]

@dosu-bot but i want add chat history in my retrivalQA by custom chat_history

[dosubot[bot]]

🤖

Based on the context provided, it seems you want to modify the _generate method in the ChatLiteLLM class to include a custom chat history. However, the ChatLiteLLM class is not directly responsible for handling the chat history. This is typically handled by a class that implements the RunnableWithMessageHistory interface.

To include a custom chat history, you would need to modify the _enter_history method in the RunnableWithMessageHistory class. This method is responsible for preparing the input for the wrapped Runnable, which includes the chat history.

Here's a general idea of how you might modify the _enter_history method to accept a list of BaseMessage instances as chat history:

def _enter_history(self, input: Any, config: RunnableConfig) -> List[BaseMessage]:
    # Instead of getting the history from the config, we'll use the input
    # Assume that the input is a dictionary with a 'chat_history' key
    chat_history = input.get('chat_history', [])

    # Check if the chat history is a list of BaseMessage instances
    if not all(isinstance(msg, BaseMessage) for msg in chat_history):
        raise ValueError("Chat history must be a list of BaseMessage instances")

    # If the history_messages_key is set, return only the historic messages
    if self.history_messages_key:
        return chat_history
    # Otherwise, return all messages
    else:
        input_val = (
            input if not self.input_messages_key else input[self.input_messages_key]
        )
        return chat_history + self._get_input_messages(input_val)

This modification assumes that the input to the RunnableWithMessageHistory instance is a dictionary with a 'chat_history' key, which contains the chat history as a list of BaseMessage instances. The _enter_history method then checks if the chat history is a list of BaseMessage instances, and raises a ValueError if it's not. If the history_messages_key is set, it returns only the historic messages; otherwise, it returns all messages.

Please note that this is a general idea and might need to be adjusted based on the specifics of the ChatLiteLLM class and its _generate method, which are not included in the provided context. You can find more details about the RunnableWithMessageHistory class in the LangChain repository.

Regarding the ChatLiteLLM class, it is a chat model that uses the LiteLLM API and inherits from the BaseChatModel class. It includes methods for generating chat completions, both synchronously and asynchronously, with options for streaming the completions. It also includes methods for creating message dictionaries from a list of messages, and for creating a chat result from a response. More details about this class can be found in the LangChain repository.

I hope this helps! If you have any other questions or need further clarification, feel free to ask.

Sources

• libs/core/langchain_core/runnables/history.py
• libs/community/langchain_community/chat_models/litellm.py

About Dosu

This response is meant to be useful and save you time. It is not meant to be a precise solution, but rather a starting point for your own research.

Help me be more useful! Please leave a 👍 if this is helpful and 👎 if it is irrelevant.

If you want to continue the conversation, start your reply with @dosu-bot.

[ngothankphuong]

I'm the same as you, do you have any ideas for that

[deepak-hl]

Use
ConversationalRetrievalChain.from_llm(
llm=llm,
retriever=retriever,
return_source_documents=True,
memory=memory,
combine_docs_chain_kwargs={"prompt": prompt},
rephrase_question=False,

[ngothankphuong]

thank bro, im trying and that work