Langchain MultiQuery Retriever is not returning any generated queries

[calicxy]

Checked other resources

• I added a very descriptive title to this question.
• I searched the LangChain documentation with the integrated search.
• I used the GitHub search to find a similar question and didn't find it.

Commit to Help

• I commit to help with one of those options 👆

Example Code

This is the RAG pipeline code:

documents = load_documents(document_directory)
embed_model = EmbeddingClient("local").get_embed()
db = Qdrant.from_documents(documents, embed_model, location='http://qdrant:6333', collection_name='my_documents', force_recreate=True)
llm = setup_chat_model(chat_model_dir)
retriever = RetrieverClient(embed_model, llm).get_retriever()
rag_chain = RetrievalChain(llm, retriever).get_chain()

And this is the RetrieverClient code where the MultiQuery retriever is implemented:

class RetrieverClient():
    '''
    Instantiates Retriever Service
    '''
    def __init__(self, embed_model, llm):
        client = QdrantClient(url="http://qdrant:6333")
        collection_name = "my_documents"
        qdrant = Qdrant(client, collection_name, embed_model)
        self.retriever = MultiQueryRetriever.from_llm(retriever=qdrant.as_retriever(), llm=llm)
    
    def get_retriever(self):
        return self.retriever

Description

I'm trying to implement a RAG pipeline via the code above, and usually the MultiQuery retriever returns something like
INFO:langchain.retrievers.multi_query:Generated queries: ['What is 1-Benzylpiperazine commonly known as?', 'Can you provide the common terminology or name for 1-Benzylpiperazine?', 'What is the everyday or familiar title for the chemical 1-Benzylpiperazine?']
However, it returns INFO:langchain.retrievers.multi_query:Generated queries: [''] instead, causing the pipeline to fail.

Is anyone able to provide me with some guidance as to why this issue is happening?
I also used langsmith to trace, and below are screenshots of the trace:

Thanks!

System Info

torch==2.3.1 -f https://download.pytorch.org/whl/torch_stable.html
accelerate==0.33.0
bitsandbytes==0.43.3
datasets==2.21.0
langchain==0.2.14
langchain-community==0.2.12
langchain-core==0.2.32
langchain-openai==0.1.21
langchain-qdrant==0.1.1
langchain-huggingface==0.0.3
langsmith==0.1.99
openai==1.40.6
ninja==1.11.1.1
pandas==2.2.2
PyMuPDF==1.24.2
python-dotenv==1.0.1
qdrant-client==1.11.0
ragas==0.1.9
safetensors==0.4.4
sentence-transformers==2.7.0
tiktoken==0.7.0
unstructured[doc,docx,ppt,pptx,csv,xlsx,msg]==0.15.3
flash-attn==2.5.9.post1

platform: linux, python version: 3.10.12

[dosubot[bot]]

Hey there, @calicxy! I'm here to help you with any bugs, questions, or contributions you have. I'll assist you while we wait for a human maintainer.

The issue with the LangChain MultiQuery Retriever returning empty generated queries in your RAG pipeline implementation could be due to several factors. Here are some steps to help you debug and resolve the issue:

1. Check the Input to the llm_chain:
   Ensure that the input to the llm_chain is correctly formatted and not empty. The question variable passed to the generate_queries method should be a non-empty string.

2. Verify the Output of the llm_chain:
   The llm_chain should generate a valid response. You can add logging to check the response from the llm_chain before it is parsed by the LineListOutputParser.

3. Inspect the LineListOutputParser:
   The LineListOutputParser is responsible for parsing the response from the llm_chain. Ensure that the response contains valid text that can be split into lines. Here is the relevant code for the parser:

   class LineListOutputParser(BaseOutputParser[List[str]]):
       """Output parser for a list of lines."""
   
       def parse(self, text: str) -> List[str]:
           lines = text.strip().split("\n")
           return list(filter(None, lines))  # Remove empty lines

4. Check the Configuration of the llm_chain:
   Ensure that the llm_chain is correctly configured with the appropriate prompt and language model. The from_llm method should be used to initialize the MultiQueryRetriever with the correct parameters.

   class MultiQueryRetriever(BaseRetriever):
       # ...
   
       @classmethod
       def from_llm(
           cls,
           retriever: BaseRetriever,
           llm: BaseLanguageModel,
           prompt: BasePromptTemplate = DEFAULT_QUERY_PROMPT,
           include_original: bool = False,
           callbacks: Callbacks = None,
           llm_chain_kwargs: Optional[dict] = None,
       ) -> "MultiQueryRetriever":
           """Initialize from llm using default template."""
           output_parser = LineListOutputParser()
           llm_chain = prompt | llm | output_parser
           return cls(
               retriever=retriever,
               llm_chain=llm_chain,
               include_original=include_original,
           )

5. Ensure Proper Environment Setup:
   Make sure that your environment is correctly set up, including the necessary API keys and configurations for the language models and retrievers.

6. Debugging with LangSmith:
   Use LangSmith for tracing and debugging. The traces you provided show that the LineListOutputParser is receiving an empty input ("") and returning an empty list ([]). This indicates that the issue might be with the llm_chain not generating any text.

By following these steps, you should be able to identify and resolve the issue with the MultiQuery Retriever returning empty generated queries. If the problem persists, consider providing more detailed logs or additional context to further diagnose the issue.

For more information, you can refer to the relevant parts of the code in the LangChain repository:

• MultiQueryRetriever class ^[1]
• LineListOutputParser class ^[2]

To continue talking to Dosu, mention @dosu.