query_similarity_score of ContextualCompressionRetriever

[choiwb]

Checked other resources

• I added a very descriptive title to this question.
• I searched the LangChain documentation with the integrated search.
• I used the GitHub search to find a similar question and didn't find it.

Commit to Help

• I commit to help with one of those options 👆

Example Code

def src_doc(prompt):
    prompt_str = str(prompt)
    source_documents = prompt_str.split("context for answer: ")[1].split("question: ")[0]
    extraction_question = prompt_str.split("question: ")[1]
    
    if len(source_documents.strip()) > 0 and source_documents.strip() != '\\n':
        source_documents_list = source_documents.split('\\n\\n')
        sample_src_doc = [[i+1, doc[:100] + '.....(이하 생략)'] for i, doc in enumerate(source_documents_list)] 
        sample_src_doc_df = pd.DataFrame(sample_src_doc,  columns=['No', '참조 문서'])
        sample_src_doc_df = sample_src_doc_df.set_index('No')

        if st.session_state.selected_db == 'user_vectordb':
            load_db = Chroma(persist_directory=os.path.join(db_save_path, user_db_name),
                                embedding_function=embeddings) 
        else:
            load_db = Chroma(persist_directory=os.path.join(db_save_path, db_name),
                                embedding_function=embeddings) 
        
        # src_doc_score_cal = load_db.similarity_search_with_score(extraction_question)
        # src_doc_score_cal = load_db.similarity_search_with_relevance_scores(prompt_str, k=db_doc_k,
        #                                                                     score_threshold=db_similarity_threshold)

        retriever = load_db.as_retriever(
                                                search_type=db_search_type,         
                                                search_kwargs={'k': db_doc_k, 'fetch_k': db_doc_fetch_k}
                                            )
        embeddings_filter = EmbeddingsFilter(embeddings=embeddings, similarity_threshold=db_similarity_threshold) 
        compression_retriever = ContextualCompressionRetriever(
                base_compressor=embeddings_filter, base_retriever=retriever
            )

        extraction_context = compression_retriever.invoke(extraction_question)
        # extraction_context의 query_similarity_score를 각 각 호출해야 함!!!!!!!!!!!!!!!!
        src_doc_score = [doc['query_similarity_score'] for doc in extraction_context]

        # src_doc_score = [doc for doc in extraction_context]
        print(src_doc_score)

        print('@@@@@@@@@@@@@@@@@@@@@@@@@@@@@@@@@@@@@@@@')
        sample_src_doc_df['유사도'] = src_doc_score
        sample_src_doc_df['유사도'] = round(sample_src_doc_df['유사도'], 2)
                
        # 참조 문서 UI 표출
        if sample_src_doc_df.shape[0] > 0:
            with st.expander('참조 문서'):
                st.table(sample_src_doc_df)
    return prompt

Description

above of code,
src_doc_score = [doc['query_similarity_score'] for doc in extraction_context]
this line code error represented.

how to get to a query_similarity_score of ContextualCompressionRetriever??

System Info

.

[dosubot[bot]]

To access the query_similarity_score from the Document objects returned by the ContextualCompressionRetriever, you need to ensure that the similarity scores are included in the document metadata. This requires modifying the method that executes the vector store search to propagate similarity scores into the document metadata.

Here's an example of how to do this:

1. Override the method that retrieves documents to include similarity scores in the metadata.
2. Access the query_similarity_score from the document metadata.

Here's how you can modify your code:

from typing import Any, Dict, List
from langchain_core.documents import Document
from langchain.retrievers.contextual_compression import ContextualCompressionRetriever

class CustomContextualCompressionRetriever(ContextualCompressionRetriever):
    def _get_relevant_documents(
        self, query: str, *, run_manager: CallbackManagerForRetrieverRun, **kwargs: Any
    ) -> List[Document]:
        """Get docs, adding score information."""
        docs_and_similarities = self.base_retriever.similarity_search_with_relevance_scores(query, **kwargs)
        docs = [doc for doc, _ in docs_and_similarities]
        for doc, similarity in docs_and_similarities:
            doc.metadata["query_similarity_score"] = similarity

        return docs

# Example usage
retriever = load_db.as_retriever(
    search_type=db_search_type,         
    search_kwargs={'k': db_doc_k, 'fetch_k': db_doc_fetch_k}
)
embeddings_filter = EmbeddingsFilter(embeddings=embeddings, similarity_threshold=db_similarity_threshold) 
compression_retriever = CustomContextualCompressionRetriever(
    base_compressor=embeddings_filter, base_retriever=retriever
)

extraction_context = compression_retriever.invoke(extraction_question)
src_doc_score = [doc.metadata['query_similarity_score'] for doc in extraction_context]

print(src_doc_score)

This code ensures that the similarity scores are included in the metadata of the documents returned by the ContextualCompressionRetriever. The similarity_search_with_relevance_scores method from the VectorStore class is used to retrieve documents along with their relevance scores, which are then added to the document metadata ^[1][2].

To continue talking to Dosu, mention @dosu.

[wo-yashsinghvi]

@dosu I am getting the error "VectorStoreRetriever object has no attribute similarity_search_with_relevance_scores." I am using the Milvus vector store retriever.