Return both query and doc similarity scores when using Maximum Marginal Relevance (MMR) search

[Louva1]

Hi, for some analysis on retrieval I want to extract the similarity scores from a Maximal Marginal Relevance search. I want to extract both the similarity score between the query and the document, as well as the similarity between the documents. When looking at the documentation I find the following two functions:

• db.max_marginal_relevance_search(query, k=5)
• db.max_marginal_relevance_search_with_score_by_vector(query_embed, k=5)

The second function only seems to return to similarity score between the query and the document. Are there ways how I can extract the similarity between the documents as well?
Thanks!

[hwchase17]

i dont think this is currently supported. moving to langchain repo as well since more related to that

[dosubot[bot]]

Hey @Louva1, I'm here to help you with bug fixes, questions, and more! Let's squash those bugs together. 🤖

To extract both the similarity score between the query and the document, as well as the similarity between the documents, using Maximal Marginal Relevance (MMR) search, you can modify the max_marginal_relevance_search_by_vector method to return the similarity scores along with the documents. Here is an example of how you can achieve this:

def max_marginal_relevance_search_by_vector(
    self,
    embedding: List[float],
    k: int = 4,
    fetch_k: int = 20,
    lambda_mult: float = 0.5,
    **kwargs: Any,
) -> List[Tuple[Document, float]]:
    # Retrieves the docs with similarity scores
    docs = self.similarity_search_with_score(
        query_embedding=embedding, k=fetch_k, **kwargs
    )

    # Re-ranks the docs using MMR
    mmr_doc_indexes = maximal_marginal_relevance(
        np.array(embedding),
        [doc.metadata[self._embedding_key] for doc, _ in docs],
        k=k,
        lambda_mult=lambda_mult,
    )

    mmr_docs_and_scores = [(docs[i][0], docs[i][1]) for i in mmr_doc_indexes]
    return mmr_docs_and_scores

def max_marginal_relevance_search(
    self,
    query: str,
    k: int = 4,
    fetch_k: int = 20,
    lambda_mult: float = 0.5,
    **kwargs: Any,
) -> List[Tuple[Document, float]]:
    # compute the embeddings vector from the query string
    embedding = self._embedding_function.embed_query(query)

    docs_and_scores = self.max_marginal_relevance_search_by_vector(
        embedding,
        k=k,
        fetch_k=fetch_k,
        lambda_mult=lambda_mult,
        **kwargs
    )
    return docs_and_scores

This modification ensures that the max_marginal_relevance_search_by_vector and max_marginal_relevance_search methods return a list of tuples, where each tuple contains a document and its corresponding similarity score ^{[1][2][3][4][5]}.

To continue talking to Dosu, mention @dosu.

[Spoertm]

If I understood you correctly, then we needed the same functionality. Thankfully I found an easy way to do this with an already existing function.

In langchain_chroma/vectorstores.py, change the call to _results_to_docs in max_marginal_relevance_search_by_vector to _results_to_docs_and_scores, then change the return types in the two following functions to List[Tuple[Document, float]].

Here's what the changes look like:

def max_marginal_relevance_search_by_vector(
    ...
) -> List[Tuple[Document, float]]: # Changed from List[Document]
    ...

    candidates = _results_to_docs_and_scores(results) # Changed from _results_to_docs(results)

    ...

def max_marginal_relevance_search(
    ...
) -> List[Tuple[Document, float]]: # Changed from List[Document]
    ...