what is the score in DocumentSearchPipline result?

[amy-why]

I have a simple document search pipeline with llm as embedding retriever.

from haystack.pipelines import DocumentSearchPipeline
retriever = EmbeddingRetriever(
    document_store=document_store,
    embedding_model="sentence-transformers/multi-qa-mpnet-base-dot-v1",
    model_format="sentence_transformers",
)
pipeline = DocumentSearchPipeline(retriever=retriever)
result = pipeline.run(query="iot", params={"Retriever": {"top_k": 1}})

result looks like this:

{'documents': [<Document: {'content': 'IoT', 'content_type': 'text', 'score': 0.5932160503183657, 'meta': {'name': 'content_132.txt', 'vector_id': '43'}, 'embedding': None, 'id': '98dbeb38e7360e062df713997b819383'}>],
 'root_node': 'Query',
 'params': {'Retriever': {'top_k': 1}},
 'query': 'iot',
 'node_id': 'Retriever'}

The scores for all my tests are very narrow range, I am trying to figure out what the score here, since this is embedding. The document_store items has score as none.

Anyone has any pointer?

[anakin87]

Hello @amy-why!

Your EmbeddingRetriever uses a Sentence Transformers model that converts both the query and the documents into embeddings (=vectors).
These vectors can be compared using similarity functions, such as cosine similarity and dot product.
(The similarity function is chosen when you initialize the Document Store, using the similarity parameter)

So, the score that the retriever assigns to each document measures how much the document is semantically similar (≈relevant) to the query.

You can read more about this topic in the Sentence Transformers docs.

Feel free to ask for clarification.