How to use "sentence_transformers/all-mpnet-base-v2" as a retriever model?

[dkbs12]

Hello,
I'd like to use the "sentence_transformers/all-mpnet-base-v2" as a embedding retriever model.
So I coded as below, but faced the error that "LocalTokenNotFoundError: Token is required (token=True), but no token found. You need to provide a token or be logged in to Hugging Face with huggingface-cli login or huggingface_hub.login "
As you can see, I input the api_key which was the access token issued from Hugging Face.

< Coding >
from haystack.nodes import EmbeddingRetriever

retriever = EmbeddingRetriever(
document_store=document_store,
embedding_model="sentence_transformers/all-mpnet-base-v2",
use_auth_token=True,
api_key=api_key
)

document_store.update_embeddings(retriever)

< Error message >
LocalTokenNotFoundError: Token is required (token=True), but no token found. You need to provide a token or be logged in to Hugging Face with huggingface-cli login or huggingface_hub.login. See https://huggingface.co/settings/tokens.

Please help me using the model "all-mpnet-base-v2" from Hugging Face.

[vblagoje]

Hey @dkbs12, I'm not 100% sure but I believe you don't need a token for that model. Just use all-mpnet-base-v2 as the embedding_model. Let me know if that worked.

[dkbs12]

Hi, vblagoje.
I tried it again as you advised, but it failed as below (just input "all-mpnet-base-v2" without api_key) ;

< Coding >
from haystack.nodes import EmbeddingRetriever

retriever = EmbeddingRetriever(
document_store=document_store,
embedding_model="all-mpnet-base-v2"
)

document_store.update_embeddings(retriever)

< Error Message >
OSError: all-mpnet-base-v2 is not a local folder and is not a valid model identifier listed on 'https://huggingface.co/models'
If this is a private repository, make sure to pass a token having permission to this repo with use_auth_token or log in with huggingface-cli login and pass use_auth_token=True.

[vblagoje]

Aha, ok, what I meant to say is that there is no need to provide use_auth_token=True, and api_key=api_key parameters. These are publicly available free models. Would you try that please?

[anakin87]

Hey @dkbs12, the following code works for me.
Please make sure to use the correct model name.

from haystack import Document
from haystack.nodes import EmbeddingRetriever
from haystack.document_stores import InMemoryDocumentStore

document_store = InMemoryDocumentStore()

doc = Document('example doc')

document_store.write_documents([doc])

retriever = EmbeddingRetriever(
document_store=document_store,
embedding_model="sentence-transformers/all-mpnet-base-v2"
)

document_store.update_embeddings(retriever)

[dkbs12]

Thank you, anakin87.
I overlooked my mistake of the name of model.
It worked well after amending the model name.
I appreciate your help.