using bm25 or e5_large on disconnected environments.

[jxadro]

Hi.

I will use pymilvus for an application.
I need to use bm25 and e5_large models to generate the vectors.

If I'm not wrong pymilvus downloads automatically the models for bm25 and e5_large on runtime.

My issue is that my environment will be disconnected from internet so it will not be possible to download it on runtime.

I couldn't find how to download them before using them. My plan is to build a container with everything needed where I will execute my python application using pymilvus.

How can I download all requirements to use bm25 and e5_large before using them?

Thank you.

[yhmo]

BM25 is built-in in Milvus v2.5: https://milvus.io/docs/full-text-search.md#Full-Text-Search-BM25, you don't need bm25 in client-side.
In 2.4, the pymilvus provided BM25EmbeddingFunction in client-side: https://milvus.io/docs/v2.4.x/embed-with-bm25.md
It is recommended to use the built-in BM25 in Milvus v2.5.

[jxadro]

Great, thank you.

Do you have any information about using e5_large on disconnected environments?
I guess it could be the same question as using BM25 on 2.4 on disconnected environments.

I don't know if may be is this line:

pip install "pymilvus[model]"

That I have to replace "model" with e5_large or the corresponding for bpm25 on 2.4.

Thank you

[yhmo]

I didn't see e5_large used in the source code of pymilvus.model: https://github.com/milvus-io/milvus-model

The pymilvus.model automatically installs libraries like this:
https://github.com/milvus-io/milvus-model/blob/2fe97ac8ba923fec5d7e3afa8b2b9e9c25ebbce0/src/pymilvus/model/dense/sentence_transformer.py#L20
And the third-party lib automatically downloads the models.
But I didn't see e5_large used in pymilvus.model. I think you can use it in huggingface way: https://huggingface.co/intfloat/e5-large

[jxadro]

I got the e5 idea from this page:

https://zilliz.com/ai-models/e5-large-v2

from pymilvus.model.dense import SentenceTransformerEmbeddingFunction
from pymilvus import MilvusClient

ef = SentenceTransformerEmbeddingFunction("intfloat/e5-large-v2")
docs = [
   "Artificial intelligence was founded as an academic discipline in 1956.",
   "Alan Turing was the first person to conduct substantial research in AI.",
   "Born in Maida Vale, London, Turing was raised in southern England."
]
# Generate embeddings for documents
docs_embeddings = ef(docs)

queries = ["When was artificial intelligence founded",
          "Where was Alan Turing born?"]
# Generate embeddings for queries
query_embeddings = ef(queries)

[yhmo]

The SentenceTransformerEmbeddingFunction calls sentence_transformers.SentenceTransformer to get the model.
This line will automatically download the model:
https://github.com/milvus-io/milvus-model/blob/2fe97ac8ba923fec5d7e3afa8b2b9e9c25ebbce0/src/pymilvus/model/dense/sentence_transformer.py#L29
It just equal to this:

from sentence_transformers import SentenceTransformer
model = SentenceTransformer("intfloat/e5-large-v2")

[jxadro]

Thank you!

[xiaofan-luan]

For BM25, you don't really to download pretrained model.
please check https://milvus.io/docs/zh/full-text-search.md

[xiaofan-luan]

For Milvus 2.6, we support integrating with other inference engines, like huggingface or VLLM

So if you know how to deploy VLLM offline and install embedding models it could be much easier and more performant