Request for a better support for RAG with large datasets

[gland1]

The issue we see is as follows:
Suppose you'd like to build a RAG system with milvus using 1B dataset.
The source content for each vector needs to be retrieved during/after performing an ANN search.
If you keep the source content along with the vector in one collection, for example like langchain does , you'll end up fetching huge amount of date from object storage to querynode local drive.
The current workaround we use is to keep it separately on the object storage and fetch it directly after retrieving the required vector id by ann search. If it was possible for milvus to avoid loading such field during load(I understand this is available in beta) and then fetch the source content by a simple get api by milvus, or any other solution by milvus that would better support the RAG case with big datasets.

[yhmo]

"fetch the source content by a simple get api by milvus" --------------------- I believe the query() interface already works for this purpose
I think what you expected is:

1. load a few fields for ANN search, the vector field, the ID field, etc.
2. ANN search to get the topk IDs
3. get content by the topk IDs

In v2.5, the approach:

1. client.load_collection(collection_name, load_fields=["vector", "id"])
2. client.search(collection_name)
3. client.query(collection_name, filter=f"id in [ids]", output_field=["content"])

[gland1]

so query api will work even though I didnt load the content field?

[yhmo]

You can choose mmap for scalar fields, scalar data file is automatically downloaded into querynode's local disk and mmap is created to reduce the memory usage.
...
queryNode:
mmap:
vectorField: false
vectorIndex: false
scalarField: true
scalarIndex: true
....

If mmap is disabled, query node will read the scalar content(a small chunk file that contains the content) from s3/minio, the latency of query() mainly depends on the IO.

[gland1]

I've tried your suggestion , after loading only 2 fields, the query method returns:
code=65535, message=field content is not loaded)>

[yhmo]

Ok, I'm wrong. As @xiaofan-luan mentioned, only loaded fields can be retrieved. The new feature of v2.6 might meet your requirements.

[xiaofan-luan]

The issue we see is as follows: Suppose you'd like to build a RAG system with milvus using 1B dataset. The source content for each vector needs to be retrieved during/after performing an ANN search. If you keep the source content along with the vector in one collection, for example like langchain does , you'll end up fetching huge amount of date from object storage to querynode local drive. The current workaround we use is to keep it separately on the object storage and fetch it directly after retrieving the required vector id by ann search. If it was possible for milvus to avoid loading such field during load(I understand this is available in beta) and then fetch the source content by a simple get api by milvus, or any other solution by milvus that would better support the RAG case with big datasets.

Data not loaded won't be able to retrieve or filters.

In Milvus 2.6, we will support a new data type named TEXT.

The text datatype won't be able to load or filterins. it only support tree operations

1. retrieve by id
2. text match when enable
3. convert to bm25 sparse index

[gland1]

thanks for the info.. I believe I saw TEXT field already in the master branch can we start and check it?

[xiaofan-luan]

it's not ready for integration test.
Our plan is to test it in the next 2-3 weeks and put it to production next month

[gland1]

ok, thanks

[xiaofan-luan]

building a 1B rag case is cool!