Does Milvus takes reclustering by own, without downtime for reads & writes?

[Yagyansh]

I see that the searches are queueing up in Milvus for me because of the increased ingestion rate. By when I perform re-clustering the search times improve drastically. Is there any way Milvus can perform reclustering itself without any downtime?

I am using version 0.10.0.

[yhmo]

I am not sure what's the meaning of "perform re-clustering", could you tell me what API you invoked from sdk?

[Yagyansh]

I ran the create index from Python on the existing running Milvus which already has a lot of data to perform the re-clustering.

from milvus import Milvus, IndexType, MetricType, Status collection_name='mycollection' client = Milvus(host='IP', port='19530') client.create_index(collection_name, IndexType.IVF_SQ8,{'nlist': 4096,'nprobe':128})

[yhmo]

Please take a look this page: https://www.milvus.io/docs/v0.10.6/storage_concept.md
Milvus data management is 3-level: collections, partitions, and segments
The api create_index() will build index for each segment file. Before you call create_index(), milvus performs brute-force search. After create_index() is done, milvus will use index to search. Index can dramatically improve the performance of search.

Milvus has two ways to build index:

1. Manually call create_index() api to force milvus to build index for each segment
2. Once a segment size exceeds "index_file_size"(this parameter is specified by create_collection(), default value is 1024MB), milvus will automatically build index for this segment in background thread. You can read this page: https://www.milvus.io/docs/v0.10.6/index.md#CPU