Milvus Sizing Tool for higher loads

[shaiktas]

Hello Team,
We are starting to ingest data of 200Million that contains 10 Vector Fields and each Vector Field is of 1024 Dimensions. But we could not see that Milvus sizing tool can provide estimate for this load. Milvus sizing tool can provide estimate up to 700 Million load with 1 Vector 10124 Dimensions. Any pointers for resources on this load? Also do we have any alternate option to access Milvus Sizing tool other than UI, like app?

[liliu-z]

Hi @shaiktas

For simple answer, you can enlarge your QueryNode 10x comparing with 1 vector field 1024 dimensions. And for Sizing tool, we only offer UI. And also, it is for general cases w/o considering customization, optimization, etc.

10 vec fields may need some finer optimizations to make Milvus run happily, and we are more happy to know more about your use cases to provide better specific usage suggestions. Plz free feel to ping me if you want to discuss more. li.liu@zilliz.com

[shaiktas]

Thank you @liliu-z for your timely response, you mean enlarge DataNodes or QueryNodes, Please note this is for data ingestion and we are thinking of using 32 Shards for this collection.

[xiaofan-luan]

Hello Team, We are starting to ingest data of 200Million that contains 10 Vector Fields and each Vector Field is of 1024 Dimensions. But we could not see that Milvus sizing tool can provide estimate for this load. Milvus sizing tool can provide estimate up to 700 Million load with 1 Vector 10124 Dimensions. Any pointers for resources on this load? Also do we have any alternate option to access Milvus Sizing tool other than UI, like app?

Hi @shaiktas ,
10 vector fields seems to be alog. is there a special reason to use that much of vector fields.
we are glad to help on the sizing. feel free to connect me at xiaofan.luan@zilliz.com