Optimal Indexing Configuration for 10 Million with 8400 Dimensions

[sctrueew]

Hi, I hope you're doing well.
I have approximately 10 million data points with a dimension of 8400. I would like to know what configuration I should set for indexing them.

Server specifications:

OS: Windows Server
RAM: 64 GB
Storage: 10 TB HDD

Thank you in advance!

[xiaofan-luan]

If you run with purely memory mode, I would guess 300GB+ memory is required.

For your current hardware, if recall is not a strict concern, I would recommend to run with IVFSQ8 or HNSWSQ8.

We don't support windows so a docker is required.

Unfortunatelhy you don't have SSDs, thus diskANN will not be an option. Running on Zilliz cloud capacity instance is another recommendation I have.