How to calculate the index memory/disk size?

[bingbb47]

There is a tag shows the approximate index size of mamory and disk in Attu, I want to know how to calculate those value.
Is there any formulas

[yhmo]

The formulas to estimate the memory size:

"index size in theory" is the theoretical size.
"expanding rate" is based on our experience.

[bingbb47]

thanks! :)

[nikita-tomilov]

Hello @yhmo!
By any chance, could you provide me the same estimation formula, but for DiskANN?

[xiaofan-luan]

what about use the sizing tool
https://milvus.io/tools/sizing

[nikita-tomilov]

I saw the sizing tool, but it doesnt have enough granularity:

• I am working with ~10k-100k vector datasets, tool suggests 1M+
• It rounds to GB, while I need bytes

Having the "getCollectionDiskSize" method in the API would be the best for my use case but as far as i am aware it is not present and there were no plans for it.

[xiaofan-luan]

don't really understand this.
why to 10k-100k even need diskANN?

we recommend to has 8GB or more memory for milvus

[nikita-tomilov]

For my research I am willing to measure, or estimate as precisely as possible, how much disk size, to the byte, would be necessary to store N vectors of dimension D in Milvus, including index data. My datasets include: one for 60k, one for 300k, and one for 28m vectors.
For every other index I am evaluating the table above was helpful, so I only need estimates for DiskANN now.

[xiaofan-luan]

Sure! Here's the breakdown of DiskANN memory usage estimation in English, based on:

• Vector count (N)
• Vector dimension (D)
• Quantization type (e.g., PQ)
• Graph max_degree
• Data type (e.g., float32, compressed)
• Whether data is memory-mapped or fully loaded into RAM

---

1. Vector Storage Size

a. Uncompressed (float32):

Each vector has D dimensions, each of which is 4 bytes:

vector_size = N × D × 4 bytes

b. Compressed (Product Quantization, PQ):

If you use PQ with M subspaces (e.g., PQ-64 means M=64), each subspace takes 1 byte:

vector_size_pq = N × M × 1 byte

Example: PQ-64 → each vector takes 64 bytes.

---

2. Graph Structure Size

Each node stores up to max_degree neighbors (usually as uint32 IDs):

graph_size = N × max_degree × 4 bytes

---

3. PQ Codebook Size (if using PQ)

Each subspace has a codebook with K=256 centroids, and each centroid is a D/M-dimensional float vector:

codebook_size = M × 256 × (D/M) × 4 = 256 × D × 4 bytes

For D=128, this equals 256 × 128 × 4 = 128 KB

This is small and usually negligible in total memory calculations.

---

4. Metadata Overhead

Includes internal indexing structures, bitmaps, offsets, etc. Usually estimated as:

meta_size ≈ N × 4~8 bytes

---

✅ Total Memory Estimate

a. Uncompressed vectors (float32):

total_memory ≈ N × (D × 4 + max_degree × 4)

b. Compressed vectors (PQ):

total_memory ≈ N × (M × 1 + max_degree × 4) + codebook_size

---

📌 Example Calculation

Suppose:

• N = 10 million vectors
• D = 128
• PQ-64 (M = 64)
• max_degree = 64

Then:

• Vectors: 10M × 64 bytes = 640 MB
• Graph: 10M × 64 × 4 = 2.56 GB
• Codebook: ~128 KB

✅ Total RAM usage ≈ 3.2 GB

---

If you share your actual values (N, D, PQ, max_degree), I can plug them in for an exact estimate. Want to try that?