collection empty best practice

[rere950303]

• milvus version: v2.4.20
• mode: cluster

---

Apologies in advance if this has already been addressed.
I expected the memory usage of the querynode to drop immediately after deleting all data in the collection, but I observed that it only decreased after a compaction occurred about an hour later, as shown below.

Milvus(by-dev) > show compactions --collectionID=455364181455730393
JobID: 456409032520826779       TaskID: 456409032520826816       Type:MixCompaction      State:completed         StartTime: 1742469346
JobID: 456409032521244809       TaskID: 456409032521244809       Type:Level0DeleteCompaction     State:completed         StartTime: 1742534956
JobID: 456409032521244865       TaskID: 456409032521244865       Type:Level0DeleteCompaction     State:completed         StartTime: 1742534966
JobID: 456409032521244921       TaskID: 456409032521244921       Type:Level0DeleteCompaction     State:completed         StartTime: 1742534976
JobID: 456409032521244977       TaskID: 456409032521244977       Type:Level0DeleteCompaction     State:completed         StartTime: 1742534986
JobID: 456409032521244978       TaskID: 456409032521245001       Type:MixCompaction      State:completed         StartTime: 1742534986
JobID: 456409032521244978       TaskID: 456409032521245002       Type:MixCompaction      State:completed         StartTime: 1742534986
...
--- Total compactions:  34       Matched compactions:  30
================================================================================

In this case, is it better to wait for automatic compaction after deleting all data, or should I manually trigger the compaction API once?
I’d like to know what the best practice is regarding querynode memory management.

[yhmo]

Just let it do compaction automatically, since it knows when is better to do it. No need to manually trigger it.

[rere950303]

@xiaofan-luan thanks for your works!
BTW, what kind of issues can we expect? Why does the disk usage increase temporarily? Could you please explain it in a bit more detail?

[xiaofan-luan]

@xiaofan-luan thanks for your works! BTW, what kind of issues can we expect? Why does the disk usage increase temporarily? Could you please explain it in a bit more detail?

because milvus is using soft delete. if you delete 5% of the data then compaction might not be able to be triggered so you will see memory increase instead of drop.

[rere950303]

@xiaofan-luan
I see. In that case, does it mean that manually calling the compaction API wouldn't resolve the issue?

[xiaofan-luan]

manually compact can for sure help to recycle memory. But it is not recommended to call it very frequently because compaction and index rebuild is expensive

[rere950303]

I understand, thank you for your advice.