Milvus and CDC use the same etcd, which is suspected to have caused etcd write exceptions due to excessive space occupation by CDC #40220
-
|
I have a question to ask. When using CDC, we used ETCD for storing CDC metadata, which resulted in a significant increase in ETCD disk usage and caused errors when writing data to ETCD. Does the metadata stored in CDC require a relatively large amount of space? Will it continue to be written? |
Beta Was this translation helpful? Give feedback.
Replies: 3 comments 2 replies
-
|
@guhezhu01 Can you identify the KVs that take up significant space? |
Beta Was this translation helpful? Give feedback.
-
|
@jaime0815 The etcd data has been cleaned up as it affects the use of milvus |
Beta Was this translation helpful? Give feedback.
-
|
@SimFG |
Beta Was this translation helpful? Give feedback.
-
|
This problem does exist at present. The reason is that there is no batch processing for replicate messages. This will cause memory problems and require the cdc pod to have a larger memory. This part will be modified later. You can temporarily use MySQL to solve this problem. |
Beta Was this translation helpful? Give feedback.
-
|
okay, thank you |
Beta Was this translation helpful? Give feedback.
This problem does exist at present. The reason is that there is no batch processing for replicate messages. This will cause memory problems and require the cdc pod to have a larger memory. This part will be modified later. You can temporarily use MySQL to solve this problem.