-
Couldn't load subscription status.
- Fork 929
[DEFLAKE] Fix/cluster slot migration flaky test #2756
New issue
Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.
By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.
Already on GitHub? Sign in to your account
base: unstable
Are you sure you want to change the base?
[DEFLAKE] Fix/cluster slot migration flaky test #2756
Conversation
Signed-off-by: Hanxi Zhang <hanxizh@amazon.com>
Signed-off-by: Hanxi Zhang <hanxizh@amazon.com>
Signed-off-by: Hanxi Zhang <hanxizh@amazon.com>
Signed-off-by: Hanxi Zhang <hanxizh@amazon.com>
Signed-off-by: Hanxi Zhang <hanxizh@amazon.com>
Signed-off-by: Hanxi Zhang <hanxizh@amazon.com>
Codecov Report✅ All modified and coverable lines are covered by tests. Additional details and impacted files@@ Coverage Diff @@
## unstable #2756 +/- ##
============================================
- Coverage 72.59% 72.46% -0.13%
============================================
Files 128 128
Lines 71301 71303 +2
============================================
- Hits 51759 51671 -88
- Misses 19542 19632 +90 🚀 New features to boost your workflow:
|
| } else { | ||
| fail "Cluster gossip hasn't propagated all node IDs" | ||
| } | ||
|
|
There was a problem hiding this comment.
Choose a reason for hiding this comment
The reason will be displayed to describe this comment to others. Learn more.
we have wait_for_cluster_size 4 in line 276, it is indedd checking the cluster_known_nodes
# Check if cluster size is consistent.
proc cluster_size_consistent {cluster_size} {
for {set j 0} {$j < $cluster_size} {incr j} {
if {[CI $j cluster_known_nodes] ne $cluster_size} {
return 0
}
}
return 1
}
There was a problem hiding this comment.
Choose a reason for hiding this comment
The reason will be displayed to describe this comment to others. Learn more.
You are right! i will remove this.
|
As binbin noted, this change is not necessary, so there is no need to review this right now. I changed it to draft to avoid maintainers attention. I don't know the plans for this PR, to update it or close it later? |
Yes, thank you for doing this! I’m trying to think through other possible causes. It seems that the code inserts the node into cluster->nodes on each server, which increases the size to 4, but some internal registration steps might not have completed yet. I’ll look into it further. |
Address: #2693
I add an additional check to ensure that all nodes recognize the full cluster (4 nodes) after a new node is added. This helps prevent the test from failing due to cluster gossip not being fully propagated.
I ran the test 100 times for both the unmodified and modified versions, and all runs passed:
Original (unmodified): CI run
Modified (with fix): CI run
For reference, this test previously failed once in https://github.com/valkey-io/valkey/actions/runs/18266295942/job/52001092528