Skip to content

Draft: Do not merge: Kv xfer updates #1

New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

Sign up for GitHub

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Jump to bottom
Open
wants to merge 36 commits into
base: main
Choose a base branch
from
Open

Draft: Do not merge: Kv xfer updates #1

wants to merge 36 commits into from
+986 −179

Conversation

wseaton
Copy link
Owner

@wseaton wseaton commented Jun 9, 2025 •
edited by github-actions bot
Loading

No description provided.

@github-actions GitHub Actions
Copy link

github-actions bot commented Jun 9, 2025

👋 Hi! Thank you for contributing to the vLLM project.

💬 Join our developer Slack at https://slack.vllm.ai to discuss your PR in #pr-reviews, coordinate on features in #feat- channels, or join special interest groups in #sig- channels.

Just a reminder: PRs would not trigger full CI run by default. Instead, it would only run fastcheck CI which starts running only a small and essential subset of CI tests to quickly catch errors. You can run other CI tests on top of those by going to your fastcheck build on Buildkite UI (linked in the PR checks section) and unblock them. If you do not have permission to unblock, ping simon-mo or khluu to add you in our Buildkite org.

Once the PR is approved and ready to go, your PR reviewer(s) can run CI to test the changes comprehensively before merging.

To run CI, PR reviewers can either: Add ready label to the PR or enable auto-merge.

🚀

@wseaton wseaton force-pushed the kv-xfer-updates branch 2 times, most recently from 2566432 to 3cd5a0c Compare June 11, 2025 18:30
wseaton added 3 commits June 11, 2025 14:48
@wseaton
initial noodling
efbd791 
Signed-off-by: Will Eaton <weaton@redhat.com>
@wseaton
attempt to background agent registration
9dd2c1c 
Signed-off-by: Will Eaton <weaton@redhat.com>
@wseaton
more simple immediate retry
a0c03cd 
Signed-off-by: Will Eaton <weaton@redhat.com>
wseaton added 20 commits June 11, 2025 14:56
@wseaton
fix bad merge
cf616b5 
Signed-off-by: Will Eaton <weaton@redhat.com>
@wseaton
implement nicks suggestions
505f586 
Signed-off-by: Will Eaton <weaton@redhat.com>
@wseaton
change retry logic and move to scheduler; simply background handshake…
a1a0918 
… processing

Signed-off-by: Will Eaton <weaton@redhat.com>
@wseaton
fixup by changing types at callsite
518a59f 
Signed-off-by: Will Eaton <weaton@redhat.com>
@wseaton
fixup popping requests
6426e3f 
Signed-off-by: Will Eaton <weaton@redhat.com>
@wseaton
debug logging
29885af 
Signed-off-by: Will Eaton <weaton@redhat.com>
@wseaton
working checkpoint
504bba7 
Signed-off-by: Will Eaton <weaton@redhat.com>
@wseaton
fix unreachable
dd49c96 
Signed-off-by: Will Eaton <weaton@redhat.com>
@wseaton
remove uv.lock
f65c1b3 
Signed-off-by: Will Eaton <weaton@redhat.com>
@wseaton
remove unused
efd655b 
Signed-off-by: Will Eaton <weaton@redhat.com>
@wseaton
flip protocol; fix scheduling order bug
dba3835 
Signed-off-by: Will Eaton <weaton@redhat.com>
@wseaton
fix bug in case of no kvconnectorgroup
85855d1 
Signed-off-by: Will Eaton <weaton@redhat.com>
@wseaton
actually use handshake timeout; simplify route
1fc1af4 
Signed-off-by: Will Eaton <weaton@redhat.com>
@wseaton
revert back to working het TP logic
9d8c15c 
Signed-off-by: Will Eaton <weaton@redhat.com>
@wseaton
Merge branch 'main' into kv-xfer-updates
fbf0630
@wseaton
Resolve merge conflicts and remove deprecated pending handshake logic
69af91c
@wseaton
Merge remote-tracking branch 'origin/main' into kv-xfer-updates
91af1cd
@wseaton
move nixl sidechannel to own entrypoint
83ec83a 
Signed-off-by: Will Eaton <weaton@redhat.com>

push missing side channel server

Signed-off-by: Will Eaton <weaton@redhat.com>
@wseaton
allow configurable handshake strategy
9a86b37
@wseaton
pre-commit fixes
cc887f5 
Signed-off-by: Will Eaton <weaton@redhat.com>
@wseaton
satisfy mypy
b960355 
Signed-off-by: Will Eaton <weaton@redhat.com>
@wseaton wseaton force-pushed the kv-xfer-updates branch 5 times, most recently from 3d5e24a to b960355 Compare June 30, 2025 19:29
wseaton added 12 commits June 30, 2025 15:46
@wseaton
add comments; retrigger fastcheck
e0e88b2 
Signed-off-by: Will Eaton <weaton@redhat.com>
@wseaton
add type hint in test
34e3e55 
Signed-off-by: Will Eaton <weaton@redhat.com>
@wseaton
revert api change
2325470 
Signed-off-by: Will Eaton <weaton@redhat.com>
@wseaton
fix usage
430ad03 
Signed-off-by: Will Eaton <weaton@redhat.com>
@wseaton
remove dead code
4f30966 
Signed-off-by: Will Eaton <weaton@redhat.com>
@wseaton
fix type declaration
54e088e 
Signed-off-by: Will Eaton <weaton@redhat.com>
@wseaton
revert spurious changes to multi-connector
ead79d6 
Signed-off-by: Will Eaton <weaton@redhat.com>
@wseaton
remove dead code for threading
1a6ac2c 
Signed-off-by: Will Eaton <weaton@redhat.com>
@wseaton
remove dead engine code
f940b6d 
Signed-off-by: Will Eaton <weaton@redhat.com>
@wseaton
fix optional ordering
7768f21 
Signed-off-by: Will Eaton <weaton@redhat.com>
@wseaton
naming, fix engine default param issue; pr feedback
3568422 
Signed-off-by: Will Eaton <weaton@redhat.com>
@wseaton
use built in util
332723a 
Signed-off-by: Will Eaton <weaton@redhat.com>
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Reviewers
No reviews
Assignees
No one assigned
Labels
None yet
Projects
None yet
Milestone
No milestone
Development

Successfully merging this pull request may close these issues.
1 participant
@wseaton