Support different tp_size - Part 1 p/d dynamic connection

[yuan-luo]

Motivation

To support different tensor parallel size on prefill and decode. This PR is Part 1, supporting prefill / decode connection establishment and register to bootstrap server.

Modifications

1. Bootstrap server support REGISTER http route service with prefill and decode
2. KVManager add connection pool cache
3. Refactor decode's connection establishment with bootstrap server
4. Refactor prefill's connection establishment with bootstrap server

Checklist

• Format your code according to the Code Formatting with Pre-Commit.
• Add unit tests as outlined in the Running Unit Tests.
• Update documentation / docstrings / example tutorials as needed, according to Writing Documentation.
• Provide throughput / latency benchmark results and accuracy evaluation results as needed, according to Benchmark and Profiling and Accuracy Results.
• For reviewers: If you haven't made any contributions to this PR and are only assisting with merging the main branch, please remove yourself as a co-author when merging the PR.
• Please feel free to join our Slack channel at https://slack.sglang.ai to discuss your PR.

[Copilot]

Copilot reviewed 2 out of 2 changed files in this pull request and generated 4 comments.