WIP: Support multi prefill instances on one node

[yuan-luo]

Motivation

Modifications

Checklist

• Format your code according to the Code Formatting with Pre-Commit.
• Add unit tests as outlined in the Running Unit Tests.
• Update documentation / docstrings / example tutorials as needed, according to Writing Documentation.
• Provide throughput / latency benchmark results and accuracy evaluation results as needed, according to Benchmark and Profiling and Accuracy Results.
• For reviewers: If you haven't made any contributions to this PR and are only assisting with merging the main branch, please remove yourself as a co-author when merging the PR.
• Please feel free to join our Slack channel at https://slack.sglang.ai to discuss your PR.

[yuan-luo]

Per offline discussed with @ShangmingCai, suspend this PR and focus on supporting different tp_size for the moment.

[Hongbosherlock]

Per offline discussed with @ShangmingCai, suspend this PR and focus on supporting different tp_size for the moment.

When do we get back to this PR?

[ShangmingCai]

Per offline discussed with @ShangmingCai, suspend this PR and focus on supporting different tp_size for the moment.

When do we get back to this PR?

@Hongbosherlock We need to discuss this with the sglang team since supporting multi-prefill on the same node requires changing the mini_lb and bootstrap impl. Currently, this is not a high-priority issue since it is not the case that real deployment will adopt.

[Hongbosherlock]

Per offline discussed with @ShangmingCai, suspend this PR and focus on supporting different tp_size for the moment.

When do we get back to this PR?

@Hongbosherlock We need to discuss this with the sglang team since supporting multi-prefill on the same node requires changing the mini_lb and bootstrap impl. Currently, this is not a high-priority issue since it is not the case that real deployment will adopt.

Is it now possible to use multiple Docker containers on one machine to achieve xPyD? Do the P nodes within the different Docker containers need to have the same port?