Skip to content

Commit e74331a

Browse files
authored
Add dp initialize patch with hccl backend (#626)
<!-- Thanks for sending a pull request! BEFORE SUBMITTING, PLEASE READ https://docs.vllm.ai/en/latest/contributing/overview.html --> ### What this PR does / why we need it? <!-- - Please clarify what changes you are proposing. The purpose of this section is to outline the changes and how this PR fixes the issue. If possible, please consider writing useful notes for better and faster reviews in your PR. - Please clarify why the changes are needed. For instance, the use case and bug description. - Fixes # --> Add dp stateless process group initialization path with hccl backend as vllm-ascend patch. ### Does this PR introduce _any_ user-facing change? <!-- Note that it means *any* user-facing change including all aspects such as API, interface or other behavior changes. Documentation-only updates are not considered user-facing changes. --> ### How was this patch tested? <!-- CI passed with new added/existing test. If it was tested in a way different from regular unit tests, please clarify how you tested step by step, ideally copy and paste-able, so that other reviewers can test and check, and descendants can verify in the future. If tests were not added, please describe why they were not added and/or why it was difficult to add. --> --------- Signed-off-by: ganyi <pleaplusone.gy@gmail.com>
1 parent 848e041 commit e74331a

File tree

2 files changed

+24
-1
lines changed

2 files changed

+24
-1
lines changed

vllm_ascend/patch/__init__.py

Lines changed: 9 additions & 1 deletion
Original file line numberDiff line numberDiff line change
@@ -87,7 +87,15 @@
8787
# Future Plan:
8888
# Its a workaround in vllm-ascend to enable multi-node dp inference, maybe removed if vllm have better plan
8989
# on multi-node dp inference implementation
90-
#
90+
# 4. `ParallelConfig.stateless_init_dp_group`
91+
# Why:
92+
# vLLM use gloo backend by default to initialize stateless dp process gourp, but we want to use hccl here to
93+
# get better performance
94+
# How:
95+
# adopt nccl backend to init process group
96+
# Related PR (if no, explain why): no related PR, we want add this ability into vllm
97+
# Future Plan:
98+
# Remove those patch when vllm merged them
9199
# * Worker Patch:
92100
# ===============
93101
# ** File: worker/patch_0_8_4/patch_metrics.py **

vllm_ascend/patch/platform/patch_common/patch_distributed.py

Lines changed: 15 additions & 0 deletions
Original file line numberDiff line numberDiff line change
@@ -152,6 +152,21 @@ def parallel_config_get_dp_port(self) -> int:
152152
return port
153153

154154

155+
def ascend_stateless_init_dp_group(self) -> "ProcessGroup":
156+
from vllm.distributed.utils import \
157+
stateless_init_torch_distributed_process_group
158+
159+
dp_group = stateless_init_torch_distributed_process_group(
160+
self.data_parallel_master_ip,
161+
self.get_next_dp_init_port(),
162+
self.data_parallel_rank,
163+
self.data_parallel_size,
164+
backend="hccl")
165+
166+
return dp_group
167+
168+
155169
vllm.distributed.parallel_state.destroy_model_parallel = ascend_destroy_model_parallel
156170
vllm.distributed.stateless_init_torch_distributed_process_group = ascend_stateless_init_torch_distributed_process_group
157171
ParallelConfig.get_next_dp_init_port = parallel_config_get_dp_port
172+
ParallelConfig.stateless_init_dp_group = ascend_stateless_init_dp_group

0 commit comments

Comments
 (0)