Skip to content

Commit 706de02

Browse files
authored
[fix] fix compatibility for non-EPLB scenarios (#1142)
### What this PR does / why we need it? Fix incompatibility problem for non-EPLB scenarios in #1116 ### Does this PR introduce _any_ user-facing change? No. ### How was this patch tested? Tested with online serving and e2e CI. Signed-off-by: linfeng-yuan <1102311262@qq.com>
1 parent 571f88f commit 706de02

File tree

1 file changed

+4
-3
lines changed

1 file changed

+4
-3
lines changed

vllm_ascend/quantization/w8a8_dynamic.py

Lines changed: 4 additions & 3 deletions
Original file line numberDiff line numberDiff line change
@@ -150,8 +150,8 @@ def fused_experts_with_mc2(hidden_states: torch.Tensor,
150150
log2phy: torch.Tensor = None,
151151
global_redundant_expert_num: int = 0,
152152
**kwargs) -> torch.Tensor:
153-
154-
topk_ids = log2phy[topk_ids]
153+
if log2phy:
154+
topk_ids = log2phy[topk_ids]
155155
global_bs = 0
156156
moe_expert_num = len(expert_map) + global_redundant_expert_num
157157
# hidden_states = hidden_states.bfloat16()
@@ -278,7 +278,8 @@ def fused_experts_with_all2all(
278278
log2phy: torch.Tensor = None,
279279
global_redundant_expert_num: int = 0,
280280
):
281-
topk_ids = log2phy[topk_ids]
281+
if log2phy:
282+
topk_ids = log2phy[topk_ids]
282283
original_shape = hidden_states.shape
283284
if len(original_shape) == 3:
284285
hidden_states = hidden_states.view(-1, hidden_states.shape[-1])

0 commit comments

Comments
 (0)