Skip to content

Commit fe1f5c0

Browse files
committed
fix: change num_tokens_across_dp type from int to torch.Tensor
Signed-off-by: Jade Zheng <zheng.shoujian@outlook.com>
1 parent d8e3159 commit fe1f5c0

File tree

1 file changed

+1
-1
lines changed

1 file changed

+1
-1
lines changed

vllm_ascend/worker/model_runner_v1.py

Lines changed: 1 addition & 1 deletion
Original file line numberDiff line numberDiff line change
@@ -1823,7 +1823,7 @@ def _dummy_run(
18231823
is_compile: bool = False,
18241824
with_prefill: bool = True,
18251825
skip_attn: bool = True,
1826-
num_tokens_across_dp: Optional[int] = None,
1826+
num_tokens_across_dp: Optional[torch.Tensor] = None,
18271827
) -> torch.Tensor:
18281828
# Set num_scheduled_tokens based on num_tokens and max_num_seqs
18291829
# for dummy run with LoRA so that the num_reqs collectively

0 commit comments

Comments
 (0)