Skip to content

Commit f79b62b

Browse files
Update vllm/v1/spec_decode/eagle.py
Co-authored-by: gemini-code-assist[bot] <176961590+gemini-code-assist[bot]@users.noreply.github.com> Signed-off-by: Andy Lo <andy@mistral.ai>
1 parent ec931bd commit f79b62b

File tree

1 file changed

+2
-2
lines changed

1 file changed

+2
-2
lines changed

vllm/v1/spec_decode/eagle.py

Lines changed: 2 additions & 2 deletions
Original file line numberDiff line numberDiff line change
@@ -199,9 +199,9 @@ def propose(
199199
# there's a multi-layer MTP module.
200200

201201
# Generate the remaining draft tokens.
202-
# [num_speculative_tokens, batch_size]
202+
# Each tensor in the list has shape [batch_size].
203203
draft_token_ids_list: list[torch.Tensor] = [draft_token_ids]
204-
# [num_speculative_tokens, batch_size, vocab_size]
204+
# Each tensor in the list has shape [batch_size, vocab_size].
205205
draft_probs_list: list[torch.Tensor] = [draft_probs]
206206

207207
positions = target_positions[last_token_indices]

0 commit comments

Comments
 (0)