Add Eagle-3 Qwen support (follow-up to #20436)

[rahul-tuli]

Summary

This PR adds support for Eagle-3 speculative decoding with Qwen models, as a follow-up to #20436.

The changes enable Qwen models to work with Eagle-3 draft models in speculators format.

Related Issues

• Follows up on feat: Add support for speculators Eagle checkpoints #20436 which added initial Eagle support for speculators

Changes

• Added Qwen support to Eagle-3 implementation
• Proper handling of Qwen model architecture in Eagle-3 speculative decoding

Testing

Tested with the following verification script:

#\!/usr/bin/env python3
"""
Verification script for Qwen Eagle-3 support.
"""

from vllm import LLM, SamplingParams

# Initialize vLLM with Qwen model and Eagle-3 draft model
llm = LLM(
    model="Qwen/Qwen2.5-7B-Instruct",
    speculative_config={
        "method": "eagle",
        "model": "nm-testing/Qwen3-8B-Eagle3-speculators-converted",
        "num_speculative_tokens": 5
    },
    max_model_len=1024,
    enforce_eager=True
)

# Test generation
outputs = llm.generate(
    ["The future of AI is"], 
    SamplingParams(max_tokens=20, temperature=0)
)

print(f"Generated: {outputs[0].outputs[0].text}")
print("✅ Eagle-3 with Qwen works\!")

Output confirms that Eagle-3 speculative decoding works correctly with Qwen models

[gemini-code-assist]

Code Review

This pull request adds support for Eagle-3 speculative decoding with Qwen models. The changes are well-structured, introducing a new config format for "speculators" and updating the model loading and execution logic accordingly. A new config class, SpeculatorsEagleConfig, is added to handle the translation from the new format to vLLM's internal format, and the Qwen model is updated to support extracting auxiliary hidden states required by Eagle-3.

I've identified one critical bug in vllm/model_executor/models/qwen2.py related to pipeline parallelism, where the layer index for collecting auxiliary hidden states is calculated incorrectly. A fix has been suggested. The rest of the changes appear to be correct and robust.

[gemini-code-assist]

There's a bug in how auxiliary hidden states are collected when pipeline parallelism is used. The loop index idx is relative to the slice of layers for the current pipeline stage (self.layers[self.start_layer:self.end_layer]), so it will always start from 0. However, self.aux_hidden_state_layers contains absolute layer indices.

When pipeline parallelism is enabled (pipeline_parallel_size > 1), self.start_layer will be non-zero for later stages, and the condition idx in self.aux_hidden_state_layers will be incorrect. For example, if self.start_layer is 10, idx will start from 0, but self.aux_hidden_state_layers might contain values like 15. The check will fail to identify the correct layer.

To fix this, you should use the absolute layer index for the check. The absolute index is self.start_layer + idx.

Suggested change

	if idx in self.aux_hidden_state_layers:
	aux_hidden_states.append(hidden_states + residual)
	if self.start_layer + idx in self.aux_hidden_state_layers:
	aux_hidden_states.append(hidden_states + residual)

[github-actions]

👋 Hi! Thank you for contributing to the vLLM project.

💬 Join our developer Slack at https://slack.vllm.ai to discuss your PR in #pr-reviews, coordinate on features in #feat- channels, or join special interest groups in #sig- channels.

Just a reminder: PRs would not trigger full CI run by default. Instead, it would only run fastcheck CI which starts running only a small and essential subset of CI tests to quickly catch errors. You can run other CI tests on top of those by going to your fastcheck build on Buildkite UI (linked in the PR checks section) and unblock them. If you do not have permission to unblock, ping simon-mo or khluu to add you in our Buildkite org.

Once the PR is approved and ready to go, your PR reviewer(s) can run CI to test the changes comprehensively before merging.

To run CI, PR reviewers can either: Add ready label to the PR or enable auto-merge.

🚀