Eagle3 mm support, enablement on qwen3vl

[jesse996]

What this PR does / why we need it?

follow pr vllm-project/vllm#20788 , Eagle3 mm support, enablement on qwen3vl
target model Qwen/Qwen3-VL-8B-Instruct
eagle3 MNN/Qwen3-VL-8B-Instruct-Eagle3

Does this PR introduce any user-facing change?

No

How was this patch tested?

pytest ./tests/e2e/singlecard/test_completion_with_prompt_embeds.py -vv

vLLM with eagle3 :

vllm serve /model/Qwen3-VL-8B-Instruct   --enforce-eager   --port 9100    --max-model-len 32768   --max-num-seqs 32   --tensor-parallel-size 2   --allowed-local-media-path /model/gx/images  --speculative-config '{
    "method": "eagle3",
    "model": "/model/hf/Qwen3-VL-8B-Instruct-Eagle3",
    "num_speculative_tokens": 3
  }'

vLLM without eagle3 :

vllm serve /model/Qwen3-VL-8B-Instruct   --enforce-eager   --port 9100    --max-model-len 32768   --max-num-seqs 32   --tensor-parallel-size 2   --allowed-local-media-path /model/gx/images 

bench:

vllm bench serve   --backend openai-chat   --base-url http://127.0.0.1:9100   --tokenizer /model/Qwen3-VL-8B-Instruct   --endpoint /v1/chat/completions   --model /model/Qwen3-VL-8B-Instruct   --dataset-name random  --num-prompts 50   --max-concurrency 5   --temperature 0   --top-p 1.0   --seed 123

eagle3 result:

without eagle3 result

• vLLM version: v0.12.0
• vLLM main: vllm-project/vllm@ad32e3e

[gemini-code-assist]

Code Review

This pull request adds support for Eagle3 speculative decoding for multi-modal models, specifically targeting Qwen3-VL. The changes primarily focus on updating the EagleProposer to handle multi-modal inputs, including M-RoPE and image embeddings, and adding a new end-to-end test to verify correctness. While the changes are in the right direction, I've identified a couple of critical issues. The new test is using an incorrect base model for generating reference outputs, which invalidates the test's purpose. Additionally, there's a critical bug in the EagleProposer where the logic for sharing vocabulary embeddings between the target and draft models is flawed, which would lead to incorrect behavior. I've also pointed out a leftover debug print statement that should be removed.

[gemini-code-assist]

The reference LLM is being initialized with model_name, which is hardcoded to LLM-Research/Meta-Llama-3.1-8B-Instruct. However, this test is intended for vision-language models and should use vl_model_name (Qwen/Qwen3-VL-8B-Instruct), which is correctly passed as a fixture. This means the test is not comparing against the correct reference model, making the test results invalid.

Suggested change

	ref_llm = LLM(model=model_name, max_model_len=2048, enforce_eager=False)
	ref_llm = LLM(model=vl_model_name, max_model_len=2048, enforce_eager=False)

[gemini-code-assist]

The logic for sharing vocabulary embeddings between the target model (model) and the draft model (self.model) is incorrect. The line self.model.model.embed_tokens = self.model.model.embed_tokens is a no-op, and the hasattr checks are performed on the draft model instead of the target model. This will result in the draft model not using the target model's embeddings. The logic should check for embed_tokens or embedding on the target model and assign it to the draft model.

Suggested change

	if hasattr(self.model.model, "embed_tokens"):
	self.model.model.embed_tokens = self.model.model.embed_tokens
	elif hasattr(self.model.model, "embedding"):
	self.model.model.embed_tokens = self.model.model.embedding
	# self.model.model.embed_tokens = model.model.embed_tokens
	if hasattr(model.model, "embed_tokens"):
	self.model.model.embed_tokens = model.model.embed_tokens
	elif hasattr(model.model, "embedding"):
	self.model.model.embed_tokens = model.model.embedding

[gemini-code-assist]

This print statement appears to be for debugging purposes and should be removed from the final code.

[github-actions]

👋 Hi! Thank you for contributing to the vLLM Ascend project. The following points will speed up your PR merge:‌‌

• A PR should do only one thing, smaller PRs enable faster reviews.
• Every PR should include unit tests and end-to-end tests ‌to ensure it works and is not broken by other future PRs.
• Write the commit message by fulfilling the PR description to help reviewer and future developers understand.

If CI fails, you can run linting and testing checks locally according Contributing and Testing.

[github-actions]

This pull request has conflicts, please resolve those before we can evaluate the pull request.

[github-actions]

This pull request has conflicts, please resolve those before we can evaluate the pull request.

[github-actions]

This pull request has conflicts, please resolve those before we can evaluate the pull request.