[Bugfix][Speculative Decoding] Extend Eagle quantization config fix to llama_eagle.py

[rahul-tuli]

This PR extends the quantization config fix from #25883 to llama_eagle.py

Background

PR #25883 fixed an issue where Eagle3 drafter models were incorrectly using the verifier model's quantization config instead of their own. This caused problems when the drafter and verifier models had different quantization configurations.

The fix introduced a get_quant_config() method in LlamaDecoderLayer that can be overridden by Eagle subclasses to use the draft model's quantization config.

Changes

This PR applies the same pattern to additional Eagle drafters:

llama_eagle.py

• Adds get_quant_config() method to the LlamaDecoderLayer subclass in llama_eagle.py
• The method retrieves the quantization config from the draft model instead of the verifier model
• Uses VllmConfig.get_quantization_config() to properly obtain the draft model's quantization config

Impact

The fix ensures that Eagle drafter models correctly use their own quantization configuration, preventing quantization mismatches when used with differently quantized verifier models.

Notes

• llama_eagle3.py was already fixed in [Bugfix][Speculative Decoding] Fix Eagle3 quantization config issue #25883
• llama4_eagle.py handles this differently by explicitly passing quantization config as a parameter, so no changes are needed there
• minicpm_eagle.py accepts a separate quant_config parameter in its decoder layer, so it doesn't need this fix

---

Fixes: Extends fix from #25883
Related: #25883

Verification

CUDA_VISIBLE_DEVICES=0 python examples/offline_inference/spec_decode.py \
  --method "eagle" \
  --tp 1 \
  --print-output \
  --model-dir "RedHatAI/Meta-Llama-3.1-8B-Instruct-FP8-dynamic" \
  --eagle-dir "yuhuili/EAGLE-LLaMA3.1-Instruct-8B" \
  --dataset_name "hf" \
  --dataset_path "philschmid/mt-bench" \
  --num-spec-tokens 5 2>&1 | tee local/acceptance-rates-eagle.txt

Output:

--------------------------------------------------
--------------------------------------------------
total_num_output_tokens: 210113
num_drafts: 87017
num_draft_tokens: 435085
num_accepted_tokens: 123333
mean acceptance length: 2.42
--------------------------------------------------
acceptance at token 0: 0.68
acceptance at token 1: 0.38
acceptance at token 2: 0.20
acceptance at token 3: 0.10
acceptance at token 4: 0.06

[chatgpt-codex-connector]

💡 Codex Review

Here are some automated review suggestions for this pull request.

ℹ️ About Codex in GitHub

Your team has set up Codex to review pull requests in this repo. Reviews are triggered when you

• Open a pull request for review
• Mark a draft as ready
• Comment "@codex review".

If Codex has suggestions, it will comment; otherwise it will react with 👍.

[jmkuebler]

Thx @rahul-tuli , the fix LGTM and solves #26402

I wouldn't expect this to correctly handle cases where the eagle head is actually quantized. For example, I beleive the ignore list would need to be written taking into account that the layers are registered as further layers in the original model etc.. But that is beyond the pressing issue.

Thanks!

[yewentao256]

LGTM, thanks for the work!

[shreyas269]

I tested these changes with quantized Eagle head. LGTM!