Fix Kimi EAGLE3 draft config under DCP

[voipmonitor]

Summary

• Restore draft-specific speculative config overrides for Kimi EAGLE3 (draft_attention_backend and draft_kv_cache_dtype).
• Apply those overrides to the draft model config instead of forcing it to inherit target attention/KV settings.
• Pass DCP-local sequence lengths into draft MLA attention metadata so DCP speculative decode does not crash with seq_lens=None.

Root Cause

Kimi EAGLE3 launch configs used by earlier Kimi images include draft_attention_backend=TRITON_MLA and draft_kv_cache_dtype=fp8. Current chthonic rejected those fields as unexpected. Removing them got past validation but failed later when DCP was enabled because the draft speculator did not provide dcp_local_seq_lens to the MLA metadata builder; the builder then replaced seq_lens with None and crashed during warmup/generation.

Validation

• python3 -m py_compile vllm/config/speculative.py vllm/v1/spec_decode/llm_base_proposer.py vllm/v1/worker/gpu/spec_decode/speculator.py
• Runtime overlay on voipmonitor/vllm:chthonic-consecration-f1190eab-b12x0ff2847-pr20-cu132 with Kimi K2.6 target, festr2/kimi-k2.6-eagle3-mla-fp8 draft, TP8/DCP4, TRITON_MLA, fp8 KV, and the original speculative config.
• Server completed target + draft loading, CUDA graph capture, speculator capture, API startup, and a short /v1/chat/completions smoke request without the previous NoneType DCP crash.

Summary by CodeRabbit

• New Features
  • Added independent KV-cache dtype configuration for draft models in speculative decoding.
  • Added independent attention backend configuration for draft models with automatic selection option.
  • Enhanced support for distributed context parallelism in draft model attention processing.

[coderabbitai]

No actionable comments were generated in the recent review. 🎉

ℹ️ Recent review info

⚙️ Run configuration

Configuration used: Organization UI

Review profile: CHILL

Plan: Pro Plus

Run ID: 433ddd3b-4ee5-42cc-ab66-15159dd5a84e

📥 Commits

Reviewing files that changed from the base of the PR and between 225f431 and 58f130b.

📒 Files selected for processing (3)

• vllm/config/speculative.py
• vllm/v1/spec_decode/llm_base_proposer.py
• vllm/v1/worker/gpu/spec_decode/speculator.py

---

📝 Walkthrough

Walkthrough

SpeculativeConfig gains two new fields: draft_kv_cache_dtype and draft_attention_backend, with a field validator and an alias copy from attention_backend. _create_draft_vllm_config is updated to apply both overrides. Separately, _build_draft_attn_metadata adds context-parallelism support by conditionally computing dcp_local_seq_lens via prepare_dcp_local_seq_lens.

Changes

Draft-model KV cache dtype and attention backend overrides

Layer / File(s)	Summary
SpeculativeConfig: new fields, validator, and alias wiring vllm/config/speculative.py	Adds draft_kv_cache_dtype: CacheDType | None and draft_attention_backend: AttentionBackendEnum | Literal["auto"] | None fields to SpeculativeConfig. A field_validator normalizes string inputs to AttentionBackendEnum or "auto". Post-validation copies attention_backend into draft_attention_backend when the latter is unset; attention_backend documentation is updated to reflect the alias role.
_create_draft_vllm_config: apply dtype and backend overrides vllm/v1/spec_decode/llm_base_proposer.py	Conditionally overrides cache_config.cache_dtype with spec_cfg.draft_kv_cache_dtype when set. Derives draft_backend from spec_cfg.draft_attention_backend (mapping "auto" to None) and assigns it to attention_config.backend, replacing direct use of spec_cfg.attention_backend.

Context parallelism seq-lens in draft attention metadata

Layer / File(s)	Summary
_build_draft_attn_metadata: CP local seq-len computation vllm/v1/worker/gpu/spec_decode/speculator.py	Imports prepare_dcp_local_seq_lens. In _build_draft_attn_metadata, caches the seq_lens slice and, when block_tables.cp_size > 1, calls prepare_dcp_local_seq_lens to produce dcp_local_seq_lens. Updates the build_attn_metadata call to pass both seq_lens and dcp_local_seq_lens.

Estimated code review effort

🎯 3 (Moderate) | ⏱️ ~20 minutes

Possibly related PRs

• local-inference-lab/vllm#19: Both PRs modify how the speculative draft model's attention backend is propagated from speculative_config into draft attention configuration inside llm_base_proposer/draft loader code.

🚥 Pre-merge checks | ✅ 4 | ❌ 1

❌ Failed checks (1 warning)

Check name	Status	Explanation	Resolution
Docstring Coverage	⚠️ Warning	Docstring coverage is 20.00% which is insufficient. The required threshold is 80.00%.	Write docstrings for the functions missing them to satisfy the coverage threshold.

✅ Passed checks (4 passed)

Check name	Status	Explanation
Description Check	✅ Passed	Check skipped - CodeRabbit’s high-level summary is enabled.
Title check	✅ Passed	The title specifically references the issue (Kimi EAGLE3 draft config) and the context (DCP), matching the changes in the PR which fix speculative decoding configuration and DCP sequence length handling.
Linked Issues check	✅ Passed	Check skipped because no linked issues were found for this pull request.
Out of Scope Changes check	✅ Passed	Check skipped because no linked issues were found for this pull request.

_{✏️ Tip: You can configure your own custom pre-merge checks in the settings.}

✨ Finishing Touches

📝 Generate docstrings

• Create stacked PR
• Commit on current branch

🧪 Generate unit tests (beta)

• Create PR with unit tests
• Commit unit tests in branch codex/kimi-eagle3-chthonic-compat-20260616

---

Thanks for using CodeRabbit! It's free for OSS, and your support helps us grow. If you like it, consider giving us a shout-out.

❤️ Share

• X
• Mastodon
• Reddit
• LinkedIn

_{Comment @coderabbitai help to get the list of available commands and usage tips.}