Skip to content

[Spec Decode][DCP] Populate draft-local sequence lengths - #54278

Closed
GirasoleY wants to merge 1 commit into
vllm-project:mainfrom
GirasoleY:specdecode-dcp-draft-metadata
Closed

GirasoleY wants to merge 1 commit into
vllm-project:mainfrom
GirasoleY:specdecode-dcp-draft-metadata

Conversation

@GirasoleY

Copy link
Copy Markdown
Contributor

Summary

  • Recompute rank-local DCP sequence lengths from the draft model's current global sequence lengths before every draft metadata build.
  • Use the finalized BlockTables CP size, rank, and interleave instead of copying parallel state into the speculator.
  • Share the logic across EAGLE, MTP, and DFlash and remove DFlash's duplicate preparation.
  • Add an EAGLE regression test covering padded DCP-local metadata.

Motivation

Draft attention owns separate persistent metadata from the target. EAGLE and autoregressive MTP currently call the shared metadata builder without dcp_local_seq_lens; MLA metadata then receives None under DCP. Reusing target-local lengths is not sufficient because draft steps advance or rewind input_buffers.seq_lens after sampling and rejection.

This completes the draft-side metadata plumbing needed by DCP-capable attention backends such as TokenSpeed MLA (#48180) and FlashInfer MLA (#54012).

Tests

Passed locally:

  • uvx --offline ruff check on changed files
  • uvx --offline ruff format --check on changed files
  • python3 -m py_compile on changed files
  • git diff --check

Focused pytest and multi-GPU EAGLE+DCP evaluation are pending because this local worktree does not contain the vLLM runtime/test dependencies. This PR is intentionally opened as a draft.

AI assistance disclosure

AI assistance was used to trace the EAGLE/DFlash metadata paths, adapt the k3-dev behavior to current upstream BlockTables state, write the initial patch and regression test, and run static validation. The submitter will review every changed line and run focused/runtime tests before requesting final review.

Move rank-local sequence-length preparation into the shared draft metadata builder so EAGLE, MTP, and DFlash recompute exact DCP boundaries after draft advances and rejections. Use the finalized BlockTables CP geometry and add an EAGLE regression test.

Signed-off-by: Summer Yang <girasoleyang@gmail.com>

@claude claude Bot left a comment

Copy link
Copy Markdown

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Claude Code Review

This pull request is from a fork — automated review is disabled. A repository maintainer can comment @claude review to run a one-time review.

@GirasoleY

Copy link
Copy Markdown
Contributor Author

Superseded by #54277, which now contains this generic draft-side DCP metadata commit together with the FlashInfer MLA DSpark+DCP backend changes.

@GirasoleY GirasoleY closed this Aug 28, 2026
@github-project-automation github-project-automation Bot moved this from Backlog to Done in Sprint - DFlash Aug 28, 2026
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

Projects

Status: Done

Development

Successfully merging this pull request may close these issues.

1 participant