Conversation
Move rank-local sequence-length preparation into the shared draft metadata builder so EAGLE, MTP, and DFlash recompute exact DCP boundaries after draft advances and rejections. Use the finalized BlockTables CP geometry and add an EAGLE regression test. Signed-off-by: Summer Yang <girasoleyang@gmail.com>
GirasoleY
marked this pull request as ready for review
August 28, 2026 22:16
GirasoleY
requested review from
WoosukKwon,
njhill and
yewentao256
as code owners
August 28, 2026 22:16
Contributor
Author
|
Superseded by #54277, which now contains this generic draft-side DCP metadata commit together with the FlashInfer MLA DSpark+DCP backend changes. |
This file contains hidden or bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
Sign up for free
to join this conversation on GitHub.
Already have an account?
Sign in to comment
Add this suggestion to a batch that can be applied as a single commit.This suggestion is invalid because no changes were made to the code.Suggestions cannot be applied while the pull request is closed.Suggestions cannot be applied while viewing a subset of changes.Only one suggestion per line can be applied in a batch.Add this suggestion to a batch that can be applied as a single commit.Applying suggestions on deleted lines is not supported.You must change the existing code in this line in order to create a valid suggestion.Outdated suggestions cannot be applied.This suggestion has been applied or marked resolved.Suggestions cannot be applied from pending reviews.Suggestions cannot be applied on multi-line comments.Suggestions cannot be applied while the pull request is queued to merge.Suggestion cannot be applied right now. Please check back later.
Summary
BlockTablesCP size, rank, and interleave instead of copying parallel state into the speculator.Motivation
Draft attention owns separate persistent metadata from the target. EAGLE and autoregressive MTP currently call the shared metadata builder without
dcp_local_seq_lens; MLA metadata then receivesNoneunder DCP. Reusing target-local lengths is not sufficient because draft steps advance or rewindinput_buffers.seq_lensafter sampling and rejection.This completes the draft-side metadata plumbing needed by DCP-capable attention backends such as TokenSpeed MLA (#48180) and FlashInfer MLA (#54012).
Tests
Passed locally:
uvx --offline ruff checkon changed filesuvx --offline ruff format --checkon changed filespython3 -m py_compileon changed filesgit diff --checkFocused pytest and multi-GPU EAGLE+DCP evaluation are pending because this local worktree does not contain the vLLM runtime/test dependencies. This PR is intentionally opened as a draft.
AI assistance disclosure
AI assistance was used to trace the EAGLE/DFlash metadata paths, adapt the k3-dev behavior to current upstream
BlockTablesstate, write the initial patch and regression test, and run static validation. The submitter will review every changed line and run focused/runtime tests before requesting final review.