[CI][Spec Decode] Add MTP placeholder-token regression coverage - #54893
Merged
Merged
Conversation
Co-authored-by: OpenAI Codex <codex@openai.com> Signed-off-by: Andreas Karatzas <Andreas.Karatzas@amd.com>
There was a problem hiding this comment.
Claude Code Review
This repository is configured for manual code reviews. Comment @claude review for a one-time review, or @claude review always to subscribe this PR to a review on every future push.
Tip: disable this comment in your organization's Code Review settings.
Member
Author
|
/ci run |
|
✅ Triggered Buildkite CI #86788 for commit |
shen-shanshan
approved these changes
Sep 2, 2026
mylibrar
pushed a commit
to tanyuqian/vllm
that referenced
this pull request
Sep 3, 2026
…-project#54893) Signed-off-by: Andreas Karatzas <Andreas.Karatzas@amd.com> Co-authored-by: OpenAI Codex <codex@openai.com>
sheralskumar
pushed a commit
to sheralskumar/vllm
that referenced
this pull request
Sep 8, 2026
…-project#54893) Signed-off-by: Andreas Karatzas <Andreas.Karatzas@amd.com> Co-authored-by: OpenAI Codex <codex@openai.com>
This file contains hidden or bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
Sign up for free
to join this conversation on GitHub.
Already have an account?
Sign in to comment
Add this suggestion to a batch that can be applied as a single commit.This suggestion is invalid because no changes were made to the code.Suggestions cannot be applied while the pull request is closed.Suggestions cannot be applied while viewing a subset of changes.Only one suggestion per line can be applied in a batch.Add this suggestion to a batch that can be applied as a single commit.Applying suggestions on deleted lines is not supported.You must change the existing code in this line in order to create a valid suggestion.Outdated suggestions cannot be applied.This suggestion has been applied or marked resolved.Suggestions cannot be applied from pending reviews.Suggestions cannot be applied on multi-line comments.Suggestions cannot be applied while the pull request is queued to merge.Suggestion cannot be applied right now. Please check back later.
PIECEWISEandFULL_AND_PIECEWISECUDA graph modes and verify that the requested mode is not silently downgraded.Speculative decoding is an exact optimization at temperature zero, so its emitted token IDs must match ordinary greedy decoding token for token. A previous MTP regression under FULL CUDA graphs instead exposed the complete padded speculative width rather than truncating it to the accepted prefix. Rejected positions use a placeholder token ID, which is converted to token ID 0 before logprob gathering and can therefore become silent output corruption if padded positions escape the output-assembly boundary. With the GLM-5.1 tokenizer, token ID 0 renders as
!, producing responses filled with repeated exclamation marks. The engine continued running normally and reported healthy acceptance statistics, so liveness checks and acceptance-rate tests could not detect the problem. PIECEWISE execution also remained correct, meaning coverage that does not explicitly exercise a FULL graph mode can miss the regression. Comparing emitted token IDs in both graph modes, while proving that rejection actually occurred, provides a deterministic and tokenizer-independent guard for this defect class.AI assistance was used for duplicate research, test design, implementation, validation, and PR preparation. The human submitter reviewed the resulting change and remains responsible for it end to end.