Repository navigation
[MRV2] Miscellaneous code structure updates - #60754
Merged
Merged
Conversation
njhill
requested review from
BoyuanFeng,
ProExpertProg,
WoosukKwon,
yewentao256,
youkaichao and
zou3519
as code owners
October 9, 2026 02:21
![claude[bot]](https://avatars.githubusercontent.com/in/1236702?s=60&v=4){kind=small}
mergify
Bot
added
speculative-decoding
mrv2
Member
Author
|
/ci run |
|
✅ Triggered Buildkite CI #93748 for commit |
Signed-off-by: Nick Hill <nickhill123@gmail.com>
Signed-off-by: Nick Hill <nickhill123@gmail.com>
Signed-off-by: Nick Hill <nickhill123@gmail.com>
Member
Author
|
/ci run |
|
✅ Triggered Buildkite CI #93917 for commit |
njhill
changed the title
njhill
commented
Oct 9, 2026
Comment on lines
-1986
to
-1991
| connector_kwargs = dict( | ||
| scheduler_output=scheduler_output, | ||
| request_state_indices=input_batch.idx_mapping, | ||
| request_ids=input_batch.req_ids, | ||
| num_tokens=input_batch.num_tokens, | ||
| ) |
Member
Author
There was a problem hiding this comment.
this also means we avoid creating this dict when a connector isn't enabled
Member
Author
|
/ci retry |
|
✅ No failed, timed-out, or expired jobs need retrying: https://buildkite.com/vllm/ci/builds/93917 |
WoosukKwon
approved these changes
Oct 10, 2026
6 tasks done
mjbrn
added a commit
to brntech/vllm
that referenced
this pull request
Oct 10, 2026
…del runner conflict and follow the dp_sync_state rename - vllm/v1/worker/gpu/model_runner.py: upstream vllm-project#60754 replaced the runner's _get_drafter_hidden_states() with the module function spec_decode.utils.get_drafter_hidden_states(model, hidden_states). Uno's post-AsyncOutput proposal path now calls that function; the proposal itself stays in _run_speculator_proposal, whose body equals upstream's inline block (watermark preparation, draft workspace lane, num_speculative_tokens, blanking past the scheduled K, adaptive confidences), and _publish_draft_tokens is unchanged. - vllm-project#60754 also renamed dp_sync to dp_sync_state in BaseSpeculator.propose, ExecuteModelState and the runner. UnoSpeculator.propose, _run_speculator_proposal and the Uno runner tests follow the new name, so upstream's keyword call in the dummy run reaches Uno's propose. Signed-off-by: MJ <121681560+mjbrn@users.noreply.github.com>
mjbrn
added a commit
to brntech/vllm
that referenced
this pull request
Oct 10, 2026
…ropose signature vllm-project#60754 renamed propose's dp_sync to dp_sync_state, and the merge followed it in UnoSpeculator.propose. No CPU test would have noticed if it had not: reverting the parameter to dp_sync left the Uno suite green, because the runner proposal tests replace propose with a stand-in that accepts any keyword, and no CPU test reaches the real method by keyword. The stand-in now binds every call against inspect.signature of the real UnoSpeculator.propose, so the runner's sample_tokens -> _run_speculator_proposal path fails on a keyword Uno does not take; and a new test pins UnoSpeculator.propose's parameter names, order, kinds and defaults to BaseSpeculator.propose, which the runner's proposal and DP dummy-run call sites are written against. Assisted-by: Claude <noreply@anthropic.com> Signed-off-by: MJ <121681560+mjbrn@users.noreply.github.com>
4 of 6 tasks
This file contains hidden or bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
Sign up for free
to join this conversation on GitHub.
Already have an account?
Sign in to comment
2 participants
Add this suggestion to a batch that can be applied as a single commit.This suggestion is invalid because no changes were made to the code.Suggestions cannot be applied while the pull request is closed.Suggestions cannot be applied while viewing a subset of changes.Only one suggestion per line can be applied in a batch.Add this suggestion to a batch that can be applied as a single commit.Applying suggestions on deleted lines is not supported.You must change the existing code in this line in order to create a valid suggestion.Outdated suggestions cannot be applied.This suggestion has been applied or marked resolved.Suggestions cannot be applied from pending reviews.Suggestions cannot be applied on multi-line comments.Suggestions cannot be applied while the pull request is queued to merge.Suggestion cannot be applied right now. Please check back later.
Various follow-on changes to recently-merged PRs, mostly to minimize/simplify the code in
model_runner.py_get_drafter_hidden_statesto utils filekv_connector.pydp_synctodp_sync_statefor clarity.item()s