Repository navigation
[Security] Defer diffusion prompt scores until output - #61467
Open
jperezdealgaba wants to merge 1 commit into
Open
jperezdealgaba wants to merge 1 commit into
jperezdealgaba wants to merge 1 commit into
Conversation
Assisted-by: OpenAI Codex Signed-off-by: Juan Pérez de Algaba <jperezde@redhat.com>
jperezdealgaba
requested review from
ApostaC,
WoosukKwon,
ZJY0516,
alexm-redhat,
heheda12345,
ivanium,
njhill,
orozery,
robertgshaw2-redhat and
ywang96
as code owners
October 13, 2026 18:15
This branch has not been deployed
This file contains hidden or bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
Sign up for free
to join this conversation on GitHub.
Already have an account?
Sign in to comment
Add this suggestion to a batch that can be applied as a single commit.This suggestion is invalid because no changes were made to the code.Suggestions cannot be applied while the pull request is closed.Suggestions cannot be applied while viewing a subset of changes.Only one suggestion per line can be applied in a batch.Add this suggestion to a batch that can be applied as a single commit.Applying suggestions on deleted lines is not supported.You must change the existing code in this line in order to create a valid suggestion.Outdated suggestions cannot be applied.This suggestion has been applied or marked resolved.Suggestions cannot be applied from pending reviews.Suggestions cannot be applied on multi-line comments.Suggestions cannot be applied while the pull request is queued to merge.Suggestion cannot be applied right now. Please check back later.
Overview
DiffusionGemma can finish prefill with prompt scores and no sampled tokens. The base scheduler treats scores on a no-output step as an invariant violation, so a normal prompt-scoring request terminates EngineCore. Keep the scores in the Diffusion scheduler until the request emits output, and discard them if that request is aborted or preempted.
Claims
Validation
VLLM_TARGET_DEVICE=cpu .venv/bin/python -m pytest tests/v1/core/test_scheduler.py -q --tb=short: 216 passed on macOS (using the existing workspace virtual environment).pre-commit run --files vllm/v1/core/sched/diffusion_scheduler.py tests/v1/core/test_scheduler.py: passed, including ruff and mypy.pre-commit run --all-files: failed on two unrelated macOS checks: mypy references Linux-onlyMADV_HUGEPAGE/posix_fadviseconstants, and the non-root entrypoint test compares/varwith its/private/varpath.Details
The model runner returns prompt scores at final prefill; DiffusionGemma samples zero tokens in that step.
Scheduler.update_from_outputthen asserts that no prompt scores exist because it emits no output. The fix is confined toDiffusionScheduler: it holds both score forms by request ID, attaches them to the first emitted output, and clears them when a request is aborted or preempted. Scores from an output the base scheduler would drop as stale are ignored. No API request fields or model computation change.Duplicate-work check: #60527 changes DiffusionGemma prompt-logit computation but leaves the scheduler assertion in place. The diffs of #58668, #52054, #50795, and #57597 address other scheduler or prompt-logprob paths. No open or merged PR in the sweep contains this missing scheduler control.
Pull Request Checklist
/pr-checklistskill.