[Feat][Stacked] Add Nemotron seeded decision reads to structured gateway

[pst2154]

Overview

Add seeded, single-forward Nemotron masked decision reads and adapt the existing structured diffusion example gateway for Choice, Noul and Score. This PR targets upstream main and depends on #58387 and #60287. Tracking: #60283.

Review the isolated decision delta. The main comparison temporarily includes the unmerged dependencies. This reuses the existing gateway instead of introducing a competing native endpoint.

Claims

• Keep fixed template tokens while scoring masked answer positions in one bidirectional forward after causal prefill, without a causal verification/commit pass.
• Return exact requested candidate logprobs through the existing indexed-logprob machinery, including candidates outside top-k.
• Validate seeded reads against the engine canvas before worker execution.
• Reuse structured_server.py --backend nemotron for the separate /v1/systemone example gateway. This is not a route built into vllm serve.

No ordering heuristics or linear-speculation code are included. The model-runner and its state interface are unchanged.

Validation

Rebased onto main 1a3a7a8f01a8c1a3184151b8185072efda11daab and the reduced dependency stack.

CPU-only tests of the complete stacked branch: 93 passed, 9 skipped. This runs the sampling-parameter suite and the Nemotron model unit/configuration suite with CUDA disabled. The nine skips are CUDA/checkpoint-gated cases, not GPU validation.

The full applicable pre-commit suite and System One OpenAPI schema validation passed. Fresh CUDA-serving and HTTP tests have not been rerun after this rebase. Upstream CI remains subject to its author/label pre-run gate.

Historical validation, before this rebase: H100 had 15 targeted checks pass; A100 and B200 each had 11. These included eager/compiled reads, low-probability indexed candidates, fixed-token preservation and slot reuse. H100 HTTP smoke returned complete Choice, Noul, Score and 12-question responses, rejected an invalid seed with HTTP 400, and preserved raw-chat passthrough. These are historical API checks, not a new accuracy benchmark.

.venv/bin/python -m pytest --noconftest tests/test_sampling_params.py \
  tests/models/language/generation/test_nemotron_dllm.py -q

NEMOTRON_DLM_MODEL_PATH=/path/to/Nemotron-Labs-Diffusion-8B \
VLLM_USE_V2_MODEL_RUNNER=1 VLLM_GPU_SYNC_CHECK=error \
  .venv/bin/python -m pytest --confcutdir=tests/models/language/generation \
  tests/models/language/generation/test_nemotron_dllm.py \
  -k 'read_only or invalid_reads' -v

.venv/bin/python examples/features/structured_diffusion/nemotron_smoke.py \
  --url http://localhost:8011

Details

The sampler uses staged writes, existing request-slot state and compute_topk_scores; read flags and seed canvases reset on slot reuse. The engine canvas is explicitly passed through admission validation. This validation plumbing belongs to the decision PR now that the former linear-speculation plumbing is removed from its dependency.

Reads require greedy masked diffusion, the full engine canvas and max_tokens <= canvas_length. The example supports text/JSON state, at most 64 questions and 26 alternatives, one compatible token per answer slot and one deterministic sample. Thinking, images and multi-step reads are unsupported. Returned probabilities are uncalibrated model scores. A graph boundary avoids the previously reproduced SM100 compiler issue but can add launches; no end-to-end speedup is claimed. RTX was not validated.

Rebase and rerun affected CUDA/HTTP validation after the dependencies land. Codex assisted with the implementation, rebase, checks and review update; human review and confirmation of relevant validation remain necessary before merge.

---

Pull Request Checklist

• I used vLLM's /pr-checklist skill.
• AI assistance was used during the creation of this PR.
• Design Fit: Existing sampler, indexed logprobs and gateway are reused; no scheduler/model-runner feature is added.
• Testing and Validation: Fresh CUDA/HTTP validation remains outstanding; current and historical checks are separated above.
• Code Quality and Style: Applicable checks and schema validation pass; documentation reflects the reduced scope.
• Pull Request Contents: Dependencies, usage, evidence and limitations are documented.

[claude]

Claude Code Review

This pull request is from a fork — automated review is disabled. A repository maintainer can comment @claude review to run a one-time review.

[mergify]

Documentation preview: https://vllm--60288.org.readthedocs.build/en/60288/

[mergify]

This pull request has merge conflicts that must be resolved before it can be
merged. Please rebase the PR, @pst2154.

https://docs.github.com/en/pull-requests/collaborating-with-pull-requests/working-with-forks/syncing-a-fork

[pst2154]

Codex update (AI-assisted): rebased the decision-read follow-up onto current upstream main and the updated fixes branch from #60287. The PR no longer has merge conflicts. Seeded reads, indexed candidate probabilities, and the existing separate System One example gateway remain intact; no ordering heuristics were added. The original model-support PR #58387 was not modified.

The complete local pre-commit suite passed, and the System One OpenAPI schema validated. Historical GPU/HTTP results in the description are not fresh validation of this rebase; GPU serving tests have not been rerun. Upstream CI remains behind its existing author/label pre-run gate. Dependency order remains #58387, then #60287, then this PR.

Updated isolated decision comparison: pst2154/vllm@37e0476...acb61e5

[pst2154]

Codex update: restacked decision reads on the reduced #58387 and updated #60287. Seeded one-forward reads, indexed candidate probabilities and the separate existing System One gateway remain. Canvas-aware request validation is retained in this PR, without the removed linear-speculation plumbing. No ordering heuristics or model-runner changes were added. The description and isolated comparison link are updated.

Current full-stack CPU/unit checks: 93 passed, 9 CUDA/checkpoint-gated skips. Full local pre-commit and OpenAPI validation passed. Fresh CUDA/HTTP serving checks remain outstanding; the description distinguishes them from historical results.