[Feat] Add Nemotron seeded decision reads to the structured gateway

[pst2154]

Purpose

Add seeded, single-forward decision reads to the Nemotron masked-diffusion sampler, and reuse the existing structured diffusion gateway for Choice, Noul and Score requests.

This is the third part of the model-support stack: upstream vllm-project/vllm#58387, then sajadn/vllm#1, then this change. The base is feat/nemotron-full-model-followup; this diff does not reintroduce the full model.

• Seed canvases keep fixed template tokens while scoring masked answer positions in one bidirectional forward after prompt prefill. Reads finish without a causal verification/commit pass.
• Requested candidate logprobs use the existing indexed-logprob machinery, including candidates outside top-k. Invalid read parameters fail before GPU execution with HTTP 400.
• structured_server.py --backend nemotron reuses the existing /v1/systemone route. This remains a separate example gateway, not a new native route in vllm serve. No ordering heuristics are included.

Duplicate-work searches on October 6, 2026 identified full-model PRs vllm-project#58387/vllm-project#48279, the existing gateway, and #59299. vllm-project#59299 currently adds native AR Choice reads and explicitly defers diffusion; this change supplies Nemotron's masked read primitive and adapts the existing example rather than duplicating that endpoint. #58429 targets Laya, not Nemotron.

Test Plan

Use the public nvidia/Nemotron-Labs-Diffusion-8B checkpoint at revision 16c67f0560b912e93e0cabb6e0c4f5c3086d95fc and a CUDA-capable editable vLLM installation. Set NEMOTRON_DLM_MODEL_PATH to its local snapshot.

VLLM_USE_V2_MODEL_RUNNER=1 VLLM_GPU_SYNC_CHECK=error \
  .venv/bin/python -m pytest \
  tests/models/language/generation/test_nemotron_dllm.py \
  --confcutdir=tests/models/language/generation \
  -k 'read_only or invalid_reads or linear_spec_rejects' -v

The tests cover a seeded read mixed with normal generation, indexed low-probability candidates, fixed-token preservation, eager/compiled execution, and slot reuse. BF16 batch-versus-single probabilities are compared within 0.02 absolute probability, not bit-exact logits.

Follow the model guide to launch the raw server and gateway, then run:

.venv/bin/python examples/features/structured_diffusion/nemotron_smoke.py \
  --url http://localhost:8011

Run the affected-file pre-commit hooks and validate the included OpenAPI schema.

Test Result

• A100 80GB: 11 targeted checks passed, including eager and compiled GPU reads.
• B200: 11 targeted checks passed, including eager and compiled GPU reads.
• H100: 15 targeted decision/generation checks passed, including eager/compiled reads, normal greedy generation, stochastic rollouts and both mixed-logprob regressions. Compiled HTTP smoke passed for Choice, Noul, Score and all 12 questions. Invalid seed length returned HTTP 400; raw chat passthrough answered 2+2 with 4..
• Shared unit/configuration checks: 31 passed, 2 skipped, 9 deselected. Skips are checkpoint/reference-gated cases, not successful GPU evaluations.
• All applicable pre-commit hooks and OpenAPI schema validation passed.

The HTTP checks establish API behavior, finite normalized scores and answer completeness, not benchmark accuracy or improved model quality. No end-to-end speedup is claimed. Previous full-model reference parity belongs to the predecessor PR, not a new quality evaluation of these decision reads.

Details and limitations

The sampler reuses staged writes, existing sampling state and compute_topk_scores; no scheduler or model-runner changes. Seed/read flags reset when slots are reused. A graph boundary separates argmax from downstream gather/reductions because Inductor generated an undefined reduction temporary on SM100 in the mixed read/generation reproducer. This can add kernel launches; no acceleration claim is made.

Native reads require greedy masked diffusion, the full engine canvas, and max_tokens <= canvas_length. Linear speculation rejects read-only/indexed-score requests. The example supports text or JSON state, at most 64 questions and 26 alternatives, one compatible token per answer slot, and one deterministic sample. Thinking, images and multi-step reads are unsupported. Probabilities are uncalibrated model scores. RTX 6000 Pro was not validated.

Codex assisted with implementation, tests, documentation and this description. The submitting human confirmed review of this diff before publication; validation results above were obtained with Codex assistance.

---

Essential Elements of an Effective PR Description Checklist

• The purpose of the PR.
• The test plan, including reproduction commands.
• The final test results.
• Necessary documentation and examples.

Tracking issue for the full three-part stack: vllm-project#60283

[pst2154]

Codex assisting @pst2154: this incorrectly targeted fork PR is superseded by vllm-project#60288, targeting upstream main. The branch and reviewed commit are preserved. Tracking: vllm-project#60283