[Feat][Stacked] Add Nemotron seeded decision reads to structured gateway

docs/models/supported_models.md

@@ -446,6 +446,7 @@ th {
 | `MixtralForCausalLM` | Mixtral-8x7B, Mixtral-8x7B-Instruct | `mistralai/Mixtral-8x7B-v0.1`, `mistralai/Mixtral-8x7B-Instruct-v0.1`, `mistral-community/Mixtral-8x22B-v0.1`, etc. | ✅︎ | ✅︎ |
 | `NemotronForCausalLM` | Nemotron-3, Nemotron-4, Minitron | `nvidia/Minitron-8B-Base`, `mgoin/Nemotron-4-340B-Base-hf-FP8`, etc. | ✅︎ | ✅︎ |
 | `NemotronHForCausalLM` | Nemotron-H | `nvidia/Nemotron-H-8B-Base-8K`, `nvidia/Nemotron-H-47B-Base-8K`, `nvidia/Nemotron-H-56B-Base-8K`, etc. | ✅︎ | ✅︎ |
+| `NemotronLabsDiffusionModel`, `NemotronLabsDiffusionForCausalLM` | Nemotron Labs Diffusion | `nvidia/Nemotron-Labs-Diffusion-3B`, `nvidia/Nemotron-Labs-Diffusion-8B` | | |
 | `OlmoHybridForCausalLM` | OLMo Hybrid | `allenai/Olmo-Hybrid-7B` | ✅︎ | ✅︎ |
 | `OlmoeForCausalLM` | OLMoE | `allenai/OLMoE-1B-7B-0924`, `allenai/OLMoE-1B-7B-0924-Instruct`, etc. | | ✅︎ |
 | `OPTForCausalLM` | OPT, OPT-IML | `facebook/opt-66b`, `facebook/opt-iml-max-30b`, etc. | ✅︎ | ✅︎ |
@@ -502,6 +503,9 @@ Some models are supported only via the [Transformers modeling backend](#transfor
 !!! note
     Currently, the ROCm version of vLLM supports Mistral and Mixtral only for context lengths up to 4096.
 
+!!! note
+    Nemotron Labs Diffusion runs masked block diffusion by default. `--diffusion-config` sets `canvas_length` (default: the checkpoint's `block_size`) and `model_options` such as `selection_policy` and `confidence_threshold`; request temperatures are applied as usual. Pass `--hf-overrides '{"ar_mode": true}'` (or select `NemotronLabsDiffusionForCausalLM`) for ordinary autoregressive generation with the same checkpoint.
+
 ## List of Multimodal Language Models
 
 The following modalities are supported depending on the model:

examples/features/structured_diffusion/README.md

@@ -73,3 +73,39 @@ thought in its prompt, so the answer slots condition on it. The noise draws
 of a decision share one thought. `diagnostics.thought` returns the text, its
 length in tokens, whether the model closed the channel itself and the
 generation time.
+
+## Nemotron Labs Diffusion
+
+The same `/v1/systemone` gateway can serve Nemotron Labs Diffusion 3B/8B with
+`--backend nemotron`; the [OpenAPI schema](systemone-openapi.yaml) lists the
+request shapes. This backend uses mask-token answer slots instead of random
+noise, one masked forward after prefill, and exact candidate logprobs. It does
+not reorder options or questions. Text/JSON state and choice, Noul and Score
+answers are supported; thinking, images and multiple samples are not.
+
+```bash
+VLLM_USE_V2_MODEL_RUNNER=1 vllm serve nvidia/Nemotron-Labs-Diffusion-8B \
+    --served-model-name nemotron --diffusion-config '{"canvas_length": 32}' \
+    --max-logprobs 128 --enable-prefix-caching
+python examples/features/structured_diffusion/structured_server.py --backend nemotron \
+    --upstream http://127.0.0.1:8000 --model nemotron \
+    --tokenizer nvidia/Nemotron-Labs-Diffusion-8B --canvas 32
+```
+
+A direct read sets `diffusion_read_only` and a `diffusion_seed_canvas` of
+exactly the served canvas length, with the mask token (`100`) at the answer
+positions, and requests the candidates with `logprob_token_ids` (up to 128).
+It needs `temperature=0`, default `top_p`/`top_k` and `max_tokens` no larger
+than the canvas; `diffusion_pinned`, a narrower `diffusion_canvas_length` and
+`diffusion_max_steps` other than 1 are not supported. Other requests are
+rejected with HTTP 400 before they reach a worker.
+
+Verify a running gateway with:
+
+```bash
+python examples/features/structured_diffusion/nemotron_smoke.py --url http://localhost:8011
+```
+
+The smoke check validates answer shapes and finite, normalized probabilities for
+all three primitives, a 100-alternative choice and a 12-question request. It is
+not an accuracy benchmark.

examples/features/structured_diffusion/nemotron_smoke.py

@@ -0,0 +1,97 @@
+# SPDX-License-Identifier: Apache-2.0
+# SPDX-FileCopyrightText: Copyright contributors to the vLLM project
+"""Exercise the existing System One gateway against a running Nemotron backend."""
+
+import argparse
+import json
+import math
+import os
+import time
+import urllib.request
+
+
+def main():
+    parser = argparse.ArgumentParser()
+    parser.add_argument("--url", default="http://localhost:8011")
+    args = parser.parse_args()
+    headers = {"Content-Type": "application/json"}
+    if key := os.environ.get("API_KEY"):
+        headers["Authorization"] = f"Bearer {key}"
+    cases = {
+        "choice": {
+            "route": {
+                "type": "choice",
+                "instructions": "Which team should handle this?",
+                "criteria": {
+                    "billing": "Payments and payouts",
+                    "technical": "Infrastructure outages",
+                },
+            }
+        },
+        "noul": {
+            "urgent": {"type": "noul", "instructions": "Has this lasted multiple days?"}
+        },
+        "score": {
+            "severity": {
+                "type": "score",
+                "instructions": "How disruptive is this problem?",
+                "criteria": [
+                    "No problem",
+                    "Minor inconvenience",
+                    "Blocked payments for multiple days",
+                ],
+            }
+        },
+        "wide_choice": {
+            "intent": {
+                "type": "choice",
+                "instructions": "Which intent matches this message?",
+                "criteria": {
+                    "payout_delay": "Payouts have been delayed or failed",
+                    **{
+                        f"intent_{i}": f"Unrelated request number {i}"
+                        for i in range(99)
+                    },
+                },
+            }
+        },
+        "twelve_questions": {
+            f"q{i}": {
+                "type": "noul",
+                "instructions": "Have payouts failed for multiple days?",
+            }
+            for i in range(12)
+        },
+    }
+    results = {}
+    for name, questions in cases.items():
+        body = {
+            "model": "jev-latest",
+            "state": "Customer payouts have failed for three days.",
+            "questions": questions,
+        }
+        started = time.perf_counter()
+        request = urllib.request.Request(
+            args.url.rstrip("/") + "/v1/systemone",
+            headers=headers,
+            data=json.dumps(body).encode(),
+        )
+        with urllib.request.urlopen(request, timeout=120) as response:
+            result = json.load(response)
+        elapsed = (time.perf_counter() - started) * 1000
+        assert result["answers"].keys() == questions.keys()
+        for answer in result["answers"].values():
+            if answer["type"] == "noul":
+                assert math.isfinite(answer["noul"]) and 0 <= answer["noul"] <= 1
+            else:
+                probabilities = answer["probabilities"]
+                assert all(
+                    math.isfinite(p) and 0 <= p <= 1 for p in probabilities.values()
+                )
+                assert math.isclose(sum(probabilities.values()), 1, abs_tol=1e-6)
+        results[name] = {"ms": round(elapsed, 2), "answers": result["answers"]}
+    print(json.dumps(results, indent=2))
+
+
+if __name__ == "__main__":
+    main()