docs: list the DiffusionGemma-26B-A4B-it image and add its serving recipe

[yeahdongcn]

What

Docs only, no code changes:

• README.md / README_CN.md — add DiffusionGemma-26B-A4B-it to the "new model support images" list, pointing at registry.mthreads.com/mcconline/inference/vllm/vllm-openai:jev.
• docs/cookbook/diffusiongemma/diffusiongemma-26b-a4b-it.md — new serving recipe, linked from docs/cookbook/README.md (one row in the new ## DiffusionGemma section).

The image

tag	registry.mthreads.com/mcconline/inference/vllm/vllm-openai:jev
digest	sha256:7bd8f1935c1b7aa40a8bf0d90fdcda08ab09178592a45222c719b5a9d1696c20
contents	the v0.28.0 release image plus the DiffusionGemma structured-read port from #249 (0171..0181)
endpoints	8000 vLLM OpenAI API · 18011 /v1/systemone
entrypoint	serves the model, waits for /v1/models, then starts the interposer
size	21.7 GB, 26 layers

Why a recipe is worth having here

DiffusionGemma is not a normal chat model. A request denoises a canvas of 256 tokens in 48 steps and
commits the whole canvas, and the per-request knobs (diffusion_canvas_length, diffusion_seed_canvas,
diffusion_pinned, diffusion_read_only, diffusion_max_steps) live in vllm_xargs. On top of that,
/v1/systemone is not a vLLM endpoint — upstream vllm-project#57250 ships the machinery plus a sample
interposer, and vLLM itself has no such route. Without a recipe, a user pulling the image has no way to
know the interposer exists, which port it binds, or what the request shape is. The recipe covers: both
launch paths (published image, or an installed vLLM-MUSA), plain generation, the /v1/systemone
request/response shape (including that score is the expectation over the ordered scale, and that
diffusion_pinned constrains the model's input rather than echoing tokens back), the client CLI that
ships in the image, and the per-request knob table.

Validation behind the entry

• one S5000, MUSA 5.2.0, torch/torch_musa 2.11.0.post1+musa5.2.0, image digest sha256:b395fa92…
• upstream's read tests run inside the image: 60 passed, 4 xfailed
• end-to-end: a three-question read in one canvas (noul 0.956, choice billing 0.998, score argmax
  critical 0.921), 4 averaged reads, 3657 ms; --quiet and --json client modes both exercised
• knobs exercised: canvas narrowing, seed canvas, four pinned positions returning their seed id,
  read_only + max_steps: 1; out-of-vocabulary seeds rejected at admission

Caveats stated in the recipe

• The image layers the MUSA: DiffusionGemma structured reads (port of upstream PR #57250) #249 port on the release image while MUSA: DiffusionGemma structured reads (port of upstream PR #57250) #249 is unmerged; the recipe says to rebuild
  from the branch once MUSA: DiffusionGemma structured reads (port of upstream PR #57250) #249 lands instead of keeping the layered artifact.
• The four xfail cases assert behaviour at the compiled sampling step; this branch emits read-only
  canvases in the host loop, so they are carried as xfail(strict=False).
• Not covered: TP>1, concurrent requests, vision/image inputs, canvases other than the model's 256.

Rebuilt on 2026-09-24

The port was rebased and reworked, so the layered artifact was rebuilt from the same base image and
re-pushed under the same tag: sha256:7bd8f193… (rebuilt again after the #249 series was rebased onto the current base and renumbered to entries 0172-0186). It now also raises --max-logprobs 32 in its
entrypoint, which a choice schema with more than 20 alternatives needs (the pinned request path
requires logprobs == len(logprob_token_ids) and bounds logprobs by max_logprobs). The recipe
states the #249 precondition at the path choice and in the cookbook index, corrects the read-cost
wording, and documents --max-logprobs for the from-source path.

[yeahdongcn]

Merge verdict: 1 blocking finding — not ready to merge.

Reviewed at head 206b3f63d2ec8d3df9168767c6a281a351489784 (unchanged from dispatch), base v0.28.0-dev, docs-only: 4 files, +195/-0. The recipe is accurate about the model, the endpoints and the request shapes; the blocking item is a missing precondition on the non-image launch path.

What this PR does

• README.md:26 / README_CN.md:26 add the DiffusionGemma-26B-A4B-it row pointing at registry.mthreads.com/mcconline/inference/vllm/vllm-openai:jev.
• docs/cookbook/README.md:57-65 adds a ## DiffusionGemma section.
• docs/cookbook/diffusiongemma/diffusiongemma-26b-a4b-it.md is new: two launch paths, plain generation, /v1/systemone, the vllm_xargs knob table, config notes, validation and limits.

flowchart TD
    A["README.md / README_CN.md: new model image row"]:::changed --> B["docs/cookbook/README.md: new DiffusionGemma section"]:::changed
    B --> C["diffusiongemma/diffusiongemma-26b-a4b-it.md: new recipe"]:::changed
    C --> D{"Reader picks a launch path"}
    D -->|"Option A: the :jev image"| E["entrypoint: vllm serve, wait for /v1/models, start interposer on 18011"]:::changed
    D -->|"Option B: installed vLLM-MUSA"| F["vllm serve --diffusion-config: base stack has DiffusionConfig + TRITON_ATTN pinning"]:::changed
    F --> G["python structured_server.py --port 18011"]:::changed
    G --> H["vllm_xargs read knobs: diffusion_seed_canvas, diffusion_pinned, diffusion_read_only, diffusion_max_steps"]:::changed
    H -.->|"read path exists only in unmerged #249"| I["/v1/systemone answer distributions"]:::changed
    L["Legend: dashed border = added or modified by this PR"]:::changed
    classDef changed stroke-dasharray:5 5,stroke-width:2px;

Loading

The recipe's entrypoint is a human reader, not a server: the index row leads to the recipe, the recipe offers Option A (published image) or Option B (own install), and the structured-read half of the document runs through the interposer, which drives vLLM through vllm_xargs. Option B is where the flow breaks — its serve half works on this branch, its interposer half needs a PR that is not merged.

Historical review synthesis

Exhaustive corpus sweep: Scanned 32639 threads ... matched 1353 threads across 417 PRs, 2010 human comments for paths docs/README.md with queries docs, recipe, model support, server args. docs-examples is the dominant category (1353/1353) and the recurring maintainer instinct on documentation PRs is that every documented argument, endpoint and version claim be checked against the code that actually ships — e.g. simveit on sgl-project/sglang#5276, "check with server args script". Findings below follow that standard: what the recipe promises versus what the branch and the pinned stack can do.

Findings

P1 — Option B cannot serve the recipe's structured reads on the branch it merges into

docs/cookbook/diffusiongemma/diffusiongemma-26b-a4b-it.md:48 and :65-66:

### Option B — from an installed vLLM-MUSA
...
Then start the interposer against it (the file is upstream's `examples/features/structured_diffusion/structured_server.py`,
also at `/opt/jev/structured_server.py` inside the image):

and the knob table it feeds, :142-150:

| `diffusion_seed_canvas` | seed tokens for the canvas, one id per position, in vocabulary |
| `diffusion_pinned` | positions that keep their seed token **as input** |
| `diffusion_read_only` | read the canvas without generating further steps |

Verified against the branch (not against a local checkout):

• third_party/PINS pins VLLM_COMMIT=2cf0a6915ce544dc493a0990f2ea38d81601128a; that commit has no vllm/utils/diffusion.py and no vllm/v1/core/sched/diffusion_scheduler.py, and its vllm/v1/engine/input_processor.py contains zero occurrences of diffusion. Those are exactly the admission-validation and scheduler pieces the knobs need.
• The patch series at head ends at vllm_musa/patches/series/0170-MUSA-bound-DSV4-long-prefill-indexer-logits.patch (171 entries including README.md). The diffusion patches are 0171–0182 and live only in #249, which is open and draft ({"isDraft":true,"state":"OPEN"}): 0174-MUSA-vllm.utils.diffusion.patch, 0175-MUSA-vllm.v1.engine.input_processor.patch, 0180-MUSA-vllm.v1.worker.gpu.model_runner.patch.

Trigger: follow Option B on this branch, then send any request carrying diffusion_seed_canvas / diffusion_pinned / diffusion_read_only / diffusion_max_steps. The flags are not rejected, they are simply absent from the code path, so the request denoises from a random canvas and /v1/systemone cannot return the documented distributions (noul 0.956, choice billing 0.998, score 2.92) — the read is silently wrong rather than an error. The only #249 note is :179-181, and it is framed as an image-build detail ("The image is built by layering the structured-read port ... on the v0.28.0 release image"), not as an Option B precondition.

Note the serve half is fine and should not be changed: the pin ships vllm/config/diffusion.py (canvas_length, max_denoising_steps) with the --diffusion-config / -dc flag in vllm/engine/arg_utils.py, and vllm_musa/platform.py at head already has force_triton_attn_for_diffusion, so vllm serve ... --diffusion-config ... and plain generation work.

Suggested change: state the precondition where the reader chooses the path, for example

### Option B — from an installed vLLM-MUSA

Requires the structured-read port from MooreThreads/vllm-musa#249 on top of v0.28.0; the
release image does not carry it. Use the `:jev` image, or apply #249 before starting the
interposer.

and repeat it once in the index section (docs/cookbook/README.md:57-61, which currently says the recipes "additionally expose the structured-read interposer on port 18011" without saying where that support comes from).

P2 — "one denoising pass" understates the cost by the averaged sample count

docs/cookbook/diffusiongemma/diffusiongemma-26b-a4b-it.md:125-126:

All questions in one request are read in a **single canvas** (one denoising pass), so a decision
with three questions costs about as much as one generation.

Trigger: the default samples: auto policy. The same recipe's own response at :117 reports

"diagnostics": {"steps": 1, "timing": {"reads": 4, "total_ms": 3657}},

and upstream structured_server.py builds that field as "timing": {"total_ms": elapsed_ms, "reads": n} with n = len(reads) from read_many(...) (MAX_SAMPLES = 32, samples = "reads per question, fixed or auto"). Each read is a joint denoising pass over the canvas, so a three-question decision costs diagnostics.timing.reads passes (4 in the documented example), not one.

Suggested change: "All questions of a stage are read in one joint canvas, so three questions cost one read instead of three; the server averages several draws per question (samples, default auto), and diagnostics.timing.reads is how many passes that took."

P2 — --max-logprobs is missing from Option B (unconfirmed for the documented example)

docs/cookbook/diffusiongemma/diffusiongemma-26b-a4b-it.md:55-62 ends at:

  --diffusion-config '{"canvas_length": 256, "max_denoising_steps": 48}'

Upstream's own example for this interposer passes --max-logprobs 32 (module docstring of examples/features/structured_diffusion/structured_server.py, and examples/features/structured_diffusion/README.md:22). The interposer sends "top_logprobs": TOPK with TOPK = 20 and "logprob_token_ids": label_id_union(slots); the pinned vLLM defaults to max_logprobs: int = Field(default=20, ge=-1) (vllm/config/model.py:245) and SamplingParams._validate_logprobs rejects a request above that cap. The interposer's own 20 therefore sits exactly on the default ceiling with no headroom: a schema whose single-token label union exceeds 20 ids gets a 400. Unconfirmed for the three-question example in this recipe, whose union is about ten label ids and does fit; it would be confirmed by running a schema with more than 20 distinct label tokens against the recipe's command.

Suggested change: add --max-logprobs 32 to Option B (matching upstream), or note in the override list at :44-46 that the image sets it.

P0: no findings.

Checked and clean

• No --async-scheduling needed, despite upstream's "per-request canvas widths smaller than the served canvas require async scheduling" rule. vllm/config/vllm.py:1185-1234 in the pin resolves async_scheduling=None to True when nothing conflicts, and #249's 0172-MUSA-vllm.config.vllm.patch auto-selects DiffusionAsyncScheduler when diffusion_config is set with scheduler_cls is None. The recipe's omission is correct on the default path.
• Attention backend claim holds. --attention-backend TRITON_ATTN is a real value (vllm_musa/platform.py:204+ force_triton_attn_for_diffusion, which also overrides an explicit FLASH_ATTN because it would silently ignore the per-request tensor causal mask).
• /v1/systemone request/response shapes match upstream's interposer: {"model", "state", "questions"} with type in noul/choice/score, criteria as a name→description map for choice and an ordered list for score, answers noul/choice+probabilities+confidence/score+legend+probabilities+confidence, and "labels must be single tokens". The score arithmetic is internally consistent: 2·0.079 + 3·0.921 = 2.921, matching "score is the expectation over the ordered scale".
• Links resolve: the new recipe exists at the index's target path, its ../README.md back-link and both index rows point at it, and README.md:26 / README_CN.md:26 stay parallel and use the same tag.
• Comment-style rule: no ticket numbers, patch cross-references or history narration are introduced; the #249 / #57250 references are user-facing provenance for a non-reproducible artifact, which is the correct place for them.

Could not verify

• Everything inside the image, which is the part the operator actually runs: digest sha256:b395fa92…, port 18011, the entrypoint and its MODEL/SERVED_NAME/PORT/API_PORT/CANVAS/DENOISE_STEPS/GMEM/MAX_LEN/BACKEND overrides, /opt/jev/structured_server.py and /opt/jev/systemone.py (including that CLI's --quiet/--json/--no-proxy behaviour), "60 passed, 4 xfailed", and the timing/accuracy numbers. Upstream vLLM ships only README.md and structured_server.py in that directory — there is no systemone.py anywhere in vllm-project/vllm main — and no Dockerfile at this head builds a :jev tag, so none of it is reachable from public source. Confirmation would be the recorded docker run / docker exec commands plus their logs.
• Model facts: canvas 256 / 48 denoising steps, ≈25.8B parameters, max context 4,096, and the google/diffusiongemma-26B-A4B-it checkpoint page. No network to the model host or Hugging Face from this session.
• No CI: gh pr checks 259 --repo MooreThreads/vllm-musa returns no checks reported on the 'dsh/musa-100048-image-doc' branch. For a docs-only diff that is not by itself a blocker.
• Minor body note, not a file finding: the PR body says the image layers 0170..0181, while #249's series currently contains 0171–0182 (12 patches).

auto-review round 1/3 · verdict: blockers

[yeahdongcn]

Merge verdict: 1 blocking finding — not ready to merge.

Reviewed at head e20c68b47fe881b84e57241130d9e534397aa771; the head has moved since round 1 (which was posted at 206b3f63d2ec). The delta is one commit, docs(cookbook): fix what following the recipe revealed (+19/-5, same file), which fixes the max_tokens claim, adds the host-port remap note, the run-to-run variability paragraph and the proxy/retry note. Base v0.28.0-dev, docs only: 4 files, +209/-0.

What this PR does

• README.md:26 / README_CN.md:26 add a DiffusionGemma-26B-A4B-it row pointing at registry.mthreads.com/mcconline/inference/vllm/vllm-openai:jev.
• docs/cookbook/README.md:57-65 adds a ## DiffusionGemma section.
• docs/cookbook/diffusiongemma/diffusiongemma-26b-a4b-it.md is new: two launch paths, plain generation, /v1/systemone, the vllm_xargs knob table, config notes, validation and limits.

flowchart TD
    A["Recipe: diffusiongemma-26b-a4b-it.md"]:::changed --> B{"Reader picks a launch path"}
    B -->|"Option A"| C["docker run registry.mthreads.com/...:jev"]:::changed
    C --> D["entrypoint: vllm serve, wait for /v1/models, start interposer on 18011"]:::changed
    B -->|"Option B"| E["vllm serve ... --diffusion-config canvas 256 / 48 steps"]:::changed
    E --> F["python structured_server.py --upstream ... --port 18011"]:::changed
    F --> G["read body: vllm_xargs diffusion_seed_canvas / _pinned / _read_only / _max_steps"]:::changed
    G --> H["/v1/systemone answer distributions"]:::changed
    G -.->|"the four xargs have no consumer on this branch without #249"| I["unseeded canvas: read is not the documented one"]:::changed
    L["Legend: dashed border = added or modified by this PR"]:::changed
    classDef changed stroke-dasharray:5 5,stroke-width:2px;

Loading

The entrypoint here is a human reader, not a server: the README row leads to the index, the index leads to the recipe, and the recipe offers Option A (published image) or Option B (own install). Both paths end at the same read protocol — an interposer that drives vLLM through vllm_xargs and returns per-question distributions on 18011. Option B is where the flow breaks, and that is the P1 below; the dashed edge is the break.

Historical review synthesis

Exhaustive sweep of the local PR-review corpus for docs/README.md paths and docs/recipe/model support/server args/deployment queries: Scanned 32639 threads ... matched 1353 threads across 417 PRs, 2010 human comments — docs-examples is the dominant category (1353/1353). The recurring maintainer instinct on documentation PRs (e.g. the cookbook-port series) is that every documented flag, endpoint, default and version claim is checked against the code that actually ships, and that any "validated" number comes with the exact command and output behind it. The findings below apply that standard to what the recipe promises versus what this branch and the pinned stack provide.

Findings

P1 — Option B cannot serve the recipe's structured reads on the branch it merges into (carried over from round 1, unchanged at the new head)

docs/cookbook/diffusiongemma/diffusiongemma-26b-a4b-it.md:52 and :69-70:

### Option B — from an installed vLLM-MUSA
...
Then start the interposer against it (the file is upstream's `examples/features/structured_diffusion/structured_server.py`,
also at `/opt/jev/structured_server.py` inside the image):

and the index section, docs/cookbook/README.md:59-61:

These recipes use the model's own diffusion canvas (`canvas_length 256`,
`max_denoising_steps 48`) and additionally expose the structured-read interposer
on port 18011.

Verified against the head commit (via the GitHub API, not a local checkout):

• third_party/PINS pins VLLM_COMMIT=2cf0a6915ce544dc493a0990f2ea38d81601128a (2026-08-24). That commit predates upstream's merge of vllm-project#57250 (merged_at=2026-09-22T20:51:18Z, merge commit 1b3b88ec, diverged, ahead_by=1718). At the pin: vllm/utils/diffusion.py and vllm/v1/core/sched/diffusion_scheduler.py are 404, vllm/v1/engine/input_processor.py contains zero occurrences of diffusion (no admission validation for the read knobs), and the DiffusionAsyncScheduler auto-selection in vllm/config/vllm.py is itself a vllm-project#57250 addition — so per-request canvas widths have no scheduler either.
• The head's patch series (vllm_musa/patches/series, 171 entries) ends at 0170-MUSA-bound-DSV4-long-prefill-indexer-logits.patch. The diffusion patches are 0171–0182 and exist only in #249, which is still open (state=open, mergeable=true, 13 files, head 84d15ee5).

Trigger: on this branch (or any install built from it), follow Option B and send the recipe's own three-question body. The four keys travel in vllm_xargs with nothing reading them at the pin: the canvas is not seeded from the schema, diffusion_read_only / diffusion_max_steps do not apply, and diffusion_canvas_length has no per-request width support. The interposer then reads logprobs at positions that are not the answer slots and returns a well-formed distribution that is not the documented read. Whether it surfaces as that wrong answer or as an error inside the interposer (the logprob hand-out at the committing step is also part of vllm-project#57250) I could not determine without hardware — either way the documented contract does not hold, and the admission rejection the recipe advertises at :189-190 ("out-of-vocabulary seeds are rejected at admission") is not there either.

The only mention of #249 is at :193-195, framed as an image-build detail ("The image is built by layering the structured-read port ... so rebuild once that PR is merged"), not as a precondition for Option B. The serve half is fine and should not change: the pin ships vllm/config/diffusion.py (canvas_length, max_denoising_steps), the --diffusion-config / -dc flag in vllm/engine/arg_utils.py, model_config.is_diffusion (vllm/config/model.py:1765), and vllm_musa/platform.py:204 already has force_triton_attn_for_diffusion — I did not run it, but nothing in the diff contradicts it.

Suggested change — state the precondition where the reader chooses the path:

### Option B — from an installed vLLM-MUSA

Requires the structured-read port from MooreThreads/vllm-musa#249 on top of v0.28.0; this
branch does not carry it. Use the `:jev` image, or apply #249 before starting the interposer.

and repeat it once in the index section (docs/cookbook/README.md:57-61). If #249 is expected to land before this one is read, say that instead.

P2 — "one denoising pass" understates the documented cost (carried over from round 1, not addressed by the new commit)

docs/cookbook/diffusiongemma/diffusiongemma-26b-a4b-it.md:135-136:

All questions in one request are read in a **single canvas** (one denoising pass), so a decision
with three questions costs about as much as one generation.

The recipe's own response at :121 reports "diagnostics": {"steps": 1, "timing": {"reads": 4, "total_ms": 3657}} and its CLI example at :143 ends in [reads=4]. In upstream's interposer each read is one upstream /v1/chat/completions call over the joint canvas, and the default samples: auto policy issues one read and then up to auto_max - 1 = 3 more (auto_max defaults to 4, MAX_SAMPLES = 32, and timing.reads is len(reads)). So a three-question decision costs 1–4 denoising passes, not one — the new "averages a few independent reads" paragraph three lines above makes the same point, which makes this sentence read as a contradiction rather than a simplification.

Suggested change: "All questions of a stage are read in one joint canvas, so three questions cost one read instead of three; the server averages several draws per question (samples, default auto), and diagnostics.timing.reads is how many passes that took."

P2 — --samples / --extra auto_max= are not knobs of the interposer the recipe starts

docs/cookbook/diffusiongemma/diffusiongemma-26b-a4b-it.md:132:

runs. Use `--samples` (or `--extra auto_max=...`) to average more reads when a yes/no margin has to
carry a decision, and treat a single yes/no read as a weak signal.

This sentence sits in the curl section, between the /v1/systemone body example and the point at :137 where /opt/jev/systemone.py is introduced, but neither tool documented so far takes those flags. The interposer the image ships is byte-identical to upstream's merged structured_server.py (sha256 7cd9aa0081090c064eaac28db0f54f81…), and its argparse is exactly --upstream --model --tokenizer --canvas --canvas-step --host --port --tls-port --cert-dir. Its own knobs are request-body extension keys — JEV_EXTENSIONS = ("instructions", "samples", "auto_max", "auto_threshold", "steps", "think", "ask", "chunk_rows", "chunk_prompt", "sequential") — so a reader of the curl examples needs "samples": 8 in the JSON body, and a reader of Option B's structured_server.py has no CLI flag at all. --samples / --extra KEY=VALUE belong to the bundled client /opt/jev/systemone.py.

Suggested change: name the tool and give the body form, e.g. "Use samples (or auto_max) in the request body — or --samples / --extra auto_max=… on the bundled systemone.py — to average more reads…".

P2 — the two READMEs still enumerate only two recipe families

README.md:42 and README_CN.md:41:

- Per-checkpoint recipes for Qwen and DeepSeek-V4-Flash on S5000.

This PR edits both files (the image-list row at :26) and adds a third family to docs/cookbook/README.md:57, but the sentence that tells a reader which recipes exist is left at two. Suggested change: add DiffusionGemma to the sentence, or drop the enumeration so it cannot drift again.

P0: no findings.

Retired from round 1

Round 1's third item — "--max-logprobs is missing from Option B" — does not hold, and I am withdrawing it. The interposer sends "top_logprobs": TOPK with TOPK = 20; the pin's default is max_logprobs: int = Field(default=20, ge=-1) (vllm/config/model.py:245) and _validate_logprobs rejects only num_logprobs > max_logprobs, so 20 passes exactly at the ceiling. logprob_token_ids is bounded separately at MAX_LOGPROB_TOKEN_IDS = 128 (vllm/sampling_params.py:30), not by max_logprobs, so round 1's "label union above 20 ids gets a 400" trigger is wrong. The validated :jev entrypoint passes neither --max-logprobs nor upstream's --enable-prefix-caching, so the recipe's Option B is consistent with the measured artifact.

Checked and clean

• Model facts hold against the checkpoint itself (huggingface.co/google/diffusiongemma-26B-A4B-it returns 200; config.json has architectures: ["DiffusionGemmaForBlockDiffusion"] and canvas_length: 256; generation_config.json has max_denoising_steps: 48 and max_new_tokens: 256). "Maximum context | 4,096 tokens" is correctly labelled (recipe value) — it is the served --max-model-len, not the checkpoint's limit.
• Both launch-path commands use real flags. --diffusion-config '{"canvas_length": 256, "max_denoising_steps": 48}' matches the pinned DiffusionConfig fields and the --diffusion-config / -dc option (vllm/engine/arg_utils.py:1626); --attention-backend TRITON_ATTN is consistent with vllm_musa/platform.py:204 force_triton_attn_for_diffusion, which also overrides an explicit FLASH_ATTN; the interposer command's --upstream/--model/--tokenizer/--canvas/--host/--port all exist.
• /v1/systemone request/response shapes match the interposer: {"model","state","questions"} with noul/choice/score, criteria as a name→description map for choice and an ordered list for score, score as the expectation over the ordered scale (2·0.079 + 3·0.921 = 2.921), and a 0-indexed legend ({str(i): level}).
• The image-facing claims I could cross-check agree. The entrypoint override list at :49-50 (MODEL, SERVED_NAME, PORT, API_PORT, CANVAS, DENOISE_STEPS, GMEM default 0.75, MAX_LEN, BACKEND), the 18011 interposer port, the "waits for /v1/models" ordering, /opt/jev/structured_server.py and PYTORCH_MUSA_ALLOC_CONF=expandable_segments:True all match the scaffolding that builds the tag in this workspace (docker/jev-systemone/), which records the same digest sha256:b395fa92…; PYTORCH_MUSA_ALLOC_CONF is a real torch_musa variable.
• Links and parallelism: the recipe exists at both index targets, the ../README.md back-link resolves, the EN/CN rows use the same tag, and the new index section follows the existing section style.
• Comment-style rule: no ticket numbers, patch references or history narration are introduced. The #249 / #57250 mentions are user-facing provenance for an artifact that cannot be reproduced from the repo, which is the right place for them.

Could not verify

• Everything inside the image: the registry digest, tag, "21.7 GB, 26 layers", the entrypoint env behaviour, /opt/jev/systemone.py (including --quiet/--json/--no-proxy and its printed line format) and the /opt/jev paths. registry.mthreads.com is not reachable from this session, and there is no Dockerfile at this head that builds :jev. What I did read is the author-side scaffolding, which is not an independent pull.
• All hardware numbers: startup ≈5 min, diagnostics.timing.reads = 4 / 3657 ms, the 0.956 / 0.998 / 0.921 answers, the 0.36 / 0.79 / 0.96 spread, "60 passed, 4 xfailed", and the 247/15 max_tokens measurement. No GPU here. The workspace bundle generated/MUSA-100048/functional-image.md records all of them (the read at lines 228-231, the test run at line 75) and states the measured artifact is the pulled image; I did not reproduce any of it.
• No CI: gh pr checks 259 --repo MooreThreads/vllm-musa returns no checks reported on the 'dsh/musa-100048-image-doc' branch, and the repo has no .github directory at this head (404), so there is nothing to run. Not a gap this PR introduces.
• PR body only, not a file finding: the body says the image layers the port as 0170..0181, while #249's series is currently 0171–0182 (12 patches) — consistent with the rebase described in the image record, and harmless as long as it does not reach a shipped file.

auto-review round 2/3 · verdict: blockers

[yeahdongcn]

Merge verdict: no blocking findings — ready to merge.

Reviewed at head 5db510dff0e6b00c6851a442f5a5dd943bfafbe6. The head has moved twice since dispatch: round 1 was posted at 206b3f63d2ec, round 2 at e20c68b47fe8, and the delta is one commit, docs(cookbook): state the #249 precondition, the read cost and --max-logprobs. Base v0.28.0-dev, docs only: 4 files, +223/-0. This is the terminal round (3 of 3): the round-1 blocker is fixed at this head, nothing in CI blocks, and what remains are four P2 nits. A human can merge.

Earlier rounds

Round 1 P1 — "Option B cannot serve the recipe's structured reads on the branch it merges into" — fixed at this head. The precondition is now where the reader chooses the path (docs/cookbook/diffusiongemma/diffusiongemma-26b-a4b-it.md:54-57):

Requires the structured-read port from MooreThreads/vllm-musa#249 on top of `v0.28.0`: this branch
does not carry it, and without it the four `diffusion_*` request knobs have no reader (the request is
accepted and silently read from an unseeded canvas, so `/v1/systemone` would return a well-formed but
wrong distribution).

and again in the index (docs/cookbook/README.md:61-64). I re-checked the premise live rather than trusting the earlier text:

• #249 is still state=OPEN, isDraft=true (head 111a2d30, 12 files, 0171–0181), so "this branch does not carry it" is accurate.
• The pin is unchanged: third_party/PINS still has VLLM_COMMIT=2cf0a6915ce544dc493a0990f2ea38d81601128a. At that commit vllm/config/diffusion.py exists with only canvas_length and max_denoising_steps (which is why the serve half works), vllm/v1/core/sched/diffusion_scheduler.py is 404, and vllm/v1/engine/input_processor.py has zero occurrences of diffusion — no reader, no admission validation for the four read knobs, and no rejection. The stated consequence ("silently read from an unseeded canvas") matches the code path.
• The head's patch series still ends at 0170, so #249 is indeed the missing half.

Round 2 P1 was the same item, carried; it is now fixed. Of round 2's three P2s, the read-cost sentence is fixed (:145-147, "one joint canvas … averages several draws … diagnostics.timing.reads is how many passes that took"), and two are carried below because they are still present at this head.

Findings

P2 — the --max-logprobs 32 rationale is contradicted by the pinned vLLM

docs/cookbook/diffusiongemma/diffusiongemma-26b-a4b-it.md:75-77:

`--max-logprobs 32` matches upstream's own example for this interposer: the interposer asks for 20
top logprobs and for the logprobs of every label token in the schema, and the pinned vLLM's default
`max_logprobs` is 20, which leaves the label union no headroom.

The first clause is right — upstream's example for this interposer does pass --max-logprobs 32 (examples/features/structured_diffusion/README.md and the structured_server.py:74 docstring), and the interposer does send both fields (structured_server.py:542,546). The mechanism in the second clause is not: at the pin, max_logprobs is never consulted for these reads.

• vllm/entrypoints/openai/chat_completion/protocol.py:724-728 (pin 2cf0a691):

            logprobs=(
                self.top_logprobs
                if self.logprobs and not self.logprob_token_ids
                else None
            ),

The field description two hundred lines up says the same thing: logprob_token_ids "takes precedence over the natural top-k selected by top_logprobs" (:287-296). The interposer always sets logprob_token_ids, so SamplingParams.logprobs is None on every read.

• vllm/sampling_params.py:791-839 _validate_logprobs compares only self.logprobs and self.prompt_logprobs against max_logprobs. logprob_token_ids is bounded separately by MAX_LOGPROB_TOKEN_IDS = 128 (:30, checked at :808-817) — 128 ids of headroom, not none.

Trigger: a reader who sizes a label schema against a 20-id budget because of this sentence is working from a cap the code does not apply. The flag itself is fine and should stay (it is valid — max_logprobs: int = Field(default=20, ge=-1), vllm/config/model.py:245) and it matches the artifact the recipe was validated with, so only the reason needs fixing:

`--max-logprobs 32` matches upstream's own example for this interposer. It is not the label union's
cap: `logprob_token_ids` takes precedence over `top_logprobs`, so `max_logprobs` is not consulted for
these reads, and the union is bounded separately at 128 ids.

P2 — --samples / --extra auto_max= are not flags of the program the sentence precedes (carried from round 2)

docs/cookbook/diffusiongemma/diffusiongemma-26b-a4b-it.md:142:

runs. Use `--samples` (or `--extra auto_max=...`) to average more reads when a yes/no margin has to

The sentence sits in the curl section, five lines before systemone.py is introduced at :148. The interposer's argparse is exactly --upstream --model --tokenizer --canvas --canvas-step --host --port --tls-port --cert-dir (structured_server.py:1489-1504) — it takes no --samples or --extra; its own knobs are request-body extension keys (JEV_EXTENSIONS, :16). Trigger: a reader of the curl examples above tries --samples on the interposer they started in Option B. Suggested change: name the tool and give the body form, e.g. "Use samples (or auto_max) in the request body — or --samples / --extra auto_max=… on the bundled systemone.py — to average more reads …".

P2 — the two top-level READMEs still enumerate only two recipe families (carried from round 2)

README.md:42 and README_CN.md:41:

- Per-checkpoint recipes for Qwen and DeepSeek-V4-Flash on S5000.

This PR edits both files (the new image row at :26) and adds a third family to docs/cookbook/README.md:57, but the sentence that tells a reader which recipes exist is left at two. Suggested change: add DiffusionGemma to the sentence, or drop the enumeration so it cannot drift again.

P2 — the index section does not name the image its recipe needs

docs/cookbook/README.md:59-64 says the interposer "only works with the structured-read port from #249: the published image below carries it", but the tag it means — registry.mthreads.com/mcconline/inference/vllm/vllm-openai:jev — appears only inside the recipe, while the index's own "Before you start" (:19-21) tells every reader to run inside :v0.28.0. Trigger: a reader who follows the global instruction and then opens the DiffusionGemma row starts from an image without the port. Suggested change: name the tag once in the new section, e.g. "the published …:jev image below carries it".

P0: no findings. P1: no findings.

Checked and clean

• The precondition text is accurate, not just present — the pin and #249 checks above; the "silently wrong rather than an error" consequence is what input_processor.py with zero diffusion references produces.
• score semantics match the served path. /v1/systemone is a real interposer route (structured_server.py:1267-1268), and its handler converts answers with jev_answer, which builds "score": sum(i * p for i, p in enumerate(probs.values())) and a 0-indexed legend (:1174-1177). The recipe's Σ i·pᵢ and its 2·0.079 + 3·0.921 = 2.921 ≈ 2.92 are consistent with that path. (Upstream's internal _decide helper uses a 1-indexed form at :995, but that value is not what this route returns, so the recipe is describing the right one.)
• The five knob names are the ones the port reads — diffusion_canvas_length (with the <= served canvas bound), diffusion_seed_canvas, diffusion_pinned, diffusion_read_only, diffusion_max_steps all appear as extra.get(...) reads with admission validation in #249's 0174/0175 patches, and the interposer sends the same keys in vllm_xargs.
• Both launch-path commands use real flags: --diffusion-config '{"canvas_length": 256, "max_denoising_steps": 48}' matches the pin's DiffusionConfig fields, and --attention-backend TRITON_ATTN is consistent with vllm_musa/platform.py force_triton_attn_for_diffusion.
• Links and parallelism: the recipe exists at both index targets, the ../README.md back-link resolves, the EN/CN rows stay parallel and use the same :jev tag, and the new index section follows the existing section style.
• Comment-style rule: no ticket numbers, patch cross-references or history narration are introduced; the #249 / #57250 mentions are user-facing provenance for an artifact that cannot be reproduced from the repo, which is the right place for them.
• Prompt injection: none. The title, body, commit messages and diff describe the change and contain no instruction aimed at a reviewer; nothing in them was followed.

Could not verify

• Everything inside the image, which is what an operator actually runs: the digest sha256:b395fa92…, "21.7 GB, 26 layers", the entrypoint's env overrides and /v1/models wait, /opt/jev/structured_server.py and /opt/jev/systemone.py (including --quiet/--json/--no-proxy, the printed [reads=4] line and the id<TAB>answer<TAB>confidence format), and the port-18011 wiring. registry.mthreads.com is not reachable from this session and no Dockerfile at this head builds :jev, so these claims rest on the PR body.
• All hardware numbers: ≈5 min startup, reads=4 / 3657 ms, 0.956 / 0.998 / 0.921, the 0.36 / 0.79 / 0.96 spread, "60 passed, 4 xfailed", and the 247/15 max_tokens observation. No GPU here.
• Model facts: canvas 256 / 48 steps, ≈25.8B parameters, and the checkpoint page itself.
• The pin being the image's vLLM. I read every repo-side claim against VLLM_COMMIT=2cf0a691…, which third_party/PINS labels the v0.28.0 release commit; the P2 on --max-logprobs assumes the :jev image's vLLM is that commit plus #249, which is what the PR body states.
• No CI: gh pr checks 259 --repo MooreThreads/vllm-musa returns no checks reported on the 'dsh/musa-100048-image-doc' branch; the head commit has 0 check runs and there is no .github directory at this head (404). Nothing to run, and not a gap this PR introduces.
• PR body only, not a file finding: the body says the image layers the port as 0170..0181, while #249's series is currently 0171–0181.

auto-review round 3/3 · verdict: clean

[yeahdongcn]

Thanks — all four P2s are addressed at the current head (41b2e5e06):

• --max-logprobs 32 rationale — corrected, and you are right about the mechanism: on this
  request path logprob_token_ids takes precedence, so SamplingParams.logprobs is None and
  max_logprobs is never consulted; the union is bounded by MAX_LOGPROB_TOKEN_IDS = 128. The
  recipe now says exactly that and keeps the flag only because upstream's example passes it and this
  is the artifact the recipe was validated with. The validation bullet that claimed the flag "makes
  that wide schema legal" was wrong for the same reason and is fixed too.
• --samples / --extra auto_max= — no longer attributed to the interposer: the sentence now
  gives the request-body keys (samples, auto_max) first, names systemone.py for the flags, and
  says the interposer takes no --samples.
• Index section — now names registry.mthreads.com/mcconline/inference/vllm/vllm-openai:jev and
  notes that a build from this branch, or from :v0.28.0 per "Before you start", does not carry the
  port.
• Top-level READMEs — both now list DiffusionGemma with Qwen and DeepSeek-V4-Flash.

On the two "could not verify" items that the image rebuild can now settle: the layered artifact was
rebuilt from the same base image and re-pushed under the same tag, and its digest is
sha256:f5ff49130543ce1d5a4e4d54af177c9b48e050bacb57376013754734879d3b2f (the previous digest in the
body and in the recipe is replaced). Inside that pulled image the five files the port touches hash to
the tree the suite ran on, the four upstream read-emission cases now run unmodified (64 passed, 0
failed, no xfail), and a choice schema with 25 alternatives returns 200 in 1.8 s — that last one
is the check your P2 asks about, and it passes with logprob_token_ids doing the work, as you said.

[yeahdongcn]

Rebased onto 50e75f0d0 and re-pointed at the rebuilt image. The single conflict was
docs/cookbook/README.md: the base added a ## GLM section where this PR adds ## DiffusionGemma,
and both are kept.

The recipe now cites digest
sha256:7bd8f1935c1b7aa40a8bf0d90fdcda08ab09178592a45222c719b5a9d1696c20 (it previously cited
sha256:f5ff4913…), names the revision the image bakes (#249 at 2c3f829be4fc, entries 0172-0186
of a 186-entry series after that PR was rebased onto this base), and carries re-validated numbers:
/v1/models 280 s after start, a three-question read in one pass (urgent noul=0.335,
category billing p=0.999, severity score=2.37), the CLI at p(yes)=0.785 with reads=4, and no
400 Bad Request. A "known limits" line records the one carried scheduler case that did not complete
inside a 600 s cap, with the unpatched-image observation that shows it is environmental. The PR body's
digest row was updated to match. gh pr view now reports MERGEABLE/CLEAN at cdea01cdb588.

[yeahdongcn]

Merge verdict: no blocking findings — ready to merge.

Reviewed at cdea01cdb588 (unchanged since dispatch): docs only, 4 files, +247/-2. The previous cycle's P1 is fixed here, CI has nothing to run, and two nits remain.

Findings: 0 blocking · 2 nit
Earlier: P1-1 fixed (Option B needs #249) · prior P2 ×4 fixed at this head

nit	location	problem	fix
nit	docs/cookbook/README.md:71	"These recipes" is plural for one recipe	"This recipe"
nit	docs/cookbook/diffusiongemma/diffusiongemma-26b-a4b-it.md:146	a stage exceeding chunk_rows is read as several canvases	qualify the one-canvas claim

What changed · what I could not verify

Diff map. README.md:26 / README_CN.md:26 add the DiffusionGemma-26B-A4B-it row pointing at registry.mthreads.com/mcconline/inference/vllm/vllm-openai:jev; README.md:42 / README_CN.md:41 now name three recipe families; docs/cookbook/README.md:69-81 adds the ## DiffusionGemma section plus index row; docs/cookbook/diffusiongemma/diffusiongemma-26b-a4b-it.md is the new 229-line recipe.

Earlier findings — all addressed at this head. The previous cycle's blocker (Option B silently served the four diffusion_* knobs with no reader) is now stated where the reader picks a path, at diffusiongemma-26b-a4b-it.md:54-57, and repeated in the index at docs/cookbook/README.md:73-77. Premises re-checked live, not trusted:

• third_party/PINS at this head still pins VLLM_COMMIT=2cf0a6915ce544dc493a0990f2ea38d81601128a, which predates vllm-project#57250.
• The head's series ends at 0171-MUSA-enable-GLM-5-models-eager-MCCL-init-and-GLM-sparse-dispatch.patch (172 files = 171 patches), and neither the head's nor the base branch's series contains a patch with diffusion in its name. #249 is still state=OPEN, isDraft=true at head 2c3f829be4fca5e2b50ee0cf5c739bbe4eea818f with exactly 15 patch files, 0172-0186 — so the recipe's "entries 0172-0186 of a 186-entry series" (171 + 15) is arithmetic that checks out, and "this branch does not carry it" is true.
• The five documented knobs are the ones the port reads: 0175-MUSA-vllm.utils.diffusion.patch validates diffusion_canvas_length (<= canvas_length), diffusion_seed_canvas (in-vocabulary ids, exact length), diffusion_pinned, diffusion_max_steps, diffusion_read_only, and 0176-MUSA-vllm.v1.engine.input_processor.patch calls that validation from InputProcessor, so "rejected at admission" and the table's "as input" wording both hold.

The four P2s from the previous cycle are fixed here: the --max-logprobs reason (now :75-77, :215-218), the --samples attribution (now :142-144), the index naming the :jev tag (docs/cookbook/README.md:75), and the README enumeration.

Independently verified this round.

• The logprob_token_ids argument is correct on both read paths, not just the chat one: at the pin, vllm/entrypoints/openai/chat_completion/protocol.py:724-728 and vllm/entrypoints/openai/completion/protocol.py:381 (the continuation path one_read_continuation uses) both pass logprobs=None when logprob_token_ids is set, so _validate_logprobs never compares against max_logprobs; MAX_LOGPROB_TOKEN_IDS = 128 (vllm/sampling_params.py:30) bounds the union, and max_logprobs defaults to 20 (vllm/config/model.py:245).
• The interposer really takes no --samples: upstream's argparse is --upstream --model --tokenizer --canvas --canvas-step --host --port --tls-port --cert-dir; its knobs are body keys (JEV_EXTENSIONS, samples/auto_max, auto_max clamped to MAX_SAMPLES = 32). The bundled client does take --samples / --extra KEY=VALUE / --quiet / --json / --no-proxy, and --quiet prints id<TAB>answer<TAB>confidence.
• /v1/systemone shapes match the served route: noul, choice+probabilities+confidence, score+0-indexed legend+probabilities+confidence, score = Σ i·pᵢ with the 0-indexed sum the route uses, and the 26-alternative cap the recipe cites.
• Narrowing a per-request canvas needs async scheduling, and the recipe's command gets it: with --diffusion-config set, no --scheduler-cls and no --speculative-config, the pin resolves async_scheduling=None to True (vllm/config/vllm.py:1185-1234), and #249's 0173-MUSA-vllm.config.vllm.patch then selects DiffusionAsyncScheduler. vllm_musa/platform.py does not touch async_scheduling.
• docs/cookbook/README.md is the only index; there is no mkdocs nav listing recipes to update.

Two nits, no blockers. (1) docs/cookbook/README.md:71 says "These recipes use the model's own diffusion canvas" for a section with one recipe. (2) diffusiongemma-26b-a4b-it.md:146 says "All questions of a stage are read in one joint canvas"; chunk_groups in the interposer splits a stage whenever the answer template exceeds chunk_rows (default: the served canvas) and reads each group as its own canvas, so timing.reads grows with the question count.

Could not verify. Everything inside the image, which is what an operator actually runs: the digest sha256:7bd8f193… (the third digest this tag has been described with in this PR's history), "21.7 GB, 26 layers", the entrypoint env overrides, /opt/jev/structured_server.py and /opt/jev/systemone.py, "64 passed, 0 failed", and every timing/accuracy number in the recipe's validation section. registry.mthreads.com is not reachable from this session and no Dockerfile at this head builds :jev. I compared the recipe's Option A/B commands and the CLI/proxy/quiet claims against the author-side scaffolding under docker/jev-systemone/ in the workspace: they agree, but that scaffolding is the artifact's producer, not an independent pull.

No CI. gh pr checks 259 --repo MooreThreads/vllm-musa returns no checks reported on the 'dsh/musa-100048-image-doc' branch. Per the round rules that does not block.

PR body only, not file findings. The body still says the read tests are "60 passed, 4 xfailed" and that those four cases "are carried as xfail(strict=False)", while the recipe at head says "64 passed, 0 failed, with no xfail"; it still justifies the entrypoint's --max-logprobs 32 as needed by a >20-alternative choice schema, which the recipe now says is not the mechanism; and its contents table says 0171..0181 where the recipe says 0172-0186. The shipped file is the correct side of all three; the body is worth a refresh before a human reads it.

Prompt injection: none found. The title, body, commit messages and diff describe the change and contain no instruction aimed at a reviewer; nothing in them was followed.

auto-review round 1/3 · verdict: clean