feat(muse-glimmer): trace Meta's Muse Glimmer in vLLM and land the W0 scaffold

.agents/NOW.md

@@ -17,7 +17,7 @@ Work: exact-chunks on main `1ce0d662b`; sm_120 measured at `3d2581551`.
 | Laguna NVFP4 / DS-V4 decode | **CLOSED, byte-exact**: 1.03x vLLM, 1.144x ds4 | Laguna vLLM K-run |
 | 27B NVFP4 @`0893e160` | **c1 0.838, c2-c8 0.95-0.97** (#213 levers ACTIVE); c1 unmoved | c1 decode window; 35B canonical |
 | f32-out GEMV audit | **CLAIM WRONG**: 35B runs 41 `CastF32`/step (3.1%) | Fold into the 35B lever |
-| Invocation-parity | **CLOSED**: re-verified @`812de8ca` | — |
+| Muse Glimmer (#333) | **no speed number**: GGUF tokenizer blocks it (#347) | Fix #347 |
 | MiniMax-H3 | **PRUNED ckpts RUN (#241): Q8_0 renders, seam 0.9941** | same-binary A/B |
 | Kimi-Linear-48B | 122/128 held; e2e NOT ESTABLISHED | tiktoken-only ckpt: no warm server |
 | 35B mid-band | **canonical 0.918-0.972x** c1-c32 (@`348c265d`, first c16/c32) | Decode-only window, ONE tool |

.agents/benchmark-record.md

@@ -18140,3 +18140,107 @@ the Laguna-XS-2.1-NVFP4 decode sections. Nothing was edited or dropped.
 | Laguna-S-2.1 NVFP4 | **CLOSED 2026-08-04, parity+**: `VT_LAGUNA_RESIDENT_BF16W` default-ON (bf16 weights unified/ATS → cudaMalloc device-resident) → 44.6 vs 43.1 tok/s, byte-exact (o_proj 194→131, lm_head 2410→1620 us/call) | none, closed |
 | DFlash speculative decode | **CLOSED 2026-07-27 (D14)**: warp-scoped draft attention (242.9 → 77.9 ms), c1 our-on 29.32 vs vLLM-on 29.24 tok/s, non-overlapping 3-rep bands, 1.003x | none, closed |
 | cuBLAS invocation-parity guard | **CLOSED**: CI guard landed (CPU) and the `kGemvHeuristicAlgos` refactor re-verified on CUDA @`812de8ca` (forced recompile, clean `-Werror`, 315/315 + 235/235) | none |
+
+## Muse Glimmer 30B — the speed attempt (2026-08-11, issue #333, `row/MUSE-BENCH`)
+
+**Outcome: NO binding number on any axis.** Three of four bars are blocked, each
+with a named cause, and the one bar that ran produced a contended single-leg
+datapoint that is explicitly not a result. No ratio appears below, because no
+cell has two quant-matched sides.
+
+Branch `row/MUSE-BENCH` off `row/MODEL-MUSE-GLIMMER` @ `8af36075`.
+Full reasoning in `.agents/specs/muse-glimmer.md` §11.
+
+### The bars
+
+| Bar | Arm | Value | Quant-matched? | Status |
+|---|---|---|---|---|
+| vLLM | any | — | n/a | **OPEN GAP by construction**: the pin carries no `muse_glimmer`. Nothing substituted for it. |
+| llama.cpp `030ebb5` | `muse-glimmer-30B-kquant-17gb.gguf` | pp32 **9.79 t/s**, tg8 **0.79 t/s** (`-r 1 -t 4`, CPU) | **no** — nothing to match it against | **NON-BINDING**, see contention below |
+| ours | same GGUF | — | — | **BLOCKED**: tokenizer refuses `tokenizer.ggml.pre "llama4"` |
+| HF transformers | bf16 safetensors | — | — | **BLOCKED**: dgx GPU lock held all window |
+| ours | bf16 safetensors | — | — | **BLOCKED**: same host; CUDA build not started, deliberately |
+
+### Blocker 1 — our GGUF arm dies in the tokenizer, not the forward
+
+```
+$ build-cpu/examples/vllm-bench \
+    --model /mnt/nas_share/checkpoints/muse-glimmer-30b-gguf/muse-glimmer-30B-kquant-17gb.gguf \
+    --num-prompts 1 --concurrency 1 --input-len 32 --output-len 4
+vllm-bench: ... engine | num_prompts=1 input_len=32 output_len=4 concurrency=1 seed=0 temp=0.00
+vllm-bench: failed: tokenizer: unsupported tokenizer.ggml.pre "llama4"
+```
+
+The file carries `tokenizer.ggml.model = gpt2` (accepted) and
+`tokenizer.ggml.pre = llama4`, refused at `src/vllm/tokenizer/tokenizer.cpp:749`.
+llama.cpp routes `llama4` to `LLAMA_VOCAB_PRE_TYPE_GPT4O`, `clean_spaces = false`
+(`src/llama-vocab.cpp:2294-2299`), whose regex pair (`:428-434`) is neither our
+`kLlama3` nor either `kQwen2`. So this is a genuinely missing pre-tokenizer, not
+a missing alias — aliasing it would mistokenize silently (issue #347). **This is why the
+quant-matched llama.cpp cell cannot be filled today**: the GGUF is the only
+artifact both engines can hold, and we cannot open its tokenizer. Our bf16
+against a 4-bit GGUF would report quantization as speed, so it is recorded
+not-comparable rather than published with a caveat.
+
+Note this is upstream of everything §10.6 listed as unproven: the k-quant
+loader's 731/731 tensor accounting still stands, but no forward can be reached
+through the engine until the pre-tokenizer lands.
+
+### Blocker 2 — dgx was correctly busy for the entire window
+
+Another session held `/tmp/gpu` running the 27B/35B online-serving gate
+(`dgx-online-serving.sh --model 27n`, evidence `348c265d…`) from 04:37 onward; it
+advanced 18 -> 23 of ~36 legs across the window. The lock was respected: no GPU
+work was started, and no CUDA build was started either, because a parallel build
+on the same 20-core host would have perturbed a live serving measurement. Both
+bf16 cells are therefore blocked on host availability, not on any technical gap.
+
+### Blocker 3 — the local box could not host a clean measurement either
+
+The measurement host carried four other agents' concurrent builds and test runs
+(`vllm.cpp-bf16out` full test suite, `tp-task25-mutation-ef36c7ad`,
+`vllm.cpp-bug335`, and others). Observed load average over the window: **39.5,
+44.3, 48.2, 63.5, 88.1, 123.2** on 20 cores. The root filesystem hit **100% (435
+MB free)** twice; a rebuildable Docker build cache was pruned (4.58 GB) to let
+the builds proceed, and an unrelated `/tmp` cleanup deleted in-flight compiler
+temporaries mid-build, which was worked around with a private `TMPDIR`.
+
+`.agents/benchmarking.md` requires the noise band to be calibrated from repeated
+identical legs before any delta is read, and requires reproduction on an idle
+box. Under this contention neither is achievable, so the single llama.cpp leg is
+recorded as an artifact of the attempt, not as a measurement. A
+`-p 512 -n 64 -r 3 -t 20` leg was started and abandoned unfinished.
+
+### Recipe (for the re-run, which is owed)
+
+```sh
+# llama.cpp, CPU, Muse support merged 2026-08-10 (PR #26841)
+git clone --depth 1 -b master https://github.com/ggml-org/llama.cpp   # 030ebb5
+cmake -B build -DCMAKE_BUILD_TYPE=Release -DLLAMA_CURL=OFF -DGGML_NATIVE=ON
+cmake --build build -j --target llama-bench
+build/bin/llama-bench -m muse-glimmer-30B-kquant-17gb.gguf -p 512 -n 64 -r 3 -t 20
+
+# ours, CPU-only (no GPU on this host)
+cmake -B build-cpu -DCMAKE_BUILD_TYPE=Release -DVLLM_CPP_CUDA=OFF \
+      -DVLLM_CPP_BUILD_TESTS=OFF -DVLLM_CPP_BUILD_EXAMPLES=ON
+cmake --build build-cpu -j6 --target vllm-bench
+build-cpu/examples/vllm-bench --model muse-glimmer-30B-kquant-17gb.gguf \
+      --num-prompts 1 --concurrency 1 --input-len 512 --output-len 64
+```
+
+Artifacts: bf16 `/mnt/nas_share/checkpoints/muse-glimmer-30b` rev `f84ecc3a0e`,
+59.55 GB over 2 shards, `model_type: muse_glimmer`, 52 layers, hidden 6656,
+32/2 heads, head_dim 128, vocab 202048, `qk_scale_factor` 3.87,
+`post_norm_eps` 1e-8. GGUF `/mnt/nas_share/checkpoints/muse-glimmer-30b-gguf`
+rev `2fb01e4e6f`, `muse-glimmer-30B-kquant-17gb.gguf` 16,756,681,056 bytes,
+arch `muse-glimmer`, 731 tensors, file_type 15 (Q4_K_M), block_count 52,
+sliding_window 2048. Both read over a CIFS `soft` mount at ~117 MB/s.
+
+### What these numbers do and do not establish
+
+They establish that llama.cpp master loads and runs this GGUF on CPU, and that
+our engine cannot yet reach a forward on it for a reason that is now named and
+located. They establish **nothing** about how fast vllm.cpp runs Muse Glimmer,
+nothing about how it compares to any engine, and nothing about vLLM — which
+remains the only bar that counts and remains unavailable. **No ceiling is
+claimed or implied anywhere in this entry.**

.agents/coordination.md

@@ -1637,6 +1637,7 @@ commit order.
 |---|---|---|---|---|---|---|---|
 | `CLAIM-SAMPLE-PROMPT-LOGPROBS-W1` | `SAMPLE-PROMPT-LOGPROBS` (`ACTIVE`; W1 runner source only) | Claude Code (claude-opus-5) | `/home/mudler/_git/vllm.cpp-prompt-logprobs`; CPU-only `build-gate` (`-DVLLM_CPP_CUDA=OFF` Release) — NO GPU, NO download, claims no benchmark device | `row/SAMPLE-PROMPT-LOGPROBS`, opened on `origin/main` `bd6b3936`, rebased onto `8a6704a2` for the 2026-08-10 review repair and onto `e63d11d3` to land, issue #223 | The runner-side prompt-logits source ported from `_get_prompt_logprobs_dict`. Owns ONLY: the `prompt_logprob_rows`/`prompt_logprob_indices` block in `prepare_inputs.{h,cpp}`, `collect_prompt_logprobs` + `in_progress_prompt_logprobs_` in `runner.{h,cpp}`, `Sampler::compute_prompt_logprobs` in `sampler.{h,cpp}`, `num_prompt_logprobs` in `input_batch.{h,cpp}`, section 9 of `tests/vllm/v1/test_llm_engine.cpp`, this claim, the row cells, `.agents/NOW.md`, `.agents/roadmap_v1.md` issue rows, one `docs/STATUS.md` sentence, and the `prompt_logprobs` paragraph in `docs/USAGE.md`. **NON-COLLISION:** NO kernel, NO vt op, NO ABI, NO CMake, NO model file — the full-logits route reuses the existing `VT_LOGITS_GATHER=0` path rather than widening any model's gather contract. EXCLUDED: the OpenAI `echo` serialization (W2) and issue #231, both split out. | `ACTIVE` | 2026-08-10 — W1 landed then REPAIRED after a fresh review returned `VERDICT: FAIL`. Review found a REAL bug: the inertness guard keyed on `prompt_logprob_indices` while the full-logits assertion keyed on `prompt_logprob_rows`, so a zero-row final chunk (the exact-prefill edge, `:5668-5673`) beside another multi-token request threw `VT_CHECK` out of `engine.step()` and killed the whole batch. Fixed by moving the check inside the `num_rows > 0` slice; regression §8(h), RED-first (the throw). Second finding: the on-vs-off inertness case could not see a change to the SHARED route (forcing full logits on every step left it 17/17·346 green), so §8(g) now asserts the route DECISION through a const runner seam and fails under that exact mutation. Third finding recorded, NOT fixed and NOT claimed: the full-logits route hands the sampler a host pointer wearing the device label — sound on unified memory, unverified on CUDA; spec risk 4 + a `PENDING` CUDA smoke gate, and `docs/USAGE.md` narrowed to say so. Three latent lifetime divergences recorded as risks 5-7. Gates on the rebased tree (`e63d11d3`): clean CPU Release build 0 warnings under `-Werror`, `test_llm_engine` **21/21 · 384**, full `ctest -j 6` **365/366** with `test_openai_conformance` (parallel starvation — connection `-1`s) passing serially 1/1 in 0.39 s. Row stays `ACTIVE`: W2 `echo` serialization is the named residual, and the CUDA smoke gate is `PENDING`. |
 | `CLAIM-MOONCAKE-STORE` | `KV-MOONCAKE-STORE` (new, `SPIKE`) | Claude Code (opus-5), helper role | isolated worktree `/home/mudler/_git/vllm.cpp-mooncake`; records-only — NO build, NO GPU, NO download, NO external dependency installed | `row/KV-MOONCAKE-STORE`, base `origin/main` `848d4a87` | The `MooncakeStoreConnector` spike, records-only in this commit: NEW [`.agents/specs/mooncake-store-connector.md`](specs/mooncake-store-connector.md), the NEW `KV-MOONCAKE-STORE` engine-matrix row + section/total counters, the `ENGINE_ROWS` 146→147 bump in `scripts/check-agent-record.py` with its justification comment, a PROSE-ONLY Mooncake-disposition correction inside the `KV-CONNECTORS` row (both NOT-SCHEDULED sentences; that row's `ANCHOR-BACKFILL` state, tier, anchors, evidence, spec link and owner are UNCHANGED, so it is not claimed here), the `ROAD-V1-D4` portfolio note + canonical-table link, the roadmap issue-table row for [#287](https://github.com/mudler/vllm.cpp/issues/287), and this claim. **NON-COLLISION:** touches NO `src/`, `include/`, `tests/`, `examples/` or CMake path; the only script touched is the record checker's own row-count constant. | `ACTIVE` | 2026-08-10 — spec committed. The `KV-CONNECTORS` spike's blanket "Mooncake NOT SCHEDULED" conflated TWO connectors: `MooncakeConnector` (P2P prefill/decode over the Transfer Engine — two nodes, fabric, proxy) KEEPS that verdict; `MooncakeStoreConnector` (shared KV object store, the LMCache analogue) is reopened because (a) Mooncake is NATIVE C++ — `mooncake::Client` in `client_service.h`, and the `MooncakeDistributedStore` vLLM imports is a pybind wrapper over it — so we LINK instead of reimplementing a wire (the inverse of the LMCache cost shape), and (b) its single-node `protocol: "tcp"` + `mooncake_master` config is gateable on one box with NO RDMA NIC. The landed W5 `KVConnector` seam needs NO change. NEXT: W0, the go/no-go link spike (build Mooncake from source, pin the revision, drive `mooncake::Client` from a standalone C++ TU against a local master over TCP) — a genuine stop point per spec §S1. Speed is recorded as an OPEN axis: the RDMA/GPUDirect path that motivates the connector is unmeasurable for want of a fabric on any box we own. PENDING developer authority: the Mooncake source build, push/PR. |
+| `CLAIM-MUSE-GLIMMER-SPEC` | `MODEL-MM-muse-glimmer-muse-glimmer-for-conditional-generation` (`SPIKE`) | Claude Code (opus-5), helper role | isolated worktree `/home/mudler/_git/vllm.cpp-muse-glimmer`; CPU-only, records+spec+W0 scaffold — NO GPU, weights on the shared NAS | `row/MODEL-MUSE-GLIMMER`, base `origin/main`; PR [#279](https://github.com/mudler/vllm.cpp/pull/279) OPEN | The Muse Glimmer port. Spec `.agents/specs/muse-glimmer.md`, the `MODEL-MM-muse-glimmer-*` matrix row + checklist + rollup, issue-table entry for [#268](https://github.com/mudler/vllm.cpp/issues/268), porting-inventory §9 deviation 16 (the OFF-PIN anchor), `check-agent-record.py` MODEL count, the W0 scaffold (registry + config parse + weight name map + refuse-by-name forward, gate 11/11), `CHECKPOINT_ROOT` in `.env.example` + `.agents/environment.md`. Those helpers ALL LANDED onto the row branch: W1 the 52-layer text forward, W3 the 50-layer perception encoder, W4 the mm wiring (tower to adapter to projection to `perception_emb_norm` to the masked scatter; released-30B accounting 1436/1436), W7 the ATEM reasoning + tool parsers. The branch is no longer "records + spec + W0 scaffold" and its forward no longer refuses. `CLAIM-MUSE-GLIMMER-FIX` (`row/MODEL-MUSE-GLIMMER-FIX`) then closed the PR #279 review findings: three COVERAGE HOLES that mutation testing found green (`perception_emb_norm` untested, the whole non-FusedChain fallback arm unexercised, RoPE theta ungated at a realistic magnitude), the stale "OPEN FINDING" that had disarmed the real-weights accounting assertion, and the false claim in both parser headers that the seam has no `skip_special_tokens` (it does; channel scoping is an OPEN GAP at server defaults, spec §6.7). WHAT IS STILL NOT ESTABLISHED, and must not be implied: the full-depth 52-layer arm never ran (evidence is reduced depth 4/52, 5 prefill argmax positions, no generated tokens), multi-step decode is untested, the perception encoder has no reference check, and nothing has run through the server. **Anchored to the UNMERGED [vllm#51655](https://github.com/vllm-project/vllm/pull/51655) head `075d645af`** on explicit developer direction 2026-08-10: Muse Glimmer exists neither at the pin `555967922` nor on vLLM `main`. **No speed axis is claimable** until #51655 merges and the pin advances — the pinned oracle cannot load the model. | `ACTIVE` | 2026-08-10 |
 | `CLAIM-SPEC-DSPARK` | `SPEC-DSPARK` (`ACTIVE`) | Claude Code (opus-5), helper role | isolated worktree `/home/mudler/_git/vllm.cpp-spec-dspark`; CPU-only so far, NO build, NO GPU, NO download | `row/SPEC-DSPARK`, base `origin/main` `bc6e3d72`; NOT PUSHED, no PR yet (remote step PENDING developer authority) | The DSpark spike, records-only in this commit: NEW `.agents/specs/dspark-spec-decode.md`, the `SPEC-DSPARK` engine-matrix row + section/total counters, the feature-matrix §8 DSpark row, the superseded grounding-note header, this claim, `.agents/NOW.md`, and the `docs/STATUS.md`/`docs/FEATURES.md`/`docs/BENCHMARKS.md` one-liners. **NON-COLLISION:** touches NO `src/`, `include/`, `tests/`, `examples/` or CMake path. Implementation slices W1-W6 follow under this same claim. | `ACTIVE` | 2026-08-09 — spike committed. DSpark = the landed DFlash lane + Markov logit-bias head + sequential block sampling + anchor-as-first-prediction layout + `d2t` reduced vocab + method/config resolution + Speculators-format translation; upstream surface is 1613 lines over 5 files, 3 of them DFlash subclasses. Draft checkpoints exist for both gate models and for the 4B pair the upstream test uses; DeepSeek-V4 DSpark is out of scope (HW-blocked). NEXT: W1 config slice (CPU, RED = `speculative.cpp:44` rejects `"dspark"` today) and R1, prove the pinned oracle `555967922` actually RUNS DSpark (it forces the V2 runner). PENDING developer authority: checkpoint downloads, dgx GPU time, push/draft-PR. |
 | `CLAIM-ENG-RELEASE-BINARIES-W1-W13` | `ENG-RELEASE-BINARIES` (`ACTIVE`; complete release matrix) | Codex (GPT-5) | `/home/mudler/_git/vllm.cpp-release-binaries`; inline execution in the existing isolated worktree | `row/ENG-RELEASE-HANDOFF-FLAT`, base `origin/main` `24306364`; follow-up PR pending | Complete W1-W13 contract in one PR; the current follow-up completes the W8/W13 hosted handoff after run `31408404388` built and uploaded all eight required tuples but aggregation failed on nested `download-artifact` extraction. Owns the workflow-wide flat-download invariant, checker and red-first mutation, hosted dry-run verification, and exact release-readiness audit; no backend/package/manifest behavior change. ROCm remains blocked | `ACTIVE` | 2026-08-09 — required W1-W11/W13 implementation complete; hosted ten-SM completion, full eight-tuple dry run, matching-hardware gates, rebase/merge, and tagged publication pending. Update 2026-08-10: all eight bundles now build and upload, while aggregation still fails before verify because `plan/release-plan.json` is nested under the artifact name; W12 optional/non-primary |
 | `CLAIM-ENG-RELEASE-BINARIES-W5` | `ENG-RELEASE-BINARIES` (`ACTIVE`; W5 only) | Codex (GPT-5) | `/home/mudler/_git/vllm.cpp-release-binaries`; CPU/build-time metadata tooling only, no GPU/download/service work | `row/ENG-RELEASE-BINARIES`, base `b38f78a7`, claim commit `29107d0b`, draft PR #141 | W5 implemented: versioned manifest schema + deterministic generator/validator, canonical synthetic CPU/CUDA fixtures, fail-closed mutation tests, release checker/registration and required record/doc checkpoints. Excludes W1-W4, W6-W13, archives, install/package/publish workflows and runtime artifacts | `ACTIVE` | 2026-08-08 — W5 19/19; fresh-review production removals 10/10 killed; accepted release suite 30/30; no archive or real runtime/correctness/performance evidence |

.agents/environment.md

@@ -20,7 +20,25 @@ environment:
 2. Copy `developer-preferences.example.md` to the untracked
    `developer-preferences.md` for the policy choices (Git integration, which
    remote hosts you may use, contention policy).
-3. Add a profile entry to this file, in the same shape as the entries below:
+3. Set `CHECKPOINT_ROOT` in your `.env` if you have shared or network storage,
+   and download model weights there rather than onto a box's system disk. A
+   30B bf16 checkpoint is ~60 GB; a build tree is ~169 GiB on its own, and a
+   full disk surfaces as unrelated test failures rather than an obvious disk
+   error. Fetching once to shared storage means every host, worktree and agent
+   reuses it instead of each pulling its own copy. It states an INTENT and
+   nothing more: no code in the tree reads `CHECKPOINT_ROOT`, so it neither
+   redirects a download nor resolves a bare directory name — you place the
+   weights under it and pass the full path onward. A setup with no shared mount
+   leaves it empty and uses whatever the tool defaults to (usually the Hugging
+   Face cache under `$HOME`).
+
+   Two rules travel with it. Pin an explicit revision when you fetch:
+   publishers re-quantize in place under an unchanged repo name, so a bare
+   branch name is not reproducible and a checkpoint you gated against can
+   change under you. And setting the variable authorizes nothing on its own —
+   a large asset download still needs authority for the task.
+
+4. Add a profile entry to this file, in the same shape as the entries below:
    hardware, arch, toolkit versions, oracle availability, and the box's
    quirks. A PR for it is welcome, so the shared record says where each gate
    can run. New accelerator classes (an AMD/ROCm box, an Intel GPU) register