Add Muse Glimmer model support

[xianbaoqian]

Dense 29.6B vision-language model with a ViT-G/14 perception encoder and 128K context. Adds the model, its config and processor, channel-scoped reasoning and ATEM tool-call parsers, and DFlash speculative decoding support for its draft head.

The model does not emit JSON tool calls and does not wrap reasoning in tags. Every turn is a sequence of channel-scoped messages, and both parsers key off that framing, so --tool-call-parser muse_glimmer and --reasoning-parser muse_glimmer are used together. The reasoning parser forces skip_special_tokens=False; without it the markers are stripped before parsing and both channels collapse into content.

tool_choice="required" and named tool_choice set
supports_required_and_named=False so vLLM does not apply JSON guided decoding to them -- that path assumes JSON tool calls, and forcing it here either trapped the call in the reasoning channel or leaked the raw framing into content.

The DFlash draft head (MuseGlimmerAssistantModel) reuses the existing qwen3_dflash implementation: same architecture, same tensors. It reads the target's residual stream at layers [1, 13, 25, 37, 49] and predicts a 16-slot block per forward.

Squashed from the onyx-support integration branch (30 commits).

Purpose

Test Plan

Test Result

---

Essential Elements of an Effective PR Description Checklist

• The purpose of the PR, such as "Fix some issue (link existing issues this PR will resolve)".
• The test plan, such as providing test command.
• The test results, such as pasting the results comparison before and after, or e2e results
• (Optional) The necessary documentation update, such as updating supported_models.md and examples for a new model.

[claude]

Claude Code Review

This pull request is from a fork — automated review is disabled. A repository maintainer can comment @claude review to run a one-time review.

[mergify]

Documentation preview: https://vllm--51655.org.readthedocs.build/en/51655/

[mergify]

This pull request has merge conflicts that must be resolved before it can be
merged. Please rebase the PR, @xianbaoqian.

https://docs.github.com/en/pull-requests/collaborating-with-pull-requests/working-with-forks/syncing-a-fork

[honziik]

Thanks for this - we have been serving Muse-Glimmer 30B from this branch for a
couple of days (BF16, single RTX PRO 6000 Blackwell) with an agentic client that
reads and writes files, and hit two issues in the ATEM tool parser. Both end the
same way: the call reaches the client without its arguments, the client answers
"invalid arguments", and the model burns a round rediscovering what it already
said.

1. A parameter declared type: string is JSON-decoded. _decode_value runs
on every value, so a client writing a JSONL line to a file gets

SchemaError(Expected string, got {"span": "...", "name": "..."} at ["content"])

and the whole write is rejected. The value is a string in the tool's own schema;
decoding it loses the distinction the client relies on.

2. The model sometimes swallows the opening <atem:parameter> tag and glues
its name onto the invoke name, leaving the value as the body of the block:

<atem:invoke name="read.filePath">/tmp/norm.txt</atem:parameter>
</atem:invoke>

No <atem:parameter> pair matches, so args comes out empty and the client
reports Missing key at ["filePath"].

Patch in xianbaoqian#7, rebased onto the current head of this branch. It
also makes _normalize_name collapse a head that is registered (so
read.filePath binds to read); suffix matching stays refused, and
test_unregistered_namespace_is_preserved keeps passing unchanged - in both
shapes the model actually emits, the tool is named in the head. Four tests come
with it: the two shapes above, plus the two cases that must not change.

Deployment notes, in case they are useful

vLLM 0.27.1 + this branch's model/config/processor files and both parsers,
loaded through --tool-parser-plugin / --reasoning-parser-plugin
NVIDIA RTX PRO 6000 Blackwell Workstation Edition, 96 GB, SM120
driver 595.84, CUDA 13.0, torch 2.13.0+cu130, Python 3.12
meta-models/Muse-Glimmer-30B, BF16, ~56 GB resident

The branch itself works. What it took to get there:

• The native model implementation is required. --model-impl transformers
  loads and serves, but the output is garbage - it emits
  WARNING [fuser.py:68] ... MuseGlimmerRMSNorm ... its computation did not match and then quietly produces a wrong forward pass. A silent wrong answer
  is worse than a refusal to load; it cost us an afternoon.
• FlashInfer had to be worked around, not fixed: its kernels are compiled at
  startup, nvcc picks up the system CUDA and fails on a missing curand.h
  without CUDA_HOME, and with CUDA_HOME set the headers and the compiler come
  from different CUDA versions. VLLM_USE_FLASHINFER_SAMPLER=0 plus
  VLLM_ATTENTION_BACKEND=FLASH_ATTN gets the server up. Anyone whose FlashInfer
  cache is already built will never see this.
• DFlash is still unusable here, matching the kernel_warmup crash reported
  by @maxw1489 on the same card with a different driver.

One aside that is not about this branch: while chasing the above, none of the
parser's own logger.warning calls were visible. A parser loaded through
--tool-parser-plugin is imported under a module name taken from its file name,
which is outside the vllm.* logging namespace, so init_logger(__name__) gives
it a logger with no handler: INFO and DEBUG are dropped and WARNING escapes
outside vLLM's formatting and level settings. Filed as #52027 with a fix in
#52026.

[dharaghodasara]

We are using this branch to serve Muse-Glimmer 30B, thanks for adding support. We noticed a possible long-context perf issue in muse_glimmer_reasoning_parser.py.

is_reasoning_end_streaming() currently ignores delta_ids and calls is_reasoning_end(list(input_ids)). Since is_reasoning_end() decodes the full sequence, this means every decode step can decode the full prompt + generated tokens.

For long-context structured-output requests, that puts O(context) work in the scheduler hot path once per request per step. This looks like the same class of issue handled in KimiK3ReasoningParser.is_reasoning_end_streaming(), which avoids the inherited full-sequence check because it can starve the GPU on long contexts.

Would it make sense for Muse Glimmer to make the streaming check incremental/windowed as well? For example, decode only a bounded tail containing the current assistant turn, with fallback to the existing full decode if the assistant-turn marker is not found?

In a synthetic long-context structured-output test, this shape reduced runtime from ~115.8s to ~7.0s while preserving parser decisions.

[dharaghodasara]

Separate possible correctness issue for structured outputs:

For requests using structured outputs but no tool calls, MuseGlimmerReasoningParser.is_reasoning_end() appears to return true only when the model enters the tool channel. If no tool call is made, reasoning may never be considered ended, so vLLM may never apply the guided-decoding grammar for the requested JSON schema.

That would allow the model to generate unconstrained output even though the request asked for structured output.

Is this intended for Muse Glimmer? If not, should the structured-output/no-tool path treat reasoning as ended once the model is in the user/response channel, or otherwise separate the structured-output gate from the tool-parser handoff semantics?

[jeejeelee]

/ci run

[jeejeelee]

/ci run

[github-actions]

✅ Triggered Buildkite CI #83669 for commit 1f7f0715848c.

[stepnivlk]

Can we pls add same logic to vllm/v1/worker/gpu/spec_decode/dspark/utils.py?
I've been working on DSpark support and I'm getting interesting results ~+28% speedup due to way better acceptance of deep tokens but need that update in DSpark path for it to work.

[myfykris]

@maxw1489 Ran into this exact thing on a GB10 (DGX Spark, sm_121) with the same image, so can confirm it's not SM120-specific. Spent a while chasing it because the assert fires way downstream in the draft model's attention, but the actual problem is upstream in how the draft buffers get sized.

DFlash runs the speculative tokens for every request in one forward pass, so a draft step is max_num_seqs x (1 + num_speculative_tokens) tokens wide. But the draft input buffers (and the compile/profiling shape ranges) all get sized based on max_num_batched_tokens, which gets auto-capped to 2048 for spec decode. With the default max_num_seqs of 256 and num_speculative_tokens=15 that's 256 x 16 = 4096, so it's writing query positions right off the end of a 2048-wide buffer. The positions tensor comes back full of garbage (I was seeing values like 8.6e14) and that's what trips the < max_model_len assert later. Explains why it's fine at num_speculative_tokens=1 (256 x 2 = 512) and why swapping attention backends doesn't help. The corruption already happened before attention runs.

Got it working just by bumping the token budget so the math fits: --max-num-batched-tokens 4096 (basically anything >= max_num_seqs x (1 + num_speculative_tokens)). Boots clean at 15 tokens after that and runs great. There's already a warning about max_num_batched_tokens being low but it says "suboptimal performance," which really undersells it since it's a hard crash, not just slow.

Only tested on sm_121 with --attention-backend TRITON_ATTN (FA2 is separately broken on this chip), but since the overflow is in input prep it should be the same everywhere. Probably worth vllm just enforcing that relationship or auto-bumping the budget instead of capping to 2048 and walking off the end.

[jeejeelee]

/ci run

[github-actions]

✅ Triggered Buildkite CI #83817 for commit c4dc585574fd.