Repository navigation
[Rust Frontend] Use the MiMo structural-tag builder - #59148
Conversation
There was a problem hiding this comment.
Claude Code Review
This repository is configured for manual code reviews. Comment @claude review for a one-time review, or @claude review always to subscribe this PR to a review on every future push.
Tip: disable this comment in your organization's Code Review settings.
|
/ci run --allow-stale |
|
✅ Triggered Buildkite CI #91772 for commit
|
Co-Authored-By: Claude Sonnet 5.5 <noreply@anthropic.com> Signed-off-by: Bugen Zhao <i@bugenzhao.com>
Co-Authored-By: Claude Sonnet 5.5 <noreply@anthropic.com> Signed-off-by: Bugen Zhao <i@bugenzhao.com>
c334f0c to
ba4bfa1
Compare
|
/ci run --allow-stale |
|
✅ Triggered Buildkite CI #91927 for commit
|
Purpose
The Rust MiMo tool parser reuses the Qwen3-Coder structural-tag builder, whose tags are newline-delimited:
<tool_call>\n<function=NAME>\n, with trigger<tool_call>\n<function=. MiMo's template and model output are compact:<tool_call><function=NAME><parameter=K>…</parameter></function></tool_call>.auto+ strict: the model's output never reaches the trigger, so the whole tool call is treated as free text and strict constrains nothing.required/ named: the grammar forces the newline layout, which is off the model's training distribution, so the model is steered off-distribution or the request cannot complete.This is the Rust counterpart of #58019 (Python side, which depends on mlc-ai/xgrammar#916). The Rust MiMo parser came from #57933.
The fix bumps
xgrammar-structural-tagto 0.4.1 (synced with XGrammar 0.2.8, which adds themimobuilder; released from Inferact/xgrammar-structural-tag; 0.4.1 also makesnormalize_tool_choicepublic, for callers that build formats directly such as the stacked Kimi K3 output grammar) and pointsMiMoToolParseratModel::Mimo.excludestoJsonSchemaFormat, so every serializedjson_schemanode now carries"excludes": []. TheKimiK3snapshot inunified/kimi_k3/structural_tag.rsis updated for that. vLLM's own K3 builder does not populateexcludesyet; that is out of scope here.xgrammar == 0.2.7pin. The 0.4 output for MiMo, Kimi K3, Qwen3.5 and Llama compiles with XGrammar 0.2.7 (excludesis ignored there), and the MiMo tags use only fields 0.2.7 already understands.Duplicate check: the open-PR search for MiMo structural tags finds only #58019 (Python frontend) and the output-grammar replay harness #59143, which this unblocks and which does not change the builder.
Test Plan
cd rust cargo nextest run -p vllm-parser -p vllm-chat -p vllm-server cargo clippy --workspace --all-targets -- -D warnings cargo fmt --all --checkTest Result
vllm-parser,vllm-chatandvllm-server. The newstructural_tag_uses_compact_mimo_tagstest asserts the<tool_call>trigger and the compact begin/end tags.libpython3.14for the PyO3tool-parserbinary, which is missing on this host, so that binary was not run.xgrammar-structural-tagitself: Python parity against XGrammar 0.2.8 for all 18 upstream models, plus matcher checks that MiMo accepts the compact call and rejects the newline form.tool_call_mixreplay for MiMo underrequired/ named. That lands in the stacked replay harness ([Rust Frontend] Replay roundtrip output grammars through XGrammar #59143).AI assistance was used (Claude Code); the change was reviewed by the submitter.
🤖 Generated with Claude Code