Skip to content

[Rust Frontend] Use the MiMo structural-tag builder - #59148

Merged
BugenZhao merged 2 commits into
mainfrom
bz/rust-mimo-structural-tag
Sep 30, 2026
Merged

BugenZhao merged 2 commits into
mainfrom
bz/rust-mimo-structural-tag

Conversation

@BugenZhao

@BugenZhao BugenZhao commented Sep 29, 2026 •

Copy link
Copy Markdown
Member

Purpose

The Rust MiMo tool parser reuses the Qwen3-Coder structural-tag builder, whose tags are newline-delimited: <tool_call>\n<function=NAME>\n, with trigger <tool_call>\n<function=. MiMo's template and model output are compact: <tool_call><function=NAME><parameter=K>…</parameter></function></tool_call>.

  • auto + strict: the model's output never reaches the trigger, so the whole tool call is treated as free text and strict constrains nothing.
  • required / named: the grammar forces the newline layout, which is off the model's training distribution, so the model is steered off-distribution or the request cannot complete.

This is the Rust counterpart of #58019 (Python side, which depends on mlc-ai/xgrammar#916). The Rust MiMo parser came from #57933.

The fix bumps xgrammar-structural-tag to 0.4.1 (synced with XGrammar 0.2.8, which adds the mimo builder; released from Inferact/xgrammar-structural-tag; 0.4.1 also makes normalize_tool_choice public, for callers that build formats directly such as the stacked Kimi K3 output grammar) and points MiMoToolParser at Model::Mimo.

  • The same sync adds excludes to JsonSchemaFormat, so every serialized json_schema node now carries "excludes": []. The KimiK3 snapshot in unified/kimi_k3/structural_tag.rs is updated for that. vLLM's own K3 builder does not populate excludes yet; that is out of scope here.
  • This PR does not bump the engine's xgrammar == 0.2.7 pin. The 0.4 output for MiMo, Kimi K3, Qwen3.5 and Llama compiles with XGrammar 0.2.7 (excludes is ignored there), and the MiMo tags use only fields 0.2.7 already understands.

Duplicate check: the open-PR search for MiMo structural tags finds only #58019 (Python frontend) and the output-grammar replay harness #59143, which this unblocks and which does not change the builder.

Test Plan

cd rust
cargo nextest run -p vllm-parser -p vllm-chat -p vllm-server
cargo clippy --workspace --all-targets -- -D warnings
cargo fmt --all --check

Test Result

  • 1287 tests passed across vllm-parser, vllm-chat and vllm-server. The new structural_tag_uses_compact_mimo_tags test asserts the <tool_call> trigger and the compact begin/end tags.
  • Clippy and fmt pass. The workspace-wide nextest run needs libpython3.14 for the PyO3 tool-parser binary, which is missing on this host, so that binary was not run.
  • In xgrammar-structural-tag itself: Python parity against XGrammar 0.2.8 for all 18 upstream models, plus matcher checks that MiMo accepts the compact call and rejects the newline form.
  • Not covered here: end-to-end tool_call_mix replay for MiMo under required / named. That lands in the stacked replay harness ([Rust Frontend] Replay roundtrip output grammars through XGrammar #59143).

AI assistance was used (Claude Code); the change was reviewed by the submitter.

🤖 Generated with Claude Code

@claude claude Bot left a comment

Copy link
Copy Markdown

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Claude Code Review

This repository is configured for manual code reviews. Comment @claude review for a one-time review, or @claude review always to subscribe this PR to a review on every future push.

Tip: disable this comment in your organization's Code Review settings.

@BugenZhao BugenZhao added the ready ONLY add when PR is ready to merge/full CI is needed label Sep 29, 2026
@BugenZhao

Copy link
Copy Markdown
Member Author

/ci run --allow-stale

@github-actions

Copy link
Copy Markdown

✅ Triggered Buildkite CI #91772 for commit 0dae078f63a7.

⚠️ This PR is 4 commits behind upstream main. Running CI at your own risk because --allow-stale was requested; outdated CI configuration may cause failures. Before merging, merge or rebase onto the latest main, then rerun /ci run on the latest PR commit.

BugenZhao and others added 2 commits September 29, 2026 11:04
Co-Authored-By: Claude Sonnet 5.5 <noreply@anthropic.com>
Signed-off-by: Bugen Zhao <i@bugenzhao.com>
Co-Authored-By: Claude Sonnet 5.5 <noreply@anthropic.com>
Signed-off-by: Bugen Zhao <i@bugenzhao.com>
@BugenZhao
BugenZhao force-pushed the bz/rust-mimo-structural-tag branch from c334f0c to ba4bfa1 Compare September 29, 2026 11:10
@njhill

njhill commented Sep 29, 2026

Copy link
Copy Markdown
Member

/ci run --allow-stale

@github-actions

Copy link
Copy Markdown

✅ Triggered Buildkite CI #91927 for commit ba4bfa12c415.

⚠️ This PR is 16 commits behind upstream main. Running CI at your own risk because --allow-stale was requested; outdated CI configuration may cause failures. Before merging, merge or rebase onto the latest main, then rerun /ci run on the latest PR commit.

@BugenZhao
BugenZhao merged commit 8039d3c into main Sep 30, 2026
41 checks passed
@BugenZhao
BugenZhao deleted the bz/rust-mimo-structural-tag branch September 30, 2026 01:34
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

k3 kimi ready ONLY add when PR is ready to merge/full CI is needed rust

Projects

None yet

Development

Successfully merging this pull request may close these issues.

2 participants