Repository navigation
[Frontend] Support strict MiMo-V2.6 tool calling - #58019
Merged
BugenZhao merged 3 commits intoSep 29, 2026
Merged
Conversation
Use compact XML structural tags with a schema-constrained 0.2.7 fallback. Preserve raw parameter whitespace through the MiMo parser and honor single-call requests. Co-authored-by: Codex <noreply@openai.com> Signed-off-by: Ubospica <ubospica@gmail.com>
Contributor
|
Documentation preview: https://vllm--58019.org.readthedocs.build/en/58019/ |
mergify
Bot
added
documentation
Ubospica
marked this pull request as ready for review
September 24, 2026 06:39
Ubospica
requested review from
aarnphm,
bbrowning,
chaunceyjiang and
sfeng33
as code owners
September 24, 2026 06:39
![claude[bot]](https://avatars.githubusercontent.com/in/1236702?s=60&v=4){kind=small}
Signed-off-by: yuchuan <yuchuan.7streams@gmail.com>
Seven-Streams
requested review from
AndreasKaratzas and
tjtanaa
as code owners
September 25, 2026 13:59
Contributor
|
This pull request has merge conflicts that must be resolved before it can be |
This was referenced Sep 29, 2026
Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com> Signed-off-by: Bugen Zhao <i@bugenzhao.com>
Member
|
/ci run --allow-stale |
|
✅ @Ubospica, CI is now available for this PR.
|
|
✅ Triggered Buildkite CI #91771 for commit
|
BugenZhao
enabled auto-merge (squash)
September 29, 2026 07:21
Contributor
|
/ci retry |
|
✅ Queued 1 failed job(s) for retry in Buildkite CI #91771. |
sfeng33
added a commit
to Ostring24/vllm
that referenced
this pull request
Sep 29, 2026
The previous commit bounds the generic JSON schema, but parsers with a structural_tag_model install a structural tag first and never reach it, so with VLLM_ENFORCE_STRICT_TOOL_CALLING=1 (the default) the grammar still allowed any number of calls. When parallel_tool_calls is false, DelegatingParser._apply_structural_tag now sets stop_after_first on every tool-call list in the tag, and raises the strict level to at least FUNCTION so auto with non-strict tools gets the call envelope in the grammar. This covers every tag builder, including hy_v4, which overrides get_structural_tag without super(), and replaces the MiMo-only override from vllm-project#58019. The Responses API has no post-hoc filter, so for structural-tag parsers this is now its only enforcement. Signed-off-by: sfeng33 <4florafeng@gmail.com>
This file contains hidden or bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
Sign up for free
to join this conversation on GitHub.
Already have an account?
Sign in to comment
4 participants
Add this suggestion to a batch that can be applied as a single commit.This suggestion is invalid because no changes were made to the code.Suggestions cannot be applied while the pull request is closed.Suggestions cannot be applied while viewing a subset of changes.Only one suggestion per line can be applied in a batch.Add this suggestion to a batch that can be applied as a single commit.Applying suggestions on deleted lines is not supported.You must change the existing code in this line in order to create a valid suggestion.Outdated suggestions cannot be applied.This suggestion has been applied or marked resolved.Suggestions cannot be applied from pending reviews.Suggestions cannot be applied on multi-line comments.Suggestions cannot be applied while the pull request is queued to merge.Suggestion cannot be applied right now. Please check back later.
Purpose
MiMo-V2.6 Pro/Flash emit compact XML tool calls, but
--tool-call-parser mimocurrently selects Qwen3-Coder structural tags with mandatory newlines. Strict required/named calls therefore use the wrong wire format. The Qwen argument converter also removes a leading/trailing newline that MiMo emits as part of the string value.Introduce a MiMo variant of the existing parser engine that preserves parameter whitespace, with a thin tool adapter selecting
mimostructural tags. Prefer the new XGrammar builtin (mlc-ai/xgrammar#916); keep a schema-constrained compatibility builder for the currently pinned 0.2.7 wheel. Honorparallel_tool_calls=False, retain server/per-tool strictness controls, and leave reasoning to the serving gate. No dependency change is required.The shared registry change only declares the supported key. MiMo parsing and compatibility behavior live in the model's own modules. Document the existing
--tool-call-parser mimo --reasoning-parser mimo --enable-auto-tool-choiceflags and strictness options.Duplicate-work check: open #54393 and #57933 add Rust frontend MiMo support. This PR changes the Python parser/structural-tag path; neither existing PR covers it. Also searched open MiMo/XGrammar/strict-tool PRs before submission.
Test Plan
The local environment used the current checkout plus an isolated XGrammar build. Repeated MiMo cases with the actual 0.2.7 wheel first on
PYTHONPATH. Supplemental replay loads revision-pinned official tokenizers, renders chat-template completions, checks each native token mask and EOS boundary, and parses the resulting calls in both non-streaming and token-streaming modes.Test Result
-k mimoacross the registry/Qwen parser suites): 34 passed, no skipped native tests.Official revisions:
XiaomiMiMo/MiMo-V2.6-Pro-RL@54b10491b1811c76aa9681a9d0ff872396a4064cXiaomiMiMo/MiMo-V2.6-Flash-RL@3b38d063180c3e4aed9691fdc735f3d10b266ee4These are CPU grammar/parser protocol evaluations. Model weights were not loaded; inference accuracy/throughput was not measured. No model layers or kernels changed.
AI assistance: Codex implemented the change, ran local validation, and prepared this draft. Human line-by-line review and independent test execution are pending.
CI after submission: DCO passed. The upstream pre-run gate requires a
verified/readylabel or at least four merged contributions (the gate reports one), so remote pre-commit has not run. Local pre-commit and tests passed. The gate explicitly asks AI agents not to request authorization labels.