Repository navigation
[Rust Frontend] Build the Kimi K3 full-output grammar from its parser mode - #59190
Merged
Merged
Conversation
BugenZhao
added this pull request to stack #59163
September 29, 2026 09:35
BugenZhao
force-pushed
the
bz/k3-full-output-grammar
branch
from
September 29, 2026 09:51
ce1219c to
2c427d7
Compare
BugenZhao
force-pushed
the
bz/k3-full-output-grammar
branch
2 times, most recently
from
September 30, 2026 01:34
1650e5d to
8b15f5d
Compare
BugenZhao
force-pushed
the
bz/k3-full-output-grammar
branch
from
September 30, 2026 07:30
8b15f5d to
ea7763b
Compare
Member
Author
|
/ci run |
|
✅ Triggered Buildkite CI #92074 for commit |
BugenZhao
force-pushed
the
bz/k3-full-output-grammar
branch
from
September 30, 2026 08:21
ea7763b to
c54a65c
Compare
BugenZhao
removed this pull request from stack #59163
September 30, 2026 08:21
BugenZhao
added this pull request to stack #59397
September 30, 2026 08:21
BugenZhao
force-pushed
the
bz/k3-full-output-grammar
branch
from
September 30, 2026 08:52
c54a65c to
d178ebb
Compare
BugenZhao
force-pushed
the
bz/k3-full-output-grammar
branch
from
September 30, 2026 09:15
d178ebb to
a116b4a
Compare
BugenZhao
force-pushed
the
bz/k3-full-output-grammar
branch
from
September 30, 2026 11:32
a116b4a to
690f651
Compare
Contributor
|
This pull request has merge conflicts that must be resolved before it can be |
BugenZhao
force-pushed
the
bz/k3-full-output-grammar
branch
from
October 1, 2026 08:43
690f651 to
f144328
Compare
BugenZhao
force-pushed
the
bz/k3-full-output-grammar
branch
from
October 7, 2026 07:07
48e4647 to
8392a1b
Compare
Contributor
|
This pull request has merge conflicts that must be resolved before it can be |
Contributor
CI selector (shadow): no narrower answer, every step runs: 194 test steps (270 jobs) instead of 7 (7 jobs)Shadow mode: this changes nothing about what CI runs. It shows what the evidence-based selector would pick for this PR, next to today's rules. How it works. Feedback welcome: reply here if it would skip a step this change needs, or runs something unrelated.
Why: rust: rust/src/chat/tests/grammar_replay/Kimi-K3.json: root crate bucket; 16 gate-env steps + 12 hardware-image steps; + the PyO3 bridge claim (vllm/tool_parsers/rust_tool_parser.py) Selector would run (194)
Would skip (today's rules run them) (0)none Would add (today's rules do not run them) (187)
AMD mirrors: would skip (0)none AMD mirrors: would add (156)
8 changed files · base |
BugenZhao
force-pushed
the
bz/k3-full-output-grammar
branch
from
October 7, 2026 16:00
8392a1b to
4583c7f
Compare
BugenZhao
marked this pull request as ready for review
October 8, 2026 09:02
![claude[bot]](https://avatars.githubusercontent.com/in/1236702?s=60&v=4){kind=small}
There was a problem hiding this comment.
Claude Code Review
This repository is configured for manual code reviews. Comment @claude review for a one-time review, or @claude review always to subscribe this PR to a review on every future push.
Tip: disable this comment in your organization's Code Review settings.
Member
Author
|
/ci run --allow-stale |
|
✅ Triggered Buildkite CI #93558 for commit
|
… mode Kimi K3 grammars only covered the output after reasoning, through a structural-tag builder whose `reasoning` option stood for both the request's thinking mode and the grammar's start. Build the whole generation from the parser's initialized mode instead, so the engine constrains it from token zero: - a prompt-opened `think` or `response` channel continues its body, and a prompt that opened no channel may emit a `think` channel; - after `think`, `auto` needs a response, tools, or both, while `required` and named choices need tools, optionally after a response; - named choices and `parallel_tool_calls=false` allow one call. Tools may now follow reasoning directly, which the builder rejected. Remove `KimiK3StructuralTagBuilder`; its call tag and XTML argument syntax move into the new grammar module. Tool-choice normalization for model-owned grammars lives in `output_grammar`. Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com> Signed-off-by: Bugen Zhao <i@bugenzhao.com>
`xgrammar-structural-tag` 0.4.1 exposes the tool-choice normalization that `build_structural_tag` applies before calling a builder. Use it for model-owned grammars instead of the local copy, which only covered `auto`, `required`, and named choices and missed the crate's validation and allowed-tools handling. Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com> Signed-off-by: Bugen Zhao <i@bugenzhao.com>
Render the K3 topology snapshots with `output_grammar::test_utils::outline` instead of a K3-only one-line renderer. Call bodies are elided to `..` first, since the argument tests cover them, so the snapshots show the channel structure with its real markers, text exclusions, and call repetition options. Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com> Signed-off-by: Bugen Zhao <i@bugenzhao.com>
BugenZhao
force-pushed
the
bz/k3-full-output-grammar
branch
from
October 9, 2026 04:42
2de412c to
028a54d
Compare
6 tasks done
njhill
approved these changes
Oct 9, 2026
Member
|
/ci run |
|
❌ This PR is 57 commits behind upstream |
Member
|
/ci run --allow-stale |
|
✅ Triggered Buildkite CI #93919 for commit
|
Member
Author
|
/ci run --allow-stale |
|
✅ Triggered Buildkite CI #94139 for commit
|
This file contains hidden or bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
Sign up for free
to join this conversation on GitHub.
Already have an account?
Sign in to comment
3 participants
Add this suggestion to a batch that can be applied as a single commit.This suggestion is invalid because no changes were made to the code.Suggestions cannot be applied while the pull request is closed.Suggestions cannot be applied while viewing a subset of changes.Only one suggestion per line can be applied in a batch.Add this suggestion to a batch that can be applied as a single commit.Applying suggestions on deleted lines is not supported.You must change the existing code in this line in order to create a valid suggestion.Outdated suggestions cannot be applied.This suggestion has been applied or marked resolved.Suggestions cannot be applied from pending reviews.Suggestions cannot be applied on multi-line comments.Suggestions cannot be applied while the pull request is queued to merge.Suggestion cannot be applied right now. Please check back later.
Purpose
Kimi K3 grammars only cover the output after reasoning.
KimiK3StructuralTagBuildertakes areasoningoption that stands for both the request's thinking mode and the grammar's starting point, so the engine's reasoning gate decides when the grammar activates. The builder also requires a response channel on every path, which rejects tool calls directly after reasoning.Step 3 of the parser-owned output grammar rollout, following the grammar replay harness (#59143), the argument grammar split (#59395), and the shared schema root (#59408), all merged. The Kimi K3 parser already derives its starting channel from the prompt. This PR builds the whole generation's grammar from that initialized mode, so the engine constrains K3 from token zero (
reasoning_ended = Some(true)). The open-PR search found no other Rust K3 full-output grammar work; token-aware K3 parsing (#58358, merged) is a separate track.New
unified/kimi_k3/grammar.rsbuilds the topology fromKimiK3Mode:thinkclosesresponseclosesautoresponse [tools], ortools[tools]required[response] toolstools[response] tools, one calltools, one callA prompt-opened
thinkorresponsechannel continues its body. A prompt that opened no channel may first emit a wholethinkchannel. Every path may end with the message close. Named choices andparallel_tool_calls=falseallow one call.Free-text regions keep today's string-level exclusions, which stay within what the token-aware parser ([Rust Frontend] Token-aware marker parsing for Kimi K3 #58358) accepts. Token-level exclusions are a follow-up.
KimiK3StructuralTagBuilderis removed. Its call tag andXtmlArgumentssyntax, with thextml_typemapping from [Rust Frontend] ShareSchemaRootbetween argument coercion and grammars #59408, move intogrammar/arguments.rs. The call tag no longer reads the builder'sany_orderandmax_whitespace_cntoptions, so it uses their defaults, as the serving path already did.Model-owned grammars normalize the request's tools with
xgrammar_structural_tag::normalize_tool_choice, public since 0.4.1 (bumped in [Rust Frontend] Use the MiMo structural-tag builder #59148). This is the same normalizationbuild_structural_tagapplies before a crate builder, so K3 sees the same tools,BuilderToolChoice, and validation errors as builder-backed models.Out of scope: answer constraints (Step 4) and the ordinary-chat grammar skeleton. Requests without a tool grammar still get none.
Test Plan
cd rust cargo nextest run -p vllm-parser kimi_k3 cargo nextest run -p vllm-chat --test roundtrip --test grammar_replay cargo nextest run -p vllm-parser -p vllm-chat cargo clippy --workspace --all-targets --all-features -- -D warningsUnit tests snapshot the grammar for each start mode and tool choice with the shared outline renderer, with call bodies elided to
..(the argument tests cover them). Example: Reasoning plusrequired:Test Result
requirements/common.txtserves.auto: 130 tokens from insidethink, throughresponseandtools, then terminated.required: 124 tokens, withtoolsdirectly afterthink, which the removed builder rejected.vllm-parser+vllm-chat: 947 tests passed. Workspace Clippy and rustfmt passed.AI assistance: implementation and validation assisted by Claude Code.
🤖 Generated with Claude Code