Repository navigation
feat: add GrammarFSMBuilder::FromRegularGrammar for regular grammars - #910
Merged
Seven-Streams merged 4 commits intoSep 20, 2026
Merged
Conversation
Signed-off-by: zhuhaoran <zhuhaoran.zhr@alibaba-inc.com>
This reverts commit 4aa48ee. The change is re-applied in separate commits so that the FSM state-limit fix, the Unicode character class fix, the regular-grammar flattener, the excludes feature, and the Kimi-K3 wiring can be reviewed independently.
Flatten the regular EBNF subset produced by RegexToEBNF into an FSM without reinterpreting its terminal expressions. Each finite rule call gets private entry and exit states so one call cannot return into another call's continuation; a direct tail self-reference loops back to that call's own entry. Mutual or non-tail recursion, non-byte terminals, and annotated rules (lookahead, max_tokens, capture, lazy, temperature, suffix stop) return an error, and max_num_states bounds the intermediate automaton. This is the fallback JSONSchemaFormat.excludes uses to intersect string constraints whose regexes go through the CFG expansion, such as the built-in formats and XML-style patterns. Extracted from mlc-ai#901.
Seven-Streams
requested review from
DarkSharpness and
Ubospica
as code owners
September 20, 2026 18:38
Contributor
There was a problem hiding this comment.
Copilot review overview
🔵 Needs a closer look
The hard-coded depth limit can reject valid grammars, and test discovery needs to be made reliable.
Review effort: Lite
Findings: None
What changed in this PR
Adds GrammarFSMBuilder::FromRegularGrammar to flatten regular EBNF grammars into FSMs.
Changes:
- Handles rule calls, repetition, tail recursion, and state limits.
- Rejects unsupported grammar constructs.
- Adds C++ regression and differential tests.
| File | Summary | Findings |
|---|---|---|
tests/cpp/test_regular_grammar_fsm.cc |
Adds coverage for recursion, regex equivalence, Unicode, and errors. | Nit (1 vote): Ensure the test is reliably included by CMake configuration. |
cpp/grammar_functor.h |
Declares the new FSM builder API. | None. |
cpp/grammar_functor.cc |
Implements regular grammar flattening and validation. | Moderate (1 vote): The hard-coded depth limit can reject valid grammars; make it configurable or document and test the limit. |
💡 Add a code-review agent skill or configure MCP servers for context-aware, tailored reviews. Learn more in the docs.
Seven-Streams
added a commit
that referenced
this pull request
Sep 21, 2026
feat: support string exclusions in JSON schema structural tags Extracted from #901 by @izhuhaoran, fourth piece after #906, #909 and #910. The branch keeps the original commit and its revert at the bottom for attribution; the net diff against `main` is the `feat: support string exclusions in JSON schema structural tags` commit plus `feat: drop string length constraints when exclusions are present`, a refactor of the exclusion helpers, and `feat: apply exclusions only to strings without pattern or format`. The Kimi-K3 builtin wiring that turns this on by default is the last piece and comes separately. ## Motivation Kimi-K3 tool arguments are raw text. The model can emit control markers such as `<|close|>response<|sep|>` inside an argument that is still open, then start another tools/call block. The schema grammar accepts all of that as string content, while the tool-call parser reads it as structure. There is no way today to tell the converter that certain substrings must never appear inside a string value. ## API `JSONSchemaFormat` gets an optional `excludes: list[str]` (C++, Python, TypeScript, docs): ```python JSONSchemaFormat(json_schema=..., style="kimi_k3_xml", excludes=["<|open|>", "<|close|>", "<|sep|>"]) ``` - Forbids each substring in string values and property names, including nested JSON strings. It applies to the strings without `pattern` or `format`: a `pattern` or `format` is the schema's own contract for the string and is matched as is. Filtering those would intersect an arbitrary regex with the exclusion automaton, which reproduces the length blowup below for any pattern with a wide repetition (`^[\s\S]{0,256}$` gives 4592 rules). - `minLength`/`maxLength` are dropped when exclusions are set, with a warning naming the string and the bounds. Bounding a filtered string's length would multiply the grammar by the bound: one rule per position and exclusion state, about 18 rules per character for the three Kimi-K3 markers, which is 18 s and 111 MiB for `maxLength: 256` and 110 s for ten `maxLength: 128` properties. Without the bounds those schemas compile in 96 ms and 71 ms, as the unbounded strings do. - Matches the emitted text before JSON/XML unescaping, may span tokens, and never spans separate strings. JSON delimiter quotes, numbers, punctuation and XML parameter wrappers are outside the scope, so `<|close|>` can still begin a valid `<|close|>argument<|sep|>` terminator. - `const`/`enum` alternatives and fixed property names that contain an excluded substring are dropped; a required one makes the schema unsatisfiable rather than silently allowed. - An omitted or empty list changes nothing: every exclusion code path is behind `!excludes_.empty()`. - Errors instead of silent bypass: XML styles reject exclusions that start or end with formatting whitespace, and `cohere_xml` and `minimax_m3_xml` reject non-empty exclusions. ## Implementation `ExcludingString` builds the FSM of the converter's own string body (the JSON string content with escapes, or the XML identifier), intersects it with the complement automaton of the excluded substrings (`Intersect` with the budgets from #906), prunes states that can no longer reach an accepting state so no dead-end prefix is ever offered in a mask, and re-emits the result as EBNF rules with codepoint transitions. UTF-8 byte paths are collapsed back to codepoint classes and never turned into a second transition for an ASCII character, which relies on the canonical encodings from #909. The closing JSON quote is appended after the intersection because it is syntax, not content. Property names go through the same intersection for `additionalProperties` keys and through a substring check for fixed keys; keys constrained by `patternProperties`/`propertyNames` patterns are matched as is like patterned values. A length-bounded string with exclusions uses the shared string rule (json) or the raw `TagDispatch` body (XML styles) instead of an intersected length bound. ## Validation - `tests/python/test_json_schema_excludes.py`: 91 cases covering every style, nested JSON, property names and literals, Unicode and overlapping exclusions, emitted-text semantics, patterned and formatted strings left unfiltered, dropped length bounds and their warning, whitespace-boundary and unsupported-style errors, compiler cache isolation, and token-mask replays that reject the original Kimi-K3 bad output at `response` while accepting the repaired output and a valid argument end. - `xgrammar_test` 82/82. Full `tests/python` with `HF_TOKEN`: 4947 passed, 3 skipped; the two CUDA kernel tests that JIT-build a torch extension fail identically on `main` in this environment (CUDA 11.1 toolkit). - Rebased onto `main` after #908: the DeepSeek key exclusion check now applies to both `deepseek_xml` and `deepseek_v4_1_xml`, which share the raw-key path. --------- Signed-off-by: zhuhaoran <zhuhaoran.zhr@alibaba-inc.com> Co-authored-by: zhuhaoran <zhuhaoran.zhr@alibaba-inc.com>
Seven-Streams
added a commit
that referenced
this pull request
Sep 21, 2026
Extracted from #901 by @izhuhaoran, last piece after #906, #909, #910 and #912. The branch keeps the original commit and its revert at the bottom for attribution; the net diff against `main` is the `feat(kimi_k3): exclude control markers from tool-argument strings` commit plus a docs commit that states the exclusion rules of #912 as a list. ## What changes The Kimi-K3 builtin now passes `excludes=["<|open|>", "<|cloase|>", "<|sep|>"]` to the `JSONSchemaFormat` of every tool-argument schema, in `auto`, `required` and forced tool modes. A control marker inside a raw string argument is rejected at the mask instead of being swallowed as text, which is the original #901 bad case: the model emitted `<|close|>response<|sep|>` inside an open `response` argument and started another tools/call block. The existing `exclude_special_tokens=False` option disables this together with the free-text exclusions; the free-text reasoning/response scope is unchanged. Per #912, the exclusions apply to argument strings without `pattern`/`format` and to property names, nested JSON strings included; `minLength`/`maxLength` on such strings are dropped with a warning. Argument and call wrappers are outside the scope, so `<|close|>argument<|sep|>` still terminates a value. `web/src/structural_tag.ts` gains the `kimi_k3_xml` member of the `JSONSchemaFormat.style` union, missed when the style was introduced in #806. ## Validation - `tests/python/test_builtin_structural_tag.py`: the instance that accepted a complete `<|close|>response<|sep|>` inside a string argument now expects rejection (`malformed-argument-end-in-string`); `test_kimi_k3_exclude_special_tokens` checks the three exclusions on every tool-argument schema across the three tool-choice modes and the opt-out; `test_kimi_k3_tool_argument_exclusions` covers raw and nested JSON string values with and without the option, including complete valid argument and call endings. - `test_builtin_structural_tag`, `test_json_schema_excludes`, `test_structural_tag_converter`: 1733 passed. The C++ tree is identical to `main`. Full `tests/python` with `HF_TOKEN`: 4946 passed, 3 skipped; the only failures are the CUDA kernel tests that JIT-build a torch extension, which fail identically on `main` in this environment (CUDA 11.1 toolkit). --------- Signed-off-by: zhuhaoran <zhuhaoran.zhr@alibaba-inc.com> Co-authored-by: zhuhaoran <zhuhaoran.zhr@alibaba-inc.com>
This file contains hidden or bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
Sign up for free
to join this conversation on GitHub.
Already have an account?
Sign in to comment
Add this suggestion to a batch that can be applied as a single commit.This suggestion is invalid because no changes were made to the code.Suggestions cannot be applied while the pull request is closed.Suggestions cannot be applied while viewing a subset of changes.Only one suggestion per line can be applied in a batch.Add this suggestion to a batch that can be applied as a single commit.Applying suggestions on deleted lines is not supported.You must change the existing code in this line in order to create a valid suggestion.Outdated suggestions cannot be applied.This suggestion has been applied or marked resolved.Suggestions cannot be applied from pending reviews.Suggestions cannot be applied on multi-line comments.Suggestions cannot be applied while the pull request is queued to merge.Suggestion cannot be applied right now. Please check back later.
Extracted from #901 by @izhuhaoran, third piece after #906 and #909. The branch keeps the original commit and its revert at the bottom for attribution; the net diff against
mainis thefeat(fsm): add GrammarFSMBuilder::FromRegularGrammarcommit, additions only.Grammar::FromRegexsends some patterns throughRegexToEBNFand produces a small CFG with sub-rules instead of a single FSM.JSONSchemaFormat.excludes(#901) needs an FSM for every string constraint so it can intersect it with the exclusion automaton, so it needs a way to flatten that CFG back.GrammarFSMBuilder::FromRegularGrammar(grammar, max_num_states)lowers the regular subsetRegexToEBNFemits without re-parsing its terminals: byte strings and character classes go through the existing FSM builders. Each finite rule call is inlined with private entry and exit states, so one call cannot return into another call's continuation, and a direct tail self-reference loops back to that call's own entry. Bounded repetitions are unrolled, unbounded ones become a loop. Mutual or non-tail recursion, non-byte terminals such asToken(...), annotated rules (lookahead,max_tokens, capture, lazy, temperature, suffix stop) and exceedingmax_num_statesreturn an error instead of a wrong automaton.Nothing on
maincalls it yet; the excludes feature is the first consumer.Validation: four new tests in
tests/cpp/test_regular_grammar_fsm.cc. Two cover the call-return isolation (part ::= "a" part | "b"used twice, and a recursive callee next to an optional exit in the caller). One is differential: six regex patterns, including nested quantifiers,[é你]+,[^你]and an email pattern, agree withGrammarMatcheron 24 samples. One checks the error cases and the state budget.xgrammar_test81/81; no Python changes.