[qimcis]

Why

Dynamo’s v1 GLM parser lacks native structural tags for schema-constrained tool calls and named tool choice. Its unquoted argument text needs value-aware typing: raw 42 under anyOf: [{const:"auto"}, {type:"integer"}] must remain an integer while matching string constants preserve their literal spelling. XGrammar ignores nonredundant $ref sibling constraints, so the builder must detect unsupported schemas before emitting a misleading grammar. Independent schema oracles and exact-output gates expose existing parser differences without allowing new output corruption.

What Change

• Enable native GLM tags using shared policy; reject unenforceable reference sibling constraints.
• Correct constant typing in v1/v2 and percent-encoded references in the v1 builder.
• Add independent schema probes and exact-capture gates, pinned to v1 10.0.1/v2 0.7.20.

Test Plan

• All 118 GLM parser controls pass; existing parser outputs remain unchanged.
• Workspace suite: 2,578 passed; conformance Python CI: 1,019 passed, two browser tests skipped. Formatting, Clippy, dependency checks, and publish dry-run passed.
• XGrammar 0.2.4 and 0.2.7: 42 native grammar-to-parser roundtrips and six invalid-wire rejections each; unsupported-reference guard tests pass.

Known limitations

Forty-three baseline parity differences are triaged: 42 documented differences and one separately pinned DS4 empty-name defect. Authored goldens stay intact, including 99 legacy schema disagreements (81 visible report comparisons, 18 duplicate Nano records). XGrammar diagnostics still expose unsupported allOf, itemless-array, and object/reference combinations; v2’s pre-existing reference-sibling limitation remains. The v1 builder returns an error for unsupported reference siblings; callers retain their existing error/fallback policy.

Conformance case inventory

Each variant below defines TOOLCALLING.batch.7.<letter>.<variant>, TOOLCALLING.streamv1.7.<letter>.<variant>, and UNIFIED.7-<number>.<variant>. Expectations are authored independently of captures, with family-native wire formats.

Legacy group	Unified group	Meaning	Exact variants
7.o	7-17	Declared JSON argument types	declared_string, declared_integer, declared_number, declared_boolean, declared_object, declared_array, declared_null, id_direct_integer
7.p	7-18	String constants retain their literal spelling	const_plain_bare, const_plain_typed, const_plain_allof, const_plain_ref, const_quoted_bare, const_quoted_typed, const_quoted_allof, const_quoted_ref, const_object_bare, const_object_typed, const_object_allof, const_object_ref, const_array_bare, const_array_typed, const_array_allof, const_array_ref, const_number_bare, const_number_typed, const_number_allof, const_number_ref, const_boolean_bare, const_boolean_typed, const_boolean_allof, const_boolean_ref, const_null_typed, const_null_allof, const_null_ref, const_space_bare, const_space_typed, const_space_allof, const_space_ref, const_unicode_bare, const_unicode_typed, const_unicode_allof, const_unicode_ref, const_entity_bare, const_entity_typed, const_entity_allof, const_entity_ref, const_null_bare
7.q	7-19	anyOf constant and typed alternatives in both orders	union_anyOf_integer_string, union_anyOf_integer_typed, union_anyOf_number_string, union_anyOf_number_typed, union_anyOf_boolean_string, union_anyOf_boolean_typed, union_anyOf_object_string, union_anyOf_object_typed, union_anyOf_array_string, union_anyOf_array_typed, union_anyOf_null_string, union_anyOf_null_typed, union_anyOf_integer_reverse_string, union_anyOf_integer_reverse_typed, union_anyOf_number_reverse_string, union_anyOf_number_reverse_typed, union_anyOf_boolean_reverse_string, union_anyOf_boolean_reverse_typed, union_anyOf_object_reverse_string, union_anyOf_object_reverse_typed, union_anyOf_array_reverse_string, union_anyOf_array_reverse_typed, union_anyOf_null_reverse_string, union_anyOf_null_reverse_typed
7.r	7-20	oneOf constant and typed alternatives in both orders	union_oneOf_integer_string, union_oneOf_integer_typed, union_oneOf_number_string, union_oneOf_number_typed, union_oneOf_boolean_string, union_oneOf_boolean_typed, union_oneOf_object_string, union_oneOf_object_typed, union_oneOf_array_string, union_oneOf_array_typed, union_oneOf_null_string, union_oneOf_null_typed, union_oneOf_integer_reverse_string, union_oneOf_integer_reverse_typed, union_oneOf_number_reverse_string, union_oneOf_number_reverse_typed, union_oneOf_boolean_reverse_string, union_oneOf_boolean_reverse_typed, union_oneOf_object_reverse_string, union_oneOf_object_reverse_typed, union_oneOf_array_reverse_string, union_oneOf_array_reverse_typed, union_oneOf_null_reverse_string, union_oneOf_null_reverse_typed
7.s	7-21	Local references and sibling type intersections	ref_union_anyOf_string, ref_union_anyOf_integer, ref_union_oneOf_string, ref_union_oneOf_integer, ref_sibling_integer, escaped_ref_const, percent_ref_const
7.t	7-22	Composition intersects union type hints	intersection_anyOf, intersection_oneOf
7.u	7-23	Enums preserve declared argument types	enum_string, enum_integer
7.v	7-24	Type arrays and nullable argument interpretation	type_array_string_integer, type_array_integer_null, type_array_integer_null_null, nullable_string
7.w	7-25	Ambiguous native strings retain established preference	null_union_ambiguous, integer_const_ambiguous, object_const_union

Every group applies to all eight Unified families: deepseek_v4, deepseek_v41, gemma4, glm47, kimi_k2, kimi_k3, muse_glimmer, qwen3.

Legacy v1 batch covers deepseek_v3, deepseek_v3_1, deepseek_v3_2, deepseek_v4, gemma4, glm47, harmony, hermes, inkling, jamba, kimi_k2, kimi_k3, llama3_json, minimax_m2, minimax_m3, mistral, nemotron_deci, nemotron_nano, phi4, pythonic, qwen25, qwen3_coder. Harmony text shares its batch grammar; Muse has no v1 parser.

Legacy v2 streaming covers deepseek_v4, gemma4, glm47, harmony_text, kimi_k2, kimi_k3, minimax_m2, minimax_m3, muse_glimmer, qwen3_coder. Batch-on-stream covers the same list except harmony_text, whose token alias requires real token IDs. Other registered families carry explicit v2 unavailability.

Unresolved/cyclic references and the deliberately invalid typed-enum control remain Rust-only controls. Duplicate schema/null-string spelling is represented once. Ambiguous native text pins existing family interpretations; repository nullable behavior is tested separately from standard schema validation. Historical captures and existing golden values remain intact.

GLM v1/v2 argument results for every new conformance case

The case suffix expands to the batch and streaming IDs above; its Unified counterpart uses the mapped numeric group. Values below are JSON, preserving their types. “Same” equals the final value. Before is refreshed author head 8c86cb8; after includes the value-aware constant fix and the reference-builder guard. Other parser families retain their baseline interpretations; their independent goldens remain unchanged.

Case suffix	v1 before	v2 before	v1/v2 after
7.o.declared_string	Same	Same	"plain"
7.o.declared_integer	Same	Same	42
7.o.declared_number	Same	Same	3.5
7.o.declared_boolean	Same	Same	true
7.o.declared_object	Same	Same	{"x":1}
7.o.declared_array	Same	Same	[1,2]
7.o.declared_null	Same	Same	null
7.p.const_plain_bare	Same	Same	"auto"
7.p.const_plain_typed	Same	Same	"auto"
7.p.const_plain_allof	Same	Same	"auto"
7.p.const_plain_ref	Same	Same	"auto"
7.p.const_quoted_bare	Same	"hello"	"\"hello\""
7.p.const_quoted_typed	Same	Same	"\"hello\""
7.p.const_quoted_allof	Same	"hello"	"\"hello\""
7.p.const_quoted_ref	Same	"hello"	"\"hello\""
7.p.const_object_bare	Same	{"x":1}	"{\"x\":1}"
7.p.const_object_typed	Same	Same	"{\"x\":1}"
7.p.const_object_allof	Same	{"x":1}	"{\"x\":1}"
7.p.const_object_ref	Same	{"x":1}	"{\"x\":1}"
7.p.const_array_bare	Same	[1,2]	"[1,2]"
7.p.const_array_typed	Same	Same	"[1,2]"
7.p.const_array_allof	Same	[1,2]	"[1,2]"
7.p.const_array_ref	Same	[1,2]	"[1,2]"
7.p.const_number_bare	Same	Same	"42"
7.p.const_number_typed	Same	Same	"42"
7.p.const_number_allof	Same	Same	"42"
7.p.const_number_ref	Same	Same	"42"
7.p.const_boolean_bare	Same	Same	"true"
7.p.const_boolean_typed	Same	Same	"true"
7.p.const_boolean_allof	Same	Same	"true"
7.p.const_boolean_ref	Same	Same	"true"
7.p.const_null_typed	Same	Same	"null"
7.p.const_null_allof	Same	Same	"null"
7.p.const_null_ref	Same	Same	"null"
7.p.const_space_bare	Same	Same	" spaced "
7.p.const_space_typed	Same	Same	" spaced "
7.p.const_space_allof	Same	Same	" spaced "
7.p.const_space_ref	Same	Same	" spaced "
7.p.const_unicode_bare	Same	Same	"café"
7.p.const_unicode_typed	Same	Same	"café"
7.p.const_unicode_allof	Same	Same	"café"
7.p.const_unicode_ref	Same	Same	"café"
7.p.const_entity_bare	Same	Same	"<"
7.p.const_entity_typed	Same	Same	"<"
7.p.const_entity_allof	Same	Same	"<"
7.p.const_entity_ref	Same	Same	"<"
7.q.union_anyOf_integer_string	Same	Same	"auto"
7.q.union_anyOf_integer_typed	"42"	Same	42
7.q.union_anyOf_number_string	Same	Same	"auto"
7.q.union_anyOf_number_typed	"3.5"	Same	3.5
7.q.union_anyOf_boolean_string	Same	Same	"auto"
7.q.union_anyOf_boolean_typed	"true"	Same	true
7.q.union_anyOf_object_string	Same	Same	"auto"
7.q.union_anyOf_object_typed	"{\"x\":1}"	Same	{"x":1}
7.q.union_anyOf_array_string	Same	Same	"auto"
7.q.union_anyOf_array_typed	"[1,2]"	Same	[1,2]
7.q.union_anyOf_null_string	Same	Same	"auto"
7.q.union_anyOf_null_typed	Same	Same	null
7.q.union_anyOf_integer_reverse_string	Same	Same	"auto"
7.q.union_anyOf_integer_reverse_typed	"42"	Same	42
7.q.union_anyOf_number_reverse_string	Same	Same	"auto"
7.q.union_anyOf_number_reverse_typed	"3.5"	Same	3.5
7.q.union_anyOf_boolean_reverse_string	Same	Same	"auto"
7.q.union_anyOf_boolean_reverse_typed	"true"	Same	true
7.q.union_anyOf_object_reverse_string	Same	Same	"auto"
7.q.union_anyOf_object_reverse_typed	"{\"x\":1}"	Same	{"x":1}
7.q.union_anyOf_array_reverse_string	Same	Same	"auto"
7.q.union_anyOf_array_reverse_typed	"[1,2]"	Same	[1,2]
7.q.union_anyOf_null_reverse_string	Same	Same	"auto"
7.q.union_anyOf_null_reverse_typed	Same	Same	null
7.r.union_oneOf_integer_string	Same	Same	"auto"
7.r.union_oneOf_integer_typed	"42"	Same	42
7.r.union_oneOf_number_string	Same	Same	"auto"
7.r.union_oneOf_number_typed	"3.5"	Same	3.5
7.r.union_oneOf_boolean_string	Same	Same	"auto"
7.r.union_oneOf_boolean_typed	"true"	Same	true
7.r.union_oneOf_object_string	Same	Same	"auto"
7.r.union_oneOf_object_typed	"{\"x\":1}"	Same	{"x":1}
7.r.union_oneOf_array_string	Same	Same	"auto"
7.r.union_oneOf_array_typed	"[1,2]"	Same	[1,2]
7.r.union_oneOf_null_string	Same	Same	"auto"
7.r.union_oneOf_null_typed	Same	Same	null
7.r.union_oneOf_integer_reverse_string	Same	Same	"auto"
7.r.union_oneOf_integer_reverse_typed	"42"	Same	42
7.r.union_oneOf_number_reverse_string	Same	Same	"auto"
7.r.union_oneOf_number_reverse_typed	"3.5"	Same	3.5
7.r.union_oneOf_boolean_reverse_string	Same	Same	"auto"
7.r.union_oneOf_boolean_reverse_typed	"true"	Same	true
7.r.union_oneOf_object_reverse_string	Same	Same	"auto"
7.r.union_oneOf_object_reverse_typed	"{\"x\":1}"	Same	{"x":1}
7.r.union_oneOf_array_reverse_string	Same	Same	"auto"
7.r.union_oneOf_array_reverse_typed	"[1,2]"	Same	[1,2]
7.r.union_oneOf_null_reverse_string	Same	Same	"auto"
7.r.union_oneOf_null_reverse_typed	Same	Same	null
7.s.ref_union_anyOf_string	Same	Same	"auto"
7.s.ref_union_anyOf_integer	"-7"	Same	-7
7.t.intersection_anyOf	Same	Same	42
7.s.ref_union_oneOf_string	Same	Same	"auto"
7.s.ref_union_oneOf_integer	"-7"	Same	-7
7.t.intersection_oneOf	Same	Same	42
7.s.ref_sibling_integer	Same	Same	42
7.s.escaped_ref_const	Same	"hello"	"\"hello\""
7.s.percent_ref_const	Same	Same	"42"
7.u.enum_string	Same	Same	"42"
7.u.enum_integer	Same	Same	42
7.v.type_array_string_integer	Same	Same	"42"
7.v.type_array_integer_null	Same	Same	42
7.v.type_array_integer_null_null	Same	Same	null
7.v.nullable_string	Same	Same	null
7.p.const_null_bare	Same	Same	"null"
7.w.null_union_ambiguous	Same	Same	null
7.w.integer_const_ambiguous	Same	42	"42"
7.w.object_const_union	Same	{"x":1}	"{\"x\":1}"
7.o.id_direct_integer	Same	Same	42

All parser-positive cases retain their expected arguments. For ref_sibling_integer, v1 now explicitly tests a builder error because the decoder ignores its narrowing sibling type; parsing remains integer 42. Unresolved/cyclic typed-reference controls also assert builder errors. Annotation-only and exactly redundant sibling constraints retain native grammar roundtrip coverage.

[copy-pr-bot]

This pull request requires additional validation before any workflows can run on NVIDIA's runners.

Pull request vetters can view their responsibilities here.

Contributors can view more details about this message here.

[rmccorm4]

Two findings from local XGrammar and parser validation.

[dynamo-review-agent]

Previously reported defects still present:

• Original discussion: The forced-call path still emits glm_xml at glm47.rs:43; no corresponding parser change preserves XML entity literals. The reported < to < argument mutation therefore remains.

[qimcis]

Previously reported defects still present:

• Original discussion: The forced-call path still emits glm_xml at glm47.rs:43; no corresponding parser change preserves XML entity literals. The reported < to < argument mutation therefore remains.

Addressed in a9fbc49: glm47_parser.rs no longer decodes entities

[dynamo-review-agent]

Previously reported defects still present:

• Original discussion: The glm_xml tag content still sets any_order: true in parsers/v1/src/tool_calling/structural_tag/glm47.rs, so the previously reported weakening of schema enforcement for required and duplicate properties remains present.
• Original discussion: any_order: true remains set unconditionally at glm47.rs:92, so GLM tool schemas can omit required properties and accept duplicate properties, defeating strict schema enforcement.

[rmccorm4]

Both earlier findings are fixed. Two new schema-normalization issues reproduce on this revision.

[dynamo-review-agent]

Previously reported defects still present:

• Original discussion: Still present: glm_schema overwrites a referenced definition's same-key constraint with the sibling value (lines 62-66), instead of preserving both constraints. For example, $ref maximum 1000 plus sibling maximum 2000 becomes maximum 2000, weakening the requested schema.
• Original discussion: The GLM builder still calls the generic resolve_tool_schema at parsers/v1/src/tool_calling/structural_tag/glm47.rs:80; with the generic policy, tools whose strict field is omitted are replaced with true, so ordinary GLM tool argument constraints are still dropped instead of being preserved unless strict: false is explicit.

[furionw]

Thank you for working on this, @qimcis . We have a few moving pieces that impact each other.

Let's iterate and merge this PR once ai-dynamo/dynamo#13654 lands 🙏

---

[P2: Low-risk, non-blocking issue such as a narrow edge case, defensive-hardening opportunity, or minor maintainability/test gap]
Rebase onto current main and add builder regression coverage

Assuming Dynamo #13654 lands first, please rebase onto current main, preserving the merged GLM reference, nullable-value, and scalar-union fixes. Frontend-crates #196 already updated the shared resolve_tool_schema: omitted/true tool strictness preserves the declared schema, explicit strict:false uses the permissive JSON Schema true, and global Strict overrides false. This PR's existing helper call can remain; a GLM-specific strictness helper is unnecessary.

The current tests cover schema normalization but do not exercise the builder's policy and control flow. Please add coverage for omitted/false strictness, global Strict, and auto/required/named calls with a prompt-opened reasoning block. These paths become more widely used once #13654 enables structural guidance by default.

[rmccorm4]

/ok to test 8c86cb8

[github-actions]

📊 Conformance matrix rendered — view in CI summary

[furionw]

/ok to test f4f357c

[furionw]

@qimcis Following the constant roundtrip discussion, I pushed f4f357c6 with three incremental behavior changes:

1. V1 constant handling: pass the raw argument text through schema traversal. A string constant supplies a string hint only when its value matches that text. Under anyOf: [{const: "auto"}, {type: "integer"}], raw 42 now reaches the existing integer selection/conversion and produces integer 42.
2. V1 reference handling: the new grammar builder reuses the parser's existing local-reference resolver, including URI percent-decoding. This fixes a reproduced grammar-compilation failure for encoded references. The helper is crate-internal (pub(crate)).
3. V2 constant handling: apply the matching-constant rule to preserve literal quotes and JSON-looking string constants. V2 already handled the integer alternative correctly; its structural-tag builder is unchanged.

The remaining changes add independently authored schema coverage, fixtures/capture tooling, and release-version metadata. Existing golden values and historical captures are preserved.

Verification against this pushed commit: all 118 authored controls pass for v1 and v2, including streaming/Unified paths. Seven focused real XGrammar 0.2.7 grammar-to-parser probes pass, with 20 invalid wires rejected. The full-corpus comparison found no output changes in 2,726 existing cases. The PR description records the remaining decoder limits and 43 reproduced baseline batch-on-stream disagreements; these are not hidden by new exceptions.

[furionw]

/ok to test a9c7589

[furionw]

@qimcis I pushed a9c7589 to address the remaining reference-sibling review, the required Rust gate failure, and the merge conflict with current main.

This adds one functional correction beyond the three changes explained above: the v1 GLM builder returns an error when a $ref sibling constraint cannot be proved redundant. Real XGrammar 0.2.4/0.2.7 probes accepted 900 for the reviewer's reference with maximum:1000 plus sibling maximum:600; retaining both keywords in JSON alone did not enforce them. The guard checks reachable nested schemas and aliases, allows annotations and exactly repeated constraints, and propagates unsupported-schema errors with tool/path context. Dynamo retains its existing builder-error fallback. The v2 grammar builder is unchanged; its pre-existing reference-sibling limitation is disclosed in the description.

The gate failure exposed 43 unchanged baseline differences. The gate now checks every live v2 result against its exact capture before applying an exception: 42 documented parity differences and one separately pinned existing DS4 empty-name defect. Independently authored goldens and every historical capture remain intact. Fresh v1 10.0.1 captures reproduce all 3,414 prior v1 results; v2 remains 0.7.20. The base update is a normal merge, preserving your branch history.

Verification: 2,578 workspace Rust tests and 1,019 conformance Python tests passed, along with formatting, Clippy, dependency checks, and publish dry-run. Against this exact pushed commit, both XGrammar versions passed 42 native grammar-to-parser roundtrips and rejected six invalid wires; unsupported reference-sibling error paths and nested/alias guards passed. The uploaded capture archive was independently downloaded and its SHA/size verified. Exact-head CI is authorized.

@furionw, the condition in your review is fulfilled: Dynamo #13654 merged first. The branch now includes current main, keeps resolve_tool_schema, and has omitted/true/false strictness, global Strict, tool-choice, and prompt-opened reasoning coverage. The description includes the schema case inventory and known compiler limitations.

[furionw]

@dynamo-review-agent, the findings in these review bodies are addressed on a9c7589:

• Entity-literal review: the parser preserves literal <. The native constant roundtrip verifies the same literal through emitted glm_xml, real XGrammar, and parsing on both 0.2.4 and 0.2.7. The original inline thread is answered and resolved.
• Ordering/required-property review: the builder no longer sets any_order:true. Missing required properties are rejected by the exact-head native decoder test; the earlier ordering thread reply covers duplicate/extra-property probes.
• Reference and strictness review: the reference guard rejects nonredundant siblings rather than dropping or overwriting constraints. Exact-head tests exercise the error route and verify allowed redundant/annotation-only references with real XGrammar. The shared policy preserves omitted/true strictness, relaxes only explicit false under Auto, and lets global Strict override it; the GLM builder still calls that helper. The omitted-strictness objection refers to the old policy.

The description records remaining XGrammar limitations and unchanged baseline parser defects; they are not represented as newly supported schema combinations.

[furionw]

/ok to test c5dc885

[furionw]

@qimcis I pushed c5dc885 as a normal merge of live main at 3f7aae69 to resolve the remaining workspace dependency conflict. GitHub's earlier PR base metadata was stale: live main had advanced to protocols 8.1.0. The resolution keeps v1 10.0.1/v2 0.7.20 and takes the upstream protocols 8.1.0 update. GitHub now reports the conflict resolved.

GLM parser/grammar source and conformance captures are unchanged from a9c7589. Verification passed: 172 protocol tests with the new protocol-schema feature, both 118-case GLM matrices, all six affected conformance gates (78 tests), workspace check/Clippy, formatting, dependency policy, doctests, and publish dry-run. Against the exact pushed commit, real XGrammar 0.2.4 and 0.2.7 each passed 42 grammar-to-parser roundtrips and rejected six invalid wires. Fresh exact-head CI is authorized; the previous head's CI passed.

[furionw]

/ok to test e642bb5

[furionw]

@qimcis I pushed e642bb5 to address the required Rust job's timeout. Its annotation confirms the 30-minute execution limit: all workspace tests, doctests, protocol-schema tests, and Clippy passed, then package verification was cancelled while compiling. The previous green run finished in 26 minutes; this run spent 25 minutes on passing tests alone.

This changes one workflow line: the Rust job gets 40 minutes. Every check, action, and permission remains identical; parser source and captures are unchanged. YAML parsing and an exact structural comparison verified that scope. Against the exact pushed commit, real XGrammar 0.2.4 and 0.2.7 each passed 42 grammar-to-parser roundtrips and six invalid-wire rejections. Fresh exact-head CI will verify the complete workflow finishes with the new limit.