[jellysnack]

Summary

This PR adds XGrammar structural-tag support to parsers v2.

It includes:

• structural-tag builders for qwen3_coder, deepseek_v4, and glm47
• support for OpenAI tool-choice and parallel tool-call semantics
• configurable tool schema, reasoning, and special-token constraints
• tool calling combined with structured output
• a shared interface for adding structural tags to additional parser families

Motivation

Structural tags are supported by parsers v1 but are not yet available with parsers v2. This PR adds that support.

Differences from v1

The v2 implementation uses v1 as a behavioral baseline while separating request policy from model-specific formatting and adding several capabilities.

Area	v1	v2
Orchestration	Tool-call formats and tool_choice=none bans are built through separate methods	A single builder resolves the request and applies tool-choice, reasoning, and exclusion policy
Model builders	Individual builders interpret parts of the request policy themselves	Tool selection and cardinality are normalized before reaching the model-specific grammar
Named choice	Restricts generation to the selected tool, while cardinality still follows parallel_tool_calls	Produces exactly one call to the selected tool, matching the OpenAI named tool_choice contract
Reasoning	Prompt-opened reasoning is wrapped only for required and named choices	Shared handling applies to every enabled tool-calling mode. Alternatively, the inference engine can wait for the reasoning close marker and activate only the post-reasoning grammar
Special markers	tool_choice=none primarily uses token-level bans	String exclusions prevent tool-call strings regardless of tokenization and can also keep tool calls out of reasoning and reasoning markers out of free text
Tools + structured output	Not supported	auto can choose between model-native tool calls and a JSON Schema-constrained final response
Argument ordering	Declared JSON Schema order	Optional any_order, disabled by default because it weakens required-property and duplicate-key validation
Extensibility	Model formats are represented by separate builder variants and configuration types	Common flat formats share TemplatedToolCallFormat, complex formats implement ToolCallGrammar directly

Request semantics are represented by StructuralTagContext and resolved into a model-independent policy before reaching the model-specific grammar.

StructuralTagOptions contains serving-time behavior settings configured by the integration and reused across requests, including:

• special-token exclusions
• structural-tag-owned or inference-engine-owned reasoning boundaries
• tool argument property ordering.

The XGrammar wire representation remains private, so it can evolve without becoming part of the public parsers v2 API.

For auto requests with both tools and a structured-output schema, the builder produces an alternative between a calls-only model-native tool-call format and a JSON Schema-constrained final response. This allows the model to call a tool first and produce the structured final response after receiving the tool result.

Supported model formats

• Qwen3 Coder tool-call XML
• DeepSeek V4 DSML tool-call blocks
• GLM 4.7-style XML tool calls

The parser-family registry now declares both the v2 parser factory and its optional structural-tag capability, keeping model-family routing in one place.

Validation

The structural-tag builders were validated end to end with downstream Dynamo changes that integrate parsers v2 structural tags into request preprocessing. The Dynamo integration will be submitted as a separate PR after this frontend-crates change is merged.

Parser compatibility improvements discovered during end-to-end validation will be submitted separately.

[copy-pr-bot]

This pull request requires additional validation before any workflows can run on NVIDIA's runners.

Pull request vetters can view their responsibilities here.

Contributors can view more details about this message here.

[dynamo-review-agent]

parsers/v2/src/structural_tag/builders/mod.rs:82 — RequiredToolChoiceFormat::TextAllowed still builds triggered_tags with a required policy, which serializes at_least_one: true and disallows the leading free text this option is meant to permit. Fix: build the text-allowed required case as leading any_text followed by a required triggered tag, or otherwise pass a non-required triggered phase that actually allows leading text.

🤖 AI Fix

In parsers/v2/src/structural_tag/builders/mod.rs, update StructuralTagBuilder::build_with_options and the ToolCallGrammar helper path so RequiredToolChoiceFormat::TextAllowed does not call build_triggered_calls with a ToolCallingMode::Required policy that produces at_least_one: true at the start; instead add a dedicated text-allowed required builder that emits a Format::Sequence with a leading Format::AnyText excluding the tool trigger(s), followed by the existing required TriggeredTagsFormat so free text may precede the required tool call.

[rmccorm4]

Thanks for the contribution @jellysnack ! Tagging @keivenchang @indrajit96 to help review

[jellysnack]

parsers/v2/src/structural_tag/builders/mod.rs:82 — RequiredToolChoiceFormat::TextAllowed still builds triggered_tags with a required policy, which serializes at_least_one: true and disallows the leading free text this option is meant to permit. Fix: build the text-allowed required case as leading any_text followed by a required triggered tag, or otherwise pass a non-required triggered phase that actually allows leading text.

🤖 AI Fix

The TextAllowed option was misleading so I removed RequiredToolChoiceFormat entirely. Required requests now consistently use TriggeredTags and start with a tool call including the model-specific prefix where needed

[indrajit96]

Thanks @jellysnack for this taking a look right now

[rmccorm4]

/ok to test 35d583c

[github-actions]

📊 Conformance matrix rendered — view in CI summary

[indrajit96]

Hi @jellysnack thanks a lot for adding this!

@keivenchang Adding you for desing review.
PR looks good to me and mostly carries the V1 structural-tag architecture into V2 with stronger encapsulation.

Belwo are few deviations I found from V1 to V2 and my notes

Design choice	V1	V2 in this PR	Review note
Parser and builder association	ToolCallConfig contains both the parser configuration and optional structural-tag builder.	FamilySpec associates the parser factory with an optional static builder.	Appropriate for V2's factory-based parser creation.
Public API and extensibility	V1 exposes constructible builder configurations and XGrammar format types.	V2 exposes semantic request types while keeping builders and XGrammar wire types private.	Better encapsulation, but external consumers cannot construct or register V2 builders.
Policy resolution	Tool selection, schema behavior, and cardinality are handled within builder paths.	ResolvedToolCallingPolicy centralizes selected tools, mode, cardinality, and schema mode.	Good separation and easier cross-family consistency.
tool_choice = none	Uses a separate build_tool_call_ban() path with token-level exclusions.	The main build() path returns an AnyText format with text-pattern exclusions.	Meaningful semantic change that should be confirmed against backend behavior.
Structured output	No equivalent generic alternation between native tool calls and a structured final response.	auto can produce an Or between native tool calls and a JSON-schema response.	New V2 behavior; confirm that Dynamo requires this public feature.
Reasoning boundary	Mainly wraps prompt-opened reasoning for required and named calls.	Supports wrapping auto, required, named, and structured-output paths, plus ReasoningBoundary::External.	Intentional expansion beyond V1; ownership of the boundary should be documented.
Deployment integration	Dynamo obtains the builder from the selected V1 ToolCallConfig.	Dynamo must call structural_tag_builder_for_family(family) after selecting V2.	A Dynamo integration change is required; this PR only exposes the crates API.
Reference validation	Tests primarily cover Dynamo's existing builder behavior.	Tests assert locally generated XGrammar JSON.	Consider parity tests against pinned XGrammar reference builders to avoid encoding incorrect assumptions.

@jellysnack can you open a corresponding PR on Dynamo showing the integragration?

Internally I can pick both the PRs create a custom build and run CI along with some custom 2e2 tests.

[jellysnack]

@jellysnack can you open a corresponding PR on Dynamo showing the integragration?

Sure. And good catches on the review comments, I’ll address those and open a draft Dynamo integration PR shortly

[jellysnack]

@indrajit96 Hi!

I’ve addressed both issues and also added a small GLM parser fix after E2E testing showed that the model could emit whitespace between XML elements when using structural tag

Here’s also the draft PR with the corresponding Dynamo integration: ai-dynamo/dynamo#13903

[rmccorm4]

LGTM aside from 1 comment

[jellysnack]

Hi @rmccorm4 @keivenchang, just a friendly ping on this PR

[indrajit96]

/ok to test
e2994f1

[indrajit96]

LGTM!
All comments addressed!

[indrajit96]

/ok to test e2994f1