[indrajit96]

Summary

This is the grammar-building half of a coordinated change addressing two
tool-calling correctness problems:

1. Kimi K3 grammar correctness. K3 tool_choice="auto" requests could begin
   a native XTML tool call but still emit undeclared tools, empty required
   arguments, incorrectly typed values, duplicate required arguments, or
   malformed XTML that the parser could not consume.

2. Schema enforcement for default-on structural guidance. The generic
   structural-tag policy treated an omitted OpenAI strict field as relaxed.
   The new policy uses the declared schema when strict is omitted or true;
   only explicit strict:false opts out.

This PR implements those parser and grammar changes. The companion
ai-dynamo/dynamo#13654
owns deployment defaults, activation policy, frontend integration, and
compatibility fallback.

Relationship to #274 and #277

• #274 addresses
  nullable Kimi K3 arguments. feat: Kimi K3 structural tag default #196 already emits a declared-key XTML
  alternative for each representable type in anyOf and type: [...]
  unions. Its grammar retains the string/null constraints and excludes
  numeric values and unknown argument keys in those cases.
• #277 addresses
  local $ref arguments. feat: Kimi K3 structural tag default #196 already resolves references against the
  complete tool schema and retains typed object and enum constraints,
  including recursive definitions. We adapted fix(parsers): resolve Kimi K3 argument schema references #277's remaining pointer
  decoding, array-index lookup, expansion limits, and reference-scope guards
  into this PR in merge commit d53f4ca. When inlining is unsafe, the
  original explicitly typed schema is retained.

Kimi K3 grammar hardening

Kimi K3 emits tool calls through its native XTML tools, call, and argument
elements. The new auto builder follows that model-native format while
preserving the semantics of automatic tool choice:

• ordinary assistant text remains unconstrained;
• guidance activates lazily only after the model emits
  <|open|>tools<|sep|>;
• calls are restricted to tool names declared in the request;
• automatic calls require positive call indices;
• parallel_tool_calls:false limits generation to one call;
• required string arguments must be non-empty;
• integer, number, boolean, array, object, and null arguments use typed content;
• $ref, $defs, anyOf, oneOf, allOf, enum, and const participate in
  XTML type inference;
• required arguments use the schema's canonical required order, avoiding
  factorial grammar expansion;
• loose text between arguments is replaced by complete declared optional
  argument tags, preventing duplicate required keys from overwriting a
  constrained value;
• schemas with only optional properties may emit an empty argument body;
• ambiguous schemas fall back to parser-compatible XTML instead of generating
  malformed calls; and
• named tool choice prevents a second generic tool-call representation from
  being emitted in the response channel.

TriggeredTagsFormat also gains optional excludes, allowing the K3 grammar to
reject invalid reasoning and call-marker transitions before the tools trigger.
Existing builders serialize unchanged when no exclusions are configured.

flowchart LR
    A[K3 begins generation] --> B{Opens tools channel?}
    B -- No --> C[Normal assistant answer]
    B -- Yes --> D[Triggered K3 grammar]
    D --> E[Select a declared tool]
    E --> F[Generate call index]
    F --> G{Schema representable?}
    G -- Yes --> H[Required typed arguments followed by declared optional arguments]
    G -- No --> I[Parser-compatible XTML fallback]
    H --> J[Valid native K3 call]
    I --> J
    J --> K[K3 parser produces OpenAI tool_calls]

Loading

tool_choice="auto" still does not force a tool call. The grammar constrains a
call only after the model chooses to begin the native tools channel.

Shared schema behavior

StructuralTagSchemaMode::Auto now follows these rules for all structural-tag
builders:

Tool setting	Argument guidance
strict omitted	Use the declared parameter schema
strict:true	Use the declared parameter schema
strict:false	Keep the native envelope but relax argument-schema enforcement
Global schema mode strict	Use the declared schema, overriding strict:false

This PR does not change parser-to-model mappings or deployment defaults.

Correctness validation

With Dynamo and SGLang, K3 KVV correctness improved:

• non-thinking: 25/49 = 51.0% to 48/49 = 98.0%;
• thinking: 69/98 = 70.4% to 94/98 = 95.9%.

The controlled native SGLang ablation provides the strongest causal evidence:

• structural guidance enabled: 47/48 = 97.9% non-thinking and
  90/92 = 97.8% thinking;
• structural guidance disabled: 22/48 = 45.8% and 61/92 = 66.3%;
• re-enabling guidance restored the original scores.

An activation-only Dynamo patch improved non-thinking to 37/49 but did not
repair thinking correctness, exposing the old empty and weakly typed argument
grammar. The complete builder was needed for the final result.

A later near-full Dynamo and vLLM run using this frontend-crates head reached
47/48 = 97.9% non-thinking and 83/97 = 85.6% thinking after excluding
context-length cases.

Performance validation

The tested Kimi workloads did not show a TTFT regression:

• concurrency-1 probe: 0.400 s without the change and 0.356 s with it;
• concurrency-24 streaming: 1.92 s without the change and 1.79 s with it.

These are two measured workload shapes and should not be interpreted as a
general performance improvement.

Unit validation

Focused tests cover:

• optional auto activation and normal-answer passthrough;
• thinking-to-tools transitions;
• declared and named tool selection;
• required and parallel calls;
• canonical required-argument ordering;
• duplicate-required-key prevention;
• optional-only schemas and empty argument bodies;
• typed string, numeric, boolean, array, object, and null arguments;
• local references and schema-composition keywords;
• ambiguous-schema fallback; and
• serialization compatibility for existing structural-tag builders.

[github-actions]

📊 Conformance matrix rendered — view in CI summary

[rmccorm4]

Local exact-head review findings.

[rmccorm4]

/ok to test 75e882f