[stu-cao]

Current scope and overlap audit (2026-11-02)

The Unified/v2 grammar is already upstream through #493. Dynamo now consumes the public builder and merged ai-dynamo/dynamo#17255; ai-dynamo/dynamo#17254 is closed. The original v2 missing-builder statements below are historical, not the current state of main.

This draft remains a legacy-v1-only follow-up, pending the maintainer-requested rebase/narrowing and real grammar-to-parser round-trip qualification, including embedded markers and exact string whitespace. Its old diff still contains pre-#493 v2 changes; those must not overwrite main's v2 implementation. No new validation of that pending rebase is claimed.

The separate later raw-string closing-prefix repair is tracked in #530 / draft #531. Backend constraint preservation is coordinated in mlc-ai/xgrammar#901; it is not solved merely by adding the V4.1 builder. Current v2 already has the required/named two-newline framing.

Historical implementation and validation record for this draft

Overview

Add the missing DeepSeek V4.1 structural-tag grammar for both parser generations. V4.1 already has Unified output parsing, but its space-prefixed DSML framing and raw-string argument encoding require a different generation grammar from V4.

Fixes #479. The contribution proposal is recorded in that issue; this draft is pending maintainer agreement on the approach and public crate release planning.

Details

• Register the legacy V4.1 DSML configuration and the Unified V4.1 grammar builder.
• Share the existing DSML tool-choice/schema policy while keeping V4's byte framing unchanged.
• Serialize the native argument style as deepseek_v4_1_xml in the shared structural-tag crate and v2 wire format.
• Cover auto, required, named and none choices, schema enforcement, tool cardinality and prompt-opened reasoning.

This does not convert or change the existing Unified output-parser state machine, captured events, fixture inputs or goldens. No private fork dependencies or vendored sources are introduced. Linked Dynamo main capability/consumer follow-up: ai-dynamo/dynamo#17255. Dynamo will need to consume the eventual public crate release; currently published versions do not include this change.

Bug Fix Evidence

1. Minimal Sample Input

On frontend-crates main at 278f7c75ad21d0411f6171bbb8a5c6d4165a5a71, ask the structural-tag registry for deepseek_v41 with an add_numbers tool whose arguments require integer a and b, tool_choice=required, and strict schema mode. Example native model output that the generated grammar must match:

<｜DSML｜ calls>
<｜DSML｜ invoke name="add_numbers">
<｜DSML｜ parameter name="a">1</｜DSML｜ parameter>
<｜DSML｜ parameter name="b">2</｜DSML｜ parameter>
</｜DSML｜ invoke>
</｜DSML｜ calls>

Reproduction/verification: cargo test -p dynamo-parsers -p dynamo-parsers-v2 --lib deepseek_v41 and cargo test -p dynamo-parsers-v2 --lib structural_tag::builders::deepseek_dsml.

2. Buggy Output (Before)

The v1 registry has no deepseek_v41 entry. The Unified family is selectable for output parsing, but structural_tag_builder_for_family("deepseek_v41") returns None. Reusing V4 instead emits <｜DSML｜tool_calls> / <｜DSML｜invoke ...> and the deepseek_xml style, which do not describe V4.1's wire format.

3. Expected/Fixed Output (After)

Both registries supply V4.1 grammar. The block begins <｜DSML｜ calls>\n; required/named calls also preserve the checkpoint's two leading newlines before that block, each invocation begins <｜DSML｜ invoke name="add_numbers">\n, and its schema content uses "style":"deepseek_v4_1_xml" with the supplied integer schema. Required choice enforces at least one invocation; named choice narrows the allowed tool; none excludes V4.1 call markers. Existing V4 grammar tests remain green.

4. Unified Regression Cases

Not an output-parser event change: no UNIFIED.* capture or GOLDEN is changed. Eight grammar regression tests exercise the missing registry/style capability and tool-choice behavior directly (four v1 and four v2). The existing Unified event suites are run unchanged.

5. Visual Conformance Verification

uv run --python 3.12 --no-project --with pyyaml --with jinja2 bash conformance/utils/render_table_v2.sh regenerated the existing-capture report for this draft; it was regenerated again after the legacy-prefix follow-up. Its status JSON reports 0 red / 0 empty for both existing deepseek_v4 and deepseek_v41 reference columns (229 cells each, with grammar/mode N/A cases retained). The overall report has one recorded kimi_k3 mismatch (5-tool_no_close) in unchanged captured data. No fixture or GOLDEN was edited. Browser visual review has not been performed; this draft does not claim new output-parser captures or model-runtime/XGrammar integration coverage from the report.

Validation

• cargo test -p dynamo-parsers -p dynamo-parsers-v2 --lib: 966 v1 passed, 6 ignored; 910 v2 passed.
• cargo test -p dynamo-parsers-v2 structural_tag --lib: 59 passed.
• cargo clippy -p dynamo-parsers -p dynamo-parsers-v2 --lib -- -D warnings: passed.
• cargo fmt --all --check and git diff --check: passed.

These are grammar/source tests, not proof that every server/backend/schema workload is supported. A downstream legacy backport identified that required/named tags must preserve the checkpoint's two leading newlines. With that focused correction, its synchronized 256-request tool probe passed at client concurrency 64 (128 auto and 128 required). This is downstream integration evidence, not validation of this public v2 crate or ARM GPUs.

Where should the reviewer start?

parsers/v2/src/structural_tag/builders/deepseek_dsml.rs and its four V4.1 tests; then the shared style enum and v1 DSML configuration/registry.

Legacy prefix follow-up

The Unified builder already preserves \n\n before required/named DSML calls. The legacy V4.1 builder now does too: XGrammar's mandatory triggered-tag format starts at offset zero, so omitting the prefix coerced a different continuation from the checkpoint's own encoder. Auto retains free text before its trigger; other families are unchanged. New legacy tests cover auto versus required/named prefix behavior.

The local conformance report was regenerated after this follow-up using Python 3.11, PyYAML and Jinja2. Both existing DeepSeek V4 and V4.1 reference columns remain 0 red / 0 empty; the same recorded Kimi K3 reference mismatch remains. No authored input, GOLDEN, capture or manifest was changed.

Downstream stress scope

The cited 256-request downstream pass is a bounded result, not a blanket GPU-readiness claim. Subsequent private-stack default-temperature stress still found rare max-token tool noncompletion; exact worker-grammar replay admits those unfinished string prefixes and permits valid closure. That investigation remains open. This public PR supplies the missing format capability and legacy prefix alignment; it does not claim to guarantee termination for unbounded strings or validate the private backend/XGrammar integration.

[copy-pr-bot]

This pull request requires additional validation before any workflows can run on NVIDIA's runners.

Pull request vetters can view their responsibilities here.

Contributors can view more details about this message here.

[rmccorm4]

Hey @stu-cao, thanks for the contribution! Can you check if this merged PR #493 meets the same needs?

[keivenchang]

hey Stu, #493 covers the v2/unified V4.1 grammar support here; the remaining contribution is legacy v1 support. After rebasing onto main, please retain main’s v2 builders and narrow this PR to that remaining work. To bring v1 into conformance with v2, the legacy parser needs payload-aware boundaries so literal DSML markers inside strings cannot create extra calls or drop calls, exact preservation of string whitespace, and consumption of the mandatory separator without leaking \n\n into content. Reuse the v2 parsing logic through the v1 interface where practical so these rules have one owner. Please add grammar-to-parser round-trip tests using actual XGrammar acceptance, compare normalized v1/v2 calls and content, and cover embedded markers, whitespace-sensitive enums, nested JSON, empty arguments, tool choice, and call-count restrictions. The existing structural tests pass while these round trips fail, so grammar snapshots alone won’t catch the remaining problems.