[II] Parse legacy direct DeepSeek V4 DSML tool calls

[voipmonitor]

Resulting behavior

The DeepSeek V4 parser accepts both fullwidth ｜DSML｜ markers and the legacy ASCII |DSML| spelling. A direct <|DSML|invoke ...> block is emitted as a structured tool call even when the optional outer tool_calls wrapper is absent.

The streaming parser preserves marker fragments across deltas, supports multiple direct invokes, and returns to ordinary assistant content after a direct invoke closes. Wrapped DSML calls retain their existing parallel-call state transitions.

Technical reason

DeepSeek V4 can emit the legacy direct form even when the prompt requests the canonical wrapped representation. Treating the legacy marker as assistant text leaks protocol markup and prevents clients from receiving tool_calls.

The parser registers the legacy terminal set and uses distinct direct-invoke states so the closing marker can return to content without changing wrapped-call behavior.

Compatibility

The change is parser-only. It does not alter prompts, schemas, token sampling, speculative decoding, or canonical fullwidth DSML output.

This pull request supersedes #399. The parser behavior is equivalent, while its compatibility tests live in test_deepseek_v4_legacy_dsml.py; this keeps the patch mergeable with the Infernal Invocation strict-tool regression stack.

Validation

• DeepSeek V4 parser and legacy compatibility tests: 67 passed.
• Composed Infernal Invocation parser, XGrammar validation, and legacy compatibility tests: 70 passed.
• ruff check: pass.
• The composition merge against dev/infernal-invocation plus the required open Infernal Invocation PR set is conflict-free.

[coderabbitai]

Caution

Review failed

An error occurred during the review process. Please try again later.

---

Thanks for using CodeRabbit! It's free for OSS, and your support helps us grow. If you like it, consider giving us a shout-out.

❤️ Share

• X
• Mastodon
• Reddit
• LinkedIn

_{Comment @coderabbitai help to get the list of available commands.}