Skip to content

Add deepseek_dsml tool parser for DeepSeek-V4's native DSML format - #1337

Open
snagnever wants to merge 1 commit into
ml-explore:mainfrom
snagnever:add-deepseek-dsml-parser
Open

Add deepseek_dsml tool parser for DeepSeek-V4's native DSML format#1337
snagnever wants to merge 1 commit into
ml-explore:mainfrom
snagnever:add-deepseek-dsml-parser

Conversation

@snagnever

Copy link
Copy Markdown

Adds a tool parser for DeepSeek-V4 (Flash/Pro)'s native DSML tool-call format, so tool calling works for these models through the OpenAI-compatible server.

Why

DeepSeek-V4 emits tool calls in its own DSML format — not Hermes <tool_call> JSON or any of the existing parsers' formats — so there is currently no tool_parser_type that parses its output, and tool calls come back as plain content.

Format

<|DSML|tool_calls>
<|DSML|invoke name="get_weather">
<|DSML|parameter name="city" string="true">Paris</|DSML|parameter>
<|DSML|parameter name="days" string="false">3</|DSML|parameter>
</|DSML|invoke>
<|DSML|invoke name="get_weather">...</|DSML|invoke>
</|DSML|tool_calls>
  • Multiple <|DSML|invoke> per block → native parallel calls.
  • string="true" → literal string value; string="false" → JSON value (number/bool/array/object).

Change

  • mlx_lm/tool_parsers/deepseek_dsml.py — new parser (modeled on minimax_m2).
  • _infer_tool_parser — one entry so the official DSML chat template (deepseek-ai/DeepSeek-V4-Flash#16) auto-selects it; no manual tool_parser_type needed.
  • Tests — single call, parallel calls, mixed string/JSON params.

Marker note

The start/end markers use the <|DSML|tool_calls prefix (dropping the trailing >). mlx-lm matches markers by exact token-id sequence, and on this tokenizer the > merges with the following byte (...calls>\n → one token), so the full marker never matches — the same class of issue as #1335. The |DSML| core is a special token, so the prefix is a stable anchor; the parser tolerates the leftover > before the first <|DSML|invoke>.

Verification

On mlx-community/DeepSeek-V4-Flash-2bit-DQ (no tool template → tools dropped → 0 tool calls as a baseline), adding the official DSML template + this parser: 39/40 (98%) on a jdhodges-style tool suite and 8/8 on the parallel multi-tool cases — competitive with full-size locals. python -m unittest tests.test_tool_parsing passes (existing parsers unaffected; this is purely additive).

Pairs with the official template PR #16. Independent of #1335/#1336 (different module), though it uses the same prefix-marker insight.

DeepSeek-V4 (Flash/Pro) emits tool calls in its native DSML format:

  <|DSML|tool_calls>
  <|DSML|invoke name="get_weather">
  <|DSML|parameter name="city" string="true">Paris</|DSML|parameter>
  </|DSML|invoke>
  ...
  </|DSML|tool_calls>

with multiple <|DSML|invoke> per block (native parallel calls). string="true"
means a literal string value, string="false" a JSON value.

Adds mlx_lm/tool_parsers/deepseek_dsml.py (modeled on minimax_m2) and an
_infer_tool_parser entry so the official DSML chat template auto-selects it. The
start/end markers use the "<|DSML|tool_calls" prefix (dropping the trailing ">"):
mlx-lm matches markers by token-id sequence and the ">" merges with the following
byte on this tokenizer (same class of issue as ml-explore#1335); the parser extracts the
invokes/parameters regardless of the leftover ">".

Verified on mlx-community/DeepSeek-V4-Flash-2bit-DQ: 0 -> 39/40 (98%) on a
jdhodges-style tool suite (the no-tool-template baseline scored 0), 8/8 on parallel
multi-tool cases. Adds tests (single, parallel, mixed string/JSON).

Co-Authored-By: Claude Opus 4.8 (1M context) <noreply@anthropic.com>
@nh13

nh13 commented Aug 19, 2026

Copy link
Copy Markdown
Contributor

Thanks for this, @snagnever — I hit the same DSML tool-calling gap on DeepSeek-V4-Flash and independently landed on the same parser shape (parallel invokes, string="true|false", the no-trailing-> marker anchor). Two things from running it against the 4/5/6-bit MLX quants that may be worth folding in:

  1. Truncated invoke close tag — those quants systematically emit </|DSML|inv> instead of </|DSML|invoke>, so the strict close in _INVOKE misses the call and it returns as content. A lenient close (</|DSML|inv[^>]*>) recovers it.
  2. Encode side — DeepSeek ships no Jinja template (only encoding_dsv4.py), so there's currently nothing to pair the parser with under chat_template_type. I ported encode_messages to a small chat_templates/deepseek_dsml.py reusing the deepseek_v32 helpers, byte-exact for single-turn prompts.

Both are on a branch stacked on this PR (#1760, draft) if useful — glad to fold them into yours instead if you'd rather. Also worth noting: #1501 (merged) makes the trailing-> marker workaround unnecessary on current main.

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

Projects

None yet

Development

Successfully merging this pull request may close these issues.

3 participants