Repository navigation
[Bugfix] Parse DSML tool calls when the model omits the tool_calls wrapper - #55954
Conversation
Signed-off-by: sfeng33 <4florafeng@gmail.com>
![claude[bot]](https://avatars.githubusercontent.com/in/1236702?s=60&v=4){kind=small}
Signed-off-by: sfeng33 <4florafeng@gmail.com>
|
/ci run |
|
✅ Triggered Buildkite CI #87770 for commit |
When thinking is on, this part would be inside reasoning block and would not parsed as a tool call. If thinking is off, it'd be parsed as tool call. Not sure I understand what 'not a real call' is doing here, but any content after the tool call end marker should get dropped, this is to match the official dsv4 implementation. |
|
I meant incomplete invoke, not content after the end marker. This input has neither |
I see, this would be parsed as 'terminal' being the tool call, as {} as its arg. 'Documentation: ' is the content. |
yewentao256
left a comment
yewentao256
left a comment
There was a problem hiding this comment.
okk, if we can accept without </｜DSML｜invoke> the tool call can still be used
github-actions
Bot
added
the
ready
…apper (vllm-project#55954) Signed-off-by: sfeng33 <4florafeng@gmail.com> Signed-off-by: Jyotirmoy Roy <jyotirmoyroy649@gmail.com>
…m-project#56482) PR vllm-project#55954 introduced recovery for tool calls when the model omits the opening <｜DSML｜tool_calls> wrapper tag by adding an eager transition (ParserState.CONTENT, "INVOKE_PREFIX") -> TOOL_NAME emitting TOOL_CALL_START. This eager transition caused several regressions: 1. Truncation / Ghost Calls: If model output ends before arguments or invoke closing tag, an empty-argument tool call ({}) was emitted. 2. Narration / Text Hijacking: Mentioning <｜DSML｜invoke name="..."> in prose or documentation triggered unintended tool calls and swallowed text. 3. Undeclared Tool Execution: Tool names not declared in the request (or requests with tools=None / tool_choice='none') were recovered as real tool calls. Fix: - Make (ParserState.CONTENT, "INVOKE_PREFIX") a provisional transition. - Buffer semantic events and raw tokens in StreamingParserEngine during the recovery hold. - Verify tool name against declared request tools when exiting TOOL_NAME. - Only commit the tool call when (ParserState.TOOL_ARGS, "INVOKE_END") is reached with commit_provisional_tool_call=True. - On any validation failure, truncation at finish(), or unexpected terminal, abort the recovery hold and flush the buffered raw tokens as plain content. Fixes vllm-project#56482. Signed-off-by: Nandan Vallamdasu <nandan.vallamdasu@outlook.com>
4 participants
Purpose
Fixes #48931.
At long context DeepSeek-V4 intermittently omits the
<｜DSML｜tool_calls>opener while still emitting a well-formed<｜DSML｜invoke ...>block. Detection anchored solely on the opener, so the whole block leaked to the client as assistant text and no tool call was produced.Behaviour change: trailing text after a well-formed tool block was previously returned as content on both V4 and V3.2, and is now dropped.
Test Plan
pytest tests/parser/engine/test_deepseek_v4.py \ tests/parser/engine/test_deepseek_v32.py \ tests/tool_parsers/test_deepseekv4_tool_parser.py \ tests/tool_parsers/test_deepseekv32_tool_parser.py \ tests/parser/engine/test_parser_engine.py