Repository navigation
[Bugfix] Buffer and validate provisional DSML tool-call recovery (#56482) - #56667
Open
nandan2003 wants to merge 1 commit into
Open
nandan2003 wants to merge 1 commit into
nandan2003 wants to merge 1 commit into
Conversation
nandan2003
requested review from
aarnphm,
bbrowning,
chaunceyjiang and
sfeng33
as code owners
September 13, 2026 05:53
![claude[bot]](https://avatars.githubusercontent.com/in/1236702?s=60&v=4){kind=small}
mergify
Bot
added
deepseek
nandan2003
force-pushed
the
fix/deepseek-v4-dsml-recovery-provisional-commit
branch
2 times, most recently
from
September 16, 2026 03:54
df39d2f to
17e7464
Compare
…m-project#56482) PR vllm-project#55954 introduced recovery for tool calls when the model omits the opening <｜DSML｜tool_calls> wrapper tag by adding an eager transition (ParserState.CONTENT, "INVOKE_PREFIX") -> TOOL_NAME emitting TOOL_CALL_START. This eager transition caused several regressions: 1. Truncation / Ghost Calls: If model output ends before arguments or invoke closing tag, an empty-argument tool call ({}) was emitted. 2. Narration / Text Hijacking: Mentioning <｜DSML｜invoke name="..."> in prose or documentation triggered unintended tool calls and swallowed text. 3. Undeclared Tool Execution: Tool names not declared in the request (or requests with tools=None / tool_choice='none') were recovered as real tool calls. Fix: - Make (ParserState.CONTENT, "INVOKE_PREFIX") a provisional transition. - Buffer semantic events and raw tokens in StreamingParserEngine during the recovery hold. - Verify tool name against declared request tools when exiting TOOL_NAME. - Only commit the tool call when (ParserState.TOOL_ARGS, "INVOKE_END") is reached with commit_provisional_tool_call=True. - On any validation failure, truncation at finish(), or unexpected terminal, abort the recovery hold and flush the buffered raw tokens as plain content. Fixes vllm-project#56482. Signed-off-by: Nandan Vallamdasu <nandan.vallamdasu@outlook.com>
nandan2003
force-pushed
the
fix/deepseek-v4-dsml-recovery-provisional-commit
branch
from
September 16, 2026 04:05
17e7464 to
2776924
Compare
Contributor
Author
|
Hey @aarnphm, @bbrowning, @chaunceyjiang and @sfeng33! could you please check this PR when you get a chance? Thanks |
This branch has not been deployed
This file contains hidden or bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
Sign up for free
to join this conversation on GitHub.
Already have an account?
Sign in to comment
1 participant
Add this suggestion to a batch that can be applied as a single commit.This suggestion is invalid because no changes were made to the code.Suggestions cannot be applied while the pull request is closed.Suggestions cannot be applied while viewing a subset of changes.Only one suggestion per line can be applied in a batch.Add this suggestion to a batch that can be applied as a single commit.Applying suggestions on deleted lines is not supported.You must change the existing code in this line in order to create a valid suggestion.Outdated suggestions cannot be applied.This suggestion has been applied or marked resolved.Suggestions cannot be applied from pending reviews.Suggestions cannot be applied on multi-line comments.Suggestions cannot be applied while the pull request is queued to merge.Suggestion cannot be applied right now. Please check back later.
Fixes #56482
PR #55954 introduced recovery for tool calls when the model omits the opening
<｜DSML｜tool_calls>wrapper tag by eagerly transitioning(ParserState.CONTENT, "INVOKE_PREFIX") -> TOOL_NAMEand emittingTOOL_CALL_START.This eager transition caused 3 critical regressions in production:
{}) was emitted.<｜DSML｜invoke name="...">in prose/documentation were intercepted as tool calls.request.tools(or requests withtool_choice="none") were promoted to executable tool calls.Solution
(ParserState.CONTENT, "INVOKE_PREFIX")as aprovisional_tool_call.StreamingParserEngineduring recovery hold.request.toolsatTOOL_NAME -> TOOL_ARGS.</｜DSML｜invoke>is reached (commit_provisional_tool_call=True).finish()), rolls back and flushes the buffered raw tokens as plain user content.Test Plan & Result
pytest tests/parser/engine/test_deepseek_v4.py(108 passed)pytest tests/tool_parsers/test_deepseekv4_tool_parser.py(12 passed)pytest tests/parser/engine/(4,290 passed, 0 regressions across all registered models)Essential Elements of an Effective PR Description Checklist
supported_models.mdandexamplesfor a new model.