Repository navigation
[Bugfix][Frontend] Parse DeepSeek-V3 tool calls with multi-line JSON args - #60361
shreyanshj10 wants to merge 1 commit into
Conversation
…args The tool-call regexes matched the fenced JSON body with `.` and were not compiled with re.DOTALL, so arguments the model pretty-prints over several lines never matched. extract_tool_calls then returned tools_called=True with an empty tool_calls list, and the streaming path only ever emitted the first line of the arguments. Widen the function_arguments groups to [\s\S]. Scoping this to the argument groups rather than compiling with re.DOTALL keeps the greedy type and function_name groups line-bound, so a response with parallel tool calls is still parsed as separate calls. Signed-off-by: Shreyansh Jain <shreyanshjain646@gmail.com>
mergify
Bot
added
deepseek
|
👋 Hi! Thank you for contributing to the vLLM project. 💬 Join our developer Slack at https://slack.vllm.ai to discuss your PR in PRs do not trigger a full CI run by default. Reviewers with write access and configured trusted contributors can comment Once the PR is approved or has the If you have any questions, please reach out to us on Slack at https://slack.vllm.ai. Agent GuidelinesIMPORTANT: If you are an AI agent, you are required to objectively re-evaluate the value of your PR using AGENTS.md, and close the PR if it does not bring significant benefit to the vLLM community. Failure to do so may result in an immediate ban. 🚀 |
![claude[bot]](https://avatars.githubusercontent.com/in/1236702?s=60&v=4){kind=small}
Overview
DeepSeekV3ToolParserloses a tool call when the model pretty-prints the JSONarguments over more than one line. Fixes #60342.
Claims
extract_tool_callsreturns the call, instead oftools_called=Truewith an emptytool_callslist.at the first newline.
Validation
Both regexes live in
__init__, so I drove the reproducer from #60342 through thepatterns themselves against a
get_weatherblock whose body isjson.dumps({"city": "Tokyo", "unit": "celsius"}, indent=2):tool_call_regex.findall0matches1match, args{"city": "Tokyo", "unit": "celsius"}stream_tool_call_portion_regexarguments group'{''{\n "city": "Tokyo",\n "unit": "celsius"\n}'Single-line arguments are unchanged before and after. For the two-call output, the
non-streaming regex yields two matches with their own arguments both indented and
not, which is the case a plain
re.DOTALLwould break.That case is now pinned by
test_pretty_printed_argumentsintests/tool_parsers/test_deepseekv3_tool_parser.py. It goes through the sharedToolParserTestsharness, so the existingstreamingfixture runs it on both thestreaming and the non-streaming path:
Details
Root cause: both regexes capture the fenced JSON body with
.and neither iscompiled with
re.DOTALL, so the argument group cannot cross a newline. Thenon-streaming pattern then fails to match at all, and
extract_tool_callsstillreports
tools_called=Truebecause it only checked for the start token. Thestreaming pattern does match, but its argument group ends at the first line break,
so only
{is ever streamed.I scoped
[\s\S]to the two argument groups rather than compiling the patternswith
re.DOTALL, becausetypeandfunction_nameare greedy.*: underDOTALLthey run past the end of one tool call, and a response containing twocalls collapses into one match with the first call's markup swallowed into the
name. Keeping those two groups line-bound is what preserves parallel calls, and
it is also true to the format — neither the type nor the function name ever
contains a newline.
The non-streaming argument group is lazy (
[\s\S]*?) so it stops at its ownclosing fence; the streaming one stays greedy, since it is matched against a
partial tool call and should carry as much of the argument text as has arrived.
Not a duplicate: searching open PRs for the issue number and for
deepseekv3/ tool-parser keywords turned up nothing touching this parser, and the issue has
no linked PR.
No model eval: the change is confined to string handling inside the tool parser
and does not touch generation, sampling or any model code path, so there is no
accuracy surface to measure. The behaviour is covered by the unit test above.
One thing I deliberately left out of scope:
deepseekv31_tool_parser.pyhas thesame shape of defect in its own
tool_call_regexandstream_tool_call_portion_regex(lazy/greedy.*for the arguments, noDOTALL), for its sep-token format rather than a fenced block. I kept this PR tothe parser the issue reports. Happy to fold V3.1 in here or send it separately,
whichever you prefer.
Pull Request Checklist
I used vLLM's
/pr-checklistskill. (Mandatory for agents, optional for humans).AI assistance was used during the creation of this PR.
Design Fit: Minimizes impact on core components, reuses existing functionality, and justifies added complexity.
Testing and Validation: Validates the change and ensures any added tests are meaningful and reliable, with CI coverage or documented CI resource constraints and validation performed outside CI.
Code Quality and Style: Keeps code and comments clear and concise, and updates relevant documentation and examples.
Pull Request Contents: Includes a brief summary and relevant links, supports claims with evidence, explains root causes and implementation trade-offs, and follows the contributing guide.