Skip to content

[Bugfix] Fix Qwen3Coder prev_tool_call_arr double-emission on parse failure #41466

New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

Sign up for GitHub

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Draft
ToastyTheBot wants to merge 1 commit into
base: main
Choose a base branch
from
+25 −0
Draft

[Bugfix] Fix Qwen3Coder prev_tool_call_arr double-emission on parse failure #41466

Changes from all commits
Commits
Show all changes
1 commit
Select commit
File filter

Filter by extension

Filter by extension
Conversations
Failed to load comments.
Loading
Jump to
Jump to file
Failed to load files.
Loading
Diff view
Diff view
25 changes: 25 additions & 0 deletions vllm/tool_parsers/qwen3coder_tool_parser.py
View file
Open in desktop
Original file line number Diff line number Diff line change
Expand Up @@ -639,6 +639,7 @@ def extract_tool_calls_streaming(
func_content_end = tool_text.find(self.function_end_token, func_start)
if func_content_end != -1:
func_content = tool_text[func_start:func_content_end]
parse_succeeded = False
try:
parsed_tool = self._parse_xml_function_call(
func_content,
Expand All @@ -649,12 +650,36 @@ def extract_tool_calls_streaming(
self.prev_tool_call_arr[self.current_tool_index][
"arguments"
] = parsed_tool.function.arguments
parse_succeeded = True
except Exception:
logger.debug(
"Failed to parse tool call during streaming: %s",
tool_text,
exc_info=True,
)
# When _parse_xml_function_call fails (returns None
# or throws), prev_tool_call_arr still has the "{}"
# placeholder from the header-sent step. Fall back
# to the arguments that were incrementally streamed so
# the serving layer's remaining-args check produces
# correct output instead of double-emitting "{}".
# Only trigger when parsing actually failed — a successful
# parse with empty parameters produces "{}" legitimately.
if (
not parse_succeeded
and self.current_tool_index < len(self.prev_tool_call_arr)
and self.current_tool_index
< len(self.streamed_args_for_tool)
):
# Append closing brace so prev_tool_call_arr
# matches streamed_args_for_tool after the "+="
# "}" below — otherwise the serving layer's
# remainder check loses the closing brace.
self.prev_tool_call_arr[self.current_tool_index][
"arguments"
] = self.streamed_args_for_tool[
self.current_tool_index
] + "}"

if self.current_tool_index < len(self.streamed_args_for_tool):
self.streamed_args_for_tool[self.current_tool_index] += "}"
Expand Down
Loading