Skip to content

fix: streaming tool calls drop for Qwen3.6 bracket format #374

New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

Sign up for GitHub

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Open
mikepixelmagic-dev wants to merge 1 commit into
base: main
Choose a base branch
from
Open

fix: streaming tool calls drop for Qwen3.6 bracket format #374

Show file tree
Hide file tree
Changes from all commits
Commits
Show all changes
1 commit
Select commit
File filter

Filter by extension

Filter by extension
Conversations
Failed to load comments.
Loading
Jump to
Jump to file
Failed to load files.
Loading
Diff view
Diff view
9 changes: 8 additions & 1 deletion vllm_mlx/server.py
View file
Open in desktop
Original file line number Diff line number Diff line change
Expand Up @@ -1365,7 +1365,14 @@ def _start_reasoning_item() -> list[str]:

content = SPECIAL_TOKENS_PATTERN.sub("", delta_text)
if tool_parser and delta_text:
if not tool_markup_possible and "<" not in delta_text:
# Fast path: skip parsing until a tool-markup marker appears.
# Use _streaming_tool_markup_possible to catch all supported
# shapes (<tool_call>, <function=, [Calling tool:, [TOOL_CALLS],
# bare bracket [func({...})], etc.) — the old `"<" not in` check
# missed bracket formats and let Qwen3.6-style tool calls leak.
if not tool_markup_possible and not _streaming_tool_markup_possible(
tool_accumulated_text + delta_text
):
tool_accumulated_text += delta_text
else:
if not tool_markup_possible:
Expand Down
8 changes: 5 additions & 3 deletions vllm_mlx/tool_parsers/qwen_tool_parser.py
View file
Open in desktop
Original file line number Diff line number Diff line change
Expand Up @@ -272,9 +272,11 @@ def extract_tool_calls_streaming(

return None

# If we're in a tool call, accumulate and parse at the end
# For simplicity, return None during accumulation
if "</tool_call>" in delta_text or ")]" in delta_text:
# If we're in a tool call, accumulate and parse at the end.
# Check current_text (accumulated), not delta_text — closing markers
# like ")]" or "</tool_call>" often span token boundaries and may
# never appear within a single delta chunk.
if "</tool_call>" in current_text or ")]" in current_text:
# Tool call complete, parse the whole thing
result = self.extract_tool_calls(current_text)
if result.tools_called:
Expand Down