Skip to content

[Bugfix] Fix forced tool_choice crash when reasoning_parser consumes all content#40529

Open
FutureSkyFly wants to merge 1 commit into
vllm-project:mainfrom
FutureSkyFly:fix/tool-choice-empty-content-crash
Open

[Bugfix] Fix forced tool_choice crash when reasoning_parser consumes all content#40529
FutureSkyFly wants to merge 1 commit into
vllm-project:mainfrom
FutureSkyFly:fix/tool-choice-empty-content-crash

Conversation

@FutureSkyFly

Copy link
Copy Markdown
Contributor

Summary

When using --reasoning-parser (e.g., glm45, deepseek_r1) with forced tool_choice, the reasoning parser may consume the entire model output into <think>...</think> tags, leaving content as None. This causes AssertionError crashes in the forced tool_choice code paths.

Fix: Normalize None content to empty string "" (consistent with the existing "required" branch which already does content = content or ""), and replace the downstream assert tool_calls with a defensive fallback.

Changes

File Change
vllm/entrypoints/openai/engine/serving.py Replace 2x assert content is not Nonecontent = content or ""
vllm/parser/abstract_parser.py Replace 2x assert content is not Nonecontent = content or ""
vllm/entrypoints/openai/chat_completion/serving.py Replace assert tool_calls is not None and len(tool_calls) > 0tool_calls = tool_calls or []

Total: 5 lines changed across 3 files.

How to reproduce

vllm serve <reasoning-model> --reasoning-parser glm45 --enable-auto-tool-choice --tool-call-parser hermes
response = client.chat.completions.create(
    model="<model>",
    messages=[{"role": "user", "content": "What's the weather?"}],
    tools=[{"type": "function", "function": {"name": "get_weather", "parameters": {}}}],
    tool_choice={"type": "function", "function": {"name": "get_weather"}}
)
# Crash: AssertionError (no message) when model output is all <think>...</think>

Why this is not a duplicate

PR #40148 fixes engine/serving.py and abstract_parser.py but does not fix the downstream assert in chat_completion/serving.py (line 1417). This PR covers all 5 crash points across all 3 files.

Fixes #40528
Related: #40147, #40148

Test

The fix is consistent with the existing "required" branch pattern which already uses content = content or "" and tool_calls = tool_calls or [].

AI Disclosure

AI assistance was used in identifying and fixing this bug.

Signed-off-by: liuchenbing liuchenbing2026@outlook.com

@claude claude Bot left a comment

Copy link
Copy Markdown

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Claude Code Review

This pull request is from a fork — automated review is disabled. A repository maintainer can comment @claude review to run a one-time review.

@gemini-code-assist gemini-code-assist Bot left a comment

Copy link
Copy Markdown
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Code Review

This pull request replaces several assert statements with default value assignments in the OpenAI chat completion serving, engine serving, and abstract parser modules. Specifically, it ensures that tool_calls defaults to an empty list and content defaults to an empty string when they are None, preventing potential assertion errors and improving robustness. I have no feedback to provide.

…all content

When using reasoning parsers (e.g. glm45, deepseek_r1) with forced
tool_choice, the parser may consume the entire model output into
<think>...</think> tags, leaving content as None. This caused
AssertionError crashes in forced tool_choice paths.

Fix: normalize None content to empty string (consistent with the
existing "required" branch behavior) and replace the downstream
assert with a defensive fallback.

Changes:
- engine/serving.py: replace 2x `assert content is not None` with
  `content = content or ""` in ToolChoiceFunction and
  ChatCompletionNamedToolChoiceParam branches
- abstract_parser.py: same fix for the Responses API parsing path
- chat_completion/serving.py: replace `assert tool_calls is not None`
  with `tool_calls = tool_calls or []` for defensive downstream handling

Fixes vllm-project#40528

Signed-off-by: liuchenbing <liuchenbing2026@outlook.com>
@FutureSkyFly FutureSkyFly force-pushed the fix/tool-choice-empty-content-crash branch from 0c51175 to e516b5e Compare April 21, 2026 16:03
@mergify

mergify Bot commented May 23, 2026

Copy link
Copy Markdown
Contributor

This pull request has merge conflicts that must be resolved before it can be
merged. Please rebase the PR, @liuchen2026fly.

https://docs.github.com/en/pull-requests/collaborating-with-pull-requests/working-with-forks/syncing-a-fork

@mergify mergify Bot added the needs-rebase label May 23, 2026
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

bug Something isn't working frontend needs-rebase

Projects

None yet

Development

Successfully merging this pull request may close these issues.

[Bug]: Forced tool_choice crashes with AssertionError when reasoning_parser consumes all content

1 participant