Skip to content

[Bug] Qwen3 reasoning detector silently swallows tool_call when </think> is missing - #22837

Closed
gucasbrg wants to merge 1 commit into
sgl-project:mainfrom
gucasbrg:fix/qwen3-reasoning-tool-call-fallback
Closed

gucasbrg wants to merge 1 commit into
sgl-project:mainfrom
gucasbrg:fix/qwen3-reasoning-tool-call-fallback

Conversation

@gucasbrg

Copy link
Copy Markdown

Motivation

Qwen3Detector forwards every constructor argument to the base class except tool_start_token. The base BaseReasoningFormatDetector already contains a fallback that, when in reasoning state and a tool start token appears, forwards the tool call to normal_text instead of keeping it inside reasoning_content. Because Qwen3Detector never sets this token, the fallback is silently disabled for every Qwen3-family model that uses the qwen3 reasoning parser.

Reproduction (production)

This is reproducible with Qwen3.5-27B (FP8, hybrid Mamba) served via SGLang:

  1. force_reasoning=True — e.g. an enable_thinking=True request, which is the default routing for Qwen3-style chat templates that auto-prepend <think>\n in add_generation_prompt.
  2. The model emits <tool_call>...</tool_call> directly without first emitting </think>. This happens with long contexts, certain code-generation tool calls, or when the model decides not to think before acting.
  3. The entire response is silently routed into reasoning_content. content is null, tool_calls is null, finish_reason is stop.
  4. SGLang server logs print Tool 'None' is not defined in the tools list. Downstream, LangChain-style frameworks raise a Pydantic validation error (ToolMessage tool_call_id None) because they try to construct a tool message from the empty tool call.

The user-facing symptom is that the agent loop silently breaks mid-conversation: the model is producing valid <tool_call> XML, but it never reaches the function-call parser.

Root cause

python/sglang/srt/parser/reasoning_parser.py:

  • BaseReasoningFormatDetector.__init__ accepts tool_start_token and uses it in both detect_and_parse and parse_streaming_increment to escape from reasoning mode early.
  • Qwen3Detector.__init__ does not pass tool_start_token to super().__init__, so it stays None and the fallback is dead code for Qwen3 models.

Fix

One line: pass tool_start_token="<tool_call>" to the base class.

The fallback only triggers when _in_reasoning=True, so behavior is unchanged for enable_thinking=False requests and for normal flows that include a proper </think>.

Tests

Added two regression tests under TestQwen3ForcedReasoningDetector:

  • test_detect_and_parse_tool_call_without_think_close (non-streaming)
  • test_streaming_tool_call_without_think_close (streaming)

Both verify that when force_reasoning=True and the model emits <tool_call> without first closing </think>, the tool call is correctly split into normal_text and the parser flips out of reasoning state.

Verified

Tested standalone against Qwen3.5-27B-FP8 + tool-call-parser=qwen3_coder + reasoning-parser=qwen3:

  • ✅ Before fix: tool call swallowed into reasoning_content, downstream sees empty tool_calls.
  • ✅ After fix: tool_calls[0].id, tool_calls[0].name, tool_calls[0].arguments all populated correctly. reasoning_content still captures the actual reasoning text when present.
  • ✅ Regression: Normal <think>...</think>... flow unchanged.
  • ✅ Regression: enable_thinking=False flow unchanged (untouched code path, since _in_reasoning starts False).

Checklist

…ssing

Qwen3Detector forwards every constructor argument to the base class
except tool_start_token. The base BaseReasoningFormatDetector already
contains a fallback that, when in reasoning state and a tool start
token appears, forwards the tool call to normal_text instead of
keeping it inside reasoning_content. Because Qwen3Detector never sets
this token, the fallback is silently disabled for every Qwen3 family
model that uses the qwen3 reasoning parser.

In production this is reproducible with Qwen3.5-27B served via SGLang:

  1. force_reasoning=True (e.g. enable_thinking=True request, the
     default for Qwen3-style chat templates that auto-prepend <think>)
  2. The model emits <tool_call>...</tool_call> directly without
     first emitting </think> (long contexts, certain tool-call code
     templates, or when the model decides not to think before acting)
  3. The entire response is silently routed into reasoning_content,
     content is null, and tool_calls is null
  4. Downstream the function-call parser sees no tool calls and
     LangChain-style frameworks raise a Pydantic validation error
     ("ToolMessage tool_call_id None") because the framework tries
     to construct a ToolMessage from the empty tool_call

Fix: pass tool_start_token="<tool_call>" to the base class so the
existing fallback is wired up. The fallback only triggers when
_in_reasoning=True, so behavior is unchanged for enable_thinking=False
requests and for normal flows that include a proper </think>.

Added two regression tests under TestQwen3ForcedReasoningDetector
covering both the non-streaming and streaming paths.
@gemini-code-assist

Copy link
Copy Markdown
Contributor

Warning

You have reached your daily quota limit. Please wait up to 24 hours and I will start processing your requests again!

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

None yet

Projects

None yet

Development

Successfully merging this pull request may close these issues.

1 participant