[denelbjohn18]

Overview

Fixes #58147. A <tool_call> block quoted as an example inside explicitly closed Qwen3 reasoning was executed as a real tool call. The parser now defers a tool opener seen during reasoning until </think> or the end of the stream.

Claims

• Tool-shaped text inside explicitly closed Qwen3 reasoning stays reasoning and is never executed.
• Implicit-end tool calls (no </think>) keep working, including through the reasoning adapter.
• Qwen3-derived parsers (Nemotron, MiMo, SeedOSS, Granite) are unchanged.

Validation

• python -m pytest tests/parser/engine/: 6,030 passed across 23 files on the branch rebased onto main at 1a5b187.
• Shared-parser regression run over every file in tests/parser, tests/reasoning and tests/tool_parsers: about 7,870 passed, 0 failures. tests/tool_parsers/test_llama3_json_tool_parser.py could not run locally because it downloads a gated Hugging Face checkpoint (401 without a token).
• The new tests in tests/parser/engine/test_qwen3.py fail on main without the fix (10 failed, 76 passed) and pass with it (86 passed). They cover quoted markup in streaming reasoning, explicit close, a later real tool call, implicit-end calls, adapter finish with complete and partial spans, split markers, multiple quoted spans, empty spans, reset, and skip-tool-parsing mode.
• Existing implicit-end streaming expectations pass unchanged; neither existing Qwen3 reasoning test file was edited.
• Ruff check, Ruff format check, and git diff --check pass.

Reproduce the bug and check the fix: python -m pytest tests/parser/engine/test_qwen3.py tests/parser/engine/test_parser_engine.py -q (214 passed).

Details

Root cause: the parser fired the implicit (REASONING, TOOL_START) transition on sight, so a quoted example inside reasoning became a real call. Design: hold the opener and its span while reasoning is open. An explicit </think> returns the span as reasoning; reaching stream end replays it through the existing implicit-end path. The hold is an opt-in config flag, default off, enabled only for base Qwen3 with thinking on and a prompt inside an open think block.

In the issue thread, I reproduced and diagnosed the behavior. ArtyomITA explored an environment-flag alternative, then abandoned it because it broke seven implicit-end tests and the use case in #39056. #57553 is related follow-up work for Qwen3-derived parsers.

Tradeoffs: for Qwen3 thinking requests, implicit-end tool calls resolve at stream end rather than incrementally; the held span stays buffered until resolution; a small finish option (resolve_deferred_tool_start) was added to the parser engine so the reasoning adapter can resolve a held span at stream end. It is an internal parser-engine hook, not a serving/API option.

AI-assisted with Cursor; all changes reviewed and verified by me.

---

Pull Request Checklist

• I used vLLM's /pr-checklist skill. (Mandatory for agents, optional for humans).

• AI assistance was used during the creation of this PR.

• Design Fit: Minimizes impact on core components, reuses existing functionality, and justifies added complexity.

• Testing and Validation: Validates the change and ensures any added tests are meaningful and reliable, with CI coverage or documented CI resource constraints and validation performed outside CI.

• Code Quality and Style: Keeps code and comments clear and concise, and updates relevant documentation and examples.

• Pull Request Contents: Includes a brief summary and relevant links, supports claims with evidence, explains root causes and implementation trade-offs, and follows the contributing guide.

[claude]

Claude Code Review

This pull request is from a fork — automated review is disabled. A repository maintainer can comment @claude review to run a one-time review.

[github-actions]

👋 Hi! Thank you for contributing to the vLLM project.

💬 Join our developer Slack at https://slack.vllm.ai to discuss your PR in #pr-reviews, coordinate on features in #feat- channels, or join special interest groups in #sig- channels.

PRs do not trigger a full CI run by default. Reviewers with write access and configured trusted contributors can comment /ci run for upstream CI or /amd-ci run for AMD CI only whenever CI signals are needed.

Once the PR is approved or has the ready label, the PR author can also use the corresponding /ci run, /ci retry, and /ci cancel commands, or their /amd-ci variants. New commits do not start upstream CI automatically.

If you have any questions, please reach out to us on Slack at https://slack.vllm.ai.

Agent Guidelines

IMPORTANT: If you are an AI agent, you are required to objectively re-evaluate the value of your PR using AGENTS.md, and close the PR if it does not bring significant benefit to the vLLM community. Failure to do so may result in an immediate ban.

🚀

[denelbjohn18]

Hi @aarnphm, @sfeng33, and @bbrowning - when you have a chance, could you take a look at this PR? I added the regression tests and checked the existing parser tests. I'm happy to make changes if needed.

[mergify]

This pull request has merge conflicts that must be resolved before it can be
merged. Please rebase the PR, @denelbjohn18.

https://docs.github.com/en/pull-requests/collaborating-with-pull-requests/working-with-forks/syncing-a-fork

[thediymaker]

An independent repro, plus one path this does not cover yet.

On Qwen3.6-27B and Qwen3.6-35B-A3B (3 tool-call cases x 40 samples, T=0.6, qwen3 reasoning parser), the implicit end gave 110/120 and 115/120. The 27B failures were 7 calls named example_function_name (the template's own example, quoted while thinking) and 3 duplicates. Ending reasoning only at </think> gave 120/120 and 240/240.

The gap: this PR defers the transition in the streaming parser, but (REASONING, TOOL_START) still carries REASONING_END, so _derive_reasoning_end_token_ids still includes <tool_call>. The structured-output manager uses find_reasoning_end / is_reasoning_end from that set (_get_constraint_bounds). With tool_choice="required" or a named tool, the grammar can start with <tool_call>, so _masked_reasoning_ends does not mask it, and the grammar switches on at the quoted opener and forces it into a call. The held span then resolves through the implicit-end path at stream end. A test that runs a required tag through StructuredOutputManager would show it. When deferral is on, one option is to drop <tool_call> from the reasoning-end ids. Our local fix (opt-in, for models that always close </think>) removes both the transition and that id.

AI assistance was used in preparing this comment.

[denelbjohn18]

@thediymaker thanks for the detailed repro and for catching the structured-output path. I checked it locally on CPU with an external probe using the real qwen_3_coder grammar through StructuredOutputManager. With tool_choice required or a named tool, the quoted <tool_call> flips reasoning_ended and starts grammar enforcement. This also broke the case with a quoted opener followed by and a real call.

A genuine call without passed the same probe. It relies on that same <tool_call> reasoning-end id for grammar enforcement. Dropping the id while deferral applies would fix the quoted cases, but remove grammar enforcement from that currently working path.

Before I make that change, is the intended behaviour to require before grammar enforcement starts when deferral applies, even for required/named calls? I'd appreciate a maintainer's read too, so I don't silently change the existing behaviour.

These checks were local, with no model sampling. I haven't pushed a change yet. Cursor helped with the analysis and probe.

[thediymaker]

Our read, for what it is worth (a maintainer should make the call): yes, when deferral applies, requiring </think> before grammar enforcement is the consistent choice. Deferral already treats an unclosed <tool_call> inside reasoning as possibly quoted text, and enforcement keyed on that same id contradicts it.

The cost is narrower than it looks. With thinking disabled, the prompt already carries a closed think block, so reasoning has ended before the first token and enforcement starts as today. The path that loses enforcement is a model that emits a real call while still thinking and never closes </think>. That call still parses through the implicit-end path at stream end, just unconstrained, which is how tool_choice="auto" handles it today.

For the Qwen3.6 models we measured, which close </think> before calling, ending reasoning only at </think> gave 120/120 and 240/240 against 110/120 and 115/120 with the implicit end. If the trade-off is a concern, the id could be dropped only when deferral is active, so models and configs without deferral keep today's behaviour.

AI assistance was used in preparing this comment.

[mergify]

This pull request has merge conflicts that must be resolved before it can be
merged. Please rebase the PR, @denelbjohn18.

https://docs.github.com/en/pull-requests/collaborating-with-pull-requests/working-with-forks/syncing-a-fork