[bvolpato]

Overview

Non-streaming responses from engine-based tool parsers can drop assistant text after a tool call, leaving clients with incomplete message.content. #59218 reported this and was closed by its author, and the bug still reproduces on main. Drain the deferred text while preserving tool-call order and arguments.

Claims

• Retain assistant prose before, between, and after tool calls.
• Preserve tool order, decoded arguments, and existing whitespace behavior.
• Leave streaming parsing unchanged.

Validation

A real OpenAI server preserves text around tool calls with this change, and the regressions fail without it. GPU results used NVIDIA B200 GPUs with driver 580.95.05 and a precompiled build of main b8c04dd3b4, which this branch includes. The comparison tree is this branch with only the production files restored to main.

.venv/bin/python -m pytest \
  tests/entrypoints/openai/chat_completion/test_completion_with_function_calling.py -q
.venv/bin/python -m pytest tests/tool_parsers tests/parser tests/reasoning -q

Without the fix: test_non_streaming_preserves_text_after_tool_call
                 assert None == 'I will check the weather.'
                 test_extract_tool_calls_keeps_text_around_tool_calls
                 assert 'Before.' == 'Before.  Between.  After.'
With the fix:    the commands above pass

This real-server run with qwen3_coder shows the client-visible result. It ran on the same hardware against main 3a90cc6fd0, before the latest commits on this branch. Tool names and arguments matched in every case.

Model output	message.content without the fix	message.content with the fix
Text after one tool call	None	'I will check the weather.'
Text before, between, and after two tool calls	'Before.'	'Before.Between.After.'
Text before one tool call	'Before.'	'Before.'
No text	None	None

Parser-level comparison of a complete parse and a stream of the same output on this branch:

Parser	Output	Stream	Complete parse without the fix	Complete parse with the fix
minimax_m2	Text after a tool call	'After.'	None	'After.'
minimax_m2	Text before and after a tool call	'Before. After.'	'Before.'	'Before. After.'
gemma4	Text after a tool call	'Done.', in chunks of 1 and 7 characters	None	'Done.'

A wider selection also passes on this branch: the parser files in tests/tool_use, tests/entrypoints/tool_parsers, and the chat, responses, and message-assembly unit tests in tests/entrypoints/openai. The only errors are the [llama] cases of test_hermes_tool_parser.py, which need a gated checkpoint that this host cannot access, and they error the same way on main.

Two existing tests recorded the dropped text, so this PR changes their expectations. test_step3p5_tool_parser.py expected 'hello' in its non-streaming case, while the same file's streaming case expects both hello and hi. test_gemma4_response_template_preserves_content_after_tool_call expected None from the registered parser and 'Done.' from the response template parser, and both now return 'Done.'.

The real-server file and the three test directories also pass with this branch merged onto main 410ab341a8, with the same build, because no native code changed between the two commits.

Model quality was not evaluated, because the change only returns text that the model already generated.

CI coverage: (CPU) Tool Parsers selects the moved adapter regression and the Step3p5 test. (H200 MIG 35GB) Entrypoints Integration (OpenAI API chat_completion) and its (MI355 DPX) mirror select the real-server regression, but ROCm was not run. No enabled step selects tests/parser, because its step in misc.yaml is commented out, so the Gemma4 test and test_inkling.py ran only in the validation above.

(H200 MIG 35GB) Rust Frontend Tool Use and its mirror exclude the new regression. A real Rust frontend build returned assert None == 'I will check the weather.' for it. Under an answer constraint the Rust grammar keeps tool calls optional (tool_grammar_applies), so the regex does not force text after a call. The rest of the file passes on that Rust frontend build. The regressions of this PR also pass with this branch merged onto main b2a9480dc7, with its own precompiled build and VLLM_WORKER_MULTIPROC_METHOD=spawn as in CI. Upstream checks are stopped at contributor authorization, so Buildkite has not run this branch.

Details

Streaming conversion defers text after tool events until the next delta. The complete-parse path returns without that next conversion, so queued prose is lost. Drain that shared buffer through the existing converter before normalizing the response. InklingParser had its own override that drained the same buffer, so this change removes that override. Qwen3 adapter and real-server regressions cover the client-visible behavior, and the tool-calling guide describes it.

The tool-calling guide names the limit of this change: parsers such as deepseek_v4 stop at the tool call by design, and their text after a tool call is still dropped in both modes.

Rust frontend: This change touches only the Python frontend. Rust qwen_coder already keeps text after a tool call in a complete parse, checked with a temporary cargo test on parse_complete, so this change makes the Python result match it. Rust minimax_m2 ignores that text in both modes (minimax_m2_parse_complete_preserves_prefix_and_ignores_trailing_text). The Python minimax_m2 stream already returned it, so the frontends already differed for streaming. With this PR, Python complete parses also return it, so the frontends now also differ for complete parses of minimax_m2. This PR keeps Python consistent with its own stream and does not change Rust.

This does not duplicate an open PR. #59234 proposed an overlapping fix, and its author closed it without a merge. #56270 fixed the same class of bug in the DeepSeek-V3 parser and also kept the text. Open #63093 changes only the dots parser, which does not use this engine.

AI assistance was used for this change.

---

Pull Request Checklist

• I used vLLM's /pr-checklist skill. (Mandatory for agents, optional for humans).

• AI assistance was used during the creation of this PR.

• Design Fit: Minimizes impact on core components, reuses existing functionality, and justifies added complexity.

• Testing and Validation: Validates the change and ensures any added tests are meaningful and reliable, with CI coverage or documented CI resource constraints and validation performed outside CI.

• Code Quality and Style: Keeps code and comments clear and concise, and updates relevant documentation and examples.

• Pull Request Contents: Includes a brief summary and relevant links, supports claims with evidence, explains root causes and implementation trade-offs, and follows the contributing guide.

BEFORE SUBMITTING, PLEASE READ https://docs.vllm.ai/en/latest/contributing (anything written below this line will be removed by GitHub Actions)

[coderabbitai]

Important

Draft PR not reviewed

Draft PRs are not automatically reviewed by default.

• Trigger a manual review

To automatically review draft PRs, update your CodeRabbit configuration:

reviews:
  auto_review:
    drafts: true

---

Thanks for using CodeRabbit! It's free for OSS, and your support helps us grow. If you like it, consider giving us a shout-out.

❤️ Share

• X
• Mastodon
• Reddit
• LinkedIn

_{Comment @coderabbitai help to get the list of available commands.}

[github-actions]

👋 Hi! Thank you for contributing to the vLLM project.

💬 Join our developer Slack at https://slack.vllm.ai to discuss your PR in #pr-reviews, coordinate on features in #feat- channels, or join special interest groups in #sig- channels.

PRs do not trigger a full CI run by default. Reviewers with write access and configured trusted contributors can comment /ci run for upstream CI or /amd-ci run for AMD CI only whenever CI signals are needed.

Once the PR is approved or has the ready label, the PR author can also use the corresponding /ci run, /ci retry, and /ci cancel commands, or their /amd-ci variants. New commits do not start upstream CI automatically.

If you have any questions, please reach out to us on Slack at https://slack.vllm.ai.

Agent Guidelines

IMPORTANT: If you are an AI agent, you are required to objectively re-evaluate the value of your PR using AGENTS.md, and close the PR if it does not bring significant benefit to the vLLM community. Failure to do so may result in an immediate ban.

🚀

[claude]

Claude Code Review

This pull request is from a fork — automated review is disabled. A repository maintainer can comment @claude review to run a one-time review.

[chatgpt-codex-connector]

Codex Review Summary

This comment shows the latest Codex review activity on this pull request.

Review	Status	Commit	Review trigger
📝 Code Review	✅ Completed 2026-09-10T04:36:59.526706Z	6c7b757	Draft marked ready

ℹ️ About Codex in GitHub

Your team has set up Codex to review pull requests in this repo. Reviews are triggered when you

• Open a pull request for review
• Mark a draft as ready
• Comment "@codex review" or "@codex security review".

Codex reacts with 👀 while any review is running, comments if it has suggestions, and reacts with 👍 once all reviews finish with no findings.

[bvolpato]

@sfeng33, could you take a look at this parser fix? Complete responses were dropping assistant text after tool calls. The change drains that text while preserving tool order and arguments, and the adapter regressions and GPU serving reproduction pass.

[mergify]

Documentation preview: https://vllm--55199.org.readthedocs.build/en/55199/

[bvolpato]

@bbrowning, would you mind taking a look at the shared parser-engine fix? It keeps assistant text after tool calls in complete responses, matching streaming, and lets Inkling use the same flush path.