[Alex-Wengg]

Overview

Non-streaming chat derender passes the parser only generate_response.prompt_token_ids, which generate sets only when return_token_ids is on. In the usual render → generate → derender flow the parser therefore never sees the prompt. Since #61857, parsers read the prompt to tell whether it already closed reasoning, so they misclassify the output here. This affects response template parsers as well as Qwen3. For example, with Qwen3 and continue_final_message after a closed </think>, the answer comes back as reasoning.

Claims

• The parser gets the engine's prompt_token_ids when the generate response has them and falls back to the caller's prompt_token_ids otherwise. The engine's take precedence because the caller may send only a suffix.
• DerenderChatRequest.prompt_token_ids and derenderer.md now say that chat derender with a parser needs the full prompt IDs in both modes. The docs also note that a suffix starting after the last reasoning marker still misparses, the same as sending no IDs.

Validation

New E2E test through the render server (Qwen/Qwen3-0.6B --reasoning-parser qwen3, no GPU):

CPU is enough here. Derender only decodes and parses token IDs that were already generated, and the render server loads just the tokenizer.

$ pytest tests/entrypoints/scale_out/derender/test_derender.py::test_e2e_parser_uses_caller_prompt_token_ids
# without the fix:  AssertionError: assert None == ' is 42.'
# with the fix:     PASSED

Response template parser given a suffix (test_prompt_suffix_parses_like_full_prompt_or_no_prompt, a two-turn prompt whose generation prompt opens reasoning, output plan<channel|>answer<turn|>):

prompt IDs given to the parser	reasoning	content
none (before this PR)	None	plan<channel|>answer
full prompt	plan	answer
suffix from the start anchor	plan	answer
suffix past the anchor, with the reasoning opener	plan	answer
suffix past the reasoning opener	None	plan<channel|>answer

A suffix parses like the full prompt while it still holds the opener and like no prompt after that. It is never worse than the empty prefix the parser had before.

The same holds through derender with a real checkpoint: google/gemma-4-E2B-it's response template (--reasoning-parser hf), 15 suffix lengths (last 1–12, 16, 24 and 32 IDs) on 4 prompts (opening reasoning, mid-reasoning, closed reasoning, and the model writing its own). Every suffix parsed like either the full prompt or no prompt. For example, a prompt ending in <|channel>thought\n parses correctly from the last 3 IDs, the first that include <|channel>.

$ pytest tests/parser/test_response_template.py
75 passed
$ pytest tests/entrypoints/scale_out/derender/test_derender.py tests/entrypoints/scale_out/derender/test_derender_stream.py
214 passed, 1 xfailed

Details

• [Bugfix][Parser] Honor a prompt-closed reasoning block in non-streaming parse and reasoning token counts #61857 made the parsers honor a prompt-closed reasoning block in non-streaming parse and reasoning token counts, but didn't touch derender.
• Not a duplicate: no open PR changes this call (searched "derender prompt_token_ids" and "derender reasoning"). [Bugfix] Report reasoning and cached tokens in chat derender usage #64041's reasoning counts build on it and [Bugfix][Frontend] Honor --response-role and echo in chat derender #63357 touches the next line, so this should go in first.
• AI assistance (Claude Code) was used. I reviewed the changed lines and ran the tests above.

---

Pull Request Checklist

• I used vLLM's /pr-checklist skill. (Mandatory for agents, optional for humans).

• AI assistance was used during the creation of this PR.

• Design Fit: Minimizes impact on core components, reuses existing functionality, and justifies added complexity.

• Testing and Validation: Validates the change and ensures any added tests are meaningful and reliable, with CI coverage or documented CI resource constraints and validation performed outside CI.

• Code Quality and Style: Keeps code and comments clear and concise, and updates relevant documentation and examples.

• Pull Request Contents: Includes a brief summary and relevant links, supports claims with evidence, explains root causes and implementation trade-offs, and follows the contributing guide.

[github-actions]

👋 Hi! Thank you for contributing to the vLLM project.

💬 Join our developer Slack at https://slack.vllm.ai to discuss your PR in #pr-reviews, coordinate on features in #feat- channels, or join special interest groups in #sig- channels.

PRs do not trigger a full CI run by default. Reviewers with write access and configured trusted contributors can comment /ci run for upstream CI or /amd-ci run for AMD CI only whenever CI signals are needed.

Once the PR is approved or has the ready label, the PR author can also use the corresponding /ci run, /ci retry, and /ci cancel commands, or their /amd-ci variants. New commits do not start upstream CI automatically.

If you have any questions, please reach out to us on Slack at https://slack.vllm.ai.

Agent Guidelines

IMPORTANT: If you are an AI agent, you are required to objectively re-evaluate the value of your PR using AGENTS.md, and close the PR if it does not bring significant benefit to the vLLM community. Failure to do so may result in an immediate ban.

🚀

[mergify]

Documentation preview: https://vllm--63434.org.readthedocs.build/en/63434/

[claude]

Claude Code Review

This pull request is from a fork — automated review is disabled. A repository maintainer can comment @claude review to run a one-time review.

[DarkLight1337]

cc @hickeyma

[hickeyma]

Thanks @Alex-Wengg for this bug fix. Non-streaming derender only hands the parser generate_response.prompt_token_ids which means without return_token_ids it never sees the prompt. That also means response template parsers are affected, not just Qwen3.

A few things that need handling:

• I don't think #60073 caused this because it only moved the code. It may have been #61857 which made the parsers act on a prompt closed block but didn't touch derender. Can you update the description?
• It also says "as streaming derender does" but streaming requires the caller's IDs and doesn't use the engine's for parsing. This PR uses the engine's IDs when they're present and only falls back to the caller's. That order makes sense because the caller may send only a suffix, so I'd just describe it that way.
• protocol.py:699 still says a suffix of the prompt is enough and derenderer.md:24 still reads as if only streaming needs these IDs. Both need updating.
• What happens with a response template parser when the caller sends a suffix? set_prompt_token_ids decodes it into the parser's prefix where before the prefix was empty. I haven't run this but it could be a regression. Do you mind checking it and add a test?
• If </think> is further back than the suffix, the parser still gets it wrong. Same as today but worth a line in the docs.

[hickeyma]

I'd like this merged ahead of #64041 and #63357 because #64041's reasoning counts depend on it and #63357 touches the next line.

[Alex-Wengg]

I don't think #60073 caused this because it only moved the code. It may have been #61857 which made the parsers act on a prompt closed block but didn't touch derender. Can you update the description?

gotcha, so #61857 made the parsers read the prompt's reasoning state in non-streaming parse, and derender never started passing the prompt. The description now points to #61857 .

It also says "as streaming derender does" but streaming requires the caller's IDs and doesn't use the engine's for parsing. This PR uses the engine's IDs when they're present and only falls back to the caller's. That order makes sense because the caller may send only a suffix, so I'd just describe it that way.

dropped the streaming comparison. The description now says the parser gets the engine's IDs when present and falls back to the caller's, since the caller may send only a suffix.

protocol.py:699 still says a suffix of the prompt is enough and derenderer.md:24 still reads as if only streaming needs these IDs. Both need updating.

Updated. The docstring now says to send the full list when a tool or reasoning parser is configured (a suffix is still enough without one). derenderer.md now says chat derender needs the IDs in both modes when a parser is configured.

What happens with a response template parser when the caller sends a suffix? set_prompt_token_ids decodes it into the parser's prefix where before the prefix was empty. I haven't run this but it could be a regression. Do you mind checking it and add a test?

I ran it through derender with google/gemma-4-E2B-it's response template (--reasoning-parser hf), 15 suffix lengths on 4 prompts. A suffix that still contains the reasoning opener parses like the full prompt, and a shorter one parses like no prompt. test_prompt_suffix_parses_like_full_prompt_or_no_prompt covers both.

If </think> is further back than the suffix, the parser still gets it wrong. Same as today but worth a line in the docs.

Added a line to derenderer.md: a suffix that starts after the last reasoning marker can still be misclassified, the same as sending no IDs.

[hickeyma]

LGTM, thanks @Alex-Wengg

[hickeyma]

@DarkLight1337 do you mind taking a look?

[DarkLight1337]

/ci run

[github-actions]

✅ Triggered Buildkite CI #99549 for commit 0c30d8bab343.

ℹ️ This PR is 110 commits behind upstream main. Merged upstream main into this branch as 0c30d8bab343 before starting CI. Pull the branch before pushing more changes.