Repository navigation
[Bugfix][Rust Frontend] Accept assistant turns without content like Python - #64492
Conversation
![claude[bot]](https://avatars.githubusercontent.com/in/1236702?s=60&v=4){kind=small}
d2d849a to
a54d14b
Compare
|
This pull request has merge conflicts that must be resolved before it can be |
b12822d to
4bc1748
Compare
|
@BugenZhao @njhill Please review |
ad8f2b3 to
60f6543
Compare
|
/ci run |
|
❌ This PR is 115 commits behind upstream |
|
/ci run |
|
✅ @hemuku90, CI is now available for this PR.
|
|
❌ This PR is 115 commits behind upstream |
|
/ci run |
|
❌ This PR is 117 commits behind upstream |
|
/ci run |
|
✅ Triggered Buildkite CI #99913 for commit ℹ️ This PR is 4 commits behind upstream |
…ython An assistant message with null, missing or `[]` content and no tool calls got a 400 from the Rust frontend, as did one whose only reasoning is `""`. Python renders all of these as an empty assistant turn, same as `content: ""`, which Rust already took. Push an empty text block instead of bailing, so both frontends build the same prompt. Signed-off-by: hemuku90 <25220495+hemuku90@users.noreply.github.com>
…xt block
An assistant message with no blocks already renders like one with an
empty text block: every renderer either joins the text blocks or skips
empty ones. So the check can go rather than be turned into a synthetic
`Text("")`, which keeps `content: null` free of a text part, like the
`[]` Python passes to `openai`-format templates.
Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
Claude-Session: https://claude.ai/code/session_01CJckCC771BVnAH8BWPyaSV
Signed-off-by: Bugen Zhao <i@bugenzhao.com>
cd2cdf0 to
0f96c5d
Compare
|
/ci run |
|
✅ Triggered Buildkite CI #99921 for commit |
Overview
The Rust frontend returns a 400 for an assistant message with no content (
content: null,contentomitted, orcontent: []) and no tool calls. The Python frontend accepts all three and renders an empty assistant turn, same ascontent: "", which Rust already accepts. This brings Rust in line.I ran into it while diffing render output between the two frontends.
{"role": "assistant", "content": null}shows up a lot in replayed history (tool calls stripped by the client, placeholder turns), so it's worth matching.Claims
[]content and no tool calls are accepted on/v1/chat/completions,/v1/chat/completions/renderand/tokenize, and render the same ascontent: "".continue_final_message.Validation
Qwen/Qwen3-0.6B,/v1/chat/completions/render, messages[user "hi", <assistant>, user "hi"]:<assistant>{"role":"assistant","content":""}{"role":"assistant","content":null}{"role":"assistant"}{"role":"assistant","content":[]}{"role":"assistant","content":null,"tool_calls":[]}The 400s are
Assistant messages must contain text, reasoning content, or tool_calls.All the 200s return the same 20 token IDs. I also checked the empty turn as the final message: 18 tokens with default flags and 13 withcontinue_final_message: true, add_generation_prompt: false, same as Python in both cases.The new route test sends all five variants (plus
content: nullwithreasoning: "") and checks for a 200 and identical token IDs. It fails on main at thecontent: nullcase and passes with the change.cargo fmt, clippy and pre-commit are clean on the changed files.After the follow-up commit that removes the check instead of adding an empty text block:
cargo nextest run -p vllm-server -p vllm-chat --libpasses (1060 tests), the route test fails with the check restored, and clippy is clean.Screenshots (Rust main, this PR, Python)
Details
convert_messageinchat_completions/convert.rsbuilds an assistant message out of reasoning, text and tool-call blocks, and it bailed when that list came out empty.content: ""produces one empty text block, so it passed. The other forms of the same empty turn produced nothing and got rejected. The check is removed, so an assistant message may now have no blocks. Every renderer already treats that like one empty text block: they either join the text blocks or skip empty ones. So these turns render the same ascontent: "". Leaving out a synthetic empty text block also keepscontent: nullwithout a text part, which is what Python passes toopenai-format templates ([]). Rust's own non-streaming response returnscontent: nullwith no tool calls when the generated text is empty, so a client replaying that history used to get this 400.Since #62424, an empty
reasoning: ""is kept as its own block, and the check also rejected a turn whose only blocks were empty reasoning. With the check gone,{"role": "assistant", "content": null, "reasoning": ""}also renders as an empty turn instead of a 400, like Python; the route test covers it as a sixth variant.I didn't find an existing issue or PR for this. #54368 is the Python side (null content with tool calls), and #63132 and #63137 are about the reasoning field aliases, so they don't overlap.
Pull Request Checklist
I used vLLM's
/pr-checklistskill. (Mandatory for agents, optional for humans).AI assistance was used during the creation of this PR.
Design Fit: Minimizes impact on core components, reuses existing functionality, and justifies added complexity.
Testing and Validation: Validates the change and ensures any added tests are meaningful and reliable, with CI coverage or documented CI resource constraints and validation performed outside CI.
Code Quality and Style: Keeps code and comments clear and concise, and updates relevant documentation and examples.
Pull Request Contents: Includes a brief summary and relevant links, supports claims with evidence, explains root causes and implementation trade-offs, and follows the contributing guide.