Repository navigation
[Rust Frontend] Accept refusal content parts in assistant messages - #107
Closed
richard-for-bugen[bot] wants to merge 1 commit into
Closed
richard-for-bugen[bot] wants to merge 1 commit into
richard-for-bugen[bot] wants to merge 1 commit into
Conversation
OpenAI's chat API allows `refusal` parts in assistant message content (`ChatCompletionContentPartRefusalParam`), and the Python frontend replays them as text. The Rust frontend rejected them with a deserialization error. Lower them onto assistant text blocks, include them in the `echo` text like Python, and reject them in other roles with OpenAI's message. Co-authored-by: Claude Opus 5.5 <noreply@anthropic.com> Signed-off-by: richard-for-bugen[bot] <342439978+richard-for-bugen[bot]@users.noreply.github.com>
BugenZhao
approved these changes
Nov 4, 2026
Author
|
Promoted as vllm-project#65621. |
Author
|
Merged upstream as vllm-project#65621. |
richard-for-bugen
Bot
deleted the
richard/parity-impl-text-like-content-parts
branch
November 5, 2026 19:12
This file contains hidden or bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
Sign up for free
to join this conversation on GitHub.
Already have an account?
Sign in to comment
Add this suggestion to a batch that can be applied as a single commit.This suggestion is invalid because no changes were made to the code.Suggestions cannot be applied while the pull request is closed.Suggestions cannot be applied while viewing a subset of changes.Only one suggestion per line can be applied in a batch.Add this suggestion to a batch that can be applied as a single commit.Applying suggestions on deleted lines is not supported.You must change the existing code in this line in order to create a valid suggestion.Outdated suggestions cannot be applied.This suggestion has been applied or marked resolved.Suggestions cannot be applied from pending reviews.Suggestions cannot be applied on multi-line comments.Suggestions cannot be applied while the pull request is queued to merge.Suggestion cannot be applied right now. Please check back later.
Motivation
OpenAI's chat API allows a
refusalcontent part in assistant messages (ChatCompletionContentPartRefusalParam; the assistantcontentarray istext | refusal), so a client that replays a model's refusal as structured history sends it. The Python frontend accepts it and replays the refusal string as text, in every chat content format. The Rust frontend rejects the whole request at deserialization (unknown variant 'refusal'), becauseContentPartonly knowstext, the three media URLs andinput_audio.This PR adds
ContentPart::Refusaland lowers it where the role is known:AssistantContentBlock::Text, like a text part, so every renderer sees it as the turn's text (the same choice the Responses conversion already makes forrefusalinput parts).echoincludes it, as Python's echo of the parsed last message does.Ownership and layering: the role-dependent rule lives in the server's DTO lowering (
convert_assistant_text_blocks/convert_contentinchat_completions/convert.rs), the only consumer that knows the role;vllm-chatalready has the text block it maps onto.Target check (runbook §5 category 4) and scope
The ledger item also listed Python's other "text-like" parts. Python accepts them on chat completions only because the chat parser is shared with its Responses path (
MM_PARSER_MAPinvllm/renderers/chat_utils/chat_message.py). A live probe of the OpenAI API (2026-11-03,gpt-6-luna, one request per case) settles which are contract:/v1/chat/completionsrefusalin an assistant messagerefusalin a user messageinput_text,output_text(any role)input_imagedetail)So only the assistant
refusalpart is implemented; the rest stay explicit 400s, proposed as accepted divergences (see Needs Bugen). Python'sthinkingpart (Mistral ThinkChunk, vllm-project#21333) is also left out: Mistral is parked. Clients with Responses-shaped content can use the Rust/v1/responsesroute (vllm-project#63902), which accepts all of these.Python's Harmony path (gpt-oss) drops an assistant refusal part (
harmony_utils.py,TODO: Support refusal); Rust's Harmony renderer keeps it as text like every other renderer. That is an intentional improvement, and gpt-oss is parked anyway.Known difference not fixed here: an assistant message with several text-like parts is joined with
"\n"by Python and concatenated by Rust's HF renderer; that is the separatechat.renderer.hf.role_content_partsitem, and applies to text parts today in the same way.Evidence
Python
vllm launch rendervs this branch'svllm-rs render, both onQwen/Qwen3-0.6B(HF renderer),POST /v1/chat/completions/render:[{"type":"refusal","refusal":"I can't."}]: identical token IDs on both, and identical to the same turn sent as"content": "I can't.".[{"type":"refusal",...}]: Python renders it as text; Rust returns the 400 above (intended, per OpenAI).[text "Sorry.", refusal "I can't."]: PythonSorry.\nI can't., RustSorry.I can't.(therole_content_partsitem above).Tests
prepare_chat_request_replays_assistant_refusal_parts_as_text: JSON request whose assistant message is one refusal part (OpenAI's documented shape), withecho; asserts one assistant text block and the echo text.prepare_chat_request_rejects_refusal_parts_outside_assistant_messages: a user refusal part returns OpenAI's message.Both fail on
main(the request does not deserialize).Validation
Rust CI set (
.buildkite/scripts/run-rust-frontend-cargo-ci.sh), on39dbf51839+ this commit:cargo fmt --all -- --check,cargo sort --workspace --check,cargo deny check bans: passcargo clippy --workspace --all-targets --all-features --locked -- -D warnings: passcargo nextest run --workspace --all-features --locked --no-fail-fast: 2512 passed, 13 skippedCodex review
scripts/rust_parity_codex_review.sh(gpt-6.1-sol, high): approve with nits; no blocking findings. Codex checked red-green on the exact base itself.Needs Bugen
input_text,output_text,input_imageand bare-string parts on/v1/chat/completions(OpenAI rejects them; Python accepts them only through the parser it shares with Responses), andrefusaloutside assistant messages (OpenAI 400, Python text). Recommendation: accept; the Responses route covers Responses-shaped content.