Skip to content

[Rust Frontend] Accept refusal content parts in assistant messages - #107

Closed
richard-for-bugen[bot] wants to merge 1 commit into
mainfrom
richard/parity-impl-text-like-content-parts
Closed

richard-for-bugen[bot] wants to merge 1 commit into
mainfrom
richard/parity-impl-text-like-content-parts

Conversation

@richard-for-bugen

Copy link
Copy Markdown

Motivation

OpenAI's chat API allows a refusal content part in assistant messages (ChatCompletionContentPartRefusalParam; the assistant content array is text | refusal), so a client that replays a model's refusal as structured history sends it. The Python frontend accepts it and replays the refusal string as text, in every chat content format. The Rust frontend rejects the whole request at deserialization (unknown variant 'refusal'), because ContentPart only knows text, the three media URLs and input_audio.

This PR adds ContentPart::Refusal and lowers it where the role is known:

  • Assistant messages: a refusal part becomes an AssistantContentBlock::Text, like a text part, so every renderer sees it as the turn's text (the same choice the Responses conversion already makes for refusal input parts). echo includes it, as Python's echo of the parsed last message does.
  • Other roles: 400 with OpenAI's message, "Only assistant messages can have a 'refusal' content type."

Ownership and layering: the role-dependent rule lives in the server's DTO lowering (convert_assistant_text_blocks / convert_content in chat_completions/convert.rs), the only consumer that knows the role; vllm-chat already has the text block it maps onto.

Target check (runbook §5 category 4) and scope

The ledger item also listed Python's other "text-like" parts. Python accepts them on chat completions only because the chat parser is shared with its Responses path (MM_PARSER_MAP in vllm/renderers/chat_utils/chat_message.py). A live probe of the OpenAI API (2026-11-03, gpt-6-luna, one request per case) settles which are contract:

Part on /v1/chat/completions OpenAI Python Rust before Rust after
refusal in an assistant message 200 text 400 text
refusal in a user message 400 "Only assistant messages can have a 'refusal' content type." text 400 (serde) 400 (OpenAI's message)
input_text, output_text (any role) 400 "Invalid value" text 400 400 (unchanged)
input_image 400 "Invalid value" image (requires detail) 400 400 (unchanged)
bare string in the parts list 400 "expected an object" text 400 400 (unchanged)

So only the assistant refusal part is implemented; the rest stay explicit 400s, proposed as accepted divergences (see Needs Bugen). Python's thinking part (Mistral ThinkChunk, vllm-project#21333) is also left out: Mistral is parked. Clients with Responses-shaped content can use the Rust /v1/responses route (vllm-project#63902), which accepts all of these.

Python's Harmony path (gpt-oss) drops an assistant refusal part (harmony_utils.py, TODO: Support refusal); Rust's Harmony renderer keeps it as text like every other renderer. That is an intentional improvement, and gpt-oss is parked anyway.

Known difference not fixed here: an assistant message with several text-like parts is joined with "\n" by Python and concatenated by Rust's HF renderer; that is the separate chat.renderer.hf.role_content_parts item, and applies to text parts today in the same way.

Evidence

Python vllm launch render vs this branch's vllm-rs render, both on Qwen/Qwen3-0.6B (HF renderer), POST /v1/chat/completions/render:

  • assistant [{"type":"refusal","refusal":"I can't."}]: identical token IDs on both, and identical to the same turn sent as "content": "I can't.".
  • user [{"type":"refusal",...}]: Python renders it as text; Rust returns the 400 above (intended, per OpenAI).
  • assistant [text "Sorry.", refusal "I can't."]: Python Sorry.\nI can't., Rust Sorry.I can't. (the role_content_parts item above).

Tests

  • prepare_chat_request_replays_assistant_refusal_parts_as_text: JSON request whose assistant message is one refusal part (OpenAI's documented shape), with echo; asserts one assistant text block and the echo text.
  • prepare_chat_request_rejects_refusal_parts_outside_assistant_messages: a user refusal part returns OpenAI's message.

Both fail on main (the request does not deserialize).

Validation

Rust CI set (.buildkite/scripts/run-rust-frontend-cargo-ci.sh), on 39dbf51839 + this commit:

  • cargo fmt --all -- --check, cargo sort --workspace --check, cargo deny check bans: pass
  • cargo clippy --workspace --all-targets --all-features --locked -- -D warnings: pass
  • cargo nextest run --workspace --all-features --locked --no-fail-fast: 2512 passed, 13 skipped

Codex review

scripts/rust_parity_codex_review.sh (gpt-6.1-sol, high): approve with nits; no blocking findings. Codex checked red-green on the exact base itself.

  • P3, test the documented refusal-only shape through a route or renderer harness: fixed in part. The positive test now uses a single refusal part (the documented shape). Not added: a route test, because the route harness renders with a fake renderer, so it would not reach the real renderer; the real-renderer evidence is the Python-vs-Rust token-ID comparison above.
  • P3, scope parity claims to HF and note Harmony: fixed in this description (Harmony paragraph above).

Needs Bugen

  • Accept as divergences: input_text, output_text, input_image and bare-string parts on /v1/chat/completions (OpenAI rejects them; Python accepts them only through the parser it shares with Responses), and refusal outside assistant messages (OpenAI 400, Python text). Recommendation: accept; the Responses route covers Responses-shaped content.

OpenAI's chat API allows `refusal` parts in assistant message content
(`ChatCompletionContentPartRefusalParam`), and the Python frontend
replays them as text. The Rust frontend rejected them with a
deserialization error. Lower them onto assistant text blocks, include
them in the `echo` text like Python, and reject them in other roles
with OpenAI's message.

Co-authored-by: Claude Opus 5.5 <noreply@anthropic.com>
Signed-off-by: richard-for-bugen[bot] <342439978+richard-for-bugen[bot]@users.noreply.github.com>
@richard-for-bugen

Copy link
Copy Markdown
Author

Promoted as vllm-project#65621.

@richard-for-bugen

Copy link
Copy Markdown
Author

Merged upstream as vllm-project#65621.

@richard-for-bugen richard-for-bugen Bot closed this Nov 5, 2026
@richard-for-bugen
richard-for-bugen Bot deleted the richard/parity-impl-text-like-content-parts branch November 5, 2026 19:12
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

None yet

Projects

None yet

Development

Successfully merging this pull request may close these issues.

1 participant