Skip to content

Send mm_payload with return_mm_kwargs on vLLM render copies - #3455

Merged
ahg-g merged 1 commit into
llm-d:mainfrom
vMaroon:render-mm-payload
Oct 15, 2026
Merged

ahg-g merged 1 commit into
llm-d:mainfrom
vMaroon:render-mm-payload

Conversation

@vMaroon

@vMaroon vMaroon commented Oct 15, 2026

Copy link
Copy Markdown
Member

What type of PR is this?
/kind bug

What this PR does / why we need it:

vllm.omitMMKwargs adds "mm_payload": "none" next to "return_mm_kwargs": false on the render copy.

vllm-project/vllm#61427 replaced the return_mm_kwargs render option with mm_payload and kept return_mm_kwargs as a deprecated alias that vLLM ignores when mm_payload is set. With only return_mm_kwargs on the render copy, a client that sends "mm_payload": "tensors" overrides the option on vLLM main, so the render response carries the processed tensors again. Routing stays correct; render time goes back to the full-tensor cost.

The vLLM v0.32 release branch has return_mm_kwargs only, so the render copy carries both fields: v0.32 reads return_mm_kwargs, and versions with vllm-project/vllm#61427 read mm_payload. Both are spliced after the client's fields, and the later duplicate key wins in vLLM's parser, so client values for either field are overridden. v0.32 logs the unknown mm_payload at debug level only. omitMMKwargs: false sends the render copy without either field, as before.

Which issue(s) this PR fixes:

Part of #1552

Test plan

  • The render copy gets both fields before its closing brace, client values for either field are overridden, and a non-object body is rejected
  • With prefillOnly, the render copy keeps the output-budget rewrite and carries both fields
  • Bodies spliced by omitMMKwargs for chat, completions, Responses and Messages, validated with vLLM's request models from releases/v0.32.0 (7960eeb9de) and main (a97d19d792): v0.32 resolves return_mm_kwargs=false and main resolves mm_payload="none", including when the client sent "return_mm_kwargs": true, "mm_payload": "tensors"; no warnings logged
  • make presubmit and make test-unit

Release note (write NONE if no user-facing change):

`vllm.omitMMKwargs` also sends vLLM's `mm_payload` render field, so render responses stay free of multimodal tensors on vLLM versions with `mm_payload` when a client sends its own value.

vLLM replaced the return_mm_kwargs render option with mm_payload in
vllm-project/vllm#61427 and kept return_mm_kwargs as an alias that is
ignored when mm_payload is set, so a client-sent mm_payload overrides
omitMMKwargs. Sending both keeps v0.32, which knows only
return_mm_kwargs, and lets the router's values win on later versions.

Signed-off-by: Maroon Ayoub <mayoub@redhat.com>
Signed-off-by: Maroon Ayoub <Maroonay@gmail.com>
@vMaroon
vMaroon requested review from a team, liu-cong and sagearc as code owners October 15, 2026 15:39
@github-actions github-actions Bot added size/M Denotes a PR that changes 30-99 lines, ignoring generated files. kind/bug Categorizes issue or PR as related to a bug. area/epp area/scheduling labels Oct 15, 2026
@ahg-g
ahg-g added this pull request to the merge queue Oct 15, 2026
Merged via the queue into llm-d:main with commit 9de126d Oct 15, 2026
48 checks passed
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

area/epp area/scheduling kind/bug Categorizes issue or PR as related to a bug. size/M Denotes a PR that changes 30-99 lines, ignoring generated files.

Projects

None yet

Development

Successfully merging this pull request may close these issues.

2 participants