[NickCao]

Contributor intent (required human answers)

All four questions must be answered by the human contributor before submission.
An agent may reuse explicit human answers with a public-safe source reference;
it must not invent, infer, or prefill missing answers. If an answer is missing
or materially ambiguous, ask the human and wait. Do not publish private
conversation content without authorization.

1. What outcome do you want to achieve?

Reduce the risk that crafted images trigger unsafe decoder behavior in publicly accessible vLLM-Omni services.

2. What is the highest-leverage variable for that outcome?

Whether untrusted image bytes reach Pillow through unrestricted plugin selection.

3. How will this work move that variable?

Route public image decoding through vLLM’s open_image, which restricts Pillow’s plugin choices at open time.

4. Whose customer or user needs does this address, and what are those needs?

Operators of public omni services need to reduce exposure to malicious uploads.

Purpose

Route public image and video ingress through the vLLM audited Pillow format allowlist.

Although listed by MiniMax H3 validation, HEIC/HEIF were not supported in a clean install because the project does not include a HEIF decoder dependency.

Test Plan

vLLM Version: 0.32.0

vLLM-Omni Commit: ce024e0

pytest -q \
  tests/entrypoints/openai_api/test_image_server.py \
  tests/entrypoints/openai_api/test_serving_chat_multistage_generation.py \
  tests/entrypoints/openai_api/test_video_api_utils.py \
  tests/entrypoints/openai_api/test_video_server.py \
  tests/entrypoints/openai_api/test_video_frame_filter.py \
  tests/entrypoints/openai_api/test_serving_video_stream.py \
  tests/model_executor/models/aura_omni/duplex/test_aura_plugin_contract.py \
  tests/model_executor/models/joyai_vl_interaction/test_duplex_plugin.py \
  tests/model_executor/models/minicpmo_4_5/duplex/test_input.py \
  tests/model_executor/models/qwen3_omni/test_duplex_plugin.py \
  tests/diffusion/models/minimax_h3/test_minimax_h3_contract.py \
  tests/entrypoints/openai_api/test_untrusted_image.py
pytest -s -v tests/e2e/online_serving/test_minicpmo_4_5.py::test_image_to_text_audio_001
pytest -s -v tests/e2e/online_serving/test_boogu_image_edit.py::test_images_edits_endpoint_001 -m "slow and diffusion and H100"
pytest -s -v tests/e2e/online_serving/test_lingbot_video.py::test_video_generation_modes_001 \
  -m "advanced_model and diffusion" --run-level=advanced_model

Test Result

passed

BEFORE SUBMITTING: read CONTRIBUTING.md and run the precheck-pr skill with the code agent for a self-check against project conventions.

Note

To avoid agent spam, vLLM-Omni limits contributors without write access to 5 open PRs. If this blocks well-intentioned critical work, please contact a committer to request bypass list access.
(anything written below this line will be removed by GitHub Actions)

[chatgpt-codex-connector]

Codex Review Summary

This comment shows the latest Codex review activity on this pull request.

Review	Status	Commit	Review trigger
📝 Code Review	⚠️ Failed 2026-10-19T14:44:48.753796Z	4791631	Draft marked ready

ℹ️ About Codex in GitHub

Your team has set up Codex to review pull requests in this repo. Reviews are triggered when you

• Open a pull request for review
• Mark a draft as ready
• Comment "@codex review" or "@codex security review".

Codex reacts with 👀 while any review is running, comments if it has suggestions, and reacts with 👍 once all reviews finish with no findings.

[chatgpt-codex-connector]

Codex usage limits have been reached for code reviews. Please check with the admins of this repo to increase the limits by adding credits.
Credits must be used to enable repository wide code reviews.

[amy-why-3459]

Thanks for routing these through the vLLM allowlist, the call-site swap itself looks correct (open_image is a thin Image.open(fp, formats=ALLOWED_IMAGE_FORMATS), and denied formats still surface as UnidentifiedImageError/OSError, so the existing except clauses keep working).

A few things worth addressing before merge:

1. Ascend image compatibility: docker/Dockerfile.npu still builds on vllm-ascend:v0.31.0, and vllm/multimodal/image.py in vLLM v0.31.0 has no open_image (it was added in v0.32.0). If that image still ships vLLM 0.31, the new module-level imports in serving_chat.py, images/helpers.py, video_api_utils.py and the duplex plugins would fail at import time and the NPU server would not start. Could you confirm the NPU image, or add a small local fallback (Image.open(fp, formats=...) with the same allowlist) until the Ascend base moves to 0.32?
2. MiniMax H3 HEIC/HEIF: see inline. The allowlist has no HEIF, so these formats can no longer be decoded even with pillow-heif installed, but they are still advertised.
3. Tests: none of the listed tests exercises a denied format. A small parametrized case that feeds an EPS header (b"%!PS-Adobe-3.0 EPSF-3.0...") into _decode_image_bytes, _load_input_images and the H3 upload validator, and asserts a 400 / UnidentifiedImageError, would lock in the security property so a future Image.open regression gets caught.

For reference, the remaining Image.open call sites I checked look fine to leave as-is: the diffusion pipelines only open str/path inputs from offline callers, and experimental/fullduplex/video_stacking.py is client-side (used by clients/duplex.py and the demo).

[amy-why-3459]

Inline notes for points 1 and 2 above.

[vllm-omni-review-bot]

This PR appears to belong to: docs/design/module/ar_runtime.md, docs/design/module/observability.md, docs/design/module/model_integration.md.

Module owners: @fake0fan @Gaohan123 @gcanlin

Routing: @fake0fan via module of the changed files; @Gaohan123 via module of the changed files; @gcanlin via module of the changed files

@NickCao, please review your own changes and leave a short self-review comment describing what you checked. PRs without author self-review may not be assigned a reviewer.

Please take a look when you have a chance. If you would like an automated review, mention @vllm-omni-review-bot in a comment.

[vllm-omni-review-bot]

Omni ReviewBot triage note

Automated triage of commit ce024e07931d produced:

• Priority: high. Prompt maintainer attention is suggested.

These are automated triage suggestions only — the final decision belongs to the maintainers.

[NickCao]

Also added some regression test for the denial case.

[amy-why-3459]

Thanks for the follow-up. Looked at the two new commits (8a2934b, e3c212e): the HEIC/HEIF cleanup in both MINIMAX_H3_REFERENCE_IMAGE_FORMATS constants and error messages looks right, and the new test_untrusted_image.py is a good regression guard. Patching EpsImageFile._open to raise proves the EPS decoder is never reached, not just that some error came back, and since Pillow re-raises non-syntax exceptions from plugin _open, a future bare Image.open would fail these tests loudly.

A few remaining items:

1. Docs still advertise HEIC/HEIF (outside the diff, so noting here): docs/serving/videos_api.md ("Images use JPEG/PNG/WEBP/HEIC/HEIF (30 MiB)") and recipes/MiniMaxAI/MiniMax-H3.md ("Reference images accept JPG/JPEG, PNG, WEBP, HEIC, or HEIF up to 30 MiB"). Worth updating them in this PR so the docs match the new behavior.
2. open_image on the Ascend image: this push doesn't touch the serving_chat.py import thread, so the vllm-ascend v0.31.0 question from the earlier review is still open. Just flagging it so it doesn't get lost before merge, since the import now sits at module level in the OpenAI entrypoints as well as the duplex plugins.
3. Minor, see inline: HEIC/HEIF uploads now get the generic "Invalid uploaded image reference" error instead of the format list.

[amy-why-3459]

Checked d7d5342f and 9f5581ee. The docs and the H3 recipe no longer advertise HEIC/HEIF, and a failed decode now returns the supported-format list, with a test for the .heic case. Thanks. The one item from my earlier review that's still open is the module-level from vllm.multimodal.image import open_image (in helpers.py and serving_chat.py). docker/Dockerfile.npu is still on vllm-ascend:v0.31.0 at this head, so that import path depends on the NPU base moving to vLLM 0.32 first.

[NickCao]

Checked d7d5342f and 9f5581ee. The docs and the H3 recipe no longer advertise HEIC/HEIF, and a failed decode now returns the supported-format list, with a test for the .heic case. Thanks. The one item from my earlier review that's still open is the module-level from vllm.multimodal.image import open_image (in helpers.py and serving_chat.py). docker/Dockerfile.npu is still on vllm-ascend:v0.31.0 at this head, so that import path depends on the NPU base moving to vLLM 0.32 first.

NPU updated in #9143

[hsliuustc0106]

@NickCao

• Please attach execution evidence for the new EPS-denial regressions at ce024e07931ddf683fcf086a3d9380865a8ab0a9. The Test Plan currently pins 47916310 and does not list test_untrusted_image.py. Please include the command (pytest -q tests/entrypoints/openai_api/test_untrusted_image.py), Python/vLLM/Pillow versions and actual result, or link the existing job/log if these tests already ran. For the claimed security property, show the same three EPS cases fail with the unrestricted baseline decoder and pass with the reviewed allowlist path; identify the baseline revision or controlled revert used. Acceptance: the EPS decoder is not entered on the reviewed head, and the negative control catches restoring unrestricted decoding.

The regression code and public-ingress call-site changes have been inspected statically; no tests were executed here. Current CUDA is green, but its aggregate status does not identify whether these named tests ran. This is a specific verification gap, not a newly demonstrated code defect.

[NickCao]

@NickCao

* [ ]  Please attach execution evidence for the new [EPS-denial regressions](https://github.com/vllm-project/vllm-omni/blob/ce024e07931ddf683fcf086a3d9380865a8ab0a9/tests/entrypoints/openai_api/test_untrusted_image.py#L31-L58) at `ce024e07931ddf683fcf086a3d9380865a8ab0a9`. The Test Plan currently pins `47916310` and does not list `test_untrusted_image.py`. Please include the command (`pytest -q tests/entrypoints/openai_api/test_untrusted_image.py`), Python/vLLM/Pillow versions and actual result, or link the existing job/log if these tests already ran. For the claimed security property, show the same three EPS cases fail with the unrestricted baseline decoder and pass with the reviewed allowlist path; identify the baseline revision or controlled revert used. Acceptance: the EPS decoder is not entered on the reviewed head, and the negative control catches restoring unrestricted decoding.

The regression code and public-ingress call-site changes have been inspected statically; no tests were executed here. Current CUDA is green, but its aggregate status does not identify whether these named tests ran. This is a specific verification gap, not a newly demonstrated code defect.

Updated and retested.

Negative control:

git revert --no-commit b7815b0d2
pytest -q tests/entrypoints/openai_api/test_untrusted_image.py

Failing as expected.

[linyueqian]

Reviewed ce024e0. Every Image.open on a request path under vllm_omni/ now routes through open_image, covering the OpenAI image, chat, and video entrypoints, the streaming video handler, the MiniMax-H3 upload and preprocessing paths, and the Qwen3-Omni, MiniCPM-o 4.5, JoyAI, and Aura duplex plugins. Current main (88c9ccd) has no other raw Image.open on those paths; the remaining calls are in diffusion pipelines that open str/path inputs from offline callers, the benchmarks, and the client-side video_stacking.py.

Existing except clauses still catch the rejection because a disallowed format raises UnidentifiedImageError, which is an OSError. Patching EpsImageFile._open to fail in the new EPS tests proves the decoder is never entered, rather than only checking that an error was returned. The earlier NPU concern is resolved on main because docker/Dockerfile.npu moved to vllm-ascend v0.32.0 in #9143, which includes open_image.

Static review only; I did not run the tests locally. Main CUDA (build 17678) and the NPU lane pass at ce024e0.