[Feature] add a new response_format is file type when image generate by lengrongfu · Pull Request #1673 · vllm-project/vllm-omni

lengrongfu · 2026-03-05T03:28:38Z

PLEASE FILL IN THE PR DESCRIPTION HERE ENSURING ALL CHECKLIST ITEMS (AT THE BOTTOM) HAVE BEEN CONSIDERED.

Purpose

Current image generate response format is json data, need user to handler this data, this is not very convenient, e.g:

curl -X POST http://localhost:8000/v1/images/generations \
  -H "Content-Type: application/json" \
  -d '{
    "prompt": "a dragon laying over the spine of the Green Mountains of Vermont",
    "size": "1024x1024",
    "seed": 42
  }' | jq -r '.data[0].b64_json' | base64 -d > dragon.png

So, i add a new response_format type, it is file type, when use set this param response_format=file in http request , user can don't handler any response data, e.g

curl -o dragon.png -X POST http://localhost:8000/v1/images/generations \
  -H "Content-Type: application/json" \
  -d '{
    "prompt": "a dragon laying over the spine of the Green Mountains of Vermont",
    "size": "1024x1024",
    "seed": 42,
    "response_format":"file"
  }'

Default save video file name is dragon.png.

Test Plan

Test Result

The purpose of the PR, such as "Fix some issue (link existing issues this PR will resolve)".
The test plan. Please provide the test scripts & test commands. Please state the reasons if your codes don't require additional test scripts. For test file guidelines, please check the test style doc
The test results. Please paste the results comparison before and after, or the e2e results.
(Optional) The necessary documentation update, such as updating supported_models.md and examples for a new model. Please run mkdocs serve to sync the documentation editions to ./docs.
(Optional) Release notes update. If your change is user-facing, please update the release notes draft.

BEFORE SUBMITTING, PLEASE READ https://github.com/vllm-project/vllm-omni/blob/main/CONTRIBUTING.md (anything written below this line will be removed by GitHub Actions)

chatgpt-codex-connector

💡 Codex Review

Here are some automated review suggestions for this pull request.

Reviewed commit: 20374714ef

ℹ️ About Codex in GitHub

Codex has been enabled to automatically review pull requests in this repo. Reviews are triggered when you

Open a pull request for review
Mark a draft as ready
Comment "@codex review".

If Codex has suggestions, it will comment; otherwise it will react with 👍.

When you sign up for Codex through ChatGPT, Codex can also answer questions or update the PR, like "@codex address that feedback".

chatgpt-codex-connector · 2026-03-05T03:30:55Z

+        else:
+                zip_buffer = io.BytesIO()
+                with zipfile.ZipFile(zip_buffer, "w", zipfile.ZIP_DEFLATED) as zf:
+                    for idx, item in enumerate(response.data):


Iterate over self.data when zipping image file responses

When response_format=file and more than one image is generated (n > 1), this branch crashes because it references response.data, but response is undefined in this method. That raises a NameError and returns a 500 instead of the expected zip download, so the new multi-image file response path is currently unusable.

Useful? React with 👍 / 👎.

chatgpt-codex-connector · 2026-03-05T03:30:55Z

+                    status_code=HTTPStatus.INTERNAL_SERVER_ERROR.value,
+                    detail="No video data available for file response.",
+                )
+        video_bytes = base64.b64decode(self.data[0].b64_json)


Return all requested videos for file response format

This file-response implementation always serves only self.data[0], so requests that ask for multiple videos (n > 1) silently lose every generated output except the first when response_format=file is used. The endpoint still accepts multi-output requests, so clients can receive incomplete results without any error.

Useful? React with 👍 / 👎.

Gaohan123

The title and PR description have some typos and a bit confusing. Please make it clear. And please supplement some Unit Tests. Besides, does it have potential to support video streaming output with the StreamingResponse?

lengrongfu · 2026-03-06T08:55:07Z

does it have potential to support video streaming output with the StreamingResponse

Is the stream you described generated and output simultaneously? the current approach is to return all the generated JSON data as a stream.

lengrongfu · 2026-03-09T08:58:41Z

@Gaohan123 PTAL, thanks ~

Gaohan123 · 2026-04-30T08:19:49Z

Please resolve conflicts. Thanks

lengrongfu · 2026-05-26T02:45:14Z

@Gaohan123 Hi, code conflicts have resolve, please take a look, thanks ~

Signed-off-by: rongfu.leng <lenronfu@gmail.com>

lengrongfu · 2026-06-01T02:52:13Z

These two failed CI runs are unrelated to my changes.

david6666666

Please fix these response_format=file issues.

david6666666 · 2026-06-02T07:33:11Z

+            filename = f"image_{uuid.uuid4().hex[:8]}.png"
+            return StreamingResponse(
+                io.BytesIO(image_bytes),
+                media_type="image/png",


Use self.output_format for the media type and filename extension here. With output_format="jpeg" or "webp", the body is encoded in that format but the download still advertises image/png and a .png filename; the ZIP member names below have the same issue.

david6666666 · 2026-06-02T07:33:11Z

+        json={
+            "prompt": "a dog",
+            "n": 1,
+            "response_format": "file",


Please add coverage for output_format="jpeg" or "webp" with response_format="file". The current tests only hit the default PNG path, so the response header / filename mismatch can slip through.

david6666666 · 2026-06-02T07:33:11Z

+
+# save to file
+content_disposition = response.headers.get("Content-Disposition", "")
+match = re.search(r'filename="?(.+)"?', content_disposition)


This regex is greedy, so attachment; filename="image.png" captures image.png" with the trailing quote. Use a non-greedy or character-class match here so the example writes the intended filename.

david6666666 · 2026-06-02T07:38:44Z

In the official OpenAI Images API, the standard value for response_format is either url or b64_json; file is not specified. The official documentation also states that GPT image models return base64 by default, and response_format is primarily a parameter of DALL-E 2/3. Therefore, response_format="file" in the PR is indeed a non-standard extension and not part of the OpenAI-compatible contract.

david6666666 · 2026-06-02T07:39:24Z

@SamitHuang @wtomin WDYT

Gaohan123

LGTM. Thanks

…llm-project#1673) Signed-off-by: rongfu.leng <lenronfu@gmail.com> Co-authored-by: Gao Han <hgaoaf@connect.ust.hk>

lengrongfu requested a review from hsliuustc0106 as a code owner March 5, 2026 03:28

chatgpt-codex-connector Bot reviewed Mar 5, 2026

View reviewed changes

Gaohan123 reviewed Mar 5, 2026

View reviewed changes

Gaohan123 added this to the v0.18.0 milestone Mar 5, 2026

lengrongfu changed the title ~~[Feature] add support response file when video or image generage~~ [Feature] add a new response_format is file type when video or image generate Mar 6, 2026

lengrongfu force-pushed the feat/add-stage_failed branch from 669c0ad to 4ef7c36 Compare March 16, 2026 03:24

lengrongfu changed the title ~~[Feature] add a new response_format is file type when video or image generate~~ [Feature] add a new response_format is file type when image generate Mar 16, 2026

lengrongfu mentioned this pull request Apr 14, 2026

[Bugfix] Handle numpy array outputs when generate image #1680

Merged

5 tasks

Gaohan123 modified the milestones: v0.18.0, v0.20.0 Apr 14, 2026

Gaohan123 modified the milestones: v0.20.0, v0.22.0 May 9, 2026

lengrongfu force-pushed the feat/add-stage_failed branch from fc70202 to 48e3df4 Compare May 25, 2026 13:44

lengrongfu requested review from tzhouam and yenuo26 as code owners May 25, 2026 13:44

hsliuustc0106 added ready label to trigger buildkite CI high priority high priority issue, needs to be done asap labels May 28, 2026

lengrongfu force-pushed the feat/add-stage_failed branch from 35ed527 to a5afd34 Compare May 28, 2026 06:16

lengrongfu requested a review from david6666666 as a code owner May 29, 2026 16:29

lengrongfu added 4 commits May 30, 2026 11:07

[Feature] add a new response_format is file type when image generate

5122b1e

Signed-off-by: rongfu.leng <lenronfu@gmail.com>

add image download docs

c5ddcac

Signed-off-by: rongfu.leng <lenronfu@gmail.com>

add docs

b32c61c

Signed-off-by: rongfu.leng <lenronfu@gmail.com>

fix docs

d343360

Signed-off-by: rongfu.leng <lenronfu@gmail.com>

lengrongfu force-pushed the feat/add-stage_failed branch from 26a13a0 to d343360 Compare May 30, 2026 11:07

hsliuustc0106 reviewed May 30, 2026

View reviewed changes

Comment thread tests/entrypoints/openai_api/test_image_server.py

add single image file unit test

6c8d070

Signed-off-by: rongfu.leng <lenronfu@gmail.com>

david6666666 reviewed Jun 2, 2026

View reviewed changes

hsliuustc0106 removed the ready label to trigger buildkite CI label Jun 10, 2026

Gaohan123 modified the milestones: v0.22.0, v0.24.0 Jun 10, 2026

Merge branch 'main' into feat/add-stage_failed

1ef22f5

Gaohan123 added the ready label to trigger buildkite CI label Jun 23, 2026

Gaohan123 approved these changes Jun 23, 2026

View reviewed changes

Gaohan123 enabled auto-merge (squash) June 23, 2026 02:34

hsliuustc0106 disabled auto-merge June 24, 2026 01:04

hsliuustc0106 merged commit a2d1daf into vllm-project:main Jun 24, 2026
7 of 9 checks passed

Conversation

lengrongfu commented Mar 5, 2026 • edited Loading Uh oh! There was an error while loading. Please reload this page.

Uh oh!

Purpose

Test Plan

Test Result

Uh oh!

chatgpt-codex-connector Bot left a comment

Choose a reason for hiding this comment

💡 Codex Review

Uh oh!

chatgpt-codex-connector Bot Mar 5, 2026

Choose a reason for hiding this comment

Uh oh!

chatgpt-codex-connector Bot Mar 5, 2026

Choose a reason for hiding this comment

Uh oh!

Gaohan123 left a comment

Choose a reason for hiding this comment

Uh oh!

lengrongfu commented Mar 6, 2026

Uh oh!

lengrongfu commented Mar 9, 2026

Uh oh!

Gaohan123 commented Apr 30, 2026

Uh oh!

lengrongfu commented May 26, 2026

Uh oh!

Uh oh!

lengrongfu commented Jun 1, 2026

Uh oh!

david6666666 left a comment

Choose a reason for hiding this comment

Uh oh!

david6666666 Jun 2, 2026

Choose a reason for hiding this comment

Uh oh!

david6666666 Jun 2, 2026

Choose a reason for hiding this comment

Uh oh!

david6666666 Jun 2, 2026

Choose a reason for hiding this comment

Uh oh!

david6666666 commented Jun 2, 2026

Uh oh!

david6666666 commented Jun 2, 2026

Uh oh!

Gaohan123 left a comment

Choose a reason for hiding this comment

Uh oh!

Uh oh!

Reviewers

Assignees

Labels

Projects

Milestone

Development

Uh oh!

4 participants

lengrongfu commented Mar 5, 2026 •

edited

Loading