Skip to content

[Bugfix] Expose base_model_paths property in _DiffusionServingModels#1771

Merged
wtomin merged 3 commits into
vllm-project:mainfrom
RuixiangMa:fixdiffusionServing
Mar 11, 2026
Merged

[Bugfix] Expose base_model_paths property in _DiffusionServingModels#1771
wtomin merged 3 commits into
vllm-project:mainfrom
RuixiangMa:fixdiffusionServing

Conversation

@RuixiangMa
Copy link
Copy Markdown
Collaborator

@RuixiangMa RuixiangMa commented Mar 10, 2026

Purpose

In pure diffusion mode, _DiffusionServingModels uses getattr to intercept undefined attributes. Accessing base_model_paths returned a _Unsupported object instead of the actual list, causing NotImplementedError when
retrieving model names for video generation endpoints

curl -s -D >(grep -i x-request-id >&2) -o >(jq -r '.data[0].b64_json' | base64 --decode > output.png) -X POST "http://localhost:8004/v1/images/edits" -F "model=black-forest-labs/FLUX.2-klein-4B" -F "image=@./test.jpg" -F "prompt=change into orange." -F "size=1024x1024" -F "output_format=png"

APIServer pid=494089)     model_name = serving_models.base_model_paths[0].name
(APIServer pid=494089)                  ~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~^^^
(APIServer pid=494089) TypeError: '_Unsupported' object is not subscriptable

Test Plan

Test Result

(APIServer pid=111918) INFO 03-10 11:50:44 [diffusion_engine.py:86] Generation completed successfully.
(APIServer pid=111918) INFO 03-10 11:50:44 [diffusion_engine.py:108] Post-processing completed in 0.1113 seconds
(APIServer pid=111918) INFO 03-10 11:50:44 [diffusion_engine.py:111] DiffusionEngine.step breakdown: preprocess=0.00 ms, add_req_and_wait=25044.09 ms, postprocess=111.33 ms, total=25156.01 ms
(APIServer pid=111918) INFO 03-10 11:50:44 [omni_diffusion.py:128] OmniDiffusion.generate total: 25156.33 ms
(APIServer pid=111918) INFO 03-10 11:50:44 [api_server.py:1395] Successfully generated 1 image(s)
(APIServer pid=111918) INFO: 127.0.0.1:52072 - "POST /v1/images/edits HTTP/1.1" 200 OK

Signed-off-by: Lancer <maruixiang6688@gmail.com>
@wtomin
Copy link
Copy Markdown
Collaborator

wtomin commented Mar 10, 2026

Is your PR going to fix a CI failures introduced by recent commit 7a56049?

@RuixiangMa
Copy link
Copy Markdown
Collaborator Author

RuixiangMa commented Mar 10, 2026

Is your PR going to fix a CI failures introduced by recent commit 7a56049?

Yes, tested against docs/serving/image_edit_api.md using the specified curl command on the main branch (7a56049 (HEAD -> main, origin/main) Add online serving to Stable Audio Diffusion and introduce v1/audio/generate endpoint (#1255))
curl -s -D >(grep -i x-request-id >&2) -o >(jq -r '.data[0].b64_json' | base64 --decode > output.png) -X POST "http://localhost:8004/v1/images/edits" -F "model=black-forest-labs/FLUX.2-klein-4B" -F "image=@./test.jpg" -F "prompt=change into orange." -F "size=1024x1024" -F "output_format=png

If my command is ok, the current failure suggests a recent regression, as tests passed after merging this PR.

@wtomin wtomin added the ready label to trigger buildkite CI label Mar 10, 2026
Copy link
Copy Markdown
Collaborator

@lishunyang12 lishunyang12 left a comment

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

LGTM — fix is correct. Left a minor note about the __getattr__ design.

def base_model_paths(self) -> list[BaseModelPath]:
return self._base_model_paths

def __getattr__(self, name):
Copy link
Copy Markdown
Collaborator

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Nit: the broader issue is that __getattr__ returns _Unsupported instead of raising AttributeError, so hasattr() always returns True for any attribute on this class. This fix is fine for the immediate bug, but any future hasattr() check will silently pass too.

@RuixiangMa
Copy link
Copy Markdown
Collaborator Author

Since the buggy commit got reverted, is this still needed?

@wtomin
Copy link
Copy Markdown
Collaborator

wtomin commented Mar 11, 2026

I think it's still needed, because there is a buggy code here:

if serving_models and hasattr(serving_models, "base_model_paths") and serving_models.base_model_paths:
model_name = serving_models.base_model_paths[0].name
else:
model_name = "unknown"

Copy link
Copy Markdown
Collaborator

@wtomin wtomin left a comment

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Please solve this conflicts.

Signed-off-by: Lancer <maruixiang6688@gmail.com>
Signed-off-by: Lancer <maruixiang6688@gmail.com>
@RuixiangMa
Copy link
Copy Markdown
Collaborator Author

Please solve this conflicts.

Done

@wtomin wtomin merged commit 0f7c9c7 into vllm-project:main Mar 11, 2026
7 checks passed
meghaagr13 pushed a commit to meghaagr13/vllm-omni that referenced this pull request Mar 12, 2026
…llm-project#1771)

Signed-off-by: Lancer <maruixiang6688@gmail.com>
Signed-off-by: Megha Agarwal <agarwalmegha1308@gmail.com>
meghaagr13 pushed a commit to meghaagr13/vllm-omni that referenced this pull request Mar 12, 2026
clodaghwalsh17 pushed a commit to clodaghwalsh17/nm-vllm-omni-ent that referenced this pull request May 12, 2026
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

ready label to trigger buildkite CI

Projects

None yet

Development

Successfully merging this pull request may close these issues.

3 participants