[Bugfix] Expose base_model_paths property in _DiffusionServingModels#1771
Conversation
Signed-off-by: Lancer <maruixiang6688@gmail.com>
|
Is your PR going to fix a CI failures introduced by recent commit |
Yes, tested against docs/serving/image_edit_api.md using the specified curl command on the main branch (7a56049 (HEAD -> main, origin/main) Add online serving to Stable Audio Diffusion and introduce If my command is ok, the current failure suggests a recent regression, as tests passed after merging this PR. |
lishunyang12
left a comment
There was a problem hiding this comment.
LGTM — fix is correct. Left a minor note about the __getattr__ design.
| def base_model_paths(self) -> list[BaseModelPath]: | ||
| return self._base_model_paths | ||
|
|
||
| def __getattr__(self, name): |
There was a problem hiding this comment.
Nit: the broader issue is that __getattr__ returns _Unsupported instead of raising AttributeError, so hasattr() always returns True for any attribute on this class. This fix is fine for the immediate bug, but any future hasattr() check will silently pass too.
|
Since the buggy commit got reverted, is this still needed? |
|
I think it's still needed, because there is a buggy code here: vllm-omni/vllm_omni/entrypoints/openai/api_server.py Lines 1400 to 1403 in 74eedd3 |
wtomin
left a comment
There was a problem hiding this comment.
Please solve this conflicts.
Signed-off-by: Lancer <maruixiang6688@gmail.com>
Done |
…llm-project#1771) Signed-off-by: Lancer <maruixiang6688@gmail.com> Signed-off-by: Megha Agarwal <agarwalmegha1308@gmail.com>
…llm-project#1771) Signed-off-by: Lancer <maruixiang6688@gmail.com>
…llm-project#1771) Signed-off-by: Lancer <maruixiang6688@gmail.com>
Purpose
In pure diffusion mode, _DiffusionServingModels uses getattr to intercept undefined attributes. Accessing base_model_paths returned a _Unsupported object instead of the actual list, causing NotImplementedError when
retrieving model names for video generation endpoints
curl -s -D >(grep -i x-request-id >&2) -o >(jq -r '.data[0].b64_json' | base64 --decode > output.png) -X POST "http://localhost:8004/v1/images/edits" -F "model=black-forest-labs/FLUX.2-klein-4B" -F "image=@./test.jpg" -F "prompt=change into orange." -F "size=1024x1024" -F "output_format=png"Test Plan
Test Result
(APIServer pid=111918) INFO 03-10 11:50:44 [diffusion_engine.py:86] Generation completed successfully.
(APIServer pid=111918) INFO 03-10 11:50:44 [diffusion_engine.py:108] Post-processing completed in 0.1113 seconds
(APIServer pid=111918) INFO 03-10 11:50:44 [diffusion_engine.py:111] DiffusionEngine.step breakdown: preprocess=0.00 ms, add_req_and_wait=25044.09 ms, postprocess=111.33 ms, total=25156.01 ms
(APIServer pid=111918) INFO 03-10 11:50:44 [omni_diffusion.py:128] OmniDiffusion.generate total: 25156.33 ms
(APIServer pid=111918) INFO 03-10 11:50:44 [api_server.py:1395] Successfully generated 1 image(s)
(APIServer pid=111918) INFO: 127.0.0.1:52072 - "POST /v1/images/edits HTTP/1.1" 200 OK