Repository navigation
[Bugfix][ROCm] Skip Qwen-Image fused QK-norm assertions on HIP - #9229
Conversation
torch.cuda.is_available() is true on ROCm, but the fused kernel requires NVIDIA CUDA and returns None. Assert that fallback. Co-authored-by: Cursor <cursoragent@cursor.com> Signed-off-by: haic0 <149741444+haic0@users.noreply.github.com>
Codex Review SummaryThis comment shows the latest Codex review activity on this pull request.
ℹ️ About Codex in GitHubYour team has set up Codex to review pull requests in this repo. Reviews are triggered when you
Codex reacts with 👀 while any review is running, comments if it has suggestions, and reacts with 👍 once all reviews finish with no findings. |
|
Codex usage limits have been reached for code reviews. Please check with the admins of this repo to increase the limits by adding credits. |
|
This PR appears to be related to model: qwen-image. Model owners: @NumberWan @NickCao @yenuo26 Routing: @NumberWan via semantic router, model owner; @NickCao via CODEOWNERS; @yenuo26 via CODEOWNERS @haic0, please review your own changes and leave a short self-review comment describing what you checked. PRs without author self-review may not be assigned a reviewer. Please take a look when you have a chance. If you would like an automated review, mention @vllm-omni-review-bot in a comment. |
Signed-off-by: andyluo7 <andy.luo@amd.com>
andyluo7
left a comment
There was a problem hiding this comment.
Reviewed at exact head 6be07c8. The ROCm guards now cover every NVIDIA fused-path assertion while preserving CPU and cross-platform fallback coverage. All GitHub checks and exact synthetic-merge Buildkite pipelines passed with no retries or soft failures; AMD #14336 also confirms the prior Qwen-Image ROCm failures are gone.
#8798 (#9251) Signed-off-by: aayc123 <2803023760@qq.com> Signed-off-by: aayc123 <115020989+aayc123@users.noreply.github.com> Signed-off-by: 曾妮 <2803023760@qq.com> Signed-off-by: haic0 <149741444+haic0@users.noreply.github.com> Signed-off-by: andyluo7 <andy.luo@amd.com> Signed-off-by: caoshijie <shijiecao4@gmail.com> Signed-off-by: wuli666 <421774554@qq.com> Signed-off-by: Iryna Boiko <iboiko@habana.ai> Signed-off-by: NumberWan <wantszkin2003@gmail.com> Co-authored-by: aayc123 <115020989+aayc123@users.noreply.github.com> Co-authored-by: haic0 <149741444+haic0@users.noreply.github.com> Co-authored-by: Cursor <cursoragent@cursor.com> Co-authored-by: andyluo7 <andy.luo@amd.com> Co-authored-by: Shijie_cao <shijiecao4@gmail.com> Co-authored-by: dengyunyang <584797741@qq.com> Co-authored-by: wuli666 <421774554@qq.com> Co-authored-by: Claude Opus 5.5 <noreply@anthropic.com> Co-authored-by: NATURE <wzliu@connect.hku.hk> Co-authored-by: Iryna Boiko <iryna.boiko@intel.com> Co-authored-by: Chendi.Xue <chendi.xue@intel.com> Co-authored-by: NumberWan <wantszkin2003@gmail.com> Co-authored-by: wangyu <53896905+yenuo26@users.noreply.github.com>
Purpose
#7594 gates the fused kernel with
current_platform.is_cuda(), which is false on ROCm. The new tests usedtorch.cuda.is_available(), which is true on HIP. They ran on MI300 and expected a non-None CUDA table. AMD builds 14204, 14330, and 14334 failed those assertions. This is not a change to enable the fused kernel on ROCm.Original Test Plan
vLLM Version: the version installed in this checkout's
.venvvLLM-Omni Commit:
857b05b52on6b35bca6ffffOriginal Test Result
On this ROCm host (
torch.version.hipset,current_platform.is_rocm()true):3 passed, 5 skipped. The five NVIDIA fused-table cases skipped.
test_packed_table_skipped_when_fused_path_unavailableandtest_packed_table_is_none_on_rocmpassed and both assert the ROCm table isNoneeven when the token gate is open.pre-commit and
mypy-3.12on the test file passed.Maintainer follow-up and current-head validation
Current head
6be07c8a2c944e2a6b4ac85f3c546493bd0500d4additionally applies the NVIDIA-only guard to every test whose assertion depends on the fused CUDA kernel and requires an available HIP GPU for the dedicated ROCm fallback test.torch.cudapolicy.0e3a3570b35b1a5c7f371ad5a448afb2ba64fe76has parents6b35bca6ffff3c6bee4723984704fa39c9ac2af0and6be07c8a2c944e2a6b4ac85f3c546493bd0500d4.mi300_1: Diffusion · Model Testjob reported 105 passed and 102 skipped; the Qwen-Image file contributed 5 passes and 16 intentional NVIDIA-only skips, with none of the six prior ROCm failures.