Skip to content

[Test] [AMD] diffusion: Skip the Kandinsky6 FA dispatch test on ROCm - #44421

Merged
yichiche merged 4 commits into
sgl-project:mainfrom
michaelzhang-ai:fix-kandinsky6-sdpa-amd
Oct 20, 2026
Merged

[Test] [AMD] diffusion: Skip the Kandinsky6 FA dispatch test on ROCm#44421
yichiche merged 4 commits into
sgl-project:mainfrom
michaelzhang-ai:fix-kandinsky6-sdpa-amd

Conversation

@michaelzhang-ai

@michaelzhang-ai michaelzhang-ai commented Oct 15, 2026 •
edited by github-actions Bot
Loading

Copy link
Copy Markdown
Collaborator

What

test_kandinsky6_bf16_dtype.py::test_attention_backend_dispatch_and_repeated_forward[*-FA] (added in #42743) fails on the scheduled AMD run (multimodal-gen-test-1-gpu-amd). Test-only fix, no runtime changes.

Failing run: https://github.com/sgl-project/sglang/actions/runs/38563492873/job/115850314797

Why

On ROCm the FA backend resolves to Torch SDPA (FA3 is CUDA-only, and flash_attn is not in the image), so the "must not silently fall back" assert fails. The test already skips FA on SM12.x for the same reason; this adds the same skip for ROCm.

This PR originally also fixed test_server_args.py::test_fasth3_selects_vsa_h3_for_the_transformer on ROCm. Main has since fixed that upstream (the shared CUDA mock now pins is_rocm), so the merge with main takes main's version of that file.

Test plan

  • MI300: the FA cases skip and the TORCH_SDPA cases pass (job).
  • CUDA: the multimodal unit suite passes (job).

CI States

Latest PR Test (Base): ✅ Run #39262941748
Latest PR Test (Extra): ❌ Run #39262941729
Latest PR Test (AMD ROCm 10): ❌ Run #39262941758

@michaelzhang-ai
[diffusion] fix: skip platform-specific attention asserts on ROCm in …
187ade9 
…unit tests

test_kandinsky6_bf16_dtype: ROCm resolves the FA backend to SDPA (FA3 is
CUDA-only), so skip the FA case there as is already done for SM12.x.

test_server_args: the FastH3 VSA-H3 default is gated on is_cuda(); patch it
in _fasth3_args alongside the mocked capability so the test does not depend
on the host platform, and cover the non-CUDA path explicitly.
@github-actions github-actions Bot added the diffusion SGLang Diffusion label Oct 15, 2026
@michaelzhang-ai michaelzhang-ai added the run-ci CI: run the baseline test suite on this PR label Oct 15, 2026
@michaelzhang-ai michaelzhang-ai changed the title [diffusion] fix: make Kandinsky6 FA and FastH3 VSA-H3 unit tests platform-aware for ROCm [Test] [AMD] diffusion: Make Kandinsky6 FA and FastH3 VSA-H3 unit tests platform-aware Oct 15, 2026
@bingxche bingxche mentioned this pull request Oct 15, 2026
Open
@michaelzhang-ai
[diffusion] fix: pin is_rocm in the FastH3 server-args tests
b311e39 
ROCm defaults attention_backend to aiter, which FastH3 treats as an explicit
transformer backend. The shared CUDA mock patches is_cuda but not is_rocm, so
patch it in _fasth3_args and cover the ROCm default as its own case.
@stayinalive181

stayinalive181 commented Oct 15, 2026

Copy link
Copy Markdown

Some MI355X data in support of keeping aiter as FastH3's transformer default on ROCm. Setup: lmsysorg/sglang-rocm:v0.5.21-rocm10-mi35x-20261014, 1× MI355X.

  • The kernel itself works on gfx950. The in-tree VSA-H3 Triton forward passes test_vsa_h3_attention.py (12 passed).

  • It is slower than dense aiter at 10% block density.

    Tokens (24 heads, head 128) VSA-H3 Triton Dense aiter
    64k 69.4 ms 42.5 ms
    32k 17.6 ms 10.6 ms

    The VSA-H3 numbers use num_stages=2, waves_per_eu=2, matrix_instr_nonkdim=16. The pinned Hopper/Blackwell launch (num_warps=4, num_stages=5) is slower still, at 91.3 ms for 64k.

  • End to end it would need more work. Routing video_sparse_attn_h3 on ROCm stops in flash_attention_v3.flash_attn_varlen_func with No module named 'flash_attn'.

  • The aiter default works. FastH3 V2 t2va at 1344×768, 5 s, 8 steps takes 34.8 s on one MI355X.

So the CUDA-only VSA-H3 default looks like the right call for ROCm today.

@michaelzhang-ai

michaelzhang-ai commented Oct 15, 2026 •
edited
Loading

Copy link
Copy Markdown
Collaborator Author

Thanks for the data. To be clear on scope, this PR is test-only and doesn't change backend selection: FastH3 still defaults to VSA-H3 only on CUDA, and ROCm keeps its aiter default. The new test_fasth3_keeps_the_rocm_default_backend case pins that, so routing ROCm to VSA-H3 later would mean updating that test on purpose. @stayinalive181

@michaelzhang-ai michaelzhang-ai added the bypass-fail-fast CI: a failing job no longer aborts its siblings (lint still gates) label Oct 15, 2026
@michaelzhang-ai
[diffusion] fix: run the FastH3 ROCm-default test only on ROCm
d354e81 
FastH3 validation resolves the aiter backend class, which cannot be imported
on a CUDA host, so mocking is_rocm is not enough there.
@michaelzhang-ai michaelzhang-ai mentioned this pull request Oct 15, 2026
Merged
This was referenced Oct 16, 2026
Open
Open
Open
Open
@yichiche

yichiche commented Oct 19, 2026

Copy link
Copy Markdown
Collaborator

@michaelzhang-ai Please fix conflict. PR looks good to me.

@michaelzhang-ai
Merge main into fix-kandinsky6-sdpa-amd
56a8c45 
Take main's test_server_args.py: upstream now pins is_rocm in the shared CUDA
mock and stubs backend resolution, which covers the FastH3 ROCm failure this
branch fixed.
@michaelzhang-ai michaelzhang-ai changed the title [Test] [AMD] diffusion: Make Kandinsky6 FA and FastH3 VSA-H3 unit tests platform-aware [Test] [AMD] diffusion: Skip the Kandinsky6 FA dispatch test on ROCm Oct 19, 2026
@michaelzhang-ai

michaelzhang-ai commented Oct 19, 2026 •
edited
Loading

Copy link
Copy Markdown
Collaborator Author

Thanks. Merged main and resolved the conflict in test_server_args.py by taking main's version, since main now covers the FastH3 ROCm fix there. The PR is down to the Kandinsky6 FA skip; title and description are updated. @yichiche

@bingxche bingxche mentioned this pull request Oct 20, 2026
Open
yichiche
yichiche approved these changes Oct 20, 2026
View reviewed changes

@yichiche yichiche left a comment

Copy link
Copy Markdown
Collaborator

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

LGTM.

@yichiche
yichiche merged commit 36978d6 into sgl-project:main Oct 20, 2026
143 of 154 checks passed
@michaelzhang-ai
michaelzhang-ai deleted the fix-kandinsky6-sdpa-amd branch October 20, 2026 06:08
@stayinalive181 stayinalive181 mentioned this pull request Oct 21, 2026
Open
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Reviewers

@yichiche yichiche yichiche approved these changes
@mickqian mickqian Awaiting requested review from mickqian mickqian is a code owner
@ping1jing2 ping1jing2 Awaiting requested review from ping1jing2 ping1jing2 is a code owner
@HaiShaw HaiShaw Awaiting requested review from HaiShaw HaiShaw is a code owner
@AgainstEntropy AgainstEntropy Awaiting requested review from AgainstEntropy AgainstEntropy is a code owner
@BBuf BBuf Awaiting requested review from BBuf BBuf is a code owner
@kevin-mii kevin-mii Awaiting requested review from kevin-mii kevin-mii is a code owner
@niehen6174 niehen6174 Awaiting requested review from niehen6174 niehen6174 is a code owner

Assignees

No one assigned

Labels

bypass-fail-fast CI: a failing job no longer aborts its siblings (lint still gates) diffusion SGLang Diffusion run-ci CI: run the baseline test suite on this PR

Projects

None yet

Milestone

No milestone

Development

Successfully merging this pull request may close these issues.

3 participants

@michaelzhang-ai @stayinalive181 @yichiche