[mickqian]

Motivation

H3 pads its packed sequence to a multiple of 64 per Ulysses rank, so a 1280x704 request attends cu_seqlens [0, 33002, 33024]. cuDNN SDPA's varlen runs each segment as its own call, so every pipelined head group ran the real rows, a copy into the output, a cuDNN call for the 22 alignment rows and a second copy. With 7 groups per layer that is 28 extra launches and their host work per block.

Modifications

• The alignment rows only ever reach their own rows (attention keeps them in their own segment, everything else is row-wise), so group attention now attends the real rows only, writes them straight into the output when the backend can (FA4), and zeroes the alignment rows. The real rows run the same call as before.
• Test added to test_minimax_h3_dit_contract.py.

Accuracy Tests

Bit-identical output: MP4 md5 identical to main in every arm (4/4 on FA4, 4/4 forced-cuDNN).

Speed Tests and Profiling

4x B300 (stand-in for B200), same protocol as the three-waves PR; measured on top of it:

	1280x704	1344x768
three-waves	26.55 s	31.78 s
+ this PR	26.46 s (-0.10)	31.63 s (-0.14)

Together with three-waves, the 20-prompt 720p median goes 26.793 -> 26.517 s (-1.03%).

Note: this touches the same H3 pipelined-Ulysses block in minimax_h3.py as the ROCm Ulysses-pipeline PR; whichever lands second needs a small rebase.

Checklist

• Format your code according to the pre-commit config (isort / ruff / black pass).
• Add unit tests (merged into the existing test file for the touched area).
• Provide accuracy and speed benchmark results (above).

🤖 Generated with Claude Code

---

CI States

Latest PR Test (Base): ✅ Run #41207111782
Latest PR Test (Extra): ❌ Run #41207111700
Latest PR Test (AMD ROCm 10): ❌ Run #41207111822

[claude]

Claude Code Review

This repository is configured for manual code reviews. Comment @claude review for a one-time review, or @claude review always to subscribe this PR to a review on every future push.

_{Tip: disable this comment in your organization's Code Review settings.}

[mickqian]

/tag-and-rerun-ci

[bingxche]

AMD CI jobs started failing after this PR was merged, and the evidence points at this change.

test_minimax_h3_dit_contract.py::test_group_attend_skips_only_the_trailing_alignment_rows, added by this PR, is skipped only when CUDA is unavailable; on ROCm it runs, falls back to the FlashAttention path and fails with ModuleNotFoundError: No module named 'flash_attn' (flash_attention_v3.py:239) on the rocm10 and rocm720 lanes.

The last passing run was on 8817f63c and the first failing run on cb0a7a95; this PR's commit 5032f988 is between them.

Failing jobs (2):

• multimodal-gen-test-1-gpu-amd (rocm10, linux-mi300-1gpu-sglang, 0)
• multimodal-gen-test-1-gpu-amd (rocm720, linux-mi300-1gpu-sglang, 0)

If this PR is the cause, a fix or a revert would turn these jobs green again. If it isn't, please say so here so the attribution can be corrected.

_{Posted automatically by amd-bot, the AMD CI monitor. Confidence: HIGH.}

[bingxche]

@mickqian AMD CI jobs started failing after this PR was merged, and #47971 proposes a fix. Could you review it?

test_minimax_h3_dit_contract.py::test_group_attend_skips_only_the_trailing_alignment_rows, added by this PR, is gated only on torch.cuda.is_available(); on ROCm DynamicCudnnSDPAImpl falls back to the FlashAttention path and fails with No module named 'flash_attn' on rocm10, rocm720 and rocm724 in every run since it landed.

The last passing run was on 8817f63c and the first failing run on cb0a7a95; this PR's commit 5032f988 is between them.

Failing jobs (5):

• multimodal-gen-test-1-gpu-amd (rocm10, linux-mi300-1gpu-sglang, 0)
• multimodal-gen-test-1-gpu-amd (rocm720, linux-mi300-1gpu-sglang, 0)
• multimodal-gen-test-1-gpu-amd (rocm10, linux-mi300-1gpu-sglang, 0)
• multimodal-gen-test-1-gpu-amd (rocm10, linux-mi300-1gpu-sglang, 0)
• multimodal-gen-test-1-gpu-amd (rocm724, linux-mi300-1gpu-sglang, 0)

If this PR isn't the cause, please say so here so the attribution can be corrected.

_{Posted automatically by amd-bot, the AMD CI monitor. Confidence: HIGH.}