Repository navigation
[Perf] [ROCM] diffusion: Pipeline MiniMax-H3's Ulysses exchange over head groups on RCCL - #47567
Merged
Merged
Conversation
…oups on RCCL Without the copy-engine transport (ROCm), each H3 layer packs q/k/v, waits on the input all-to-all, attends, then waits on the output all-to-all: ~2.5 ms of exposed RCCL per layer at 1280x704 on 4xMI355X. Split the local heads into two groups: queue both input exchanges up front on the collective stream and send group g's output while group g + 1 attends. aiter attends a head group with the same bytes as inside the full call, so the output is bit-identical. Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
![claude[bot]](https://avatars.githubusercontent.com/in/1236702?s=60&v=4){kind=small}
There was a problem hiding this comment.
Claude Code Review
This repository is configured for manual code reviews. Comment @claude review for a one-time review, or @claude review always to subscribe this PR to a review on every future push.
Tip: disable this comment in your organization's Code Review settings.
mickqian
requested review from
AgainstEntropy,
BBuf,
kevin-mii,
niehen6174 and
yichiche
as code owners
October 31, 2026 02:15
This comment has been minimized.
This comment has been minimized.
Collaborator
Author
|
/tag-and-rerun-ci |
mickqian
changed the title
This was referenced Nov 1, 2026
niehen6174
added a commit
to niehen6174/sglang
that referenced
this pull request
Nov 5, 2026
Reviewed the six upstream minimax_h3.py changes since the last pin (sgl-project#46047, sgl-project#46709, sgl-project#47567, sgl-project#47569, sgl-project#47566 and the BCG output buffers). None touches what h3_comfyui_v1 relies on in its single-GPU eager torch_sdpa profile: DiTBlock.forward is unchanged, _attend still calls fused_inplace_qknorm_rope on the _use_fused_qknorm_rope branch with the same arguments (the new pipelined branch needs Ulysses, which the profile rejects), and _embed still returns (embeddings, t_emb) in eager mode.
Yuxingwang-intel
pushed a commit
to Yuxingwang-intel/sglang
that referenced
this pull request
Nov 6, 2026
…head groups on RCCL (sgl-project#47567) Co-authored-by: Mick Qian <mickqian@users.noreply.github.com> Co-authored-by: Claude Opus 5.5 <noreply@anthropic.com>
This file contains hidden or bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
Sign up for free
to join this conversation on GitHub.
Already have an account?
Sign in to comment
Add this suggestion to a batch that can be applied as a single commit.This suggestion is invalid because no changes were made to the code.Suggestions cannot be applied while the pull request is closed.Suggestions cannot be applied while viewing a subset of changes.Only one suggestion per line can be applied in a batch.Add this suggestion to a batch that can be applied as a single commit.Applying suggestions on deleted lines is not supported.You must change the existing code in this line in order to create a valid suggestion.Outdated suggestions cannot be applied.This suggestion has been applied or marked resolved.Suggestions cannot be applied from pending reviews.Suggestions cannot be applied on multi-line comments.Suggestions cannot be applied while the pull request is queued to merge.Suggestion cannot be applied right now. Please check back later.
Motivation
Without the copy-engine transport (ROCm), each H3 layer packs q/k/v, waits on the input all-to-all, attends, then waits on the output all-to-all: ~2.5 ms of exposed RCCL per layer at 1280x704 on 4xMI355X (14.5% of a denoise step).
Modifications
test_usp_pipelined_attention_multi_gpu.py.Accuracy Tests
Bit-identical output: 22/22 MP4 md5s equal to main in the final run.
Speed Tests and Profiling
Setup for all numbers below: 4x MI355X, MiniMax-H3 T2VA, 1280x704 (SemiAnalysis "720p"), 124 frames, 49 forwards, Ulysses 4 + aiter, single stream. Each arm is a fresh server (1 warmup + 3 timed requests), ABAB across two rounds, base = main
75a5da7f.720p median -3.24 s (the largest single item). Per step, ~63 ms of RCCL is now hidden behind attention; 849.5 -> 738 ms/step together with the QK norm change.
Note: this touches the same H3 pipelined-Ulysses block in
minimax_h3.pyas the CUDA-side PR that skips the packing's alignment rows; whichever lands second needs a small rebase.Checklist
🤖 Generated with Claude Code
CI States
Latest PR Test (Base): ✅ Run #41159187045
Latest PR Test (Extra): ❌ Run #41159187112
Latest PR Test (AMD ROCm 10): ❌ Run #41159187001