Skip to content

[ROCm] Disable AITER allreduce fusion #41866

New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

Sign up for GitHub

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Closed
jpvillam-amd wants to merge 4 commits into from
+2 −8
Closed

[ROCm] Disable AITER allreduce fusion #41866

Changes from all commits
Commits
Show all changes
4 commits
Select commit Hold shift + click to select a range
4918b15
Subset of #41816
jpvillam-amd May 6, 2026
6d6697e
Always disable for rocm
jpvillam-amd May 6, 2026
7bbd3bb
precommit changes
jpvillam-amd May 6, 2026
297f0ff
Merge branch 'main' into jpvillam/disable-rs-fusion
dllehr-amd May 6, 2026
File filter

Filter by extension

Filter by extension
Conversations
Failed to load comments.
Loading
Jump to
Jump to file
Failed to load files.
Loading
Diff view
Diff view
10 changes: 2 additions & 8 deletions vllm/compilation/passes/pass_manager.py
View file
Open in desktop
Original file line number Diff line number Diff line change
Expand Up @@ -19,9 +19,6 @@
from .vllm_inductor_pass import VllmInductorPass, VllmPatternMatcherPass

if rocm_aiter_ops.is_enabled():
from .fusion.allreduce_rms_fusion import (
RocmAiterAllReduceFusionPass,
)
from .fusion.rocm_aiter_fusion import (
MLADualRMSNormFusionPass,
RocmAiterRMSNormQuantFusionPass,
Expand Down Expand Up @@ -142,11 +139,8 @@ def configure(self, config: VllmConfig) -> None:
if self.pass_config.fuse_gemm_comms:
self.passes += [AsyncTPPass(config)]

if self.pass_config.fuse_allreduce_rms:
if rocm_aiter_ops.is_enabled():
Copy link
Copy Markdown
Collaborator

@tjtanaa tjtanaa May 7, 2026 •
edited
Loading

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

@jpvillam-amd @gshtras may I know what is the reason to disable all reduce fusion when we can control through python argument by setting --compilation_config.pass_config.fuse_allreduce_rms=false

Copy link
Copy Markdown
Collaborator

@tjtanaa tjtanaa May 7, 2026 •
edited
Loading

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

And it is preferred to not remove the path completely as it was working with older aiter version, and we should should disable else where and add a TODO to fix it.

self.passes += [RocmAiterAllReduceFusionPass(config)]
else:
self.passes += [AllReduceFusionPass(config)]
if self.pass_config.fuse_allreduce_rms and current_platform.is_cuda():
self.passes += [AllReduceFusionPass(config)]
Comment thread
jpvillam-amd marked this conversation as resolved.

if self.pass_config.fuse_minimax_qk_norm:
self.passes += [MiniMaxQKNormPass(config)]
Expand Down
Loading