Skip to content

Revert "Fix RMSNorm API CALL mismatch issue. (#10032)" #13727

New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

Sign up for GitHub

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Merged
Fridge003 merged 2 commits into from
Nov 22, 2025
Merged

Revert "Fix RMSNorm API CALL mismatch issue. (#10032)" #13727

Changes from all commits
Commits
Show all changes
2 commits
Select commit Hold shift + click to select a range
ba9cf4d
Revert "Fix RMSNorm API CALL mismatch issue. (#10032)"
ErsongWang Nov 21, 2025
92f5eb2
Merge branch 'main' into fused_rms_args_mismatch
Fridge003 Nov 22, 2025
File filter

Filter by extension

Filter by extension
Conversations
Failed to load comments.
Loading
Jump to
Jump to file
Failed to load files.
Loading
Diff view
Diff view
21 changes: 2 additions & 19 deletions python/sglang/srt/layers/layernorm.py
View file
Open in desktop
Original file line number Diff line number Diff line change
Expand Up @@ -19,7 +19,6 @@
import torch
import torch.nn as nn
import torch.nn.functional as F
from packaging.version import Version

from sglang.srt.batch_invariant_ops import (
is_batch_invariant_mode_enabled,
Expand Down Expand Up @@ -70,11 +69,8 @@
from aiter import rmsnorm2d_fwd as rms_norm
from aiter import rmsnorm2d_fwd_with_add as fused_add_rms_norm
elif _is_hip:
import vllm
from vllm._custom_ops import fused_add_rms_norm, rms_norm

_vllm_version = Version(vllm.__version__)

logger = logging.getLogger(__name__)

if _is_npu:
Expand Down Expand Up @@ -169,21 +165,8 @@ def forward_hip(
# NOTE: Remove this if aiter kernel supports discontinuous input
x = x.contiguous()
if residual is not None:
if _vllm_version < Version("0.9"):
fused_add_rms_norm(x, residual, self.weight.data, self.variance_epsilon)
return x, residual
else:
residual_out = torch.empty_like(x)
output = torch.empty_like(x)
fused_add_rms_norm(
output,
x,
residual_out,
residual,
self.weight.data,
self.variance_epsilon,
)
return output, residual_out
fused_add_rms_norm(x, residual, self.weight.data, self.variance_epsilon)
return x, residual
out = torch.empty_like(x)
rms_norm(out, x, self.weight.data, self.variance_epsilon)
return out
Expand Down
Loading