Scaling for MuP over Muon optimizer. - #3715

Merged

BoxiangW merged 13 commits into

NVIDIA:mainfrom

plugyawn:feature/mup-muon

Mar 23, 2026

Commits on Mar 17, 2026

mup: avoid double-scaling Muon-managed matrices and warn on non-principled mode
plugyawn
committed
b3f865e
mup+muon: default to unit_rms_norm only when MuP is enabled
plugyawn
committed
fa5aa86
mup+muon: warn on spectral mode without changing defaults
plugyawn
committed
10a46ad
mup+muon: clarify warning to explicitly set unit_rms_norm
plugyawn
committed
b0c933b
mup+muon: document spectral mode usage with mup
plugyawn
committed
4baa863
mup+muon: remove redundant muon scale fallback
plugyawn
committed
cf65b09
mup+muon: clarify spectral warning semantics
plugyawn
committed
b483008
mup+muon: align warning wording with megatron style
plugyawn
committed
6d638ab
Merge branch 'main' into feature/mup-muon
asolergi-nv
authored
c8c9078

Commits on Mar 19, 2026

tests: patch single-rank MuP warning assertions
plugyawn
committed
962b945
Merge branch 'main' into feature/mup-muon
plugyawn
authored
aac1e1b

Commits on Mar 22, 2026

Merge branch 'main' into feature/mup-muon
BoxiangW
authored
9849e84

Commits on Mar 23, 2026

Merge branch 'main' into feature/mup-muon
BoxiangW
authored
a86e420