Skip to content

Add per-head Muon QKV orthogonalization and fix TP>num_query_groups with Muon. Generalize to MLA and GQA - #6326

Open
mkhona-nvidia wants to merge 8 commits into
NVIDIA:mainfrom
mkhona-nvidia:agent/muon-split-qkv-per-head
Open

Add per-head Muon QKV orthogonalization and fix TP>num_query_groups with Muon. Generalize to MLA and GQA#6326
mkhona-nvidia wants to merge 8 commits into
NVIDIA:mainfrom
mkhona-nvidia:agent/muon-split-qkv-per-head

Address QKV layout review feedback

febf2f1
Select commit
Loading
Failed to load commit list.
DCO / DCO succeeded Aug 19, 2026 in 1s

DCO

All commits are signed off!

View more details on DCO