Skip to content

feat(optim): per-head Muon for attention q/k/v (heads_per_block, default off) - #4114

Closed
snimu wants to merge 4 commits into
mainfrom
feat/muon-heads-per-block
Closed

feat(optim): per-head Muon for attention q/k/v (heads_per_block, default off)#4114
snimu wants to merge 4 commits into
mainfrom
feat/muon-heads-per-block

refactor(optim): head blocks from a model-level muon_head_partitions …

e4a6e09
Select commit
Loading
Failed to load commit list.

Select a check to view from the sidebar