Add MoE layer support for NemotronH (Nemotron-3 30B/120B) - #188

Merged

justinchuby merged 7 commits into

mainfrom

justinchu/nemotron3-moe-support

Apr 22, 2026

Commits on Apr 22, 2026

Add MoE layer support for NemotronH (Nemotron-3 30B/120B)

justinchuby
and
Copilot
committed
8e32f86
Add NemotronH MoE L4 test cases and fix HF dt_bias corruption

justinchuby
and
Copilot
committed
0fa4cc9
Update skill docs: NemotronH MoE architecture and dt_bias corruption bug

justinchuby
and
Copilot
committed
ee5e162
Document com.microsoft.MoE incompatibility with NemotronH

justinchuby
and
Copilot
committed
c7210f5
Address PR review: float32 gate routing, dt_bias fix, MoE weight test

justinchuby
and
Copilot
committed
261e4f1
Add NemotronH L5 golden tests and fix _init_weights corruption

justinchuby
and
Copilot
committed
0ba4fc1
Merge origin/main into nemotron3-moe-support

justinchuby
and
Copilot
committed
bc5c613