Skip to content

Add MoE layer support for NemotronH (Nemotron-3 30B/120B) - #188

Merged
justinchuby merged 7 commits into
mainfrom
justinchu/nemotron3-moe-support
Apr 22, 2026
Merged

Add MoE layer support for NemotronH (Nemotron-3 30B/120B)#188
justinchuby merged 7 commits into
mainfrom
justinchu/nemotron3-moe-support

Conversation

@justinchuby

Copy link
Copy Markdown
Member

Summary

Adds MoE (Mixture of Experts) support for NemotronH architecture, enabling ONNX graph construction for NVIDIA Nemotron-3 30B and 120B models.

Changes

MoE implementation:

  • NemotronHMoEGate: Sigmoid routing with correction bias and top-k selection
  • NemotronHMoEBlock: Non-gated FCMLP experts + shared expert + optional latent projection
  • NemotronHMoELayer: Standalone MoE decoder layer (RMSNorm → MoE → residual)
  • Updated preprocess_weights() for 3D→2D expert weight splitting
  • Updated cache utils to handle MoE as stateless layer type

L4 test cases:

  • nemotron-h-nano-4b (4B dense), nemotron-3-nano-30b (30B, 128 experts), nemotron-3-super-120b (120B, 512 experts)
  • Golden reference data generated with dt_bias corruption fix

HF dt_bias corruption fix:

  • NemotronH remote-code _init_weights re-initializes dt_bias with torch.rand() after checkpoint loading
  • Added _fix_nemotron_h_dt_bias() to read correct values from safetensors

Fused MoE op investigation:

  • com.microsoft.MoE is incompatible: squared ReLU activation not supported, sigmoid routing incompatible

Testing

  • L1 tests pass for both dense and MoE variants (2544 passed)
  • L4 golden tests created for all 3 models (ci_skip due to model size)

justinchuby and others added 4 commits April 22, 2026 04:45
Add NemotronHMoELayer with:
- NemotronHMoEGate: sigmoid routing with score correction bias
  (bias affects selection only, not final routing weights)
- NemotronHMoEBlock: non-gated FCMLP experts, shared expert,
  optional latent projection (120B)
- Stacked 3D expert tensor splitting in preprocess_weights

Config changes:
- Add 'E' -> 'moe' to hybrid_override_pattern char_map
- Fix type_map: 'moe' was incorrectly mapped to 'mlp'
- Add moe_latent_size field to NemotronHConfig
- Extract moe_shared_expert_intermediate_size from HF config

Cache changes:
- Handle 'moe' as stateless layer type (like 'mlp')

Test config:
- Add MoE variant with 4 experts for L1 graph build tests

Co-authored-by: Copilot <223556219+Copilot@users.noreply.github.com>
Signed-off-by: Justin Chu <justinchu@microsoft.com>
- Add L4 YAML test cases for Nemotron-3 Nano 30B (MoE, 128 experts)
  and Nemotron-3 Super 120B (MoE, 512 experts)
- Update Nemotron-H Nano 4B YAML: skip_reason → ci_skip_reason, L4 only
- Generate golden reference data for all three NemotronH models
- Fix HF NemotronH _init_weights dt_bias corruption: the remote code
  re-initialises dt_bias with torch.rand after from_pretrained loads
  checkpoint weights, silently corrupting the model. Added
  _fix_nemotron_h_dt_bias() to torch_reference.py to restore correct
  values from safetensors files.
- Add NemotronH models to _XFAIL_REASONS (HF non-determinism from
  mamba-ssm CUDA kernels); remove bamba (now passes)

Co-authored-by: Copilot <223556219+Copilot@users.noreply.github.com>
Signed-off-by: Justin Chu <justinchu@microsoft.com>
- Document NemotronH sigmoid routing, shared experts, latent projection,
  and graph size impact in moe-models skill
- Document com.microsoft.MoE compatibility considerations for sigmoid gates
- Add HF _init_weights corruption pitfall (#6) to adding-a-new-model skill
  with diagnosis pattern and fix reference

Co-authored-by: Copilot <223556219+Copilot@users.noreply.github.com>
Signed-off-by: Justin Chu <justinchu@microsoft.com>
NemotronH experts use squared ReLU (relu2) activation, which the fused
MoE op does not support. Additionally, the sigmoid routing with correction
bias and shared expert/latent projection patterns are not compatible.
Document this in both the model docstring and MoE skill file.

Co-authored-by: Copilot <223556219+Copilot@users.noreply.github.com>
Signed-off-by: Justin Chu <justinchu@microsoft.com>
@github-actions

github-actions Bot commented Apr 22, 2026

Copy link
Copy Markdown

Performance Comparison

Comparing 8d04ba8bc5c613

Model Metric Baseline Current Delta
bert (feature-extraction) model_size_bytes 359 KB 359 KB +0.0%
bert (feature-extraction) num_nodes 60 60 +0.0%
falcon model_size_bytes 364 KB 364 KB +0.0%
falcon num_nodes 66 66 +0.0%
gemma2 model_size_bytes 428 KB 428 KB +0.0%
gemma2 num_nodes 107 107 +0.0%
gpt2 model_size_bytes 388 KB 388 KB +0.0%
gpt2 num_nodes 53 53 +0.0%
llama model_size_bytes 425 KB 425 KB +0.0%
llama num_nodes 61 61 +0.0%
llama (static-cache) model_size_bytes 425 KB 425 KB +0.0%
llama (static-cache) num_nodes 58 58 +0.0%
mamba (ssm-text-generation) model_size_bytes 360 KB 360 KB +0.0%
mamba (ssm-text-generation) num_nodes 98 98 +0.0%
phi3 model_size_bytes 421 KB 421 KB +0.0%
phi3 num_nodes 59 59 +0.0%
phi3 (static-cache) model_size_bytes 421 KB 421 KB +0.0%
phi3 (static-cache) num_nodes 56 56 +0.0%
qwen2 model_size_bytes 425 KB 425 KB +0.0%
qwen2 num_nodes 61 61 +0.0%
qwen2 (static-cache) model_size_bytes 425 KB 425 KB +0.0%
qwen2 (static-cache) num_nodes 58 58 +0.0%
qwen3_5_moe (hybrid-text-generation) model_size_bytes 506 KB 506 KB +0.0%
qwen3_5_moe (hybrid-text-generation) num_nodes 275 275 +0.0%
qwen3_5_text (hybrid-text-generation) model_size_bytes 458 KB 458 KB +0.0%
qwen3_5_text (hybrid-text-generation) num_nodes 129 129 +0.0%
qwen3_5_vl (hybrid-qwen-vl) model_size_bytes 977 KB 977 KB +0.0%
qwen3_5_vl (hybrid-qwen-vl) num_nodes 408 408 +0.0%
t5 (seq2seq) model_size_bytes 836 KB 836 KB +0.0%
t5 (seq2seq) num_nodes 166 166 +0.0%
whisper (speech-to-text) model_size_bytes 1008 KB 1008 KB +0.0%
whisper (speech-to-text) num_nodes 128 128 +0.0%

No performance regressions.

@codecov

codecov Bot commented Apr 22, 2026

Copy link
Copy Markdown

Codecov Report

❌ Patch coverage is 61.87050% with 53 lines in your changes missing coverage. Please review.

Files with missing lines Patch % Lines
src/mobius/_testing/torch_reference.py 8.69% 41 Missing and 1 partial ⚠️
src/mobius/models/nemotron_h.py 89.65% 4 Missing and 5 partials ⚠️
src/mobius/_configs.py 33.33% 2 Missing ⚠️

📢 Thoughts on this report? Let us know!

@github-actions

github-actions Bot commented Apr 22, 2026

Copy link
Copy Markdown

🏗️ Architecture Diff

Comparing 8d04ba8bc5c613

Model Sub-model Changes Status
bert (feature-extraction) model 0
falcon model 0
gemma2 model 0
gemma4 (gemma4) decoder 0
gemma4 (gemma4) embedding 0
gemma4 (gemma4) vision 0
gemma4_text model 0
gpt2 model 0
llama model 0
llama (static-cache) model 0
mamba (ssm-text-generation) model 0
phi3 model 0
phi3 (static-cache) model 0
qwen model 0
qwen (static-cache) model 0
qwen2 model 0
qwen2 (static-cache) model 0
qwen2_moe model 0
qwen2_moe (static-cache) model 0
qwen3 model 0
qwen3 (static-cache) model 0
qwen3_5_moe (hybrid-text-generation) model 0
qwen3_5_text (hybrid-text-generation) model 0
qwen3_5_vl (hybrid-qwen-vl) decoder 0
qwen3_5_vl (hybrid-qwen-vl) embedding 0
qwen3_5_vl (hybrid-qwen-vl) vision 0
qwen3_moe model 0
qwen3_moe (static-cache) model 0
qwen3_next (hybrid-text-generation) model 0
t5 (seq2seq) decoder 0
t5 (seq2seq) encoder 0
whisper (speech-to-text) decoder 0
whisper (speech-to-text) encoder 0

No architecture changes detected.


Legend: ⚪ No change · 🔵 Minor (attrs/inits) · 🟡 Moderate (nodes added/removed) · 🔴 Major (interface changed)

Copilot AI left a comment

Copy link
Copy Markdown
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Pull request overview

Adds NemotronH MoE (Mixture-of-Experts) support to Mobius’ ONNX graph builder, enabling Nemotron-3 30B/120B hybrid (Mamba2 + MoE + Attention) model construction and associated golden/L4 test artifacts.

Changes:

  • Implemented NemotronH MoE gate/block/layer and updated NemotronH weight preprocessing to split stacked expert tensors.
  • Updated hybrid cache utilities and build-graph tests to treat moe layers as stateless (no KV/state I/O).
  • Added/updated L4 golden test cases + documentation for NemotronH MoE and the HF dt_bias corruption workaround.

Reviewed changes

Copilot reviewed 15 out of 15 changed files in this pull request and generated 3 comments.

Show a summary per file
File Description
src/mobius/models/nemotron_h.py Adds NemotronH MoE modules, integrates moe into layer dispatch, and splits stacked expert weights in preprocess_weights().
src/mobius/_configs.py Extends NemotronH config parsing to recognize MoE (E / "moe") and adds MoE-specific config fields.
src/mobius/tasks/_cache_utils.py Treats MoE layers as stateless in hybrid cache input/output wiring.
tests/build_graph_test.py Adjusts cache-output assertions for MoE and adds a tiny NemotronH MoE config case.
tests/_test_configs.py Adds NemotronH MoE variant tiny config and removes outdated “MoE not implemented” note.
tests/e2e_golden_test.py Marks NemotronH golden comparisons as xfail (per PR rationale).
src/mobius/_testing/torch_reference.py Adds _fix_nemotron_h_dt_bias() and applies it after HF model load for reference inference.
testdata/cases/causal-lm/*.yaml Adds Nemotron-3 30B/120B cases and adjusts NemotronH nano case to L4-only w/ CI skip reason.
testdata/golden/causal-lm/*.json Adds golden references for the three NemotronH/Nemotron-3 cases.
.agents/skills/moe-models/SKILL.md Documents NemotronH MoE architecture, routing differences, and graph-size implications.
.agents/skills/adding-a-new-model/SKILL.md Adds troubleshooting guidance for HF _init_weights corruption (NemotronH dt_bias).

Comment thread src/mobius/models/nemotron_h.py
Comment thread src/mobius/_testing/torch_reference.py Outdated
Comment thread src/mobius/models/nemotron_h.py Outdated
@justinchuby
justinchuby force-pushed the justinchu/nemotron3-moe-support branch from 88b2c29 to f3834ed Compare April 22, 2026 17:15
- Cast NemotronHMoEGate routing computation to float32 for numerical
  stability (eps=1e-20 underflows in fp16/bf16). Cast routing_weights
  back to original dtype before returning.
- Use huggingface_hub.snapshot_download(local_files_only=True) to
  resolve exact snapshot path in _fix_nemotron_h_dt_bias instead of
  unreliable lexicographic glob sort. Add warning when no dt_bias
  params found.
- Add test_nemotron_h_moe_preprocess_weights verifying 3D stacked
  expert tensor splitting into per-expert 2D weights with correct
  keys and shapes.

Co-authored-by: Copilot <223556219+Copilot@users.noreply.github.com>
Signed-off-by: Justin Chu <justinchu@microsoft.com>
@justinchuby
justinchuby force-pushed the justinchu/nemotron3-moe-support branch from f3834ed to 261e4f1 Compare April 22, 2026 17:16
justinchuby and others added 2 commits April 22, 2026 17:34
- Extend _fix_nemotron_h_init_weights to also restore out_proj.weight
  params corrupted by HF _init_weights (rescale_prenorm_residual)
- Set rescale_prenorm_residual=False before loading NemotronH models
  in load_torch_model to prevent corruption at load time
- Patch NemotronH prepare_inputs_for_generation cache_position bug
  in generate_golden.py for transformers 5.x compatibility
- Enable L5 generation tests for nemotron-h-nano-4b
- Regenerate L4 golden with corrected weights (deterministic now)

Co-authored-by: Copilot <223556219+Copilot@users.noreply.github.com>
Signed-off-by: Justin Chu <justinchu@microsoft.com>
Resolve conflict in tests/e2e_golden_test.py: keep empty _XFAIL_REASONS
dict since NemotronH non-determinism was fixed by _fix_nemotron_h_init_weights.

Co-authored-by: Copilot <223556219+Copilot@users.noreply.github.com>
Signed-off-by: Justin Chu <justinchu@microsoft.com>
@justinchuby
justinchuby merged commit e6cdfe3 into main Apr 22, 2026
19 of 22 checks passed
@justinchuby
justinchuby deleted the justinchu/nemotron3-moe-support branch April 22, 2026 20:25
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

None yet

Projects

None yet

Development

Successfully merging this pull request may close these issues.

3 participants