Skip to content

Fix MXFP4 expert zero weight loading - #31196

Open
robbiemu wants to merge 3 commits into
sgl-project:mainfrom
robbiemu:agent/fix-mxfp4-expert-zero-loader
Open

robbiemu wants to merge 3 commits into
sgl-project:mainfrom
robbiemu:agent/fix-mxfp4-expert-zero-loader

Conversation

@robbiemu

@robbiemu robbiemu commented Jul 14, 2026

Copy link
Copy Markdown

Summary

  • distinguish the all-expert None sentinel from valid expert ID 0 in the static MXFP4 weight loader
  • add a focused CPU regression for a 2-D per-expert MXFP4 weight with expert ID 0

Root cause

FusedMoE.weight_loader used not expert_id to detect an all-expert tensor. That condition is also true for expert ID 0, so the loader entered the 3-D all-expert path and indexed loaded_weight.shape[2] on a 2-D per-expert tensor.

This small fix is also present in the broader open #21529; this PR splits the loader correctness change and its regression test into an independently reviewable patch.

Impact

Static MXFP4 checkpoints loaded one expert at a time can fail while loading expert zero with IndexError: tuple index out of range.

Validation

  • python3 -m pytest -q test/registered/unit/layers/moe/test_mxfp4_expert_weight_loader.py
  • python3 -m ruff check python/sglang/srt/layers/moe/fused_moe_triton/layer.py test/registered/unit/layers/moe/test_mxfp4_expert_weight_loader.py

CI States

Latest PR Test (Base): ❌ Run #29341502637
Latest PR Test (Extra): ❌ Run #29341502576

@gemini-code-assist

Copy link
Copy Markdown
Contributor

Warning

You have reached your daily quota limit. Please wait up to 24 hours and I will start processing your requests again!

@robbiemu
robbiemu marked this pull request as ready for review July 14, 2026 14:18
@gemini-code-assist

Copy link
Copy Markdown
Contributor

Warning

You have reached your daily quota limit. Please wait up to 24 hours and I will start processing your requests again!

@robbiemu

robbiemu commented Aug 31, 2026

Copy link
Copy Markdown
Author

This probably should have been merged before #32541 . You can feel free to close it.

Maybe there is a norm I'm not conforming to that my PRs get no review and the same code gets merged later? (Of course, I understand things slip through the cracks sometimes! Just want to make sure I understand if and when contributions are accepted)

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

None yet

Projects

None yet

Development

Successfully merging this pull request may close these issues.

1 participant