Skip to content

[MoE] Unify DeepGEMM MegaMoE experts behind one feature-matrix layer; add NVFP4 #61538

New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

Sign up for GitHub

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Merged
zyongye merged 7 commits into from
Oct 15, 2026
Merged

[MoE] Unify DeepGEMM MegaMoE experts behind one feature-matrix layer; add NVFP4 #61538

Show file tree
Hide file tree
Changes from 5 commits
Commits
Show all changes
7 commits
Select commit Hold shift + click to select a range
f369e66
[MoE] Unify DeepGEMM MegaMoE experts behind one layer with a feature …
zyongye Oct 13, 2026
127fcef
[MoE] Add NVFP4 routed experts with MXFP8 or BF16 shared experts to M…
zyongye Oct 13, 2026
ce8a54b
[MoE] MegaMoE: run kernel tests on B200 CI, reloadable NVFP4 L2 scale
zyongye Oct 14, 2026
603db01
[MoE] MegaMoE NVFP4: skip meta loads in host scale state, check w1/w3…
zyongye Oct 14, 2026
5550180
Merge branch 'main' into refactor/deep-gemm-mega-moe-experts
zyongye Oct 14, 2026
c587e24
[MoE] MegaMoE: run the input staging tests on SM100 only
zyongye Oct 15, 2026
d83c149
Merge branch 'main' into refactor/deep-gemm-mega-moe-experts
khluu Oct 15, 2026
File filter

Filter by extension

Filter by extension
Conversations
Failed to load comments.
Loading
Jump to
Jump to file
Failed to load files.
Loading
Diff view
Diff view
4 changes: 4 additions & 0 deletions .buildkite/test_areas/kernels.yaml
View file
Open in desktop
Original file line number Diff line number Diff line change
Expand Up @@ -138,11 +138,15 @@ steps:
- csrc/fused_deepseek_v4_qnorm_rope_kv_insert_kernel.cu
- vllm/models/deepseek_v4/common/ops/
- vllm/models/deepseek_v4/nvidia/
- vllm/model_executor/layers/fused_moe/deep_gemm_mega_moe.py
- vllm/model_executor/layers/fused_moe/prepare_megamoe.py
- vllm/utils/deep_gemm.py
- tests/kernels/test_fused_deepseek_v4_qnorm_rope_kv_insert.py
- tests/kernels/moe/test_deep_gemm_mega_moe.py
- tests/models/test_deepseek_v4_mega_moe.py
commands:
- pytest -v -s kernels/test_fused_deepseek_v4_*.py
- pytest -v -s kernels/moe/test_deep_gemm_mega_moe.py
- pytest -v -s models/test_deepseek_v4_mega_moe.py
mirror:
amd:
Expand Down
Loading
Loading