feat(moe_ep): restructure MegaMoE kernel sources into kernel_src + latest CuTeDSL MegaMoE kernels with improved performance - #3980

Merged

Anerudhan merged 28 commits into

flashinfer-ai:mainfrom

mhoqueanik:new_cutedsl_kernels

Jul 16, 2026

Commits on Jul 15, 2026

Refactor kernel_src/cutedsl_megamoe: relocate cutedsl_backend_kernels into canonical kernel_src layout

authored and
mhoqueanik
committed
f674ad4
Restructure cutedsl_megamoe into a shim/ layer behind the package __init__

authored and
mhoqueanik
committed
1d60dba
cutedsl_megamoe: enforce shim boundary and make src/ a verbatim drop zone

authored and
mhoqueanik
committed
6467fc2
cutedsl_megamoe: drop new kernel src verbatim + port shim to the new kernel API

authored and
mhoqueanik
committed
1144ee6
cutedsl_megamoe/tuner: default token-count knob heuristic in the kernel call

authored and
mhoqueanik
committed
34053e6
tests/moe_ep: add large-token (>=2048) coverage for the tuner LARGE profile

authored and
mhoqueanik
committed
f54af21
moe_ep/cutedsl: knob tuning (config + online autotune) + launch-path overhead fixes

mhoqueanik
and
claude
committed
d50eff5
cutedsl_megamoe: validated-once fast launch path + async fused-launch wrappers

mhoqueanik
and
claude
committed
b4b07eb
cutedsl_megamoe/tuner: fix the NVFP4 mid-range dip with two measured profiles

mhoqueanik
and
claude
committed
6215d31
cutedsl_megamoe: TUNING.md -- tuning surface, measured profiles, bench methodology

mhoqueanik
and
claude
committed
d1c2a49
cutedsl_megamoe/shim: robust peer-offset resolution on nvshmem heap reuse

mhoqueanik
and
claude
committed
768d5bd
moe_ep/cutedsl: plumb in_kernel_fc2_reduce + combine_dtype (TRT-LLM import)

mhoqueanik
and
claude
committed
4d968f2
cutedsl_megamoe/autotune: session-aware candidates incl. the ikr axis

mhoqueanik
and
claude
committed
56e8bc8
tests/moe_ep: ikr + quantized-combine multirank coverage

mhoqueanik
and
claude
committed
7fc9468
cutedsl_megamoe: TUNING.md -- variants, measured sweep, runbook, knob flow

mhoqueanik
and
claude
committed
4dc01a7
cutedsl_megamoe: ACKNOWLEDGEMENT.md for the kernel authors

mhoqueanik
and
claude
committed
fcd7eca
cutedsl_megamoe: TUNING.md -- fix GitHub mermaid rendering + smoother intro

mhoqueanik
and
claude
committed
fbf8f0f
design doc updated
mhoqueanik
committed
8ed9f2d
cutedsl_megamoe: kernel-drop updates + lint-exempt upstream src/

mhoqueanik
and
claude
committed
d0ff4b6
moe_ep/cutedsl_megamoe: fix confirmed review findings in shim + backends

mhoqueanik
and
claude
committed
eae1384
moe_ep: fix nvfp4 mega weight preprocess (strides + norm_const) and split bf16 weight prep

mhoqueanik
and
claude
committed
327171e
tests/moe_ep: single-GPU torch-oracle correctness for every compute path

mhoqueanik
and
claude
committed
2d2cca9
cutedsl_megamoe: TUNING.md corrected-layout results (2026-07-15)

mhoqueanik
and
claude
committed
0a201a0
cutedsl_megamoe: credit the MegaMoE kernel team by name in ACKNOWLEDGEMENT.md

mhoqueanik
and
claude
committed
205b182
cutedsl_megamoe: re-derive default knob profiles on the corrected weight layout

mhoqueanik
and
claude
committed
31e2d7a
cutedsl_megamoe: TUNING.md — expand headline sweep table to full variant breakdown

mhoqueanik
and
claude
committed
8930199
cutedsl_megamoe: TUNING.md — split sweep into variants table + backend summary

mhoqueanik
and
claude
committed
5f0fc8a
tests/moe_ep: ruff-format the oracle test files

mhoqueanik
and
claude
committed
e6c44e5

Navigation Menu

Uh oh!

feat(moe_ep): restructure MegaMoE kernel sources into kernel_src + latest CuTeDSL MegaMoE kernels with improved performance - #3980

feat(moe_ep): restructure MegaMoE kernel sources into kernel_src + latest CuTeDSL MegaMoE kernels with improved performance#3980
Anerudhan merged 28 commits into
flashinfer-ai:mainfrom
mhoqueanik:new_cutedsl_kernels

Commits on Jul 15, 2026

Refactor kernel_src/cutedsl_megamoe: relocate cutedsl_backend_kernels into canonical kernel_src layout

Restructure cutedsl_megamoe into a shim/ layer behind the package init

cutedsl_megamoe: enforce shim boundary and make src/ a verbatim drop zone

cutedsl_megamoe: drop new kernel src verbatim + port shim to the new kernel API

cutedsl_megamoe/tuner: default token-count knob heuristic in the kernel call

tests/moe_ep: add large-token (>=2048) coverage for the tuner LARGE profile

moe_ep/cutedsl: knob tuning (config + online autotune) + launch-path overhead fixes

cutedsl_megamoe: validated-once fast launch path + async fused-launch wrappers

cutedsl_megamoe/tuner: fix the NVFP4 mid-range dip with two measured profiles

cutedsl_megamoe: TUNING.md -- tuning surface, measured profiles, bench methodology

cutedsl_megamoe/shim: robust peer-offset resolution on nvshmem heap reuse

moe_ep/cutedsl: plumb in_kernel_fc2_reduce + combine_dtype (TRT-LLM import)

cutedsl_megamoe/autotune: session-aware candidates incl. the ikr axis

tests/moe_ep: ikr + quantized-combine multirank coverage

cutedsl_megamoe: TUNING.md -- variants, measured sweep, runbook, knob flow

cutedsl_megamoe: ACKNOWLEDGEMENT.md for the kernel authors

cutedsl_megamoe: TUNING.md -- fix GitHub mermaid rendering + smoother intro

design doc updated

cutedsl_megamoe: kernel-drop updates + lint-exempt upstream src/

moe_ep/cutedsl_megamoe: fix confirmed review findings in shim + backends

moe_ep: fix nvfp4 mega weight preprocess (strides + norm_const) and split bf16 weight prep

tests/moe_ep: single-GPU torch-oracle correctness for every compute path

cutedsl_megamoe: TUNING.md corrected-layout results (2026-07-15)

cutedsl_megamoe: credit the MegaMoE kernel team by name in ACKNOWLEDGEMENT.md

cutedsl_megamoe: re-derive default knob profiles on the corrected weight layout

cutedsl_megamoe: TUNING.md — expand headline sweep table to full variant breakdown

cutedsl_megamoe: TUNING.md — split sweep into variants table + backend summary

tests/moe_ep: ruff-format the oracle test files