perf: optimize MXFP4xBF16 & INT4xFP8 and add MXFP4xFP8 CUTLASS MoE backend for SM90 - #3738

Merged

jiahanc merged 30 commits into

flashinfer-ai:mainfrom

StudyingShao:jiangs/hopper_mixed_gemm

Jul 15, 2026

Commits on Jul 13, 2026

fix(moe): enable Hopper FP4 kernels without native CUDA FP4
StudyingShao
committed
b343133
perf(moe): optimize Hopper MoE W4A16 CUTLASS backend
StudyingShao
committed
15d7c99
perf(moe): optimize Hopper MoE W4A8 CUTLASS backend
StudyingShao
committed
1e70bdc
perf(moe): precompute Hopper grouped GEMM scheduling
StudyingShao
committed
34255b0
fix(moe): harden Hopper grouped GEMM precompute path
StudyingShao
committed
6700035
perf(moe): optimize Hopper MoE W4 grouped GEMM backend
StudyingShao
committed
a39f3dd
feat(moe): add Hopper MXFP4 FP8 pre-MMA scale path
StudyingShao
committed
b20f559
feat(moe): add Hopper MXFP4 FP8 runtime quantization path
StudyingShao
committed
f0c0d38
fix(moe): use 32-bit precomputed scheduler indices
StudyingShao
committed
93af1e7
feat(moe): enable Hopper MXFP4 FP8 full tactics
StudyingShao
committed
a8d52a7
fix(moe): validate Hopper mixed-input autotune
StudyingShao
committed
bce0189
style(moe): apply pre-commit formatting
StudyingShao
committed
05f5617
perf(moe): optimize Hopper MXFP4 FP8 pre-MMA small-K GEMM
StudyingShao
committed
2da88ae
docs(moe): attribute Humming preprocessing approach
StudyingShao
committed
420cd06
fix(moe): isolate SM90 MXFP4 FP8 scale modes
StudyingShao
committed
1360798
fix(moe): keep GEMM2 tactic IDs absolute
StudyingShao
committed
1ac2b00
fix(moe): correct mixed-input type and trace metadata
StudyingShao
committed
f9f2431
fix(moe): validate Humming preprocessing inputs
StudyingShao
committed
b425c84
fix(moe): validate SM90 MXFP4 inputs
StudyingShao
committed
4534599
fix(moe): validate SM90 W4 inputs and reference
StudyingShao
committed
faa5b87
fix(moe): align W4A8 shared prequant scales
StudyingShao
committed
4321923
fix(moe): handle empty batches and safe FP4 packing
StudyingShao
committed
c9d58c3
fix(moe): validate SM90 MXFP4 FP8 dispatch
StudyingShao
committed
48117b4
fix(moe): constrain Humming E8M0 scale range
StudyingShao
committed
cd60770
precommit
StudyingShao
committed
4ee51cd
fix(moe): restore SM100 MXFP4 runner instantiations
StudyingShao
committed
f129244
fix(moe): preserve MXFP8 profiler configuration
StudyingShao
committed
e7431af

Navigation Menu

perf: optimize MXFP4xBF16 & INT4xFP8 and add MXFP4xFP8 CUTLASS MoE backend for SM90 - #3738

perf: optimize MXFP4xBF16 & INT4xFP8 and add MXFP4xFP8 CUTLASS MoE backend for SM90#3738
jiahanc merged 30 commits into
flashinfer-ai:mainfrom
StudyingShao:jiangs/hopper_mixed_gemm

Commits on Jul 13, 2026

fix(moe): enable Hopper FP4 kernels without native CUDA FP4

perf(moe): optimize Hopper MoE W4A16 CUTLASS backend

perf(moe): optimize Hopper MoE W4A8 CUTLASS backend

perf(moe): precompute Hopper grouped GEMM scheduling

fix(moe): harden Hopper grouped GEMM precompute path

perf(moe): optimize Hopper MoE W4 grouped GEMM backend

feat(moe): add Hopper MXFP4 FP8 pre-MMA scale path

feat(moe): add Hopper MXFP4 FP8 runtime quantization path

fix(moe): use 32-bit precomputed scheduler indices

feat(moe): enable Hopper MXFP4 FP8 full tactics

fix(moe): validate Hopper mixed-input autotune

style(moe): apply pre-commit formatting

perf(moe): optimize Hopper MXFP4 FP8 pre-MMA small-K GEMM

docs(moe): attribute Humming preprocessing approach

fix(moe): isolate SM90 MXFP4 FP8 scale modes

fix(moe): keep GEMM2 tactic IDs absolute

fix(moe): correct mixed-input type and trace metadata

fix(moe): validate Humming preprocessing inputs

fix(moe): validate SM90 MXFP4 inputs

fix(moe): validate SM90 W4 inputs and reference

fix(moe): align W4A8 shared prequant scales

fix(moe): handle empty batches and safe FP4 packing

fix(moe): validate SM90 MXFP4 FP8 dispatch

fix(moe): constrain Humming E8M0 scale range

precommit

fix(moe): restore SM100 MXFP4 runner instantiations

fix(moe): preserve MXFP8 profiler configuration

Commits on Jul 14, 2026

fix(sm100): restore native FP4 type in MoE utilities

Merge branch 'main' into jiangs/hopper_mixed_gemm

Merge branch 'main' into jiangs/hopper_mixed_gemm