[Fridge003]

MegaMoE gains optional post-expert RMS normalization before combining routed expert outputs, including an optional FP32 gain and weight-first normalization. This ports the MegaMoE changes from the Muse Spark DeepGEMM patch onto current dev.

• Add normalization support to the BF16 and FP8/FP4 paths, with workspace accumulation for fused partial sums.
• Include the NVFP4 scale-inverse lookup table, store-tile tuning and DG_MEGA_MOE_TILES override, normalization test mode, DG_SKIP_LEGACY option, and recipe fix.
• Adapt the original TVM-FFI changes to the current TORCH_LIBRARY wrappers and schemas, retaining the SiTU template parameters introduced after the patch's cbe1727 base.

Validation:

• Reconstructed the original patch on cbe1727 and checked the port for retained changes.
• git diff --check and Python syntax checks pass.
• Both operator schemas parse, and Python/C++ argument counts and ordering agree.
• The two adapted C++ bindings pass clang++ -std=c++20 -fsyntax-only against local PyTorch headers and the current MegaMoE API declarations. This checks the host bindings, not CUDA compilation.
• Rebuilt the wheel and JIT kernels at b1c6fb0 on 4 x GB300 (SM103, driver 580.178.04), PyTorch 2.13.0+cu130, CUDA 13.0, GCC 13.3, in an isolated environment.
• All 18 targeted GPU validation runs passed: post-expert norm on 1/4 ranks for BF16, FP8xFP4 and FP8xFP8; 4-rank PyTorch normalization/combine reference checks including NVFP4 and FP8 combine; zero/masked inputs, signed routing weights, epsilon, gain ordering, repeated calls and CUDA Graph replays; BF16 hidden=8192; NVFP4 alphas; BF16/NVFP4 TRT-LLM layouts; SiTU; and balanced/uneven/idle-rank global-token cases. The maximum normalization relative L2 error across 28 reference cases was 0.168%.
• The NVFP4 scale-inverse lookup matched the original reciprocal implementation bitwise on 1,048,576 float2 samples and all 256 table entries.
• Single-GPU memcheck and synccheck passed for BF16, FP8xFP4 and NVFP4, each with 0 reported errors. Unfiltered synccheck hit an internal nvjet launch error in the PyTorch reference matmul, also reproduced without DeepGEMM. The final synccheck runs use the repository's existing --kernel-name-exclude kns=nvjet setting; all DeepGEMM kernels remain instrumented.
• This was targeted kernel correctness validation, not a full release-suite or performance benchmark run.