[tlrmchlsmth]

Purpose

Extend the deep_gemm_mega_moe backend to DeepSeek-derived GLM model
implementations without changing compressed-tensors NVFP4 weights into a
different FP4 encoding.

This change:

• adds MegaMoE adapters for DeepSeek V2/V3.2 and GLM-5-next, including their
  MTP paths;
• supports unquantized BF16 and checkpoint-native MXFP4 expert weights;
• rejects compressed-tensors NVFP4 expert checkpoints instead of performing a
  lossy NVFP4-to-E2M1/UE8M0 conversion;
• preserves grouped routing, deferred-reduction, and sequence-parallel
  semantics;
• puts the reusable expert implementation in a shared module; DSv4 and Kimi K3
  model files and the existing DSv4 tests are unchanged;
• covers the adapters, checkpoint-format selection, and explicit NVFP4
  rejection with focused tests.

Duplicate-work check

This does not duplicate the open related work:

• [Kernels] Integration for DeepGEMM MegaMoE kernel #40833 introduces the general DeepGEMM MegaMoE kernel integration; it does
  not provide these GLM/DeepSeek model adapters.
• [Model][Experimental] DeepSeek V4 w4a4 MegaMoE support #42844 experiments with FP4 activation dispatch for DeepSeek V4 using an
  external DeepGEMM package; this PR uses the existing FP8-activation/FP4-weight
  MegaMoE path.
• [DSV4][SM90] add dsv4 fp8 weight use mega_moe backend #53527 targets FP8 DeepSeek V4 on SM90; this PR adds model integration for
  MegaMoE on SM100.

Test plan and results

Local checks after splitting the shared implementation from DSv4:

pre-commit run
# all hooks passed on the staged split

.venv/bin/python -m pytest tests/models/test_deep_gemm_mega_moe.py tests/models/test_deepseek_v4_mega_moe.py -q
# 11 passed, 29 skipped (CUDA/SM100 unavailable locally)

The CPU tests cover checkpoint-format selection, NVFP4 rejection, mocked
BF16/MXFP4 loading and finalization, expert mapping, and deferred-reduction
scaling. Non-compressed-tensors quantization configurations are not rejected
by the MXFP4 format predicate.

DSv4 migration is kept separately on
feature/dsv4-shared-megamoe,
stacked on this PR. No follow-up PR has been opened. Kimi K3 file changes are
omitted from both branches.

The following results predate this extraction; no new GPU or model-accuracy
evaluation was run for the split.

GB200 kernel validation performed earlier on the PR:

• Random FP8 x FP4 MegaMoE, four ranks: relative L2 0.0252-0.0276 and cosine
  similarity greater than 0.9996.
• Exact GLM topology (EP8 across two GB200 nodes, hidden size 6144,
  intermediate size 2048, 256 experts, top-k 8): all ranks produced finite
  outputs.

MXFP4 performance validation used RedHatAI/GLM-5.2-MXFP4xMXFP8 on two GB200
nodes with TP8/EP8. At concurrency 64, DeepGEMM MegaMoE reached 927.8 output
tokens/s, compared with 562.2 for TP8, 540.2 for EP8 with AGRS and CUTLASS
MXFP4, and 237.8 for EP8 with DeepEP-v2 and CUTLASS MXFP4. All 2,269 recorded
requests across the four runs completed successfully. These are single-run
client-side measurements, not repeated trials; Prometheus/DCGM series were not
exported.

No NVFP4 model evaluation is claimed: compressed-tensors NVFP4 is deliberately
unsupported by this PR because conversion into DeepGEMM's different FP4
representation showed unacceptable source-level error.

AI assistance

AI assistance was used to implement, debug, test, and draft this change.
Human review of the latest split and relevant GPU/model validation are pending.

[coderabbitai]

Important

Draft PR not reviewed

Draft PRs are not automatically reviewed by default.

• Trigger a manual review

To automatically review draft PRs, update your CodeRabbit configuration:

reviews:
  auto_review:
    drafts: true

---

Thanks for using CodeRabbit! It's free for OSS, and your support helps us grow. If you like it, consider giving us a shout-out.

❤️ Share

• X
• Mastodon
• Reddit
• LinkedIn

_{Comment @coderabbitai help to get the list of available commands.}

[mergify]

This pull request has merge conflicts that must be resolved before it can be
merged. Please rebase the PR, @tlrmchlsmth.

https://docs.github.com/en/pull-requests/collaborating-with-pull-requests/working-with-forks/syncing-a-fork

[github-actions]

✅ Triggered Buildkite CI #88564 for commit 80893b14f903.

[tlrmchlsmth]

/ci run

[github-actions]

✅ Triggered Buildkite CI #88670 for commit c4cab4255685.

[mergify]

This pull request has merge conflicts that must be resolved before it can be
merged. Please rebase the PR, @tlrmchlsmth.

https://docs.github.com/en/pull-requests/collaborating-with-pull-requests/working-with-forks/syncing-a-fork

[tlrmchlsmth]

/ci run

[github-actions]

✅ Triggered Buildkite CI #89108 for commit 79941a31a28e.

[mergify]

This pull request has merge conflicts that must be resolved before it can be
merged. Please rebase the PR, @tlrmchlsmth.

https://docs.github.com/en/pull-requests/collaborating-with-pull-requests/working-with-forks/syncing-a-fork