revert(moe): revert #3738 SM90 CUTLASS MoE backend (+ dependents #4025, #4080) on release-v0.6.17

[aleozlx]

📌 Description

Reverts #3738 (5823159c) and its two dependents on release-v0.6.17, to unblock the release after a critical crash reported by a vLLM user that escaped QA.

This is a release-unblocking revert, not a judgement on the feature. The proper fix is expected in 0.6.18. The same revert is being applied to release-v0.6.16 separately by another engineer — note #3738 shipped in v0.6.16 and v0.6.16.post2, so 0.6.17 is not the first release carrying it.

Why three commits

#3738 cannot be reverted alone — two later commits are built on it:

commit	PR	why it must go first
2475121f	#4080	fixes getProfilerWorkspaces in code #3738 reworked (the quant_1 && … && quant_6 assert)
c83607a9	#4025	exists only to re-export the interleave helpers that #3738 moved core → prepare
5823159c	#3738	the target

Reverted newest-first. #4080 and #4025 reverted with zero conflicts; #3738 conflicted in 3 files.

Conflict resolution — later work is preserved

This reverts #3738 only, not the features layered on top of it:

• core.py — kept profile_ids / workspace_buffer and the CUDA-device guard (feat(moe): support caller-owned CUTLASS MoE workspace #4057 and follow-ups); dropped only use_wfp4afp8_humming and its plumbing.
• __init__.py — kept cutlass_fused_moe_workspace_size (feat(moe): support caller-owned CUTLASS MoE workspace #4057); restored the two interleave_moe_*_for_sm90_mixed_gemm exports, which move back to core once perf: optimize MXFP4xBF16 & INT4xFP8 and add MXFP4xFP8 CUTLASS MoE backend for SM90 #3738 is gone.
• prepare.py — dropped the SM90 Humming docstring and the then-unused functools / struct imports; kept the TrtllmBf16Config wording added after perf: optimize MXFP4xBF16 & INT4xFP8 and add MXFP4xFP8 CUTLASS MoE backend for SM90 #3738.

Net: 47 files, +1701 / -10557 — a near-exact mirror of #3738's +10546 / -1704, the delta being the later features deliberately kept.

✅ Verification

• No wfp4afp8_humming / Humming reference remains anywhere in flashinfer/, csrc/ or tests/.
• Every cutlass_kernels/ and cutlass_backend/ source matches the pre-perf: optimize MXFP4xBF16 & INT4xFP8 and add MXFP4xFP8 CUTLASS MoE backend for SM90 #3738 parent except two files, and both residuals are fully attributable to later commits that correctly survive:
  • flashinfer_cutlass_fused_moe_binding.cu → feat(moe): support caller-owned CUTLASS MoE workspace #4057 (caller-owned workspace; 16 workspace-API references retained)
  • one std::remove_reference_t<…> line in moe_gemm_tma_ws_launcher.inl → SM107 work (Adds SM107 support #4122 / SM 107 Reland + Merge Back from v0.6.16 Release Branch #4280)
• Every relative import in fused_moe/__init__.py resolves (AST-checked).
• py_compile passes on all touched Python files.
• pre-commit (clang-format, mypy, ruff check, ruff format) passes.

✅ GPU-validated — see the validation comment. A/B contrast on B300 / sm103 with the reporter's repro, same node / container / install, only the commit differing:

	commit	result
baseline	4e1206b3 (branch head, unreverted)	FAIL_IN_AUTOTUNE — dies at gemm1 tactic 1/21
revert	57d74695 (this PR)	PASS — gemm1 21/21, gemm2 21/21

The without autotune control passes on both builds, isolating the autotuned path.

Still not run: the MoE unit-test suites. CI remains the gate for regression coverage; this validates the specific reported crash only.

Reviewer notes

• Worth a targeted look at the core.py conflict resolutions, since that is where perf: optimize MXFP4xBF16 & INT4xFP8 and add MXFP4xFP8 CUTLASS MoE backend for SM90 #3738 and feat(moe): support caller-owned CUTLASS MoE workspace #4057 interleaved in the same parameter lists.
• If the crash turns out to be reproducible on v0.6.16 too, that confirms perf: optimize MXFP4xBF16 & INT4xFP8 and add MXFP4xFP8 CUTLASS MoE backend for SM90 #3738 as the cause rather than a 0.6.17-specific interaction.

🤖 Generated with Claude Code

[coderabbitai]

Important

Review skipped

Auto reviews are disabled on base/target branches other than the default branch.

Please check the settings in the CodeRabbit UI or the .coderabbit.yaml file in this repository. To trigger a single review, invoke the @coderabbitai review command.

⚙️ Run configuration

Configuration used: defaults

Review profile: CHILL

Plan: Pro Plus

Run ID: 00356ae2-8773-4328-a423-8b45bfa1cffb

You can disable this status message by setting the reviews.review_status to false in the CodeRabbit configuration file.

Use the checkbox below for a quick retry:

• 🔍 Trigger review

---

Thanks for using CodeRabbit! It's free for OSS, and your support helps us grow. If you like it, consider giving us a shout-out.

❤️ Share

• X
• Mastodon
• Reddit
• LinkedIn

_{Comment @coderabbitai help to get the list of available commands.}

[aleozlx]

✅ Silicon validation — the revert fixes the reported crash

Validated by A/B contrast on B300 / sm103 (compute_cap 10.3), using the vLLM-free repro script from the reporting framework engineer (gpt-oss-20b shapes, TP=1, mxfp4 weights + mxfp8 activation, M=16384).

Same node, same container, same editable install — only the commit differs:

	commit	result
baseline	4e1206b3 (release-v0.6.17 head, unreverted)	FAIL_IN_AUTOTUNE
revert	57d74695 (this PR)	PASS

RESULT baseline FAIL_IN_AUTOTUNE rc=1 without_autotune=1 with_autotune=0
RESULT revert   PASS              rc=0 without_autotune=1 with_autotune=1

Baseline reproduces the reported signature exactly

Dies at the first tactic of trtllm::fused_moe::gemm1 (1/21 profiles), surfacing where the repro script predicts — at the victim, not the culprit, because the null TMA descriptor corrupts the CUDA context:

File "flashinfer/autotuner/initializers.py", line 69, in _rand_scaled
    return (torch.rand(shapes, device=device) * 10 - 5).to(dtype)
torch.AcceleratorError: CUDA error: an illegal instruction was encountered
[AutoTuner]: Tuning trtllm::fused_moe::gemm1:   5%|▍ | 1/21 [00:00<00:07, 2.71profile/s]

With the revert, autotuning completes

[AutoTuner]: Tuning trtllm::fused_moe::gemm1: 100%|██████████| 21/21 [00:12<00:00, 1.63profile/s]
[AutoTuner]: Tuning trtllm::fused_moe::gemm2: 100%|██████████| 21/21 [00:12<00:00, 1.62profile/s]
[Autotuner]: Autotuning process ends
with autotune: OK

Both GEMMs sweep all 21 tactics and the call returns. The without autotune control passes on both builds, matching the script's note that only the autotuned path fails — so the contrast isolates the autotune path rather than a general breakage.

Method notes

• Outcomes were classified explicitly (PASS / FAIL_IN_AUTOTUNE / FAIL_OTHER) rather than by exit code. That mattered: an earlier attempt returned FAIL_OTHER from a missing-CCCL-header JIT compile error (CUDA 13 moved cuda/std/* under include/cccl/), which an exit-code check would have mis-scored as the bug reproducing. Fixed with FLASHINFER_EXTRA_CFLAGS/CUDAFLAGS and re-run from scratch.
• Baseline chosen as 4e1206b3 (current branch head) rather than the rc4 tag, so the two runs differ by exactly this PR's three reverts.

This supersedes the "no GPU validation" caveat in the PR description, which I have updated.

Independent corroboration

Cross-checked against the parallel 0.6.16 revert (bkryu/revert-3738-release-v0.6.16): the reverted cutlass_kernels/ sources are byte-identical between the two branches, and neither retains any wfp4afp8_humming reference — two independently produced reverts of a 10k-line change agreeing exactly on the C++ half.

🤖 Generated with Claude Code

[aleozlx]

Cross-validation against the parallel 0.6.16 revert — closing summary

The equivalent revert for release-v0.6.16 was produced independently (bkryu/revert-3738-release-v0.6.16). We diffed the two branches against each other specifically to catch mistakes that a single author (or a single agent) would be blind to. Outcome:

Agreed — the C++ half is corroborated. The reverted cutlass_kernels/ sources are byte-identical between the two branches, and neither retains any wfp4afp8_humming / Humming reference anywhere in flashinfer/, csrc/ or tests/. Two independently produced reverts of a ~10k-line change converging exactly on the kernel state is the strongest signal available for the part that is impractical to eyeball.

Diverged — all in the Python layer, all now resolved or assigned:

finding	branch	status
core.py still imported the interleave_moe_* helpers from .prepare after #3738 was reverted, while prepare.py no longer defined them → ImportError on import flashinfer.fused_moe	0.6.16	fixed there by also reverting #4025 (which this PR had already done)
missing get_compute_capability import in tests/moe/test_trtllm_cutlass_fused_moe.py	0.6.16	fixed there; not applicable here — this branch already imports it
#4057 (cutlass_fused_moe_workspace_size / workspace_buffer=) reverted on 0.6.16, kept here	both	being aligned on the 0.6.16 side

On the last point, for the record of why this PR keeps #4057: it shipped in v0.6.16, v0.6.16.post1, v0.6.16.post2 and every 0.6.17 rc, and is exported in fused_moe.__all__, so removing it would drop a released public API. Its entire entanglement with #3738 was a single keyword argument (use_wfp4afp8_humming) that only ever selected the reverted Humming path, so stripping just that argument keeps the feature intact with no behavioural ambiguity.

Combined with the silicon result above (baseline FAIL_IN_AUTOTUNE → revert PASS on B300/sm103), this PR is considered ready from our side. Remaining gate is CI, which covers the regression surface this validation deliberately does not.

🤖 Generated with Claude Code