[NVFP4 4over6] Add the grouped-experts four-over-six converter

[wolfcomos]

Stack (emulated ghstack, oldest at bottom):

• -> [NVFP4 4over6] Add the grouped-experts four-over-six converter #14
• [NVFP4 4over6] Add the dense four-over-six linear converter #13

Depends on the torchao grouped GEMM dispatcher PR: wolfcomos/ao#14.

Summary

Adds NVFP4GroupedExpertsConverter with recipe="four_over_six", applying four-over-six NVFP4 quantization to MoE routed-expert grouped GEMMs. The NVFP4 RL recipes in the miles RL framework (https://github.com/radixark/miles) quantize only the routed-expert projections; with no Linear converter alongside, this converter alone reproduces that allow-list (attention, dense MLP, shared experts, router, embeddings, and the LM head all stay bf16).

Also included: nvfp4_bf16_first_last_fqns (the miles recipes' --first-last-layers-bf16 analog), the deepseek_v3_debugmodel_nvfp4_four_over_six{,_dequantized} debug recipes (high-precision-backward base point, and the dequantized-backward variant with first/last decoder layers kept bf16), has_quantization coverage of the four-over-six experts cache and linear config, and CPU unit tests.

Design notes

• recipe="four_over_six" is the only wired grouped recipe in torchao today: the config carries the same recipe selector as the dense NVFP4LinearConverter ([NVFP4 4over6] Add the dense four-over-six linear converter #13), but torchao wires no RHT/SR grouped GEMM into its dispatcher, so there is no "default" grouped path; any other value — including "default" — is rejected at converter build with a ValueError explaining exactly that.
• Mirrors MXFP8GroupedExpertsConverter exactly: a class factory over the _grouped_mm(self, *, A, B_t, offs) seam plus swap_token_dispatcher(pad_multiple). The quantized subclass builds an NVFP4FourOverSixTrainingOpConfig (config object) and calls torchao's shared _quantize_then_scaled_grouped_mm dispatcher — identically to how the MXFP8 subclass calls it with MXFP8TrainingOpConfig. The torchao dispatcher branch owns tail handling, so the earlier titan-side slice/pad glue and its per-forward int(offs[-1]) D2H sync were deleted outright; nothing here reads offsets on the host.
• pad_multiple validated as a multiple of 128 at converter build: the four-over-six grouped GEMM's swizzle contract requires 128-row-aligned token groups, and violating it produces silently wrong numerics rather than an error, so it is rejected at config time.
• model_compile_enabled rejected unconditionally at converter build: torchao's dispatcher branch host-reads offs[-1] in every configuration (the row-scaled loop adds more host reads), which fullgraph capture cannot handle; the copied "compile recommended for performance" warning was removed as it recommended an unsupported configuration.
• Both debug recipes set training.disable_cuda_graphs = True: the quantized grouped GEMM host-reads group offsets, which CUDA-graph capture forbids; the miles recipes likewise run with graphs off.
• Requires the companion torchao PR: the installed torchao build must include the ao-side dispatcher change adding NVFP4FourOverSixTrainingOpConfig and its _quantize_then_scaled_grouped_mm branch ([nvfp4 training][rl] Add the four-over-six grouped GEMM and dispatcher wiring ao#14). Without it the converter raises an actionable ImportError at build.

Numerics evidence

Numerics live in the companion torchao PRs; parity results are maintained in a local TE harness (not in-tree): pure-torch quantizer 384/384 bitwise vs TE 2.17 kernels; CuTe DSL kernel 1920/1920 bitwise vs the torch oracle plus 12/12 vs TE; re-verified 2026-08-24 vs TE main.

Test plan

CPU-only (container without --gpus; requires a torchao build providing NVFP4FourOverSixTrainingOpConfig on PYTHONPATH, otherwise the new tests skip):

python -m pytest tests/unit_tests/cpu/test_quantization.py -q
41 passed, 14 warnings in 11.61s

Import checks of every touched module pass (torchtitan.components.quantization{,.nvfp4,.utils}, torchtitan.models.deepseek_v3.config_registry). Lint: ufmt (black 22.12.0 / usort 1.0.5), flake8 with the repo plugin pins, pydoclint, and codespell all clean on the changed files.

GPU results come from the assembled stack tip on GB200 (RL-parity phases A-D2 plus a 6-arm Qwen3 recipe ablation), since CI has no SM100 runner for the four-over-six kernels.

Draft on the fork to stage the upstream submission; supersedes the exploratory stack (#8)

Review pass (2026-08-30)

• Compile rejection is now unconditional (was row-scaled-only), with the test updated accordingly; the misleading perf warning is gone.
• The dense-linear has_quantization coverage (utils.py hunk + test tail) moved down the stack into the dense PR; this PR keeps only the grouped-experts cache coverage.
• Grouped knob validation split into its own test (a torchao with only the dense prototype no longer skips the dense assertions) and now validates err_mode/e4m3_scale_bound/row_scaled through the shared helper.
• FourOverSixGroupedExperts.__init__ imports NVFP4FourOverSixTrainingOpConfig locally and raises an actionable ImportError for direct factory/config use (MXFP8-analog pattern).
• CPU suite re-run in the vLLM 26.08 container: 43 passed / 0 failed.