[nvfp4 training][rl] Add the four-over-six NVFP4 dense training recipe

[wolfcomos]

Summary

Adds the four-over-six NVFP4 training recipe (dense path): an adaptive block-scaling recipe where every 16-value quantization block is encoded twice — the standard map-to-6 encoding, and a 1.5x-scale map-to-4 candidate whose denser FP4 grid lowers error for blocks with mass below the amax — and the lower-error candidate is stored.

The goal of this PR series is recipe support with a correctness-first reference implementation and minimal kernel prototypes; kernel optimization lands in future PRs.

What's added

• four_over_six_quantize: pure-PyTorch quantizer (per-tensor or row-scaled global scales; 1x16 / 16x16 blocks), with every rounding step pinned so faster implementations can be validated bitwise against it.
• nvfp4_dequantize: the standard NVFP4 decode (recipe-agnostic — four-over-six only changes the encode side). A pure-PyTorch correctness helper, not an optimized kernel; a fused decode kernel is future work.
• four_over_six_mm / four_over_six_linear: differentiable matmul with quantized, high_precision, and dequantized backward modes. dequantized differentiates the quantized-forward function itself from the saved 4-bit codes and scales — the RL train/inference-consistency mode.
• NVFP4FourOverSixLinear plus recipe="default"|"four_over_six" on NVFP4TrainingConfig, so quantize_() installs the recipe's module. The existing NVFP4Linear is untouched: it owns RHT/SR state (sign-vector/seed buffers, TP machinery) that four-over-six deliberately has none of, so the recipes stay separate leaf modules behind one config, and each recipe's fields are rejected under the other at config build.

Tests
quantizer scale/selection invariants, row-scaled semantics, dequantize round-trips and validation, linear forward/backward for all three backward modes, and config swap/validation — 41 passed / 1 parametrization skip on GB200 (B200-class required for the FP4 GEMMs; the quantizer itself is pure PyTorch).

Reference
NVFP4 RL training recipe roadmap references from Ziang Li from humans&: https://humansand.ai/blog/nvfp4-rl, radixark/miles#615, NVIDIA/TransformerEngine#2972

Stack: this PR (base) ← #4852 (CuTe DSL quantize kernel) ← #4853 (grouped GEMM + dispatcher wiring).

[pytorch-bot]

🔗 Helpful Links

🧪 See artifacts and rendered test results at hud.pytorch.org/pr/pytorch/ao/4851

• 📄 Preview Python docs built from this PR

Note: Links to docs will display an error until the docs builds have been completed.

This comment was automatically generated by Dr. CI and updates every 15 minutes.