Skip to content

[nvfp4 training][rl] Add the four-over-six NVFP4 dense training recipe - #12

Closed
wolfcomos wants to merge 1 commit into
mainfrom
4over6/ao1-dense
Closed

[nvfp4 training][rl] Add the four-over-six NVFP4 dense training recipe#12
wolfcomos wants to merge 1 commit into
mainfrom
4over6/ao1-dense

Conversation

@wolfcomos

@wolfcomos wolfcomos commented Aug 30, 2026

Copy link
Copy Markdown
Owner

Summary

Adds the four-over-six NVFP4 training recipe (dense path): an adaptive block-scaling recipe where every 16-value quantization block is encoded twice — the standard map-to-6 encoding, and a 1.5x-scale map-to-4 candidate whose denser FP4 grid lowers error for blocks with mass below the amax — and the lower-error candidate is stored.

References: https://humansand.ai/blog/nvfp4-rl, radixark/miles#615.

The goal of this PR series is recipe support with a correctness-first reference implementation and minimal kernel prototypes; kernel optimization lands in future PRs.

What's added

  • four_over_six_quantize: pure-PyTorch quantizer (per-tensor or row-scaled global scales; 1x16 / 16x16 blocks), with every rounding step pinned so faster implementations can be validated bitwise against it.
  • nvfp4_dequantize: the standard NVFP4 decode (recipe-agnostic — four-over-six only changes the encode side). A pure-PyTorch correctness helper, not an optimized kernel; a fused decode kernel is future work.
  • four_over_six_mm / four_over_six_linear: differentiable matmul with quantized, high_precision, and dequantized backward modes. dequantized differentiates the quantized-forward function itself from the saved 4-bit codes and scales — the RL train/inference-consistency mode.
  • NVFP4FourOverSixLinear plus recipe="default"|"four_over_six" on NVFP4TrainingConfig, so quantize_() installs the recipe's module. The existing NVFP4Linear is untouched: it owns RHT/SR state (sign-vector/seed buffers, TP machinery) that four-over-six deliberately has none of, so the recipes stay separate leaf modules behind one config, and each recipe's fields are rejected under the other at config build.

Tests
quantizer scale/selection invariants, row-scaled semantics, dequantize round-trips and validation, linear forward/backward for all three backward modes, and config swap/validation — 41 passed / 1 parametrization skip on GB200 (B200-class required for the FP4 GEMMs; the quantizer itself is pure PyTorch).

Stack: this PR (base: main) ← #13 (CuTe DSL quantize kernel) ← #14 (grouped GEMM + dispatcher wiring).

@wolfcomos
wolfcomos force-pushed the 4over6/ao5-cutedsl-compat branch from ab985fb to 20c78fe Compare August 31, 2026 02:00
@wolfcomos
wolfcomos force-pushed the 4over6/ao1-dense branch 4 times, most recently from e47975f to 8ac6288 Compare August 31, 2026 06:35
@wolfcomos wolfcomos changed the title [NVFP4 4over6] Add the four-over-six NVFP4 training recipe (dense) [nvfp4 training][rl] Add the four-over-six NVFP4 dense training recipe Aug 31, 2026
@wolfcomos
wolfcomos changed the base branch from 4over6/ao5-cutedsl-compat to main August 31, 2026 06:35
Four-over-six is an adaptive NVFP4 block-scaling recipe: every
quantization block is encoded twice — the standard map-to-6 encoding
and a 1.5x-scale map-to-4 candidate whose denser FP4 grid lowers error
for blocks with mass below the amax — and the lower-error candidate is
stored. It is part of the NVFP4 RL training recipe roadmap (miles).

This PR adds the recipe surface and a correctness-first reference
implementation; optimized kernels land separately:

- four_over_six_quantize: pure-PyTorch quantizer with per-tensor or
  row-scaled global scales and 1x16/16x16 blocks, every rounding step
  pinned so faster implementations can be validated bitwise against it.
- nvfp4_dequantize: the standard NVFP4 decode (recipe-agnostic), a
  pure-PyTorch correctness helper — an optimized kernel is future work.
- four_over_six_mm / four_over_six_linear: differentiable matmul with
  quantized, high_precision, and dequantized backward modes; the
  dequantized mode differentiates the quantized-forward function itself
  (the RL train/inference-consistency mode) while saving only 4-bit
  codes and scales.
- NVFP4FourOverSixLinear: a stateless leaf module for the recipe, and
  recipe="default"|"four_over_six" on NVFP4TrainingConfig so
  quantize_() installs the right module. The existing NVFP4Linear is
  untouched: it owns RHT/SR state (sign-vector and seed buffers, TP
  machinery) that four-over-six deliberately has none of, so the
  recipes stay separate leaf modules behind one config.

Co-Authored-By: Claude Fable 5 <noreply@anthropic.com>
@wolfcomos

Copy link
Copy Markdown
Owner Author

Opened upstream as pytorch#4851 (same head). This fork draft stays as the staging copy.

@wolfcomos

Copy link
Copy Markdown
Owner Author

Live upstream as pytorch#4851 — closing the staging draft. The branch stays: it is the upstream PR's head.

@wolfcomos wolfcomos closed this Aug 31, 2026
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

None yet

Projects

None yet

Development

Successfully merging this pull request may close these issues.

1 participant