[nvfp4 training][rl] Add the four-over-six NVFP4 dense training recipe - #12
Closed
wolfcomos wants to merge 1 commit into
Closed
[nvfp4 training][rl] Add the four-over-six NVFP4 dense training recipe#12wolfcomos wants to merge 1 commit into
wolfcomos wants to merge 1 commit into
Conversation
This was referenced Aug 30, 2026
wolfcomos
force-pushed
the
4over6/ao5-cutedsl-compat
branch
from
August 31, 2026 02:00
ab985fb to
20c78fe
Compare
wolfcomos
force-pushed
the
4over6/ao1-dense
branch
from
August 31, 2026 02:00
7bcd854 to
f114077
Compare
wolfcomos
force-pushed
the
4over6/ao1-dense
branch
4 times, most recently
from
August 31, 2026 06:35
e47975f to
8ac6288
Compare
Four-over-six is an adaptive NVFP4 block-scaling recipe: every quantization block is encoded twice — the standard map-to-6 encoding and a 1.5x-scale map-to-4 candidate whose denser FP4 grid lowers error for blocks with mass below the amax — and the lower-error candidate is stored. It is part of the NVFP4 RL training recipe roadmap (miles). This PR adds the recipe surface and a correctness-first reference implementation; optimized kernels land separately: - four_over_six_quantize: pure-PyTorch quantizer with per-tensor or row-scaled global scales and 1x16/16x16 blocks, every rounding step pinned so faster implementations can be validated bitwise against it. - nvfp4_dequantize: the standard NVFP4 decode (recipe-agnostic), a pure-PyTorch correctness helper — an optimized kernel is future work. - four_over_six_mm / four_over_six_linear: differentiable matmul with quantized, high_precision, and dequantized backward modes; the dequantized mode differentiates the quantized-forward function itself (the RL train/inference-consistency mode) while saving only 4-bit codes and scales. - NVFP4FourOverSixLinear: a stateless leaf module for the recipe, and recipe="default"|"four_over_six" on NVFP4TrainingConfig so quantize_() installs the right module. The existing NVFP4Linear is untouched: it owns RHT/SR state (sign-vector and seed buffers, TP machinery) that four-over-six deliberately has none of, so the recipes stay separate leaf modules behind one config. Co-Authored-By: Claude Fable 5 <noreply@anthropic.com>
wolfcomos
force-pushed
the
4over6/ao1-dense
branch
from
August 31, 2026 07:01
8ac6288 to
f2f7647
Compare
Owner
Author
|
Opened upstream as pytorch#4851 (same head). This fork draft stays as the staging copy. |
Owner
Author
|
Live upstream as pytorch#4851 — closing the staging draft. The branch stays: it is the upstream PR's head. |
This file contains hidden or bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
Sign up for free
to join this conversation on GitHub.
Already have an account?
Sign in to comment
Add this suggestion to a batch that can be applied as a single commit.This suggestion is invalid because no changes were made to the code.Suggestions cannot be applied while the pull request is closed.Suggestions cannot be applied while viewing a subset of changes.Only one suggestion per line can be applied in a batch.Add this suggestion to a batch that can be applied as a single commit.Applying suggestions on deleted lines is not supported.You must change the existing code in this line in order to create a valid suggestion.Outdated suggestions cannot be applied.This suggestion has been applied or marked resolved.Suggestions cannot be applied from pending reviews.Suggestions cannot be applied on multi-line comments.Suggestions cannot be applied while the pull request is queued to merge.Suggestion cannot be applied right now. Please check back later.
Summary
Adds the four-over-six NVFP4 training recipe (dense path): an adaptive block-scaling recipe where every 16-value quantization block is encoded twice — the standard map-to-6 encoding, and a 1.5x-scale map-to-4 candidate whose denser FP4 grid lowers error for blocks with mass below the amax — and the lower-error candidate is stored.
References: https://humansand.ai/blog/nvfp4-rl, radixark/miles#615.
The goal of this PR series is recipe support with a correctness-first reference implementation and minimal kernel prototypes; kernel optimization lands in future PRs.
What's added
four_over_six_quantize: pure-PyTorch quantizer (per-tensor or row-scaled global scales; 1x16 / 16x16 blocks), with every rounding step pinned so faster implementations can be validated bitwise against it.nvfp4_dequantize: the standard NVFP4 decode (recipe-agnostic — four-over-six only changes the encode side). A pure-PyTorch correctness helper, not an optimized kernel; a fused decode kernel is future work.four_over_six_mm/four_over_six_linear: differentiable matmul withquantized,high_precision, anddequantizedbackward modes.dequantizeddifferentiates the quantized-forward function itself from the saved 4-bit codes and scales — the RL train/inference-consistency mode.NVFP4FourOverSixLinearplusrecipe="default"|"four_over_six"onNVFP4TrainingConfig, soquantize_()installs the recipe's module. The existingNVFP4Linearis untouched: it owns RHT/SR state (sign-vector/seed buffers, TP machinery) that four-over-six deliberately has none of, so the recipes stay separate leaf modules behind one config, and each recipe's fields are rejected under the other at config build.Tests
quantizer scale/selection invariants, row-scaled semantics, dequantize round-trips and validation, linear forward/backward for all three backward modes, and config swap/validation — 41 passed / 1 parametrization skip on GB200 (B200-class required for the FP4 GEMMs; the quantizer itself is pure PyTorch).
Stack: this PR (base: main) ← #13 (CuTe DSL quantize kernel) ← #14 (grouped GEMM + dispatcher wiring).