feat(x86): AVX-512 low-precision float quantize lane — fp8/fp6/fp4 (S9) - #168
Merged
Merged
Conversation
…9, was 0/0)
PR-D of the campaign. Adds quantize/dequantize for fp8 (e4m3/e5m2), fp6
(e2m3/e3m2) and fp4 (e2m1) to x86 — 6 ops that were reference-only on both
devices. New x86_fpquant_compiled lane.
Kernel (avx512_fpquant_f32.cpp, new file) — format-agnostic float-grid snap,
parameterized by (max_normal, mantissa_bits):
ax=min(|x|,max_normal); ulp=2^(floor(log2 ax)−mantissa_bits);
out=sign·min(round_RNE(ax/ulp)·ulp, max_normal)
mapped to the AVX-512 getexp (floor log2) / roundscale (RNE) / scalef (2^n)
intrinsics — no polynomial. The runtime applies the per-tensor symmetric scale
(amax/max_normal, floored) around it; dequantize is the fp32 passthrough (the
forward already rescaled). Fake-quant in f32 storage.
Validation:
- Standalone C++ test_fpquant.cpp — ALL PASSED vs the scalar mantissa-snap across
fp4/fp6/fp8 grids (incl. the ax==0 path).
- tests/unit/test_x86_fpquant_compiled.py — 15 passed; the quantize path matches
tessera.ops.quantize_fp{8,6,4} EXACTLY (0 abs err — the AVX-512 RNE grid-snap
matches even the ml_dtypes fp8 cast), explicit-scale, dequant passthrough,
bad-format + unknown-op rejects.
Wiring: runtime `_X86_FPQUANT_OPS` + `_execute_x86_compiled_fpquant` + symbol
binding + executor table; execution_matrix catalog + row; backend_manifest
`_X86_KERNELS` (fused, 6 ops) + `_NUMERICAL_FIXTURES`; dashboards regenerated.
x86 lane count 93 → 99.
(nvfp4 uses the block-scaled microscaling path — a separate follow-up lane.)
Co-Authored-By: Claude Opus 4.8 <noreply@anthropic.com>
![chatgpt-codex-connector[bot]](https://avatars.githubusercontent.com/in/1144995?s=60&v=4){kind=small}
There was a problem hiding this comment.
💡 Codex Review
Here are some automated review suggestions for this pull request.
Reviewed commit: 7e8f5d4385
ℹ️ About Codex in GitHub
Your team has set up Codex to review pull requests in this repo. Reviews are triggered when you
- Open a pull request for review
- Mark a draft as ready
- Comment "@codex review".
If Codex has suggestions, it will comment; otherwise it will react with 👍.
Codex can also answer questions or update the PR. Try commenting "@codex address that feedback".
…eview) Two P2 review fixes on the fp-quant lane: 1. NaN propagation: the kernel turned NaN lanes into max_normal (min(abs(NaN), max) / std::fmin saturate), silently destroying NaN sentinels the public fp4/fp6 fallback preserves. The kernel now detects NaN (_CMP_UNORD_Q / std::isnan) and propagates it through both the vector and scalar paths. 2. fp8 vs ml_dtypes subnormals: the generic mantissa-snap diverged from the ml_dtypes float8 cast in the subnormal range (e.g. e4m3, scale=1, ~1e-3) — the prior test only hit the normal range. Added a per-format `min_exp` clamp so the subnormal grid is FLAT below the smallest normal (e4m3 −6, e5m2 −14), matching IEEE gradual underflow / ml_dtypes exactly; fp4/fp6 pass a very negative min_exp (no clamp) to keep their reference's pure mantissa-snap. Tests: + fp8 subnormal-range match vs ml_dtypes (scale=1, ~1e-3 inputs) + NaN-sentinel propagation; standalone test_fpquant.cpp now exercises NaN and the clamp. 17 passed. Co-Authored-By: Claude Opus 4.8 <noreply@anthropic.com>
Owner
Author
|
Thanks — both addressed in b76d9bb:
17 passed; ruff/mypy clean. |
gstoner
added a commit
that referenced
this pull request
Jun 28, 2026
…m loss/quant mirror (#172) Final ROCm-mirror PR for the loss/quant families — closes the cross-device pair for the S11 class-axis losses and the S9 quantize ops (x86 in #167/#168/#169). class-axis loss (rocm_class_loss_compiled, pure composition — no new C++): cross_entropy / kl / js / focal / label_smoothed_cross_entropy / z_loss; exp/log run on the rocm unary lane (gfx1151), class-axis max/sum/gather/one-hot on the host (same pattern as the x86 class-loss + ROCm attention/gemm-family lanes). fpquant + nvfp4 (new GenerateROCMFpQuantKernel.cpp): tessera_rocm.fpquant — flat 1-operand float-grid mantissa-snap (ax=min(|x|,max); e=max(floor(log2 ax), min_exp); ulp=2^(e-mant); roundeven; sign+NaN-propagate), log2/floor/exp2/ roundeven via math->ROCDL, parameterized for fp8 e4m3/e5m2, fp6 e2m3/e3m2, fp4 e2m1. rocm_fpquant_compiled (per-tensor scale) + rocm_nvfp4_compiled (per-block fp8-E4M3 scale + E2M1 codes, host block structure). Validation (gfx1151): - tessera-opt codegen + ROCDL lowering verified. - test_rocm_class_loss_compiled.py + test_rocm_fpquant_compiled.py — 15 passed vs tessera.losses/tessera.ops (class-loss 2e-4; fpquant/nvfp4 2e-3 — the GPU log2/roundeven grid matches the reference on random data). Wiring: runtime executors (class-loss / fpquant / nvfp4) + `_rocm_fpgrid` / `_rocm_unary_t` / `_rocm_log_softmax` helpers + executor table; execution_matrix catalog + 3 rows; backend_manifest _ROCM_COMPILED (6+6+2 ops) + _NUMERICAL_FIXTURES; dashboards regenerated. With this, loss (29) + rl_loss (4) + quantize (8) all run on BOTH gfx1151 and AVX-512 — three of the four "0/0" categories fully cross-device. Only spectral remains. Co-authored-by: gstoner <angstroms01@gmail.com> Co-authored-by: Claude Opus 4.8 <noreply@anthropic.com>
This file contains hidden or bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
Sign up for free
to join this conversation on GitHub.
Already have an account?
Sign in to comment
1 participant
Add this suggestion to a batch that can be applied as a single commit.This suggestion is invalid because no changes were made to the code.Suggestions cannot be applied while the pull request is closed.Suggestions cannot be applied while viewing a subset of changes.Only one suggestion per line can be applied in a batch.Add this suggestion to a batch that can be applied as a single commit.Applying suggestions on deleted lines is not supported.You must change the existing code in this line in order to create a valid suggestion.Outdated suggestions cannot be applied.This suggestion has been applied or marked resolved.Suggestions cannot be applied from pending reviews.Suggestions cannot be applied on multi-line comments.Suggestions cannot be applied while the pull request is queued to merge.Suggestion cannot be applied right now. Please check back later.
PR-D of the campaign. Adds quantize/dequantize for fp8 (e4m3/e5m2), fp6 (e2m3/e3m2), fp4 (e2m1) to x86 — 6 ops that were reference-only on both devices. New
x86_fpquant_compiledlane.Kernel (
avx512_fpquant_f32.cpp, new file)Format-agnostic float-grid snap, parameterized by
(max_normal, mantissa_bits):mapped to the AVX-512
getexp(floor log2) /roundscale(RNE) /scalef(2ⁿ) intrinsics — no polynomial. The runtime applies the per-tensor symmetric scale (amax/max_normal, floored); dequantize is the fp32 passthrough (the forward already rescaled). Fake-quant in f32 storage.Validation (real hardware)
test_fpquant.cpp— ALL PASSED vs the scalar mantissa-snap across fp4/fp6/fp8 grids (incl. ax==0).test_x86_fpquant_compiled.py— 15 passed; the quantize path matchestessera.ops.quantize_fp{8,6,4}EXACTLY (0 abs err — the AVX-512 RNE grid-snap matches even the ml_dtypes fp8 cast), explicit-scale, dequant passthrough, bad-format + unknown-op rejects.Wiring
runtime
_X86_FPQUANT_OPS+_execute_x86_compiled_fpquant+ symbol binding + executor table;execution_matrixcatalog + row;backend_manifest_X86_KERNELS(fused, 6 ops) +_NUMERICAL_FIXTURES; dashboards regenerated. x86 lane count 93 → 99.nvfp4uses the block-scaled microscaling path — a separate follow-up lane (PR-D2).🤖 Generated with Claude Code