[MXFP8] Add fqns filtering and a guarded backward_override to the grouped experts converter - #15
Draft
wolfcomos wants to merge 1 commit into
Draft
[MXFP8] Add fqns filtering and a guarded backward_override to the grouped experts converter#15wolfcomos wants to merge 1 commit into
wolfcomos wants to merge 1 commit into
Conversation
This was referenced Aug 30, 2026
…ed experts converter Two additions to MXFP8GroupedExpertsConverter: - fqns: like the Linear converters, only GroupedExperts.Config entries whose FQN contains a match are converted; an empty list keeps the previous convert-everything behavior. - backward_override: plumbs torchao's MXFP8 grouped-GEMM backward override (None/"quantized" keeps the quantized MXFP8 backward, "high_precision" uses the saved high-precision operands, "dequantized" uses the dequantized forward operands). The value is validated at converter build. The op config is only dataclasses.replace()d when the knob is actually set, and the replace is guarded by a field check with an actionable error when the installed torchao's MXFP8TrainingOpConfig predates backward_override -- stock recipes keep working on every released torchao, and only setting the new knob requires a torchao build with the matching MXFP8 backward_override support. Add CPU unit tests for the older-torchao guard, fqns filtering, and backward_override plumbing/validation. Co-Authored-By: Claude Fable 5 <noreply@anthropic.com>
wolfcomos
force-pushed
the
4over6/titan3-mx-override
branch
from
August 31, 2026 02:02
74661f3 to
054c4d4
Compare
This file contains hidden or bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
Sign up for free
to join this conversation on GitHub.
Already have an account?
Sign in to comment
Add this suggestion to a batch that can be applied as a single commit.This suggestion is invalid because no changes were made to the code.Suggestions cannot be applied while the pull request is closed.Suggestions cannot be applied while viewing a subset of changes.Only one suggestion per line can be applied in a batch.Add this suggestion to a batch that can be applied as a single commit.Applying suggestions on deleted lines is not supported.You must change the existing code in this line in order to create a valid suggestion.Outdated suggestions cannot be applied.This suggestion has been applied or marked resolved.Suggestions cannot be applied from pending reviews.Suggestions cannot be applied on multi-line comments.Suggestions cannot be applied while the pull request is queued to merge.Suggestion cannot be applied right now. Please check back later.
Independent PR (no stack). Companion torchao PR: wolfcomos/ao#10.
Summary
Two additions to
MXFP8GroupedExpertsConverter:fqns: like the Linear converters, onlyGroupedExperts.Configentries whose FQN contains a match are converted; an empty list keeps the previous convert-everything behavior.backward_override: plumbs torchao's MXFP8 grouped-GEMM backward override through the converter (None/"quantized"keeps the quantized MXFP8 backward,"high_precision"computes both gradients with plain grouped GEMMs on the saved high-precision operands,"dequantized"computes them from the dequantized forward operands). The value is validated at converter build.Design notes
dataclasses.replace()only runs when the knob is set, and is guarded: the op config isreplace()d withbackward_overrideonly whenconfig.backward_override is not None, behind a field-presence check that raises an actionableValueError("the installed torchao does not support backward_override for MXFP8 grouped GEMMs") when the installed torchao'sMXFP8TrainingOpConfigpredates the field. Stock MXFP8 recipes therefore keep working on every released torchao; only actually setting the new knob requires a torchao build with the matching field.backward_overridePR at runtime ([moe training] Add backward_override to the MXFP8 grouped GEMM ao#10), but this titan change merges safely ahead of any torchao pin bump because the default path never touches the field.fqnsfiltering follows the exact substring-match convention of the Linear converters and the NVFP4 four-over-six converters, so per-layer mixed-precision recipes (e.g. first/last decoder layers in bf16) work for MXFP8 grouped experts too.Test plan
CPU-only (container without
--gpus):New tests:
test_mxfp8_backward_override_guard_on_older_torchao(older-torchao guard: unset knob builds, set knob raises the actionable error),test_mxfp8_grouped_converter_fqns_filtering(targeted layer converts, others stay stock; empty fqns converts all), andtest_mxfp8_grouped_converter_backward_override_plumbs(invalid value rejected at build; valid value reaches every converted expert config). Import checks oftorchtitan.components.quantization{,.mx}pass. Lint: ufmt (black 22.12.0 / usort 1.0.5), flake8 with the repo plugin pins, pydoclint, and codespell all clean on the changed files.GPU results for the dequantized override come from the assembled stack tip on GB200, since CI has no SM100 runner for MXFP8 grouped GEMMs.
Draft on the fork to stage the upstream submission; supersedes the MXFP8 portions of the exploratory branch
Review pass (2026-08-30)
backward_overrideis typedLiteral["quantized", "high_precision", "dequantized"] | Noneon the converter Config (config-parse-time validation, matchingrecipe_name's idiom)."quantized"now normalizes to the stock quantized backward before the torchao field probe, so it works on torchao builds that predate the knob (tested against a simulated older torchao).__init__fail-fasts when an actual override is set but the installed torchao'sMXFP8TrainingOpConfiglacks the field; the guard message is capitalized and names the remediation.mxfp8.mddocuments the grouped converter'sfqnsandbackward_overrideoptions.