Repository navigation
[HIP] [FlyDSL] add mxfp8 w_scale 1*32 (scalea/b nonpreshuffle) - #6407
Merged
Merged
Conversation
solinzby1
requested review from
a team and
amd-ruitang3
and
a balanced review from Copilot
October 12, 2026 03:15
Contributor
🏷️ CI GuideRuns automatically on every PR:
Extended tests (opt-in via labels):
One backend per PR: PR title tags & labels: |
github-actions
Bot
changed the title
Contributor
There was a problem hiding this comment.
🔵 Needs a closer look
The low-level GPU and AOT changes have unresolved findings and no supplied hardware validation results.
3 open findings
What changed in this PR
Adds gfx950 support for MXFP8 GEMM using unshuffled row-major 1×32 scales.
Changes:
- Adds FlyDSL row-scale loading and dispatch.
- Adds gfx950 tuning configurations and tuner.
- Extends tests and AOT compilation support.
| File | Description |
|---|---|
op_tests/test_gemm_a8w8_mxfp8_bpreshuffle.py |
Extends correctness testing to gfx950. |
csrc/gemm_a8w8_mxfp8_bpreshuffle/gemm_a8w8_mxfp8_bpreshuffle_tune.py |
Adds the gfx950 tuner. |
aiter/ops/gemm_op_a8w8.py |
Dispatches gfx950 to FlyDSL. |
aiter/ops/flydsl/mxscale_preshuffle_kernels.py |
Adds row-scale host dispatch. |
aiter/ops/flydsl/kernels/gemm/mxscale_preshuffle.py |
Implements LDS-staged row-scale loading. |
aiter/ops/flydsl/gemm_tune/flydsl_gemm_mxscale_preshuffle_common.py |
Adds scale-layout-aware candidate filtering. |
aiter/ops/flydsl/gemm_a8w8_mxfp8.py |
Adds architecture-aware GEMM routing. |
aiter/configs/model_configs/dsv4_a8w8_mxfp8_bpreshuffle_tuned_gemm.csv |
Adds gfx950 tuned rows. |
aiter/aot/flydsl/gemm.py |
Adds AOT support for row-scale specializations. |
🧠 Review effort: Balanced
💡 Add a code-review agent skill or configure MCP servers for context-aware, tailored reviews. Learn more in the docs.
| batched_skipped = set() | ||
| # The mxfp8 1x32 table's gfx950 rows name mxpsh kernels that read the | ||
| # unshuffled per-1x32 scales; every other table's mxpsh rows are blockscale. | ||
| mxfp8_table = "a8w8_mxfp8_bpreshuffle" in os.path.basename(csv_path) |
| f"1x32 scales (needs K%128==0 and N a multiple of a tile_n)" | ||
| ) | ||
| kernel_name = ki.name | ||
| _warn_untuned(M, N, K, "fp8", "fp8", kernel_name) |
Contributor
There was a problem hiding this comment.
🟡 Changes recommended
Cross-XCD cache coherence, retained large workspaces, and gfx950 availability reporting need correction.
6 open findings
🧠 Review effort: Balanced
| M, | ||
| N, | ||
| num_tiles, | ||
| torch.float32 if fused_f32 else torch.bfloat16, |
Comment on lines
+2254
to
+2255
| gfx950 rows are FlyDSL mxpsh kernels (unshuffled 1x32 scales); an untuned | ||
| gfx950 shape runs the mxpsh heuristic tile. |
Contributor
There was a problem hiding this comment.
🟡 Changes recommended
The TDM predicate overwrites multicast masks, while the API default silently changes the gfx1250 scale contract and tuned rows show a substantial regression.
9 open findings
🧠 Review effort: Balanced
Comment on lines
+143
to
+145
| dgroup0=Vec.from_elements( | ||
| [fx.Int32(pred), g0[1], g0[2], g0[3]], fx.Int32 | ||
| ).ir_value(), |
| w_scale: Tensor, | ||
| dtype: torch.dtype = dtypes.bf16, | ||
| out: Tensor | None = None, | ||
| a_scale_preshuffled: bool = False, |
| gfx1250,256,256,7168,16384,flydsl,44,4,19.036,flydsl_mxfp8_32_bpreshuffle_compute_wmma_t128x256x128_mw2_nw2_nb3_sk4_cm4_cn4,3158.73,6782.20,0.0 | ||
| gfx1250,256,512,7168,16384,flydsl,38,2,27.797,flydsl_mxfp8_32_bpreshuffle_compute_wmma_t128x256x128_mw2_nw2_nb3_sk2_cm4_cn2_fsk,4326.39,4932.30,0.0 | ||
| gfx1250,256,1024,7168,16384,flydsl,-1,1,48.453,flydsl_mxfp8_32_bpreshuffle_compute_wmma_t128x256x128_mw2_nw2_nb3_sk1_cm4_cn4,4963.99,3159.63,0.0 | ||
| gfx1250,256,2048,7168,16384,flydsl,-1,1,87.731,flydsl_mxfp8_32_bpreshuffle_compute_wmma_t256x256x128_mw2_nw2_nb2_sk1_cm4_cn4,5483.06,2109.55,0.0 |
Contributor
There was a problem hiding this comment.
🟡 Changes recommended
The review found scale-layout compatibility and descriptor-bound correctness issues that can produce incorrect results.
10 open findings
🧠 Review effort: Balanced
valarLip
approved these changes
Oct 14, 2026
This was referenced Oct 16, 2026
This file contains hidden or bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
Sign up for free
to join this conversation on GitHub.
Already have an account?
Sign in to comment
Add this suggestion to a batch that can be applied as a single commit.This suggestion is invalid because no changes were made to the code.Suggestions cannot be applied while the pull request is closed.Suggestions cannot be applied while viewing a subset of changes.Only one suggestion per line can be applied in a batch.Add this suggestion to a batch that can be applied as a single commit.Applying suggestions on deleted lines is not supported.You must change the existing code in this line in order to create a valid suggestion.Outdated suggestions cannot be applied.This suggestion has been applied or marked resolved.Suggestions cannot be applied from pending reviews.Suggestions cannot be applied on multi-line comments.Suggestions cannot be applied while the pull request is queued to merge.Suggestion cannot be applied right now. Please check back later.
Motivation
Technical Details
Test Plan
Test Result
Submission Checklist