Skip to content

[Config] [Perf] Add gfx950 a8w8 bpreshuffle and BF16 GEMM tunings for Qwen3.8 Flash Next - #6769

Merged
zufayu merged 1 commit into
ROCm:mainfrom
amd-cgilli:qwen38-flash-next-gfx950-gemm-tunings
Oct 22, 2026
Merged

zufayu merged 1 commit into
ROCm:mainfrom
amd-cgilli:qwen38-flash-next-gfx950-gemm-tunings

Conversation

@amd-cgilli

Copy link
Copy Markdown
Contributor

Tuned on MI355X for amd/Qwen3.8-Flash-Next-Quark-MXFP4-AttnFP8-PLEFP8 at TP1, decode and prefill M values:

  • a8w8 bpreshuffle (PTPC-FP8 attention / GDN projections): +119 gfx950 rows.
  • BF16: +77 gfx950 rows (router incl. the 528-wide fused shared-expert router, lm_head 248320x2560, MTP and hyper-connection GEMMs). 5 existing gfx950 HC rows (10240x320 / 320x10240, M=32..4096) are replaced with faster solutions found by an exhaustive retune (e.g. 54.4 -> 41.3 us, 16.8 -> 13.6 us).
  • Untuned shape lists updated to match.

No gfx942 rows are touched; no existing row is removed.

…Qwen3.8-Flash-Next

Tuned on MI355X (gfx950, 256 CUs) for amd/Qwen3.8-Flash-Next-Quark-MXFP4-AttnFP8-PLEFP8
at TP1, decode and prefill M values:

- a8w8 bpreshuffle (PTPC-FP8 attention / GDN projections): +119 gfx950 rows.
- BF16: +77 gfx950 rows (router incl. the 528-wide fused shared-expert router,
  lm_head 248320x2560, MTP and hyper-connection GEMMs). 5 existing gfx950 HC rows
  (10240x320 / 320x10240, M=32..4096) are replaced with faster solutions found by
  an exhaustive retune (e.g. 54.4 -> 41.3 us, 16.8 -> 13.6 us).
- Untuned shape lists updated to match.

No gfx942 rows are touched; no existing row is removed.
@amd-cgilli
amd-cgilli requested review from a team, mjkvaak-amd and zovonoir October 21, 2026 10:19
@github-actions

Copy link
Copy Markdown
Contributor

🏷️ CI Guide

Runs automatically on every PR:

  • ✅ Pre-checks (submodule verification, code formatting)
  • ✅ Aiter op tests (gfx942 + gfx950)
  • ✅ Triton tests on MI35X (only when aiter/ops/triton/** or related paths are changed)

Extended tests (opt-in via labels):

Label Tests
ci:gfx1250-ffm-triton Run the five-shard gfx1250 FFM Triton test suite
ci:triton-300x Run an additional Triton test job on MI300X in PRs (added automatically when gfx942 configs change); main branch always runs both MI35X and MI300X
ci:triton-355 Run the full Triton test suite on MI35X, not only the tests the change affects
ci:windows Windows JIT op smoke test on the gfx1201 ROCm runner
ci:multi-gpu Aiter multi-GPU tests on the 8-GPU runner
ci:sglang SGLang integration tests: DeepSeek-R1-MXFP4 accuracy, Qwen 3.5 accuracy
ci:atom ATOM benchmark: DeepSeek-R1-0528, GPT-OSS-120B
ci:atom_full ATOM accuracy suite for PR and main models from ATOM models_accuracy.json
ci:vllm vLLM benchmark: DeepSeek-V4-Pro, MiniMax-M3-MXFP4
ci:all All standard extended tests (excludes ci:atom_full)

Only add ci:atom_full for FlyDSL or Triton upgrades.
Add labels via the sidebar or gh pr edit 6769 --add-label <label>

One backend per PR:
A PR changes one kernel backend: [Triton/Gluon] (Triton and Gluon count as one), [HIP], [ASM], [CK], [OPUS] or [FlyDSL]. If the title ends up with two backend tags, split the PR -- as stacked pull requests when one part cannot merge without the other.

PR title tags & labels:
Component tags ([Triton/Gluon], [HIP], [CK], [ASM], ...) are added to the PR title and as PR labels automatically from the changed files and re-synced on every push — change-type tags like [fix]/[Perf], op tags like [MLA], and human labels (ci:*) are left untouched. Add the no-auto-title label to stop the title rewrites; labels stay in sync either way.

@github-actions github-actions Bot changed the title [Config][Perf] Add gfx950 a8w8 bpreshuffle and BF16 GEMM tunings for Qwen3.8 Flash Next [Config] [Perf] Add gfx950 a8w8 bpreshuffle and BF16 GEMM tunings for Qwen3.8 Flash Next Oct 21, 2026
@amd-cgilli
amd-cgilli requested a review from coderfeli October 21, 2026 14:27
@zufayu
zufayu merged commit 8bdd00f into ROCm:main Oct 22, 2026
ZLkanyo009 added a commit that referenced this pull request Oct 26, 2026
#6769 tuned the same Qwen3.8-Flash-Next bf16 GEMMs on gfx950.  Resolve
qwen38_flash_next_bf16_{tuned,untuned}_gemm.csv as:

- keep main's rows by default, in place;
- add the 244 keys only this branch has;
- of the 40 overlapping keys whose rows differ, take this branch's row where
  an A/B re-measurement on MI355X (same tensors, the tuned_gemm.solMap
  backend, run_perftest 10 warmup / 101 iters, median of 9 alternating
  rounds) shows it more than 3% faster: 12 keys (N=640 M=8..128, N=2560
  K=2560 M=1/4/16/32/64, (128, 248320, 2560), (128, 13312, 2560));
- main's 33 gfx950 hipblaslt rows carry solution indices that do not exist
  in hipBLASLt 1.2.70204 (ROCm 7.2.4, the version aiter CI uses); hipb_mm
  exit()s on them, e.g. Qwen3.8 in ATOM dies at warmup on
  (16384, 10240, 2560) and the LM head (M<=64, N=248320) does the same.
  None of the kernel names map to a current solution.  For the 18 that
  overlap, take this branch's row (tuned on ROCm 7.2.4, verified to run and
  match torch); re-tune the other 15 with gemm_a16w16_tune.py restricted to
  hipblaslt (the library #6769 chose for them) on ROCm 7.2.4;
- untuned: main's list plus this branch's extra keys, plus the two tuned keys
  main's untuned list was missing, so every tuned key is listed.

The 6 MiniMax-M3 rows in minimax_m3_eagle_bf16_tuned_gemm.csv (N=2560,
K=6144) still share keys with this branch's rows on the new main, so they
are dropped again as in the original commit; the merged bf16_tuned_gemm
family has no duplicate shape.

Co-authored-by: Cursor <cursoragent@cursor.com>
ZLkanyo009 added a commit that referenced this pull request Oct 26, 2026
…le-tuning

#6769 tuned the same Qwen3.8-Flash-Next a8w8 bpreshuffle GEMMs on gfx950.
Resolve qwen38_flash_next_a8w8_bpreshuffle_{tuned,untuned}_gemm.csv as:

- keep every row from main as is;
- add the 10 keys only this branch has (N=2560, K=6144);
- of the 31 overlapping keys whose rows differ, take this branch's row only
  where an A/B re-measurement on MI355X (same tensors, aiter's own launcher,
  run_perftest 10 warmup / 101 iters, median of 9 alternating rounds) shows
  it more than 3% faster: (128, 13312, 2560) and (256, 16384, 2560);
- untuned: main's list plus this branch's extra keys, plus the two tuned keys
  main's untuned list was missing, so every tuned key is listed.

The 9 MiniMax-M3 rows in a8w8_bpreshuffle_tuned_gemm_minimax_m3.csv that
share keys with the N=2560, K=6144 rows are still duplicates on the new
main, so they are dropped again as in the original commit; the merged
a8w8_bpreshuffle_tuned_gemm family has no duplicate shape.

Co-authored-by: Cursor <cursoragent@cursor.com>
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

Projects

None yet

Development

Successfully merging this pull request may close these issues.

2 participants