Skip to content

[Config] Tune a8w8 bpreshuffle GEMMs - #6547

Merged
demonsan merged 2 commits into
mainfrom
farlukas/a8w8-bpreshuffle-tuning
Oct 15, 2026
Merged

demonsan merged 2 commits into
mainfrom
farlukas/a8w8-bpreshuffle-tuning

Conversation

@farlukas

@farlukas farlukas commented Oct 14, 2026 •

Copy link
Copy Markdown
Collaborator

Motivation

Tune the input projection GEMM kernels for DSR1 FP4 TP8 for gfx950.

Test Result

M Before (µs) After (µs) Speedup
8 11.1 6.7 1.66x
16 11.4 6.8 1.68x
32 17.5 7.4 2.36x
64 10.6 9.1 1.16x
128 12.7 11.2 1.13x
256 19.1 14.9 1.28x
512 22.6 19.5 1.16x
2048 46.1 39.6 1.16x

Submission Checklist

@github-actions

Copy link
Copy Markdown
Contributor

🏷️ CI Guide

Runs automatically on every PR:

  • ✅ Pre-checks (submodule verification, code formatting)
  • ✅ Aiter op tests (gfx942 + gfx950)
  • ✅ Triton tests on MI35X (only when aiter/ops/triton/** or related paths are changed)

Extended tests (opt-in via labels):

Label Tests
ci:gfx1250-ffm-triton Run the five-shard gfx1250 FFM Triton test suite
ci:triton-300x Run an additional Triton test job on MI300X in PRs (added automatically when gfx942 configs change); main branch always runs both MI35X and MI300X
ci:triton-355 Run the full Triton test suite on MI35X, not only the tests the change affects
ci:windows Windows JIT op smoke test on the gfx1201 ROCm runner
ci:multi-gpu Aiter multi-GPU tests on the 8-GPU runner
ci:sglang SGLang integration tests: DeepSeek-R1-MXFP4 accuracy, Qwen 3.5 accuracy
ci:atom ATOM benchmark: DeepSeek-R1-0528, GPT-OSS-120B
ci:atom_full ATOM accuracy suite for PR and main models from ATOM models_accuracy.json
ci:vllm vLLM benchmark: DeepSeek-V4-Pro, MiniMax-M3-MXFP4
ci:all All standard extended tests (excludes ci:atom_full)

Only add ci:atom_full for FlyDSL or Triton upgrades.
Add labels via the sidebar or gh pr edit 6547 --add-label <label>

One backend per PR:
A PR changes one kernel backend: [Triton/Gluon] (Triton and Gluon count as one), [HIP], [ASM], [CK], [OPUS] or [FlyDSL]. If the title ends up with two backend tags, split the PR -- as stacked pull requests when one part cannot merge without the other.

PR title tags & labels:
Component tags ([Triton/Gluon], [HIP], [CK], [ASM], ...) are added to the PR title and as PR labels automatically from the changed files and re-synced on every push — change-type tags like [fix]/[Perf], op tags like [MLA], and human labels (ci:*) are left untouched. Add the no-auto-title label to stop the title rewrites; labels stay in sync either way.

@farlukas
farlukas marked this pull request as ready for review October 14, 2026 19:48
@farlukas
farlukas requested a review from a team October 14, 2026 19:48
@github-actions github-actions Bot changed the title Tune a8w8 bpreshuffle GEMMs [Config] Tune a8w8 bpreshuffle GEMMs Oct 14, 2026
@farlukas
farlukas force-pushed the farlukas/a8w8-bpreshuffle-tuning branch from 4850da9 to 6562786 Compare October 14, 2026 20:01
@zufayu
zufayu requested a review from demonsan October 15, 2026 02:10

@demonsan demonsan left a comment

Copy link
Copy Markdown
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

LGTM

@demonsan
demonsan merged commit 3a80ec0 into main Oct 15, 2026
@demonsan
demonsan deleted the farlukas/a8w8-bpreshuffle-tuning branch October 15, 2026 07:09
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

Projects

None yet

Development

Successfully merging this pull request may close these issues.

2 participants