[Triton/Gluon] Satya/dsr1 tuning oct4th by Boss2002n · Pull Request #6157 · ROCm/aiter

Boss2002n · 2026-10-05T18:29:37Z

Motivation

Technical Details

Test Plan

Test Result

Submission Checklist

Look over the contributing guidelines at https://github.com/ROCm/TheRock/blob/main/GOVERNANCE.md#pull-requests.

…=7168) Configs from gfx1250-gluon-a16w16-dsr1-qkv-a-tuning (3e0460f), merged with main per bucket. Each bucket keeps the branch's config only where it beat main's in a paired CUDA-graph A/B on gfx1250 at the bucket's top M (M=1536 for M_LEQ_2048; 16384 for any); buckets where main was as fast or faster keep main's config. Branch config kept (main -> branch): M=1: 7.52 -> 6.53 us (1.154x, 18/18) M=4: 7.74 -> 6.68 us (1.156x, 18/18) M=8: 7.70 -> 6.96 us (1.104x, 18/18) M=16: 8.03 -> 7.16 us (1.125x, 18/18) M=64: 11.68 -> 9.69 us (1.200x, 18/18) M=1024: 47.04 -> 28.45 us (1.651x, 12/12) M=2048 (timed at M=1536): 48.15 -> 47.51 us (1.015x, 18/18) M=4096: 99.59 -> 76.15 us (1.305x, 12/12) M=8192: 165.91 -> 148.92 us (1.114x, 12/12) M=16384: 318.51 -> 253.93 us (1.251x, 12/12) Main's config kept: M=32: 7.95 -> 8.95 us (0.887x, 0/18) M=128: 11.88 -> 14.06 us (0.848x, 0/18) M=256: 14.79 -> 17.56 us (0.843x, 0/18) M=512: 21.30 -> 27.13 us (0.787x, 0/12)

…6, K=1536) Configs from gfx1250-dsr1-qb-mxfp4-tuning (ae925ef), merged with main per bucket. Every bucket sets num_ctas, so this shape no longer raises KeyError: 'num_ctas' in main's multicast wrapper; main's configs were timed with num_ctas 1. Each bucket keeps the branch's config only where it beat main's in a paired CUDA-graph A/B on gfx1250 at the bucket's top M (M=1536 for M_LEQ_2048; 16384 for any); buckets where main was as fast or faster keep main's config. Branch config kept (main -> branch): M=128: 8.11 -> 5.61 us (1.447x, 18/18) M=256: 7.32 -> 7.13 us (1.029x, 18/18) M=512: 10.75 -> 10.24 us (1.049x, 12/12) M=1024: 18.36 -> 15.87 us (1.150x, 12/12) M=2048 (timed at M=1536): 25.34 -> 22.45 us (1.125x, 18/18) M=4096: 55.59 -> 46.30 us (1.198x, 12/12) M=8192: 104.82 -> 87.44 us (1.200x, 12/12) Main's config kept: M=16: 4.49 -> 4.69 us (0.952x, 1/18) M=64: 5.12 -> 5.02 us (1.006x, 12/18)

Pass B to get_gemm_config so BATCHED_GEMM-A16W8-B=<B>-N=<N>-K=<K>.json is read before the N/K file, as batched_gemm_bf16 already does. Without it the DSR1 K-/V-absorb B=128 configs are never used.

…(B=128, N=512, K=128) Configs from dsr1-gfx1250-kabsorb-bmm-a16w8-tuning (e156200), merged with main per bucket. Timed in ATOM's layout: X is the q_nope view of the (M, 128, 192) q buffer, output (M, 128, 512) with transpose_bm. Read only with the wrapper passing B (previous commit). Each bucket keeps the branch's config only where it beat main's in a paired CUDA-graph A/B on gfx1250 at the bucket's top M (M=1536 for M_LEQ_2048; 16384 for any); buckets where main was as fast or faster keep main's config. Branch config kept (main -> branch): M=1: 4.21 -> 4.08 us (1.033x, 14/18) M=256: 8.42 -> 8.21 us (1.025x, 16/18) M=4096: 81.61 -> 74.52 us (1.096x, 12/12) M=8192: 157.97 -> 138.52 us (1.139x, 12/12) Main's config kept: M=4: 3.64 -> 4.31 us (0.834x, 3/18) M=8: 4.11 -> 4.12 us (1.000x, 10/18) M=16: 4.32 -> 4.31 us (1.000x, 9/18) M=32: 4.38 -> 4.28 us (1.026x, 13/18) M=128: 5.83 -> 5.88 us (0.994x, 6/18) M=512: 13.39 -> 13.65 us (0.978x, 0/12) M=1024: 21.85 -> 23.05 us (0.949x, 0/12) M=2048 (timed at M=1536): 29.85 -> 33.24 us (0.899x, 0/18) M=16384: 313.60 -> 500.49 us (0.626x, 0/12)

…(B=128, N=128, K=512) Configs from dsr1-gfx1250-a16w8-v-absorb-tuning (ad0b582), merged with main per bucket. Timed in ATOM's layout: X is the (M, 128, 512) attention output viewed as (128, M, 512), output (M, 128, 128) with transpose_bm. DEFAULT.json gains waves_per_eu 1 so the tuner can sweep it; the wrapper already defaults it to 1. Each bucket keeps the branch's config only where it beat main's in a paired CUDA-graph A/B on gfx1250 at the bucket's top M (M=1536 for M_LEQ_2048; 16384 for any); buckets where main was as fast or faster keep main's config. Branch config kept (main -> branch): M=4: 3.96 -> 3.59 us (1.101x, 18/18) M=16: 4.39 -> 4.19 us (1.046x, 18/18) M=32: 4.54 -> 4.47 us (1.019x, 14/18) M=512: 10.12 -> 9.90 us (1.021x, 12/12) M=8192: 108.11 -> 105.79 us (1.020x, 10/12) Main's config kept: M=1: 4.18 -> 4.37 us (0.960x, 4/18) M=8: 4.22 -> 4.41 us (0.971x, 6/18) M=64: 4.68 -> 4.73 us (0.990x, 1/18) M=128: 5.58 -> 5.59 us (1.001x, 11/18) M=256: 7.28 -> 7.28 us (0.997x, 7/18) M=1024: 17.33 -> 18.44 us (0.943x, 0/12) M=2048 (timed at M=1536): 24.07 -> 25.26 us (0.954x, 0/18) M=4096: 55.71 -> 57.74 us (0.969x, 0/12) M=16384: 214.82 -> 354.78 us (0.605x, 0/12)

… K=7168) Configs from gfx1250-dsr1-gate-tune-register-prune (54a209c), merged with main per bucket. Each bucket keeps the branch's config only where it beat main's in a paired CUDA-graph A/B on gfx1250 at the bucket's top M (M=1536 for M_LEQ_2048; 16384 for any); buckets where main was as fast or faster keep main's config. Branch config kept (main -> branch): M=8192: 33.12 -> 27.55 us (1.203x, 12/12) Main's config kept: M=512: 7.85 -> 9.01 us (0.874x, 0/12) M=1024: 9.69 -> 9.74 us (0.993x, 1/12) M=2048 (timed at M=1536): 12.17 -> 14.68 us (0.828x, 0/18) M=16384: 74.15 -> 102.56 us (0.724x, 0/12)

…tas 1 The multicast wrapper (#6118) reads config["num_ctas"], and this file predates it, so every o_proj call raised KeyError: 'num_ctas'. num_ctas 1 keeps each bucket as it was tuned (one CTA, no cluster).

… (N=7168, K=16384) at M=1536 M_LEQ_2048: 256x256x256 w4 nb3 -> 512x1024x256 w4 nb4 num_ctas 8 (multicast, 256x256 per CTA). Swept all 7,216 configs of the search space at M=1536 on 4 gfx1250 GPUs; the final round on one GPU timed 46.07 -> 41.83 us, and a paired A/B against main's config (num_ctas 1) gave 46.16 -> 42.04 us (1.098x, 18/18 pairs). Other buckets unchanged.

github-actions · 2026-10-05T18:30:10Z

🏷️ CI Guide

Runs automatically on every PR:

✅ Pre-checks (submodule verification, code formatting)
✅ Aiter op tests (gfx942 + gfx950)
✅ Triton tests on MI35X (only when aiter/ops/triton/** or related paths are changed)

Extended tests (opt-in via labels):

Label	Tests
`ci:gfx1250-ffm-triton`	Run the five-shard gfx1250 FFM Triton test suite
`ci:triton-300x`	Run an additional Triton test job on MI300X in PRs (added automatically when gfx942 configs change); main branch always runs both MI35X and MI300X
`ci:triton-355`	Run the full Triton test suite on MI35X, not only the tests the change affects
`multigpu`	Aiter multi-GPU tests on the 8-GPU runner
`ci:sglang`	SGLang integration tests: DeepSeek-R1-MXFP4 accuracy, Qwen 3.5 accuracy
`ci:atom`	ATOM benchmark: DeepSeek-R1-0528, GPT-OSS-120B
`ci:atom_full`	ATOM accuracy suite for PR and main models from ATOM `models_accuracy.json`
`ci:vllm`	vLLM benchmark: GPT-OSS-120B, DeepSeek-R1-0528, Kimi-K2.5
`ci:all`	All standard extended tests (excludes `ci:atom_full`)

Only add ci:atom_full for FlyDSL or Triton upgrades.
Add labels via the sidebar or gh pr edit 6157 --add-label <label>

One backend per PR:
A PR changes one kernel backend: [Triton/Gluon] (Triton and Gluon count as one), [HIP], [ASM], [CK], [OPUS] or [FlyDSL]. If the title ends up with two backend tags, split the PR -- as stacked pull requests when one part cannot merge without the other.

PR title tags & labels:
Component tags ([Triton/Gluon], [HIP], [CK], [ASM], ...) are added to the PR title and as PR labels automatically from the changed files and re-synced on every push — change-type tags like [fix]/[Perf], op tags like [MLA], and human labels (ci:*) are left untouched. Add the no-auto-title label to stop the title rewrites; labels stay in sync either way.

Refactor gemm configuration retrieval to remove unnecessary conditional check.

Copilot

Copilot review overview

🟡 Changes recommended

Independent retunes must be split, and several changed Gluon configurations lack active correctness coverage.

Review effort: Balanced
Findings: 4 · 1

Open (5)

Add Gluon correctness tests for N=2112, K=7168 tiers · New
Add A16W16 Gluon tests for M=4096 and M=8192 tiers · New
Enable Gluon coverage for the new 16-CTA AFP4 configuration · New
Enable Gluon coverage for the new 8-CTA clustered configuration · New
Split unrelated kernel retunes into separate follow-up PRs · New

What changed in this PR

Tunes gfx1250 Gluon GEMMs for DSR1 workloads and enables batch-specific A16W8 configuration lookup.

Changes:

Adds batch-aware A16W8 config selection and specialized configurations.
Retunes AFP4 and A16W16 GEMM dispatch tiers.
Adds cluster and wave launch parameters.

File	Description
`aiter/ops/triton/gemm/batched/batched_gemm_a16w8.py`	Passes batch size into config lookup.
`.../gemm_afp4wfp4_preshuffled/GEMM-AFP4WFP4_PRESHUFFLED-N=7168-K=16384.json`	Adds clustered launch tuning.
`.../gemm_afp4wfp4_preshuffled/GEMM-AFP4WFP4_PRESHUFFLED-N=24576-K=1536.json`	Retunes tiles and clustered launches.
`.../gemm_a16w16/GEMM-A16W16-N=256-K=7168.json`	Adds large-M tuning tiers.
`.../gemm_a16w16/GEMM-A16W16-N=2112-K=7168.json`	Retunes small and large M ranges.
`.../batched_gemm_a16w8/DEFAULT.json`	Adds default wave occupancy settings.
`.../batched_gemm_a16w8/BATCHED_GEMM-A16W8-B=128-N=512-K=128.json`	Adds specialized W_UK tuning.
`.../batched_gemm_a16w8/BATCHED_GEMM-A16W8-B=128-N=128-K=512.json`	Adds specialized W_UV tuning.

💡 Add a code-review agent skill or configure MCP servers for context-aware, tailored reviews. Learn more in the docs.

Boss2002n added 8 commits October 5, 2026 18:23

batched_gemm_a16w8: look up the B-specific config first

ad7c9ea

Pass B to get_gemm_config so BATCHED_GEMM-A16W8-B=<B>-N=<N>-K=<K>.json is read before the N/K file, as batched_gemm_bf16 already does. Without it the DSR1 K-/V-absorb B=128 configs are never used.

[Config] gfx1250 MXFP4 preshuffle o_proj (N=7168, K=16384): add num_c…

9ba64b8

…tas 1 The multicast wrapper (#6118) reads config["num_ctas"], and this file predates it, so every o_proj call raised KeyError: 'num_ctas'. num_ctas 1 keeps each bucket as it was tuned (one CTA, no cluster).

vgokhale previously approved these changes Oct 5, 2026

View reviewed changes

Simplify gemm config retrieval in batched_gemm_a16w8.py

7b629fa

Refactor gemm configuration retrieval to remove unnecessary conditional check.

Boss2002n dismissed vgokhale’s stale review via 7b629fa October 5, 2026 21:00

Boss2002n marked this pull request as ready for review October 5, 2026 21:01

Boss2002n requested review from a team and vgokhale and a balanced review from Copilot October 5, 2026 21:01

Boss2002n self-assigned this Oct 5, 2026

Boss2002n changed the title ~~Satya/dsr1 tuning oct4th~~ dsr1 tuning oct4th Oct 5, 2026

github-actions Bot changed the title ~~dsr1 tuning oct4th~~ [Triton/Gluon] Satya/dsr1 tuning oct4th Oct 5, 2026

github-actions Bot added the Triton/Gluon label Oct 5, 2026

vgokhale approved these changes Oct 5, 2026

View reviewed changes

Copilot AI reviewed Oct 5, 2026

View reviewed changes

Copilot started reviewing on behalf of Boss2002n October 5, 2026 22:05

View session

zufayu requested review from a team and vgokhale October 6, 2026 00:32

Boss2002n removed the request for review from a team October 6, 2026 02:37

Boss2002n merged commit 6a5ac94 into main Oct 6, 2026

Boss2002n deleted the satya/dsr1_tuning_oct4th branch October 6, 2026 02:38

Boss2002n mentioned this pull request Oct 25, 2026

[Triton/Gluon] [Config] Use the M_LEQ_8192 config as "any" in the tuned gfx1250 Gluon GEMM files #6921

Merged

Navigation Menu

Repository navigation

[Triton/Gluon] Satya/dsr1 tuning oct4th - #6157

[Triton/Gluon] Satya/dsr1 tuning oct4th#6157
Boss2002n merged 9 commits into
mainfrom
satya/dsr1_tuning_oct4th

Boss2002n commented Oct 5, 2026

Uh oh!

github-actions Bot commented Oct 5, 2026

Uh oh!

Copilot AI left a comment

Uh oh!

Uh oh!

Uh oh!

Uh oh!

Uh oh!

Uh oh!

Reviewers

Uh oh!

Assignees

Labels

Projects

Milestone

Development

Uh oh!

3 participants

Conversation

Boss2002n commented Oct 5, 2026

Motivation

Technical Details

Test Plan

Test Result

Submission Checklist

Uh oh!

github-actions Bot commented Oct 5, 2026

🏷️ CI Guide

Uh oh!

Copilot AI left a comment

Choose a reason for hiding this comment

Copilot review overview

🟡 Changes recommended

Uh oh!

Uh oh!

Uh oh!

Uh oh!

Uh oh!

Uh oh!

Reviewers

Uh oh!

Assignees

Labels

Projects

Milestone

Development

Uh oh!

3 participants