[Triton/Gluon] DSR1 GEMM tuning

[azaidy]

Perf tuning for DSR1 kernels on gfx1250

Kernel	Shape	Bucket	Speedup vs baseline
gemm_a16w16 (router, gluon)	N=256 K=7168	M_LEQ_16	4.1–4.8x
gemm_a16w16 (router, gluon)	N=256 K=7168	M_LEQ_64	3.5–4.2x
gemm_a16w16 (router, gluon)	N=256 K=7168	M_LEQ_512	2.3–2.5x
gemm_a16w16 (router, gluon)	N=256 K=7168	M_LEQ_2048	1.40–1.50x
gemm_a16w16 (router, gluon)	N=256 K=7168	any	0.99–1.72x
gemm_a8w8	N=2112 K=7168	M_LEQ_512	1.49x
gemm_a8w8	N=7168 K=16384	M_LEQ_512	1.26x
gemm_a8w8	N=7168 K=16384	M_LEQ_2048	1.10x
gemm_a8w8	N=24576 K=1536	M_LEQ_256	1.17x
gemm_a8w8	N=6144 K=1536	M_LEQ_1	1.39x
gemm_a8w8	N=6144 K=1536	M_LEQ_4	1.04x
gemm_a8w8	N=6144 K=1536	M_LEQ_64	1.12x
batched_gemm_a8w8 (B=128)	N=512 K=128	M_LEQ_2048	1.06x

[github-actions]

🏷️ CI Guide

Runs automatically on every PR:

• ✅ Pre-checks (submodule verification, code formatting)
• ✅ Aiter op tests (gfx942 + gfx950)
• ✅ Triton tests on MI35X (only when aiter/ops/triton/** or related paths are changed)

Extended tests (opt-in via labels):

Label	Tests
ci:gfx1250-ffm-triton	Run the five-shard gfx1250 FFM Triton test suite
ci:triton-300x	Run an additional Triton test job on MI300X in PRs; main branch always runs both MI35X and MI300X
multigpu	Aiter multi-GPU tests on the 8-GPU runner
ci:sglang	SGLang integration tests: DeepSeek-R1-MXFP4 accuracy, Qwen 3.5 accuracy
ci:atom	ATOM benchmark: DeepSeek-R1-0528, GPT-OSS-120B
ci:atom_full	ATOM accuracy suite for PR and main models from ATOM models_accuracy.json
ci:vllm	vLLM benchmark: GPT-OSS-120B, DeepSeek-R1-0528, Kimi-K2.5
ci:all	All standard extended tests (excludes ci:atom_full)

Only add ci:atom_full for FlyDSL or Triton upgrades.
Add labels via the sidebar or gh pr edit 5867 --add-label <label>

One backend per PR:
A PR changes one kernel backend: [Triton/Gluon] (Triton and Gluon count as one), [HIP], [ASM], [CK], [OPUS] or [FlyDSL]. If the title ends up with two backend tags, split the PR -- as stacked pull requests when one part cannot merge without the other.

PR title tags & labels:
Component tags ([Triton/Gluon], [HIP], [CK], [ASM], ...) are added to the PR title and as PR labels automatically from the changed files and re-synced on every push — change-type tags like [fix]/[Perf], op tags like [MLA], and human labels (ci:*) are left untouched. Add the no-auto-title label to stop the title rewrites; labels stay in sync either way.

[Copilot]

Copilot review overview

🟡 Changes recommended

Unresolved critical and moderate correctness, configuration, and dispatch issues remain.

Review effort: Lite
Findings: 2 · 2 · 2

Open (6)

• Add barrier before TDM descriptor creation · New
• Size B layout using w element width · New
• Define waves_per_eu in Gluon configs · New
• Require explicit opt-in for alternate backend · New
• Include waves_per_eu in kernel representation · New
• Move tuning candidates into Gluon config files · New

Resolved since last review (5)

• Tuned GEMM entry missing matrix_instr_nonkdim parameter
• Specialized entry missing matrix_instr_nonkdim parameter
• Specialized entry missing matrix_instr_nonkdim parameter
• Tuned entry omits required matrix_instr_nonkdim parameter
• M_LEQ_2048 entry missing required GEMM parameters

[Copilot]

Copilot review overview

🔵 Needs a closer look

Unresolved moderate correctness, dispatch, configuration, benchmark, and test issues remain.

Review effort: Lite
Findings: 4 · 2 · 2

Open (8)

• Avoid invalid descriptor construction for inactive pipeline loads
• Preserve A8W8 quantization semantics for non-default group sizes
• Size B layout using w element width
• Add barrier before TDM descriptor creation
• Require explicit opt-in for alternate backend
• Define waves_per_eu in Gluon configs
• Move tuning candidates into Gluon config files
• Include waves_per_eu in kernel representation

[Copilot]

Copilot review overview

🟡 Changes recommended

Unresolved moderate issues affect dispatch correctness, kernel safety, configuration validity, and test/config integration.

Review effort: Lite
Findings: 4 · 2 · 3

Open (9)

• Avoid invalid descriptor construction for inactive pipeline loads
• Preserve A8W8 quantization semantics for non-default group sizes
• Size B layout using w element width
• Add barrier before TDM descriptor creation
• Require explicit opt-in for alternate backend
• Define waves_per_eu in Gluon configs
• Document the wrapper's config argument · New
• Move tuning candidates into Gluon config files
• Include waves_per_eu in kernel representation

[Copilot]

Copilot review overview

🟡 Changes recommended

Unresolved kernel safety, configuration schema, API behavior, and numerical-semantics issues remain.

Review effort: Lite
Findings: 5 · 2 · 4

Open (11)

• Retune tiles to fit gfx1250 TDM-accessible LDS · New
• Avoid invalid descriptor construction for inactive pipeline loads
• Preserve A8W8 quantization semantics for non-default group sizes
• Size B layout using w element width
• Add barrier before TDM descriptor creation
• Require explicit opt-in for alternate backend
• Define waves_per_eu in Gluon configs
• Separate new kernel work from existing kernel retuning · New
• Document the wrapper's config argument
• Move tuning candidates into Gluon config files
• Include waves_per_eu in kernel representation

Previously missed (2)

In code that hasn't changed since last review

Retain matrix_instr_nonkdim in the modified GEMM entry

aiter/​ops/​triton/​configs/​gfx1250/​triton/​gemm/​gemm_a8w8/​GEMM-A8W8-N=7168-K=16384.json:103

The retuned Triton GEMM entry removes the required matrix_instr_nonkdim field. Keep this field (16, as in the neighboring entries) in every modified entry; otherwise the config no longer follows the GEMM schema and tuning validation will reject it.

Reject Gluon benchmarks with unsupported bias settings

op_tests/​op_benchmarks/​triton/​bench_batched_gemm_a8w8_a_per_token_group_prequant_w_per_batched_tensor_quant.py:209

The help text says Gluon requires --no-bias, but this guard only checks the architecture. Running the advertised command with --backend gluon and the default bias setting reaches batched_gemm_a16w8, which asserts on the bias and aborts instead of producing a benchmark. Reject this argument combination before launching (or automatically disable bias).

[Copilot]

Copilot review overview

🟡 Changes recommended

Unresolved moderate issues remain in kernel safety, configuration completeness, dispatch semantics, and model selection.

Review effort: Lite
Findings: 5 · 3 · 4

Open (12)

• Retune tiles to fit gfx1250 TDM-accessible LDS
• Avoid invalid descriptor construction for inactive pipeline loads
• Preserve A8W8 quantization semantics for non-default group sizes
• Size B layout using w element width
• Add barrier before TDM descriptor creation
• Restore DSV3 tuned Opus rows changed outside the DSR1 scope · New
• Require explicit opt-in for alternate backend
• Define waves_per_eu in Gluon configs
• Separate new kernel work from existing kernel retuning
• Document the wrapper's config argument
• Move tuning candidates into Gluon config files
• Include waves_per_eu in kernel representation

[Copilot]

Copilot review overview

🟡 Changes recommended

Unresolved correctness, configuration, scope, and tuning-record issues remain.

Review effort: Lite
Findings: 6 · 1 · 2

Open (9)

• Missing matrix_instr_nonkdim in Triton GEMM config · New
• Retune tiles to fit gfx1250 TDM-accessible LDS
• Avoid invalid descriptor construction for inactive pipeline loads
• Preserve A8W8 quantization semantics for non-default group sizes
• Size B layout using w element width
• Add barrier before TDM descriptor creation
• Require explicit opt-in for alternate backend
• Move tuning candidates into Gluon config files
• Include waves_per_eu in kernel representation

Resolved since last review (4)

• Restore DSV3 tuned Opus rows changed outside the DSR1 scope
• Define waves_per_eu in Gluon configs
• Separate new kernel work from existing kernel retuning
• Document the wrapper's config argument

[Copilot]

Copilot review overview

🟡 Changes recommended

Unresolved configuration, kernel-safety, API, benchmark, and dispatch issues remain.

Review effort: Lite
Findings: 1 · 1

Open (2)

• Preserve matrix_instr_nonkdim in M_LEQ_512 and M_LEQ_2048 · New
• Split unrelated DSV3 backend change or retain faster OPUS · New

Resolved since last review (9)

• Missing matrix_instr_nonkdim in Triton GEMM config
• Retune tiles to fit gfx1250 TDM-accessible LDS
• Avoid invalid descriptor construction for inactive pipeline loads
• Preserve A8W8 quantization semantics for non-default group sizes
• Size B layout using w element width
• Add barrier before TDM descriptor creation
• Require explicit opt-in for alternate backend
• Move tuning candidates into Gluon config files
• Include waves_per_eu in kernel representation