[Config] Modify kimik3_bf16_tuned_gemm.csv

[hmahmad26]

Summary

• Modify aiter/configs/model_configs/kimik3_bf16_tuned_gemm.csv

Adds BF16-tuned GEMMs for shapes collected using Conc 1 for the Kimi-K3 model.

How the shapes were collected

The shapes come from a Kimi-K3 concurrency-1 agentic run with AITER_TUNE_GEMM=1, which records every gemm_a16w16 shape the model executes. They were then tuned on gfx950 (cu_num=256), with the aiter build from the pinned vLLM ROCm image.

What this adds

1,464 tuned rows, all gfx950 / cu_num=256, by M band:

Band	Count
decode M<=64	572
M 65-512	77
M 513-4096	557
M>4096	258

Backends: flydsl 902, asm 379, torch 146, triton 26, opus 11.

Rows already in the file are kept, including all gfx1250 entries. 23 gfx950 shapes existed in both; those take the newly tuned kernel.

Validation

We timed each BF16 GEMM by itself. The tuner builds random A and W tensors and calls gemm_a16w16 (gemm_a16w16_tune.py --run_config). This is kernel time only. It is not a full e2e model run.

Default is the kernel aiter already picks from the shipped model_configs tables. For small token counts it often pads M (for example 9 → 16). Tuned is the kernel from this PR's CSV. The last column is how much faster tuned is: (default − tuned) / default.

The microbenchmark results are shown below. The MLA / KDA / MoE Shared / DSpark breakdown is a granular analysis for decode GEMMs only.

• MLA / KDA / MoE shared: the BF16 GEMMs in one target decoder layer of that type (projections and the shared expert). Same ops at every repeating layer.
• DSpark: the draft model's BF16 GEMMs, counted once (input concat, in-proj, q_b, o_proj, MLP, lm_head, Markov head). The draft has 5 layers; this row is not 5× that cost.
• Total: sum of the four decode rows above.
• Prefill GEMMs: timed at prefill M. Not included in Total.

	Dtype: BF16
	Default µs	Tuned µs	vs default
MLA GEMMs	28.89	27.40	+5.2%
KDA GEMMs	26.19	25.12	+4.1%
MoE Shared GEMMs	14.57	14.18	+2.7%
DSpark GEMMs	223.84	213.97	+4.4%
Total	293.49	280.67	+4.4%
Prefill GEMMs	206.13	201.77	+2.1%

[github-actions]

🏷️ CI Guide

Runs automatically on every PR:

• ✅ Pre-checks (submodule verification, code formatting)
• ✅ Aiter op tests (gfx942 + gfx950)
• ✅ Triton tests on MI35X (only when aiter/ops/triton/** or related paths are changed)

Extended tests (opt-in via labels):

Label	Tests
ci:gfx1250-ffm-triton	Run the five-shard gfx1250 FFM Triton test suite
ci:triton-300x	Run an additional Triton test job on MI300X in PRs; main branch always runs both MI35X and MI300X
multigpu	Aiter multi-GPU tests on the 8-GPU runner
ci:sglang	SGLang integration tests: DeepSeek-R1-MXFP4 accuracy, Qwen 3.5 accuracy
ci:atom	ATOM benchmark: DeepSeek-R1-0528, GPT-OSS-120B
ci:atom_full	ATOM accuracy suite for PR and main models from ATOM models_accuracy.json
ci:vllm	vLLM benchmark: GPT-OSS-120B, DeepSeek-R1-0528, Kimi-K2.5
ci:all	All standard extended tests (excludes ci:atom_full)

Only add ci:atom_full for FlyDSL or Triton upgrades.
Add labels via the sidebar or gh pr edit 5353 --add-label <label>

PR title tags & labels:
Component tags ([Triton/Gluon], [HIP], [CK], [ASM], ...) are added to the PR title and as PR labels automatically from the changed files and re-synced on every push — change-type tags like [fix]/[Perf], op tags like [MLA], and human labels (ci:*) are left untouched. Add the no-auto-title label to opt this PR out.

[yifehuan]

LGTM

[XiaobingSuper]

@hmahmad26 I have reverted this PR because the tuned config has the older flydsl kernel name(flydsl_gemm*) which can't be passed(will fallback the native path)