Repository navigation
[Config] [tune] DSv4 a8w8 blockscale: add gfx950 configs for three MI355X shapes - #4664
Merged
Merged
Conversation
Adds 45 tuned rows to dsv4_a8w8_blockscale_tuned_gemm.csv for gfx950, covering N,K = 6144x7168, 65536x1536 and 7168x3072 with an M sweep of 1..256 each. None of these shapes had an entry, so they previously fell back to an untuned default. Rows are merged into the existing DSv4 file rather than added as a new CSV: get_config_file() globs model_configs/*<name>*.csv and merges every match, so a separate file would be redundant and the naming would be inconsistent with dsv4_a8w8_blockscale_untuned_gemm.csv. Append-only: no existing rows are modified or removed. Signed-off-by: jiacao-amd <jiahui.cao@amd.com>
Contributor
🏷️ CI GuideRuns automatically on every PR:
Extended tests (opt-in via labels):
|
Adds 61 gfx950 rows for the (N=7168, K=7168) non-preshuffle blockscale
GEMM -- the most frequently hit untuned shape in DSv4 MI355X runs. It
appears in every DSv4 server log we sampled, across 61 distinct M values
(4 .. 65536), and every one of them was falling back to the heuristic
default kernel.
The bpreshuffle variant of this shape is already tuned (75 gfx950 rows in
dsv4_a8w8_blockscale_bpreshuffle_tuned_gemm.csv); only the non-preshuffle
path was missing.
Tuned on MI355X (gfx950, cu_num=256) with
csrc/ck_gemm_a8w8_blockscale/gemm_a8w8_blockscale_tune.py --mp 8
--shape_grouped. 61/61 shapes tuned successfully, errRatio 0.0 on all.
Measured with --run_config against the production operator (heuristic
default vs tuned), same harness for both sides:
M range speedup
4 - 32 1.21 - 1.23x
40 - 64 1.01 - 1.05x
72 - 128 1.46 - 1.73x
136 - 144 1.10x
152 - 256 1.48 - 1.96x
288 - 512 1.62 - 2.55x
544 - 1024 2.20 - 3.24x
8192 - 65536 5.06 - 5.59x
total 35998 us -> 8244 us = 4.37x (77.1% saved)
No shape regresses. Weighting each M by how often it appears in the
sampled logs gives the same picture: 4.40x.
All 10 selected kernel names (43 ck rows + 18 cktile rows) are already in
the compiled kernel registry of module_gemm_a8w8_blockscale.so /
module_gemm_a8w8_blockscale_cktile.so, so this is a pure config change --
no codegen and no JIT rebuild. Verified 0 key collisions against the
existing non-preshuffle tuned set.
Signed-off-by: jiacao-amd <jiahui.cao@amd.com>
jiacao-amd
marked this pull request as ready for review
September 10, 2026 21:59
coderfeli
approved these changes
Sep 11, 2026
zufayu
approved these changes
Sep 11, 2026
This was referenced Sep 12, 2026
Closed
This file contains hidden or bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
Sign up for free
to join this conversation on GitHub.
Already have an account?
Sign in to comment
Add this suggestion to a batch that can be applied as a single commit.This suggestion is invalid because no changes were made to the code.Suggestions cannot be applied while the pull request is closed.Suggestions cannot be applied while viewing a subset of changes.Only one suggestion per line can be applied in a batch.Add this suggestion to a batch that can be applied as a single commit.Applying suggestions on deleted lines is not supported.You must change the existing code in this line in order to create a valid suggestion.Outdated suggestions cannot be applied.This suggestion has been applied or marked resolved.Suggestions cannot be applied from pending reviews.Suggestions cannot be applied on multi-line comments.Suggestions cannot be applied while the pull request is queued to merge.Suggestion cannot be applied right now. Please check back later.
Summary
Adds 106 tuned rows to
dsv4_a8w8_blockscale_tuned_gemm.csvfor gfx950, covering four DeepSeek-V4 shape families:These shapes previously fell back to an untuned default.
The
M=8192, N=65536, K=1536entry uses the peer-provided CKTile config 11. On a local MI355X A/B run, it reduced latency from 1303.55 us to 947.08 us (27.35% lower, 1.376x speedup), witherrRatio=0.0.Why merge into the existing file
These rows are merged into the existing
dsv4_a8w8_blockscale_tuned_gemm.csv, becauseAITER_CONFIG.get_config_file()globsmodel_configs/*<tuned_file_name>*.csvand merges every match. A second file would be redundant at load time, anda8w8_..._dsv4would also be inconsistent with the existingdsv4_a8w8_blockscale_{tuned,untuned}_gemm.csvnaming.Details
main: 106 insertions, 0 deletions.(gfx, cu_num, M, N, K)keys or malformed rows.Test Plan
Correctness test passed on MI355X (gfx950, 256 CUs): 106/106 OK, 0 mismatches, 0 errors.
python3 csrc/ck_gemm_a8w8_blockscale/gemm_a8w8_blockscale_tune.py \ -i aiter/configs/a8w8_blockscale_untuned_gemm.csv \ --run_config .pr4664_added_shapes_plus_peer.csv \ --warmup 2 --iters 5