Repository navigation
a16w16 gemm tuned dsv4 pro shapes - #4222
Open
ahmed-bsod wants to merge 9 commits into
Open
ahmed-bsod wants to merge 9 commits into
ahmed-bsod wants to merge 9 commits into
Conversation
Contributor
🏷️ CI GuideRuns automatically on every PR:
Extended tests (opt-in via labels):
|
Contributor
There was a problem hiding this comment.
Pull request overview
This PR adds tuned configuration coverage for A16W16 GEMM shapes used by DSv4/DSv4pro on gfx1250 (via Triton/Gluon configs), and tightens the default “skinny” fallback so it only applies on specific CDNA-class gfx targets.
Changes:
- Restricts the skinny default-shape fallback in
get_GEMM_A16W16_config()togfx90a/gfx942/gfx950only. - Adds gfx1250 GEMM-A16W16 (N,K-specialized) JSON configs for K=7168 across several N values, keyed by M-bounds.
- Adds gfx1250 DSv4/DSv4pro bf16 tuned GEMM CSV entries that route these shapes to the Triton backend.
Reviewed changes
Copilot reviewed 9 out of 9 changed files in this pull request and generated 2 comments.
Show a summary per file
| File | Description |
|---|---|
| aiter/tuned_gemm.py | Restricts skinny fallback dispatch to specific gfx targets. |
| aiter/ops/triton/configs/gemm/gfx1250-GEMM-A16W16-N=64-K=7168.json | Adds gfx1250 (N=64,K=7168) Gluon tuning params by M bounds. |
| aiter/ops/triton/configs/gemm/gfx1250-GEMM-A16W16-N=384-K=7168.json | Adds gfx1250 (N=384,K=7168) Gluon tuning params by M bounds. |
| aiter/ops/triton/configs/gemm/gfx1250-GEMM-A16W16-N=512-K=7168.json | Adds gfx1250 (N=512,K=7168) Gluon tuning params by M bounds. |
| aiter/ops/triton/configs/gemm/gfx1250-GEMM-A16W16-N=1024-K=7168.json | Adds gfx1250 (N=1024,K=7168) Gluon tuning params by M bounds. |
| aiter/ops/triton/configs/gemm/gfx1250-GEMM-A16W16-N=2048-K=7168.json | Adds gfx1250 (N=2048,K=7168) Gluon tuning params by M bounds. |
| aiter/ops/triton/configs/gemm/gfx1250-GEMM-A16W16-N=32320-K=7168.json | Adds gfx1250 (N=32320,K=7168) Gluon tuning params by M bounds (DSv4pro). |
| aiter/configs/model_configs/dsv4pro_bf16_tuned_gemm.csv | New DSv4pro gfx1250 bf16 tuned GEMM shapes routed to Triton backend. |
| aiter/configs/model_configs/dsv4_bf16_tuned_gemm.csv | Adds DSv4 gfx1250 bf16 tuned GEMM shapes (multiple N values, K=7168) routed to Triton backend. |
💡 Add Copilot custom instructions for smarter, more guided reviews. Learn how to get started.
Comment on lines
+277
to
+280
| elif ( | ||
| gfx in ("gfx90a", "gfx942", "gfx950") | ||
| and is_skinny_default_shape(M, N, K, dtype, cu_num) | ||
| ): |
Comment on lines
+1
to
+3
| gfx,cu_num,M,N,K,bias,dtype,outdtype,scaleAB,bpreshuffle,libtype,solidx,splitK,us,kernelName,err_ratio,tflops,bw | ||
| gfx1250,256,1,32320,7168,False,torch.bfloat16,torch.bfloat16,False,False,triton,0,0,0.0,auto,0.0,0.0,0.0 | ||
| gfx1250,256,2,32320,7168,False,torch.bfloat16,torch.bfloat16,False,False,triton,0,0,0.0,auto,0.0,0.0,0.0 |
Contributor
Author
There was a problem hiding this comment.
python3 -m unittest op_tests.tuning_tests.test_config_shape_collision
----------------------------------------------------------------------
Ran 12 tests in 0.304s
OK
ahmed-bsod
force-pushed
the
ahmed/dsv4-tune-bf16gemm
branch
from
July 15, 2026 21:17
5d93596 to
11d7fa1
Compare
…e buffer as the ds_load without waiting for all the ds_loads to drain. we leave one buffer gap between tdm load and lds read now
yzhou103
reviewed
Jul 22, 2026
Comment on lines
+1031
to
+1032
| gfx1250,256,2,2048,7168,False,torch.bfloat16,torch.bfloat16,False,False,triton,0,0,0,auto,0.0,0.0,0.0 | ||
| gfx1250,256,1,2048,7168,False,torch.bfloat16,torch.bfloat16,False,False,triton,0,0,0,auto,0.0,0.0,0.0 |
Contributor
There was a problem hiding this comment.
could you please add the actual op duration here ?
Contributor
|
No activity for 15 days, so this is now labelled |
This branch has not been deployed
This file contains hidden or bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
Sign up for free
to join this conversation on GitHub.
Already have an account?
Sign in to comment
Add this suggestion to a batch that can be applied as a single commit.This suggestion is invalid because no changes were made to the code.Suggestions cannot be applied while the pull request is closed.Suggestions cannot be applied while viewing a subset of changes.Only one suggestion per line can be applied in a batch.Add this suggestion to a batch that can be applied as a single commit.Applying suggestions on deleted lines is not supported.You must change the existing code in this line in order to create a valid suggestion.Outdated suggestions cannot be applied.This suggestion has been applied or marked resolved.Suggestions cannot be applied from pending reviews.Suggestions cannot be applied on multi-line comments.Suggestions cannot be applied while the pull request is queued to merge.Suggestion cannot be applied right now. Please check back later.
GEMM A16W16 tuned kernel time (µs), K=7168