Skip to content

[DSV4] Add fp32-output untuned GEMM shapes for indexer kv_score #3836

New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

Sign up for GitHub

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Open
AMD-yanfeiwang wants to merge 1 commit into
base: main
Choose a base branch
from
+229 −0
Open

[DSV4] Add fp32-output untuned GEMM shapes for indexer kv_score #3836

Changes from all commits
Commits
Show all changes
1 commit
Select commit
File filter

Filter by extension

Filter by extension
Conversations
Failed to load comments.
Loading
Jump to
Jump to file
Failed to load files.
Loading
Diff view
Diff view
229 changes: 229 additions & 0 deletions aiter/configs/model_configs/dsv4_bf16_untuned_gemm.csv
View file
Open in desktop
Original file line number Diff line number Diff line change
Expand Up @@ -349,3 +349,232 @@ M,N,K,bias,dtype,outdtype,scaleAB,bpreshuffle
14336,2048,7168,False,torch.bfloat16,torch.bfloat16,False,False
16384,2048,7168,False,torch.bfloat16,torch.bfloat16,False,False
32768,2048,7168,False,torch.bfloat16,torch.bfloat16,False,False
1,512,7168,False,torch.bfloat16,torch.float32,False,False
2,512,7168,False,torch.bfloat16,torch.float32,False,False
4,512,7168,False,torch.bfloat16,torch.float32,False,False
8,512,7168,False,torch.bfloat16,torch.float32,False,False
16,512,7168,False,torch.bfloat16,torch.float32,False,False
32,512,7168,False,torch.bfloat16,torch.float32,False,False
48,512,7168,False,torch.bfloat16,torch.float32,False,False
64,512,7168,False,torch.bfloat16,torch.float32,False,False
80,512,7168,False,torch.bfloat16,torch.float32,False,False
96,512,7168,False,torch.bfloat16,torch.float32,False,False
112,512,7168,False,torch.bfloat16,torch.float32,False,False
128,512,7168,False,torch.bfloat16,torch.float32,False,False
144,512,7168,False,torch.bfloat16,torch.float32,False,False
160,512,7168,False,torch.bfloat16,torch.float32,False,False
176,512,7168,False,torch.bfloat16,torch.float32,False,False
192,512,7168,False,torch.bfloat16,torch.float32,False,False
208,512,7168,False,torch.bfloat16,torch.float32,False,False
224,512,7168,False,torch.bfloat16,torch.float32,False,False
240,512,7168,False,torch.bfloat16,torch.float32,False,False
256,512,7168,False,torch.bfloat16,torch.float32,False,False
288,512,7168,False,torch.bfloat16,torch.float32,False,False
320,512,7168,False,torch.bfloat16,torch.float32,False,False
352,512,7168,False,torch.bfloat16,torch.float32,False,False
384,512,7168,False,torch.bfloat16,torch.float32,False,False
416,512,7168,False,torch.bfloat16,torch.float32,False,False
448,512,7168,False,torch.bfloat16,torch.float32,False,False
480,512,7168,False,torch.bfloat16,torch.float32,False,False
512,512,7168,False,torch.bfloat16,torch.float32,False,False
544,512,7168,False,torch.bfloat16,torch.float32,False,False
576,512,7168,False,torch.bfloat16,torch.float32,False,False
608,512,7168,False,torch.bfloat16,torch.float32,False,False
640,512,7168,False,torch.bfloat16,torch.float32,False,False
672,512,7168,False,torch.bfloat16,torch.float32,False,False
704,512,7168,False,torch.bfloat16,torch.float32,False,False
736,512,7168,False,torch.bfloat16,torch.float32,False,False
768,512,7168,False,torch.bfloat16,torch.float32,False,False
800,512,7168,False,torch.bfloat16,torch.float32,False,False
832,512,7168,False,torch.bfloat16,torch.float32,False,False
864,512,7168,False,torch.bfloat16,torch.float32,False,False
896,512,7168,False,torch.bfloat16,torch.float32,False,False
928,512,7168,False,torch.bfloat16,torch.float32,False,False
960,512,7168,False,torch.bfloat16,torch.float32,False,False
992,512,7168,False,torch.bfloat16,torch.float32,False,False
1024,512,7168,False,torch.bfloat16,torch.float32,False,False
1088,512,7168,False,torch.bfloat16,torch.float32,False,False
1152,512,7168,False,torch.bfloat16,torch.float32,False,False
1216,512,7168,False,torch.bfloat16,torch.float32,False,False
1280,512,7168,False,torch.bfloat16,torch.float32,False,False
1344,512,7168,False,torch.bfloat16,torch.float32,False,False
1408,512,7168,False,torch.bfloat16,torch.float32,False,False
1472,512,7168,False,torch.bfloat16,torch.float32,False,False
1536,512,7168,False,torch.bfloat16,torch.float32,False,False
1600,512,7168,False,torch.bfloat16,torch.float32,False,False
1664,512,7168,False,torch.bfloat16,torch.float32,False,False
1728,512,7168,False,torch.bfloat16,torch.float32,False,False
1792,512,7168,False,torch.bfloat16,torch.float32,False,False
1856,512,7168,False,torch.bfloat16,torch.float32,False,False
1920,512,7168,False,torch.bfloat16,torch.float32,False,False
1984,512,7168,False,torch.bfloat16,torch.float32,False,False
2048,512,7168,False,torch.bfloat16,torch.float32,False,False
4096,512,7168,False,torch.bfloat16,torch.float32,False,False
6144,512,7168,False,torch.bfloat16,torch.float32,False,False
8192,512,7168,False,torch.bfloat16,torch.float32,False,False
10240,512,7168,False,torch.bfloat16,torch.float32,False,False
12288,512,7168,False,torch.bfloat16,torch.float32,False,False
14336,512,7168,False,torch.bfloat16,torch.float32,False,False
16384,512,7168,False,torch.bfloat16,torch.float32,False,False
32768,512,7168,False,torch.bfloat16,torch.float32,False,False
1,1024,7168,False,torch.bfloat16,torch.float32,False,False
2,1024,7168,False,torch.bfloat16,torch.float32,False,False
4,1024,7168,False,torch.bfloat16,torch.float32,False,False
8,1024,7168,False,torch.bfloat16,torch.float32,False,False
16,1024,7168,False,torch.bfloat16,torch.float32,False,False
24,1024,7168,False,torch.bfloat16,torch.float32,False,False
32,1024,7168,False,torch.bfloat16,torch.float32,False,False
40,1024,7168,False,torch.bfloat16,torch.float32,False,False
48,1024,7168,False,torch.bfloat16,torch.float32,False,False
56,1024,7168,False,torch.bfloat16,torch.float32,False,False
64,1024,7168,False,torch.bfloat16,torch.float32,False,False
72,1024,7168,False,torch.bfloat16,torch.float32,False,False
80,1024,7168,False,torch.bfloat16,torch.float32,False,False
88,1024,7168,False,torch.bfloat16,torch.float32,False,False
96,1024,7168,False,torch.bfloat16,torch.float32,False,False
104,1024,7168,False,torch.bfloat16,torch.float32,False,False
112,1024,7168,False,torch.bfloat16,torch.float32,False,False
120,1024,7168,False,torch.bfloat16,torch.float32,False,False
128,1024,7168,False,torch.bfloat16,torch.float32,False,False
136,1024,7168,False,torch.bfloat16,torch.float32,False,False
144,1024,7168,False,torch.bfloat16,torch.float32,False,False
152,1024,7168,False,torch.bfloat16,torch.float32,False,False
160,1024,7168,False,torch.bfloat16,torch.float32,False,False
168,1024,7168,False,torch.bfloat16,torch.float32,False,False
176,1024,7168,False,torch.bfloat16,torch.float32,False,False
184,1024,7168,False,torch.bfloat16,torch.float32,False,False
192,1024,7168,False,torch.bfloat16,torch.float32,False,False
200,1024,7168,False,torch.bfloat16,torch.float32,False,False
208,1024,7168,False,torch.bfloat16,torch.float32,False,False
216,1024,7168,False,torch.bfloat16,torch.float32,False,False
224,1024,7168,False,torch.bfloat16,torch.float32,False,False
232,1024,7168,False,torch.bfloat16,torch.float32,False,False
240,1024,7168,False,torch.bfloat16,torch.float32,False,False
248,1024,7168,False,torch.bfloat16,torch.float32,False,False
256,1024,7168,False,torch.bfloat16,torch.float32,False,False
264,1024,7168,False,torch.bfloat16,torch.float32,False,False
272,1024,7168,False,torch.bfloat16,torch.float32,False,False
280,1024,7168,False,torch.bfloat16,torch.float32,False,False
288,1024,7168,False,torch.bfloat16,torch.float32,False,False
296,1024,7168,False,torch.bfloat16,torch.float32,False,False
304,1024,7168,False,torch.bfloat16,torch.float32,False,False
312,1024,7168,False,torch.bfloat16,torch.float32,False,False
320,1024,7168,False,torch.bfloat16,torch.float32,False,False
328,1024,7168,False,torch.bfloat16,torch.float32,False,False
336,1024,7168,False,torch.bfloat16,torch.float32,False,False
344,1024,7168,False,torch.bfloat16,torch.float32,False,False
352,1024,7168,False,torch.bfloat16,torch.float32,False,False
376,1024,7168,False,torch.bfloat16,torch.float32,False,False
384,1024,7168,False,torch.bfloat16,torch.float32,False,False
416,1024,7168,False,torch.bfloat16,torch.float32,False,False
448,1024,7168,False,torch.bfloat16,torch.float32,False,False
480,1024,7168,False,torch.bfloat16,torch.float32,False,False
512,1024,7168,False,torch.bfloat16,torch.float32,False,False
544,1024,7168,False,torch.bfloat16,torch.float32,False,False
576,1024,7168,False,torch.bfloat16,torch.float32,False,False
608,1024,7168,False,torch.bfloat16,torch.float32,False,False
640,1024,7168,False,torch.bfloat16,torch.float32,False,False
672,1024,7168,False,torch.bfloat16,torch.float32,False,False
704,1024,7168,False,torch.bfloat16,torch.float32,False,False
736,1024,7168,False,torch.bfloat16,torch.float32,False,False
768,1024,7168,False,torch.bfloat16,torch.float32,False,False
800,1024,7168,False,torch.bfloat16,torch.float32,False,False
832,1024,7168,False,torch.bfloat16,torch.float32,False,False
864,1024,7168,False,torch.bfloat16,torch.float32,False,False
896,1024,7168,False,torch.bfloat16,torch.float32,False,False
928,1024,7168,False,torch.bfloat16,torch.float32,False,False
960,1024,7168,False,torch.bfloat16,torch.float32,False,False
992,1024,7168,False,torch.bfloat16,torch.float32,False,False
1024,1024,7168,False,torch.bfloat16,torch.float32,False,False
1088,1024,7168,False,torch.bfloat16,torch.float32,False,False
1152,1024,7168,False,torch.bfloat16,torch.float32,False,False
1216,1024,7168,False,torch.bfloat16,torch.float32,False,False
1280,1024,7168,False,torch.bfloat16,torch.float32,False,False
1344,1024,7168,False,torch.bfloat16,torch.float32,False,False
1408,1024,7168,False,torch.bfloat16,torch.float32,False,False
1472,1024,7168,False,torch.bfloat16,torch.float32,False,False
1536,1024,7168,False,torch.bfloat16,torch.float32,False,False
1600,1024,7168,False,torch.bfloat16,torch.float32,False,False
1664,1024,7168,False,torch.bfloat16,torch.float32,False,False
1728,1024,7168,False,torch.bfloat16,torch.float32,False,False
1792,1024,7168,False,torch.bfloat16,torch.float32,False,False
1856,1024,7168,False,torch.bfloat16,torch.float32,False,False
1920,1024,7168,False,torch.bfloat16,torch.float32,False,False
1984,1024,7168,False,torch.bfloat16,torch.float32,False,False
2048,1024,7168,False,torch.bfloat16,torch.float32,False,False
4096,1024,7168,False,torch.bfloat16,torch.float32,False,False
6144,1024,7168,False,torch.bfloat16,torch.float32,False,False
8192,1024,7168,False,torch.bfloat16,torch.float32,False,False
10240,1024,7168,False,torch.bfloat16,torch.float32,False,False
12288,1024,7168,False,torch.bfloat16,torch.float32,False,False
14336,1024,7168,False,torch.bfloat16,torch.float32,False,False
16384,1024,7168,False,torch.bfloat16,torch.float32,False,False
32768,1024,7168,False,torch.bfloat16,torch.float32,False,False
1,2048,7168,False,torch.bfloat16,torch.float32,False,False
2,2048,7168,False,torch.bfloat16,torch.float32,False,False
4,2048,7168,False,torch.bfloat16,torch.float32,False,False
8,2048,7168,False,torch.bfloat16,torch.float32,False,False
16,2048,7168,False,torch.bfloat16,torch.float32,False,False
32,2048,7168,False,torch.bfloat16,torch.float32,False,False
48,2048,7168,False,torch.bfloat16,torch.float32,False,False
64,2048,7168,False,torch.bfloat16,torch.float32,False,False
80,2048,7168,False,torch.bfloat16,torch.float32,False,False
96,2048,7168,False,torch.bfloat16,torch.float32,False,False
112,2048,7168,False,torch.bfloat16,torch.float32,False,False
128,2048,7168,False,torch.bfloat16,torch.float32,False,False
144,2048,7168,False,torch.bfloat16,torch.float32,False,False
160,2048,7168,False,torch.bfloat16,torch.float32,False,False
176,2048,7168,False,torch.bfloat16,torch.float32,False,False
192,2048,7168,False,torch.bfloat16,torch.float32,False,False
208,2048,7168,False,torch.bfloat16,torch.float32,False,False
224,2048,7168,False,torch.bfloat16,torch.float32,False,False
240,2048,7168,False,torch.bfloat16,torch.float32,False,False
256,2048,7168,False,torch.bfloat16,torch.float32,False,False
288,2048,7168,False,torch.bfloat16,torch.float32,False,False
320,2048,7168,False,torch.bfloat16,torch.float32,False,False
352,2048,7168,False,torch.bfloat16,torch.float32,False,False
384,2048,7168,False,torch.bfloat16,torch.float32,False,False
416,2048,7168,False,torch.bfloat16,torch.float32,False,False
448,2048,7168,False,torch.bfloat16,torch.float32,False,False
480,2048,7168,False,torch.bfloat16,torch.float32,False,False
512,2048,7168,False,torch.bfloat16,torch.float32,False,False
544,2048,7168,False,torch.bfloat16,torch.float32,False,False
576,2048,7168,False,torch.bfloat16,torch.float32,False,False
608,2048,7168,False,torch.bfloat16,torch.float32,False,False
640,2048,7168,False,torch.bfloat16,torch.float32,False,False
672,2048,7168,False,torch.bfloat16,torch.float32,False,False
704,2048,7168,False,torch.bfloat16,torch.float32,False,False
736,2048,7168,False,torch.bfloat16,torch.float32,False,False
768,2048,7168,False,torch.bfloat16,torch.float32,False,False
800,2048,7168,False,torch.bfloat16,torch.float32,False,False
832,2048,7168,False,torch.bfloat16,torch.float32,False,False
864,2048,7168,False,torch.bfloat16,torch.float32,False,False
896,2048,7168,False,torch.bfloat16,torch.float32,False,False
928,2048,7168,False,torch.bfloat16,torch.float32,False,False
960,2048,7168,False,torch.bfloat16,torch.float32,False,False
992,2048,7168,False,torch.bfloat16,torch.float32,False,False
1024,2048,7168,False,torch.bfloat16,torch.float32,False,False
1088,2048,7168,False,torch.bfloat16,torch.float32,False,False
1152,2048,7168,False,torch.bfloat16,torch.float32,False,False
1216,2048,7168,False,torch.bfloat16,torch.float32,False,False
1280,2048,7168,False,torch.bfloat16,torch.float32,False,False
1344,2048,7168,False,torch.bfloat16,torch.float32,False,False
1408,2048,7168,False,torch.bfloat16,torch.float32,False,False
1472,2048,7168,False,torch.bfloat16,torch.float32,False,False
1536,2048,7168,False,torch.bfloat16,torch.float32,False,False
1600,2048,7168,False,torch.bfloat16,torch.float32,False,False
1664,2048,7168,False,torch.bfloat16,torch.float32,False,False
1728,2048,7168,False,torch.bfloat16,torch.float32,False,False
1792,2048,7168,False,torch.bfloat16,torch.float32,False,False
1856,2048,7168,False,torch.bfloat16,torch.float32,False,False
1920,2048,7168,False,torch.bfloat16,torch.float32,False,False
1984,2048,7168,False,torch.bfloat16,torch.float32,False,False
2048,2048,7168,False,torch.bfloat16,torch.float32,False,False
4096,2048,7168,False,torch.bfloat16,torch.float32,False,False
6144,2048,7168,False,torch.bfloat16,torch.float32,False,False
8192,2048,7168,False,torch.bfloat16,torch.float32,False,False
10240,2048,7168,False,torch.bfloat16,torch.float32,False,False
12288,2048,7168,False,torch.bfloat16,torch.float32,False,False
14336,2048,7168,False,torch.bfloat16,torch.float32,False,False
16384,2048,7168,False,torch.bfloat16,torch.float32,False,False
32768,2048,7168,False,torch.bfloat16,torch.float32,False,False
Loading