[Config] [Tuning] Fill remaining Kimi-K3 bf16 GEMM shapes - #5139
Merged
Merged
Conversation
PR #5124 tuned 51 new bf16 shapes but --update_improved only wrote the 20 that beat the default by the 3% threshold, leaving the rest to fall back to default dispatch. The tuner had picked a best backend for all 51; the other 31 were ties rather than losses. A/B those 31 against default dispatch on a single gfx950, 3 rounds of 100 iters after 20 warmup, and add the 26 that are not slower: N=896 K=7168 M=1024,4096,16384,32768 N=7168 K=35840 M=1,2,4,8,32,48,256,1024,2048,16384,32768 N=20480 K=7168 M=1,2,8,32,64,128,256,512,1024,4096,4096 Largest gains are at N=20480 K=7168 small M (+12~16% over default). 5 shapes measured slower than default and are left untuned. Co-Authored-By: Claude Opus 5 <noreply@anthropic.com>
Contributor
🏷️ CI GuideRuns automatically on every PR:
Extended tests (opt-in via labels):
PR title tags & labels: |
github-actions
Bot
changed the title
The new shapes from #5124 and this PR were appended at the tail, so each (N, K) group ended up split across several places in the file. Gather the rows of every group this round touched into one contiguous block sorted by M, at the position where the group first appears. Row contents are unchanged - reordering only. Co-Authored-By: Claude Opus 5 <noreply@anthropic.com>
XiaobingSuper
force-pushed
the
tune/kimi-k3-bf16-fill
branch
from
August 31, 2026 07:40
c37cdd7 to
d18788e
Compare
For M=16/128/512 at N=7168 K=35840 the tuner's best candidate is torch, which is also what default dispatch runs, so the A/B measured the same kernel on both sides and the sub-1% deltas were noise. Record them. Co-Authored-By: Claude Opus 5 <noreply@anthropic.com>
M=8192 at N=896 K=7168 and M=48 at N=20480 K=7168 measured 2.2% and 3.9% slower with the tuner's pick (triton / flydsl) than with default dispatch, so record what default already runs - torch - rather than a slower kernel. Completes the 51 new shapes. Co-Authored-By: Claude Opus 5 <noreply@anthropic.com>
zufayu
approved these changes
Sep 1, 2026
This file contains hidden or bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
Sign up for free
to join this conversation on GitHub.
Already have an account?
Sign in to comment
Add this suggestion to a batch that can be applied as a single commit.This suggestion is invalid because no changes were made to the code.Suggestions cannot be applied while the pull request is closed.Suggestions cannot be applied while viewing a subset of changes.Only one suggestion per line can be applied in a batch.Add this suggestion to a batch that can be applied as a single commit.Applying suggestions on deleted lines is not supported.You must change the existing code in this line in order to create a valid suggestion.Outdated suggestions cannot be applied.This suggestion has been applied or marked resolved.Suggestions cannot be applied from pending reviews.Suggestions cannot be applied on multi-line comments.Suggestions cannot be applied while the pull request is queued to merge.Suggestion cannot be applied right now. Please check back later.
Follow-up to #5124.
#5124 tuned 51 new bf16 shapes, but
--compare --update_improvedonly writes rows that beat the default by--min_improvement_pct(3%), so only 20 landed. The tuner had selected a best backend for all 51 — the remaining 31 were ties, not losses (the tuner's own profile CSV showed its winner ahead of the default by 0.4–1.9% while the compare benchmark flipped the sign by a comparable margin).Every other shape group in these configs has near-full coverage (gfx950 old-shape coverage in this file is 127/137; the a8w8 half of #5124 is 85/85), so 20/51 was inconsistent.
Method: A/B each of the 31 shapes against default dispatch via
gemm_a16w16_tune.py --run_config, single gfx950, 3 rounds of 100 iters after 20 warmup. Kept the 26 whose median and best-of-3 are both within 1% of default or better; dropped the 5 that measured slower.Result: +26 rows, no existing row modified or removed (verified key-wise on
[gfx,cu_num,M,N,K,bias,dtype,outdtype,scaleAB,bpreshuffle]). Largest gains at N=20480 K=7168 small M, +12~16% over default.op_tests/tuning_tests/test_config_shape_collision.pyandtest_csv_validation.py: 30 passed, 37 subtests.🤖 Generated with Claude Code