Repository navigation
perf(rocke): hotfix - gate bf16 D128 exp2_fast to seqlen>=4096 (gfx942 dense prefill) - #11079
Merged
AviralGoelAMD merged 2 commits intoAug 25, 2026
Conversation
github-actions
Bot
added
project: hip-kernel-provider
rocke-lib: attention
AviralGoelAMD
force-pushed
the
users/avirgoel/rocke/gfx942-dense-bf16-d128-exp2fast-seqlen-guard
branch
from
August 20, 2026 17:37
e58bde9 to
fe09d86
Compare
✅ All Checks Passed — Ready for Review
📖 Need help? See the Policy FAQ for details on every check and how to fix failures. |
|
🎉 All checks passed! This PR is ready for review. |
…42 dense prefill exp2_fast (enabled for all configs when the bf16-D128 holdout was removed) regresses short-sequence bf16 head_dim=128 prefill on the develop dense builder: it clearly regresses at seqlen<=1024 and clearly wins at seqlen>=4096, with seqlen==2048 inside the same-node run-to-run noise band. fp16 head_dim=128 -- byte-identical across the exp2_fast boundary -- is flat at every seqlen, pinning the regression to this kernel. Short sequences are occupancy/latency-bound, where exp2_fast's register/schedule shift costs more than its VALU saving. Gate exp2_fast for bf16 head_dim=128 to seqlen>=4096 (conservative cut: reverts every clear regressor plus the noisy 2048 case to plain exp2, so no shape regresses; forgoes at most a noise-level 2048 win). head_dim!=128 and fp16 unaffected; gfx950 makes an independent exp2_fast decision and is not covered here. Verified same-node on gfx942 ROCm 7.2.2 against the develop merge base. Re-blessed the 4 bf16-D128 golden cases (both llvm flavors); updated the exp2_fast policy matrix test to cover the seqlen guard.
AviralGoelAMD
force-pushed
the
users/avirgoel/rocke/gfx942-dense-bf16-d128-exp2fast-seqlen-guard
branch
from
August 20, 2026 18:17
fe09d86 to
d3eae60
Compare
AnaghaRaoAMD
approved these changes
Aug 20, 2026
iq136boy
approved these changes
Aug 21, 2026
AviralGoelAMD
enabled auto-merge (squash)
August 25, 2026 14:47
…28-exp2fast-seqlen-guard
BrianHarrisonAMD
added a commit
that referenced
this pull request
Aug 28, 2026
The 145-variant set had no tuning axis at all: 107 of 126 shape tuples had
exactly one candidate, so the oracle sweep had nothing to rank and P1/P2 came
back 0.05% apart.
Mined the kernel's commit history for knob verdicts (now a documented step in
rocke-mining.md). Five axes, each with measured evidence:
block_m {128,256,512} source: 'an OPEN occupancy axis ... a sweep knob'
waves_per_eu +-1 author swept per (D,dtype); narrow window re-opened
use_exp2_fast both PR #11079 gates bf16-D128 to Sq>=4096 (-15..-21% short,
+8..+10% long) and says the proper fix is seqlen-aware
kernel selection -- which is what this engine does
block_n {64,32} selection coverage at Skv%64!=0
persistent both on swept shapes; dispatcher rule elsewhere
Also emits the five valued knobs that were silently taking dataclass defaults
(num_persistent, lds_k_group_pad, lds_row_pad, lazy_rescale, iglp). The three
None tri-states stay omitted on purpose so they track the shipping policy.
20 tuning families get a representative shape swept at 48-72 variants; the
other 256 shapes keep the policy default. Coverage stays 283/283.
BrianHarrisonAMD
added a commit
that referenced
this pull request
Aug 28, 2026
The 145-variant set had no tuning axis at all: 107 of 126 shape tuples had
exactly one candidate, so the oracle sweep had nothing to rank and P1/P2 came
back 0.05% apart.
Mined the kernel's commit history for knob verdicts (now a documented step in
rocke-mining.md). Five axes, each with measured evidence:
block_m {128,256,512} source: 'an OPEN occupancy axis ... a sweep knob'
waves_per_eu +-1 author swept per (D,dtype); narrow window re-opened
use_exp2_fast both PR #11079 gates bf16-D128 to Sq>=4096 (-15..-21% short,
+8..+10% long) and says the proper fix is seqlen-aware
kernel selection -- which is what this engine does
block_n {64,32} selection coverage at Skv%64!=0
persistent both on swept shapes; dispatcher rule elsewhere
Also emits the five valued knobs that were silently taking dataclass defaults
(num_persistent, lds_k_group_pad, lds_row_pad, lazy_rescale, iglp). The three
None tri-states stay omitted on purpose so they track the shipping policy.
20 tuning families get a representative shape swept at 48-72 variants; the
other 256 shapes keep the policy default. Coverage stays 283/283.
This file contains hidden or bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
Sign up for free
to join this conversation on GitHub.
Already have an account?
Sign in to comment
Add this suggestion to a batch that can be applied as a single commit.This suggestion is invalid because no changes were made to the code.Suggestions cannot be applied while the pull request is closed.Suggestions cannot be applied while viewing a subset of changes.Only one suggestion per line can be applied in a batch.Add this suggestion to a batch that can be applied as a single commit.Applying suggestions on deleted lines is not supported.You must change the existing code in this line in order to create a valid suggestion.Outdated suggestions cannot be applied.This suggestion has been applied or marked resolved.Suggestions cannot be applied from pending reviews.Suggestions cannot be applied on multi-line comments.Suggestions cannot be applied while the pull request is queued to merge.Suggestion cannot be applied right now. Please check back later.
Hot fix for a short-sequence regression from #10885 (AICK-1991), which enabled
exp2_fastfor bf16head_dim=128unconditionally. On the develop merge base (gfx942, ROCm 7.2.2, same-node A/B) it clearly regresses short-seq dense prefill (~-15 to -21% at Sq <= 1024) while winning at Sq >= 4096 (+8-10%); Sq 2048 sits in the same-node run-to-run noise band.Guard: bf16
head_dim=128usesexp2_fastonly whenseqlen >= 4096; shorter sequences revert to plainexp2(their original, pre-#10885 perf). The conservative cut never regresses a shape (it forgoes at most a noise-level Sq 2048 win). fp16 and D64 are unaffected; gfx950 makes an independent exp2_fast decision and is not covered here.The proper fix is seqlen-aware kernel selection in the rocKE dispatcher; that work is WIP, so this lands the guard in the kernel builder as a hot fix.
Jira: AICK-1991