Skip to content

Attention_dense lds_k_group_pad Lever - #11222

Merged
dlangbe merged 1 commit into
developfrom
users/dlangbe/rocke-attention_dense-lds_k_group_pad
Aug 26, 2026
Merged

dlangbe merged 1 commit into
developfrom
users/dlangbe/rocke-attention_dense-lds_k_group_pad

Conversation

@dlangbe

@dlangbe dlangbe commented Aug 24, 2026

Copy link
Copy Markdown
Contributor

JIRA ID : AICK-1895

Motivation

lds_k_group_pad was added to AttentionDenseSpec in PR #10102 with a fixed
default of 8. That PR committed to a follow-up sweep over the full pad space to
confirm the default and record whether any shape benefits from a different value.
This PR closes that follow-up by documenting the sweep methodology, results, and
decision in-repo, and making the knob visible in the runbook's knob catalog.

Technical Details

Three documentation edits:

library/builders/gfx950/attention/prefill/README.md

  • Fixed stale line: "block_n and waves_per_eu are the only performance knobs"
    — there are now three (block_n, waves_per_eu, lds_k_group_pad).
  • Added ## Tuning — lds_k_group_pad section documenting:
    • Sweep axes: pad ∈ {0, 8, 16, 24, 32} × bn ∈ {64, 128} × 3 GQA ratios ×
      causal/swa/varlen/persistent × S ∈ {2048, 4096, 8192} (840 configs,
      all correctness-passed).
    • The whole-wave bank-conflict model explaining why pad=8, 16, and 24 tie at
      the 4-lanes-per-bank floor, and what it would take to separate them.
    • Decision: default stays at 8. pad=24's plurality is sub-1% and noise-grade
      without an ABBA repeat; pad=16 has an unacceptable regression on a bn=128
      sliding-window shape that makes it unsafe as a global or shape-conditioned
      default.
    • The pad=16 regression recorded as a named constraint for future work.
    • The multiple-of-8 invariant (smem_load_vN stamps align 16
      unconditionally; __post_init__ enforces it).

platform/dsl_docs/optimization/optimization_runbook.md §12.1.F

  • Added lds_k_group_pad knob-catalog row, sibling to the existing lds_k_pad
    and lds_layout rows. Points to the gfx950 README for the full decision
    record. Makes the knob visible to the Step 0 exhaustive lever sweep.

library/builders/gfx942/attention/prefill/README.md

  • Added one row to the "Lever record → Adopted" table noting that gfx942 reuses
    the shared AttentionDenseSpec.lds_k_group_pad field and inherits default 8,
    with a cross-reference to the gfx950 sweep.

Per platform/AGENTS.md §Compliance, absolute TFLOPS figures and per-lever
deltas are kept on the protected results page only; the in-repo docs contain
methodology, axes, decision, and qualitative direction.

Test Plan

  • Documentation-only change; no kernel IR or dispatch logic modified.
  • Verified the stale knob-count sentence (L26-28 of the gfx950 README) is
    corrected to reflect three knobs.
  • Confirmed the new runbook row is correctly placed in §12.1.F alongside sibling
    LDS-layout knobs.
  • Confirmed the gfx942 lever table row accurately describes the inheritance
    relationship and points to the canonical decision record.

Test Result

No functional changes. All existing parity and byte-identity gates are unaffected.

Submission Checklist

@dlangbe
dlangbe requested review from a team as code owners August 24, 2026 18:59
@github-actions github-actions Bot added documentation ci:docs-only Docs only changes rocke: platform Touches rocKE platform labels Aug 24, 2026
@therock-pr-bot

therock-pr-bot Bot commented Aug 24, 2026 •

Copy link
Copy Markdown

✅ All Checks Passed — Ready for Review

Check Status Details
📝 PR Description ✅ Pass —
⛔ Forbidden Files ✅ Pass —
🧪 Unit Test ✅ Pass PR does not contain code files — Unit Test auto-passed
🔎 pre-commit ✅ Pass —
🚫 Draft PR 🔜 To Be Enabled —
🚩 Feature Flag 🔜 To Be Enabled —
📊 Code Coverage 🔜 To Be Enabled —
🤖 therock-pr-bot ✅ Pass —

🎉 All checks passed! This PR is ready for review.

📖 Need help? See the Policy FAQ for details on every check and how to fix failures.

🙋 Wish to Override Policy?

@therock-pr-bot

Copy link
Copy Markdown

🎉 All checks passed! This PR is ready for review.

@dlangbe
dlangbe force-pushed the users/dlangbe/rocke-attention_dense-lds_k_group_pad branch from 27a80db to 17135ad Compare August 24, 2026 20:05
@dlangbe
dlangbe merged commit 7f33e74 into develop Aug 26, 2026
32 of 34 checks passed
@dlangbe
dlangbe deleted the users/dlangbe/rocke-attention_dense-lds_k_group_pad branch August 26, 2026 19:02
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

ci:docs-only Docs only changes documentation organization: ROCm project: hip-kernel-provider rocke: platform Touches rocKE platform rocke-lib: attention Touches rocKE library attention/FMHA kernels

Projects

None yet

Development

Successfully merging this pull request may close these issues.

2 participants