Repository navigation
feat(hip-kernel-provider): paged/ragged SDPA in Gfx1151WmmaAttention - #14254
dpeabody-amd wants to merge 11 commits into
Conversation
Codecov Report✅ All modified and coverable lines are covered by tests. Additional details and impacted files@@ Coverage Diff @@
## develop #14254 +/- ##
========================================
Coverage 78.16% 78.16%
========================================
Files 2977 2977
Lines 508843 508843
Branches 79012 79012
========================================
Hits 397712 397712
Misses 83900 83900
Partials 27231 27231
*This pull request uses carry forward flags. Click here to find out more. 🚀 New features to boost your workflow:
|
ef6ca8e to
11a5c83
Compare
✅ All Checks Passed — Ready for Review
📖 Need help? See the Policy FAQ for details on every check and how to fix failures. |
|
🎉 All checks passed! This PR is ready for review. |
…PI (#14358) ## Summary This PR brings the hipDNN SDPA **CPU and GPU reference implementations** up to everything the hipDNN SDPA graph API can express, forward and backward. It also adds the tests and docs that let the references serve as the oracle every attention engine is compared against. cuDNN's semantics are the contract. Each place where hipDNN deliberately differs from cuDNN is recorded in `projects/hipdnn/docs/SdpaSemanticsRegister.md` (SEM-xxx). The design, the full support table and the testing strategy are in **`projects/hipdnn/docs/SdpaReferenceImplementations.md`**. Builds on #14483 (cuDNN's `left_bound` meaning: L keeps the L keys ending at the diagonal) and #14312 (SdpaBackward O and LSE derived from the CPU forward), both merged. JIRA ID : ALMIOPEN-2863 Related: ALMIOPEN-2928, AICK-1847 ## What the references support Both references (CPU `CpuFpReferenceSdpa`; GPU `GpuFpReferenceSdpa`, hipRTC kernels) and both graph executors support: **Forward (`SdpaAttributes`)** | Feature | Notes | |---|---| | dtypes | FP32, FP16, BF16; FP8 E4M3 / E4M3 FNUZ / E5M2 / E5M2 FNUZ inputs with BF16 or same-format fp8 output | | Layout, GQA/MQA, `D_qk != D_v` | any strides; independent `H_k` / `H_v` | | Additive bias | FLOAT/HALF/BF16, rank 1-4 broadcast | | Diagonal band | `left_bound` / `right_bound` / alignment and the deprecated causal flags | | Padding | `seq_len_q/kv`; padded rows give O = 0, LSE = -inf | | Paged K/V | K, V or both, each with its own block size; extent inferred in cuDNN's order when unset | | Ragged (RFC 0014) | native packed forward (BF16, FP8 E4M3); every other ragged graph, including backward and ragged Q over paged K/V, through a ragged-as-dense rewrite | | Attention sink | FLOAT `[1, H, 1, 1]` | | ALiBi | cuDNN slopes, top-left causal | | Block-sparse mask | cuDNN's UINT8 bitmask, one bit per 128x128 tile | | Dropout | explicit mask; Philox through the engine's `rng_dump` | | fp8 scaling | Q/K/V descales (scalar or per KV head); `scale_s` / `descale_s` / `scale_o`; `amax_s` / `amax_o` | | Outputs | LSE, Max, Sum_exp | **Backward (`SdpaBackwardAttributes`)**: a new GPU backward reference. Both references support FP32/FP16/BF16, GQA, `D_qk != D_v`, LSE or recompute, bias + dBias, band, padding, ALiBi, explicit-mask dropout, and ragged through the rewrite. **Reference-only**: logit softcap, and an opt-in GPU P-rounding mode that models provider kernels. **Not supported because the graph API lacks it**: backward Philox (no backward `rng_dump`), backward sinks / dSink, the fp8 backward node, and `score_mod`. **Structure**: each direction is one pipeline (score: scale, softcap, bias, ALiBi, band, block mask; then softmax with sink; then P @ V), configured by a named options struct. fp8, paging and ragged are thin layers around that core. The CPU and GPU references share no code, only the semantics. The graph-translation layer (`PlanUtils.hpp`, the ragged rewrite) is shared by both executors, and the dense CPU and GPU plans use one applicability gate per direction. ## Behavior changes - **Frontend validation follows cuDNN (user-visible).** `SdpaFwdNode` now rejects: `right_bound < -1`; `max_seq_len_kv` without paging; paging without both `seq_len_q` and `seq_len_kv`; page tables that are not INT32 `[B, 1, M, 1]`; M != `ceil(max_seq_len_kv / block_size)` when the extent is set; `seq_len_*` not INT32 `[B, 1, 1, 1]`; `sink_token` not FLOAT `[1, H_q, 1, 1]`. One-sided paging, per-side block sizes, an inferred extent and packed page tables stay allowed. Each rule cites cuDNN and has a decline test. In-repo callers that built such graphs are fixed. - **Breaking `test_sdk` change (deliberate).** The positional `CpuFpReferenceSdpa::forward` / `backward` overloads are removed in favor of `SdpaFwdOptions` / `SdpaBwdOptions` (`.setScale`, `.setMask`, `.setBand(left, right, topLeft)`; causal is `.setBand(-1, 0, topLeft)`). There is no forwarding shim on purpose: `left_bound` changed meaning (SEM-001), so a shim would silently change old callers' results. Every in-repo caller is migrated. - **gfx1151 WMMA attention (`Gfx1151WmmaAttentionNative.cpp` `maskFor`)**: the deprecated `causal_mask` / `causal_mask_bottom_right` flags compose with explicit bounds and alignment, as the references do, instead of overriding them. Graphs that set only the flags bind as before. ## Absorbed PRs (authorship kept) - **#14004** (EwanC, ALMIOPEN-2863): SDPA backward golden bundles and generator, extended with GQA, padding, dBias, explicit dropout and an FP64 autograd cross-check. - **#13935** (scerzh, ALMIOPEN-2928): CPU ragged SDPA forward plan. - **#14245** (dpeabody-amd, AICK-1847): the frontend paged / sink / seq_len validation, the GROUP-mode ASM test fix, and the paged golden generator. Its separate paged plans are not taken: paging goes through the dense reference path, so every other feature composes with it. Stacked work on #14245 (#14247, #14254) may need to follow the stricter `seq_len_q` rule. ## Testing How correctness is established (detail in `SdpaReferenceImplementations.md` §3): 1. **Simple by construction**: FP32, no tiling, three-pass softmax, one code path per feature. 2. **Closed-form oracles and properties**: uniform scores, single key, dead rows; GQA = repeated heads, band and block mask = -inf bias, padding = truncated batches, paged = dense, sink = extra zero-value key, ragged = per-batch dense. 3. **Finite-difference gradient checks** of the backward. 4. **CPU vs GPU parity** (`hipdnn_gpu_ref_tests`, `TestGpuSdpa*Plan`): every feature at small shapes with NaN-filled padding; Standard-tier medium sweeps up to 1100 keys; fp8 device codecs bit-exact against the host. 5. **PyTorch golden bundles** (`hipdnn_golden_data_tests` checks every bundle against both references, with no engine involved). The backward generator is FP64-autograd checked; the fp8-scaling and block-mask/ALiBi paths reproduce cuDNN's own reference code. Golden data is 78 DVC pointers (26.8 MiB, every bundle under 1 MiB, pushed to `hipdnn/golden-data`). 6. **Mutation testing**: injected reference and gate bugs each fail a test. 7. **Applicability**: every schema field is gated, with one decline test per rule; a graph is declined, never accepted and then thrown on. **Results on gfx942** (MI300X, `-Werror`; the current head is covered by CI): | Suite | Passed | |---|---| | test_sdk / frontend / public_frontend | 2609 (1 skipped, pre-existing) / 2247 / 487 | | gpu_ref | 13836 | | integration-tests unit | 1149 | | golden (every bundle with golden data, both references) | 220 | | ASM_SDPA_ENGINE SdpaBackward bundles under #14312's tolerances | 31 (41 graphs the engine does not accept) | | hip-kernel-provider / its SDPA integration / ASM SDPA external quick suite | 1393 / 45 / 26 | | flatbuffers_sdk / plugin_sdk / generator pytest | 272 / 1293 / 146 | **Devices**: gfx942 runs everything above. gfx950: no run. gfx1151 has host matcher tests only (no usable device node; see Known issues). Windows and the other families are covered by TheRock CI. GPU reference kernels are hipRTC source, so there is no `GPU_TARGETS`-dependent host configuration. **Test time**: the new medium sweeps and long bundles are Standard tier (~18 s). The GPU reference forward is faster than on `develop` (masked keys skip the dot product), with byte-identical outputs. ## Known issues - **fp8 scaling over 128 keys (SEM-024)**: cuDNN quantizes P per 128-key tile against a running max; the references are exact (and bit-exact against cuDNN's `fp8_ref`) up to 128 keys, and the bundles stay within that. - **gfx1151 (accepted risk)**: the `maskFor` composition change has host matcher tests only; a 13-graph device check is prepared for when a gfx1151 node is available. - Once this lands, #14004, #13935 and #14245 are closed as superseded. ## Pre-merge - [ ] CI green on the current head 🤖 Generated with [Claude Code](https://claude.com/claude-code) --------- Co-authored-by: Claude Opus 5.5 <noreply@anthropic.com> Co-authored-by: Ewan Crawford <ewan.crawford@streamhpc.com> Co-authored-by: Sergei Grigoriev <102019268+scerzh@users.noreply.github.com> Co-authored-by: dpeabody-amd <dpeabody-amd@users.noreply.github.com>
43d0852 to
1c25c6d
Compare
There was a problem hiding this comment.
PR description: tests claim is broader than the tests
The description and commit message say every decline's logged reason is asserted. Only the paging cases check a reason; the others pass an empty reason, so only their cause is checked. Please assert the reasons.
| - {dense_q: true, seqused_q: true, tag: denseq_used} | ||
| - {seqused_q: true, tag: raggedq} | ||
| shapes: | ||
| - {head_dim: 64, head_dim_v: 64, knobs: query, resolved: {kind: prefill, head_size: 64, head_size_v: 64, mask_mode: causal, bias: none, block_size: 256, block_m: 128, dv_split: 1, dense_q: 0, seqused_q: 0, sinks: 0}} |
There was a problem hiding this comment.
Paged resolved values don't match the output
The paged shapes list dense_q: 0, seqused_q: 0, but every arm overrides both, so the shipped objects carry other values. Please drop the two fields from the paged resolved entries (lines 206-209, 230-233), as the dense groups already do for bias.
There was a problem hiding this comment.
I dropped seqused_q from the eight paged resolved entries, since every arm sets it. dense_q: 0 has to stay: the raggedq arms do not set dense_q, so resolved is where their shipped dense_q = 0 comes from, and the loader refuses the config without it. Setting dense_q: false on those arms instead would let both go, but it adds an explicit dense_q to 48 kernel records in the shipped KDP, so I kept the descriptors as they are. The descriptors regenerate byte for byte and the render digest does not move. Happy to switch to the arm form if you prefer it. (5f4c910)
There was a problem hiding this comment.
Nothing new here since my last reply: seqused_q is gone from every paged resolved entry, and dense_q: 0 stays because the raggedq arms take it from there. The paged head-96 objects are now split into their own groups (block_m 64 on gfx11, see the spill item on the PR), and those groups follow the same rule. I'm still happy to switch to setting dense_q: false on the arms if you prefer that form.
|
Automated review: Model - qwen3.6:35b-a3b-coding-mtp-q4_K_M, effort: max PR #14254 — paged/ragged SDPA in Gfx1151WmmaAttention (dpeabody-amd, AICK-1847) — 🟠 First-round: conflict, ABI bump, sink dtype
Utility: 🟢 Positive — paged and ragged SDPA for RDNA prefill; GPU cases pass on gfx110X CI Legend: 🔴 blocking · 🟠 should-fix · 🟡 minor nit · 🟢 info/good · ⚪ retracted First-round review: first pass by a local qwen3.6 model, verified by the orchestrator (Claude Opus 5.5). Checks: merge and conflict resolution, native build + ctest, byte identity, goldens and re-bless diff, IR hot spots, ABI layouts, changed and generator tests, descriptor regeneration, CI logs, reviewer items, hipDNN contract vs 14247, code quality. Flags whether a closer look is needed; not an approval or rejection. |
|
Automated overlap notice (rocKE merge monitor) Hi @dpeabody-amd, this PR and #14247 (also yours) each write the same cuDNN paged-graph checks:
The copies already differ in two ways:
Owner: #14247. Neither PR puts the checks in a shared home and the merge plan orders neither, so the PR that wrote them first owns them: #14247's are dated 21 October, this PR's 22 October. Resolution: #14247 moves the paged-graph contract checks, with one cause vocabulary (its keys above), into develop's Suggested order: the merge plan (develop Rebase, if #14247 lands first (checked on develop
On those trees the resolved If this PR lands first, take the owner's keys now, so #14247's move keeps every key this engine logs:
#14247 then lands second and does the move in its rebase: it resolves If you think this PR should own the checks instead, reply here and on #14247, and the reviews will follow the new decision. Automated overlap check: one owner per shared piece of code, coordinated on the PRs. Nothing else changed. |
1c25c6d to
5f4c910
Compare
|
Updated to 5f4c910, rebased onto develop. Replies to the inline threads are on each thread. @BrianHarrisonAMD, review comment on the tests claim Thanks. Every decline test now asserts the engine name, the cause and a part of the logged reason ( @cgmillette, first-round review
@cgmillette, overlap notice Thanks. I have taken the "if this PR lands first" path. The paged matcher now logs #14247's keys: |
|
Automated review: Model - qwen3.6:35b-a3b-coding-mtp-q4_K_M, effort: max PR #14254 — paged/ragged SDPA in Gfx1151WmmaAttention (dpeabody-amd, AICK-1847) — 🟠 Follow-up: head-96 spills, sinks, open threads
Utility: 🟢 Positive — paged and ragged SDPA for RDNA prefill; most earlier items fixed Legend: 🔴 blocking · 🟠 should-fix · 🟡 minor nit · 🟢 info/good · ⚪ retracted Follow-up review: first pass by a local qwen3.6 model, verified by the orchestrator (Claude Opus 5.5). Checks: range-diff, merge with today's develop, build and ctest, byte identity, library parity, both goldens, changed and installed tests, catalog compile sweep, descriptor regeneration, ABI layouts, saved probes, CI logs, reviewer items, code quality. Flags whether a closer look is needed; not an approval or rejection. |
…SdpaGraphMatch.hpp hipkernel:AttentionPagedUnified's graph_match carried its own copies of the SDPA match helpers that packs/SdpaGraphMatch.hpp now shares (findTensor, sdpaNode, hasSdpaForwardNode, describeOperand, the decline head and the axis constants). It now uses the header's, with sdpa_graph::DeclinedBy. Its local overflow-checked product is renamed spanProduct: unlike pack_features::checkedProduct it takes zero factors, which every extent of 1 gives. The stricter operand checks it needs move into the header too (usesOverrideShapes, hasNonNegativeStrides, declaresAlignment, isInt32Vector), together with the cuDNN paged-K/V graph contract checks (SEM-009), so every pack serving a paged graph applies one set of rules with one cause vocabulary, logged with the calling engine's name: - pagesKAndV: one_sided_paging; - pagedContainersAgree: block_size, shape (one block size, one block count; head counts and head sizes stay the engine's to check); - pageTablesFor: page_table (INT32 [B, 1, M, 1], unit entry stride, not packed; separate K and V tables share M and the batch stride); - sequenceLengthsFor: seq_len_kv, seq_len_q (both required, INT32 [B,1,1,1]); - raggedQueryFor, countsOffsetsInTokens: ragged (Q and O through one [B+1,1,1,1] offset table counted in tokens; K/V never ragged); - declaresServedAlignment: alignment (vector operands at the engine's vector width, INT32 operands at 4 bytes); - sinkFor: sinks, and alignment below the engine's own minimum sink alignment (16 bytes for this engine's kernels); - takesNoAttentionMask: attn_mask. The reasons they log are the texts hipkernel:Gfx1151WmmaAttention's paged matcher (#14254) logs for the same rules, so that engine can call these checks without changing what it reports. The decline lines go through HIPDNN_PLUGIN_LOG_INFO((DeclinedBy{...}) << ...): the braces' comma would otherwise split the macro argument. The engine keeps its kernel limits (head and block sizes, GQA ratios, strides and address ranges) and the order of its declines; it checks itself that V has K's head count and head size. A malformed page-table tensor now logs page_table rather than operand. The header's file comment covers paged graphs.
5f4c910 to
89ee88a
Compare
|
Automated review: Model - qwen3.6:35b-a3b-coding-mtp-q4_K_M, effort: max PR #14254 — paged/ragged SDPA in Gfx1151WmmaAttention (dpeabody-amd, AICK-1847) — 🟠 Follow-up: rebase; threads, spills, sinks open
No net-new findings. The push re-cut only the ABI bump; the quality lane's points sit in the N4, N7 and Code quality rows. Utility: 🟢 Positive — paged and ragged SDPA for RDNA prefill; the rebase kept every fix. Legend: 🔴 blocking · 🟠 should-fix · 🟡 minor nit · 🟢 info/good · ⚪ retracted Follow-up review: first pass by a local qwen3.6 model, verified by the orchestrator (Claude Opus 5.5). Checks: changes since |
…SdpaGraphMatch.hpp hipkernel:AttentionPagedUnified's graph_match carried its own copies of the SDPA match helpers that packs/SdpaGraphMatch.hpp now shares (findTensor, sdpaNode, hasSdpaForwardNode, describeOperand, the decline head and the axis constants). It now uses the header's, with sdpa_graph::DeclinedBy. Its local overflow-checked product is renamed spanProduct: unlike pack_features::checkedProduct it takes zero factors, which every extent of 1 gives. The stricter operand checks it needs move into the header too (usesOverrideShapes, hasNonNegativeStrides, declaresAlignment, isInt32Vector), together with the cuDNN paged-K/V graph contract checks (SEM-009), so every pack serving a paged graph applies one set of rules with one cause vocabulary, logged with the calling engine's name: - pagesKAndV: one_sided_paging; - pagedContainersAgree: block_size, shape (one block size, one block count; head counts and head sizes stay the engine's to check); - pageTablesFor: page_table (INT32 [B, 1, M, 1], unit entry stride, not packed; separate K and V tables share M and the batch stride); - sequenceLengthsFor: seq_len_kv, seq_len_q (both required, INT32 [B,1,1,1]); - raggedQueryFor, countsOffsetsInTokens: ragged (Q and O through one [B+1,1,1,1] offset table counted in tokens; K/V never ragged); - declaresServedAlignment: alignment (vector operands at the engine's vector width, INT32 operands at 4 bytes); - sinkFor: sinks, and alignment below the engine's own minimum sink alignment (16 bytes for this engine's kernels); - takesNoAttentionMask: attn_mask. The reasons they log are the texts hipkernel:Gfx1151WmmaAttention's paged matcher (#14254) logs for the same rules, so that engine can call these checks without changing what it reports. The decline lines go through HIPDNN_PLUGIN_LOG_INFO((DeclinedBy{...}) << ...): the braces' comma would otherwise split the macro argument. The engine keeps its kernel limits (head and block sizes, GQA ratios, strides and address ranges) and the order of its declines; it checks itself that V has K's head count and head size. A malformed page-table tensor now logs page_table rather than operand. The header's file comment covers paged graphs.
…SdpaGraphMatch.hpp hipkernel:AttentionPagedUnified's graph_match carried its own copies of the SDPA match helpers that packs/SdpaGraphMatch.hpp now shares (findTensor, sdpaNode, hasSdpaForwardNode, describeOperand, the decline head and the axis constants). It now uses the header's, with sdpa_graph::DeclinedBy. Its local overflow-checked product is renamed spanProduct: unlike pack_features::checkedProduct it takes zero factors, which every extent of 1 gives. The stricter operand checks it needs move into the header too (usesOverrideShapes, hasNonNegativeStrides, declaresAlignment, isInt32Vector), together with the cuDNN paged-K/V graph contract checks (SEM-009), so every pack serving a paged graph applies one set of rules with one cause vocabulary, logged with the calling engine's name: - pagesKAndV: one_sided_paging; - pagedContainersAgree: block_size, shape (one block size, one block count; head counts and head sizes stay the engine's to check); - pageTablesFor: page_table (INT32 [B, 1, M, 1], unit entry stride, not packed; separate K and V tables share M and the batch stride); - sequenceLengthsFor: seq_len_kv, seq_len_q (both required, INT32 [B,1,1,1]); - raggedQueryFor, countsOffsetsInTokens: ragged (Q and O through one [B+1,1,1,1] offset table counted in tokens; K/V never ragged); - declaresServedAlignment: alignment (vector operands at the engine's vector width, INT32 operands at 4 bytes); - sinkFor: sinks, and alignment below the engine's own minimum sink alignment (16 bytes for this engine's kernels); - takesNoAttentionMask: attn_mask. The reasons they log are the texts hipkernel:Gfx1151WmmaAttention's paged matcher (#14254) logs for the same rules, so that engine can call these checks without changing what it reports. The decline lines go through HIPDNN_PLUGIN_LOG_INFO((DeclinedBy{...}) << ...): the braces' comma would otherwise split the macro argument. The engine keeps its kernel limits (head and block sizes, GQA ratios, strides and address ranges) and the order of its declines; it checks itself that V has K's head count and head size. A malformed page-table tensor now logs page_table rather than operand. The header's file comment covers paged graphs.
yraparti
left a comment
There was a problem hiding this comment.
rocKE checklist review
ROCm/rocm-libraries#14254 · 89ee88af237d · model gemini-4-argon, fallback gpt-5.5
PASS · graded 238/238 · BLOCKER 0, HIGH 2, MEDIUM 0, LOW 0, NEEDS-INFO 15
The diff was truncated before grading. Items that needed a missing hunk are NEEDS-INFO.
Findings
- HIGH §B ·
dnn-providers/hip-kernel-provider/rocke/library/kernels/gfx1151/wmma_fmha_fwd_v2.py:123· InWmmaFmhaFwdV2Spec.__post_init__(dnn-providers/hip-kernel-provider/rocke/library/kernels/gfx1151/wmma_fmha_fwd_v2.py:168), validateif type(self.dense_q) is not bool or type(self.seqused_q) is not bool: raise TypeError(...)before checking layout compatibility. - HIGH §D ·
dnn-providers/hip-kernel-provider/rocke/library/kernels/gfx1151/wmma_fmha_fwd_v2.py:123· Exposedense_qandseqused_q(or the corresponding query layout/length presence) onAttentionRequestand wire the RDNA prefill spec builder to populate them from the request, or document a stacked PR deferral in the PR description.
Needs info
- §A.10 dnn-providers/hip-kernel-provider/rocke/platform/tests/instances/differential/golden/llvm_gfx_all.json is truncated at line 1040, so the new golden hash entries for indices 170..179 (QUERY_LENGTHS flag-on cases across LLVM flavors and arches) are not visible in the diff.
- §C.7 The diff is truncated and does not show enough of the large omitted generated/test/config hunks to verify whether any cleanup-only portion was proven as a no-op.
- §C.8 The diff adds a new helper/primitive, but it does not include evidence that current develop was grepped for an existing equivalent before adding it.
- §E.10 Merge-simulation against other open PRs touching the same files (e.g. abi.h, llvm_gfx_all.json, Gfx1151WmmaAttentionNative.cpp) cannot be verified from the PR diff alone.
- §F.6 Local pre-commit execution status cannot be verified from the diff alone.
- §F.7 Bot, Copilot, and human review thread states are not included in the prompt.
- §F.11 While the provided title and body are populated in the prompt metadata, there is no diff path:line to cite for API field verification.
- §G.2 On-device numerical tests with fp64/fp32 references and bf16/fp16 tolerances are added (e.g. dnn-providers/hip-kernel-provider/rocke/library/tests/test_wmma_fmha_fwd_v2_features.py:643,685 and dnn-providers/hip-kernel-provider/src/integration_tests/kernel_ingestor_engine/IntegrationGpuGfx1151WmmaAttentionPaged.cpp:867), but the PR body is truncated before the test execution summary so on-silicon pass confirmation is missing.
Section notes
- §A: Two-engine parity and byte-identity verified across Python and C++ kernels, helpers, parity emitters, bindings, and goldens.
- §C: Diff is truncated; items that require omitted hunks or non-diff evidence are marked NEEDS-INFO.
- §D: Items 1–11 of 21 in Section D.
- §D: Items 12-21 of Section D
- §G: Items 1-17 of 34 in section G.
- §G: Items 18-34 of 34.
Graded with the rocKE PR checklist, one model pass per section. FAIL rows that sit on a diff line are inline comments.
§A graded 14/14
| # | Verdict | Evidence |
|---|---|---|
| 1 | PASS | dnn-providers/hip-kernel-provider/rocke/library/kernels/gfx1151/wmma_fmha_fwd_v2.py:457 and dnn-providers/hip-kernel-provider/rocke/platform/cpp/instances/gfx1151/wmma_fmha_fwd_v2.cpp:408 keep the Python and C++ kernel-body emitters in sync (along with wmma_fmha_decode.py:748 and wmma_fmha_decode.cpp:1096). |
| 2 | N/A | No core lowering-op or op-handler files (core/lower_*, core/ir) are modified in this PR. |
| 3 | PASS | dnn-providers/hip-kernel-provider/rocke/library/kernels/gfx1151/wmma_fmha_fwd_v2.py:123, dnn-providers/hip-kernel-provider/rocke/platform/cpp/include/rocke/instance_gfx1151_wmma_fmha_fwd_v2.h:63, dnn-providers/hip-kernel-provider/rocke/platform/cpp/bindings/rocke_engine.cpp:4650, and dnn-providers/hip-kernel-provider/rocke/platform/cpp/bindings/prove_parity_binding.py:813 mirror the new spec fields and parity binding cases. |
| 4 | PASS | dnn-providers/hip-kernel-provider/rocke/platform/cpp/helpers/rdna_attention.cpp:220 and dnn-providers/hip-kernel-provider/rocke/platform/cpp/instances/gfx1151/wmma_fmha_fwd_v2.cpp:473 bind operands left-to-right to temporaries to match the exact emit order in Python (rdna_attention.py:219, wmma_fmha_fwd_v2.py:536). |
| 5 | PASS | dnn-providers/hip-kernel-provider/rocke/platform/tests/instances/differential/golden/llvm_gfx_all.json:1016 re-blesses only the packed/paged gfx1151_wmma_fmha_fwd_v2 entries affected by the intentional IR changes while leaving dense/unrelated entries untouched. |
| 6 | PASS | dnn-providers/hip-kernel-provider/rocke/library/tests/parity/gfx1151_wmma_fmha_fwd_v2_emit.c:101, dnn-providers/hip-kernel-provider/rocke/library/tests/parity/gfx1151_wmma_fmha_fwd_v2_emit.py:93, and dnn-providers/hip-kernel-provider/rocke/platform/cpp/bindings/prove_parity_binding.py:813 add cross-engine parity cases alongside the golden updates. |
| 7 | PASS | dnn-providers/hip-kernel-provider/rocke/library/kernels/gfx1151/wmma_fmha_fwd_v2.py:504 and dnn-providers/hip-kernel-provider/rocke/platform/python/rocke/helpers/rdna_attention.py:818 guard new ops behind layout != 'dense', spec.dense_q, spec.seqused_q, and live_rows is not None so covered dense shapes emit identical IR. |
| 8 | PASS | dnn-providers/hip-kernel-provider/rocke/platform/cpp/helpers/rdna_attention.cpp:702 and dnn-providers/hip-kernel-provider/rocke/platform/cpp/instances/gfx1151/wmma_fmha_fwd_v2.cpp:410 were reviewed against their Python twins and wired into gfx1151_wmma_fmha_fwd_v2_emit.c:235. |
| 9 | N/A | No shared core op-handler fix or lowerer-only arch divergence is introduced in this PR. |
| 10 | NEEDS-INFO | dnn-providers/hip-kernel-provider/rocke/platform/tests/instances/differential/golden/llvm_gfx_all.json is truncated at line 1040, so the new golden hash entries for indices 170..179 (QUERY_LENGTHS flag-on cases across LLVM flavors and arches) are not visible in the diff. |
| 11 | N/A | No new IR opcodes are added in this PR. |
| 12 | N/A | core/backend.py and ROCKE_BACKEND lowering fallback logic are not touched in this PR. |
| 13 | N/A | LLVM .ll emission format and ROCKE_DEBUG_LOC handling are not modified in this PR. |
| 14 | PASS | dnn-providers/hip-kernel-provider/rocke/platform/python/rocke/helpers/rdna_attention.py:213, dnn-providers/hip-kernel-provider/rocke/platform/cpp/helpers/rdna_attention.cpp:214, and dnn-providers/hip-kernel-provider/rocke/platform/cpp/include/rocke/helper_rocke.helpers.rdna_attention.h:86 update the Python helper and its C++ twin and header in the same PR. |
§B graded 6/6
| # | Verdict | Evidence |
|---|---|---|
| 1 | PASS | New rejections in dnn-providers/hip-kernel-provider/src/engines/kernel_ingestor_engine/packs/Gfx1151WmmaAttentionNative.cpp:848-888 (unservedFeature) and Gfx1151WmmaAttentionNative.cpp:1080-1324 (gfx1151WmmaAttentionPagedGraphMatches), as well as dnn-providers/hip-kernel-provider/rocke/library/dispatch/attention/bindings.py:683-705 (_check_page_extent_i32), clearly classify and log hardware/kernel limits vs unserved graph contracts. |
| 2 | PASS | WmmaFmhaFwdV2Spec.__post_init__ in dnn-providers/hip-kernel-provider/rocke/library/kernels/gfx1151/wmma_fmha_fwd_v2.py:168-173 and C++ v2_post_init_ok in dnn-providers/hip-kernel-provider/rocke/platform/cpp/instances/gfx1151/wmma_fmha_fwd_v2.cpp:144-154 reject the exact same dense_q and seqused_q layout combinations. |
| 3 | PASS | No shapes are demoted to a slow fallback; paged graphs are routed to the prefill engine (dnn-providers/hip-kernel-provider/src/engines/kernel_ingestor_engine/packs/Gfx1151WmmaAttentionNative.cpp:1339-1342) and unsupported graph variants are loudly declined at INFO (Gfx1151WmmaAttentionNative.cpp:836-839). |
| 4 | FAIL | dnn-providers/hip-kernel-provider/rocke/library/kernels/gfx1151/wmma_fmha_fwd_v2.py:123-124 and 168-173 add public spec flags dense_q: bool = False and seqused_q: bool = False that change the kernel parameter ABI (wmma_fmha_fwd_v2.py:416-425), but WmmaFmhaFwdV2Spec.__post_init__ does not enforce type(self.dense_q) is bool or type(self.seqused_q) is bool, allowing truthy non-bools (e.g. "false") to silently enable the ABI. |
| 5 | PASS | Newly opened gates (dense_q, seqused_q, separate block_table_v, and >2 GiB paged prefill K/V caches) are tested in dnn-providers/hip-kernel-provider/rocke/library/tests/test_wmma_fmha_fwd_v2_features.py:168-225, test_wmma_fmha_fwd_v2_features.py:685-855, dnn-providers/hip-kernel-provider/rocke/library/tests/dispatch/attention/test_gfx1151_binding_i32_guard.py:119-175, and… |
| 6 | PASS | Spec validation in dnn-providers/hip-kernel-provider/rocke/library/kernels/gfx1151/wmma_fmha_fwd_v2.py:168-173 and dnn-providers/hip-kernel-provider/rocke/platform/cpp/instances/gfx1151/wmma_fmha_fwd_v2.cpp:144-154 only rejects functionally invalid layout combinations (dense_q outside paged, seqused_q on dense). |
§C graded 10/10
| # | Verdict | Evidence |
|---|---|---|
| 1 | PASS | dnn-providers/hip-kernel-provider/rocke/platform/python/rocke/helpers/rdna_attention.py:210 defines the shared rdna_attention_scaled_ptr helper, and dnn-providers/hip-kernel-provider/rocke/library/kernels/gfx1151/wmma_fmha_decode.py:745 uses it instead of repeated local pointer-scaling logic. |
| 2 | PASS | dnn-providers/hip-kernel-provider/rocke/library/kernels/gfx1151/wmma_fmha_fwd_v2.py:457 keeps the single public build_wmma_fmha_fwd_v2(spec) builder while adding the new behavior through spec fields declared at dnn-providers/hip-kernel-provider/rocke/library/kernels/gfx1151/wmma_fmha_fwd_v2.py:120. |
| 3 | PASS | dnn-providers/hip-kernel-provider/rocke/library/kernels/gfx1151/wmma_fmha_fwd_v2.py:210 adds kernel-name suffixes for codegen/layout options (_denseq and _sequsedq), not workload sizes. |
| 4 | PASS | dnn-providers/hip-kernel-provider/rocke/platform/python/rocke/helpers/rdna_attention.py:210 and dnn-providers/hip-kernel-provider/rocke/platform/cpp/helpers/rdna_attention.cpp:211 put the arch-independent pointer-scaling operation in shared helpers rather than duplicating it per arch instance. |
| 5 | PASS | dnn-providers/hip-kernel-provider/src/engines/kernel_ingestor_engine/packs/Gfx1151WmmaAttentionNative.cpp:58 documents the scope as the existing Gfx1151WmmaAttention/Gfx1151WmmaAttentionDecode engines, and dnn-providers/hip-kernel-provider/rocke/library/dispatch/attention/README.md:515 confines the documented routing changes to paged prefill versus dense decode. |
| 6 | PASS | dnn-providers/hip-kernel-provider/rocke/platform/python/rocke/helpers/rdna_attention.py:210 adds a named helper first, with the C++ twin at dnn-providers/hip-kernel-provider/rocke/platform/cpp/helpers/rdna_attention.cpp:211, instead of ad-hoc pointer math at each call site. |
| 7 | NEEDS-INFO | The diff is truncated and does not show enough of the large omitted generated/test/config hunks to verify whether any cleanup-only portion was proven as a no-op. |
| 8 | NEEDS-INFO | The diff adds a new helper/primitive, but it does not include evidence that current develop was grepped for an existing equivalent before adding it. |
| 9 | PASS | dnn-providers/hip-kernel-provider/rocke/library/kernels/gfx1151/wmma_fmha_decode.py:604 updates the decode sibling to use the shared helper, and dnn-providers/hip-kernel-provider/rocke/platform/cpp/instances/gfx1151/wmma_fmha_decode.cpp:1094 updates the C++ twin as well. |
| 10 | PASS | dnn-providers/hip-kernel-provider/rocke/platform/cpp/include/rocke/abi.h:98 documents the widened ABI structs and bumps ROCKE_ABI_VERSION, while dnn-providers/hip-kernel-provider/rocke/platform/python/rocke/portable_ir/src/abi.py:75 bumps the Python binary ABI mirror. |
§D graded 21/21
| # | Verdict | Evidence |
|---|---|---|
| 1 | PASS | New query layout options dense_q and seqused_q are declared as explicit WmmaFmhaFwdV2Spec fields with validation, name tags, and cache-key entries without reading os.environ (dnn-providers/hip-kernel-provider/rocke/library/kernels/gfx1151/wmma_fmha_fwd_v2.py:123, dnn-providers/hip-kernel-provider/rocke/library/kernels/gfx1151/wmma_fmha_fwd_v2.py:213, dnn-providers/hip-kernel-provider/rocke/library/dispatch/attention/common.py:997). |
| 2 | N/A | No Capability declarations or candidate arches lists are added or modified in this PR. |
| 3 | N/A | No _supports predicates or admits call sites are added or modified in this PR. |
| 4 | N/A | No Python dispatcher selector or spec-builder routing logic on the selection path is modified in this PR. |
| 5 | PASS | dense_q and seqused_q are baked into kernel_name() (dnn-providers/hip-kernel-provider/rocke/library/kernels/gfx1151/wmma_fmha_fwd_v2.py:213) and included in the spec cache key / identity() when enabled while preserving existing hashes when off (dnn-providers/hip-kernel-provider/rocke/library/dispatch/attention/common.py:997, dnn-providers/hip-kernel-provider/rocke/library/tests/dispatch/attention/test_gfx1151_attention_selection.py:900). |
| 6 | N/A | No dispatcher cohort routing is added or modified in this PR. |
| 7 | N/A | No heuristic tables or hand-pinned overrides in the Python dispatcher are modified in this PR. |
| 8 | N/A | No cohort definitions or spec override predicates are added or modified in this PR. |
| 9 | N/A | No new KernelCandidate or architecture is registered in the Python dispatcher registries. |
| 10 | FAIL | dense_q and seqused_q are added to WmmaFmhaFwdV2Spec (dnn-providers/hip-kernel-provider/rocke/library/kernels/gfx1151/wmma_fmha_fwd_v2.py:123) and classified under WMMA_PROBLEM_FIELDS as coming from the request (dnn-providers/hip-kernel-provider/rocke/library/dispatch/attention/axes.py:346), but common.py explicitly notes dispatch never sets them (dnn-providers/hip-kernel-provider/rocke/library/dispatch/attention/common.py:991) and tests must manually construct WmmaFmhaFwdV2Spec(dense_q=True, seqused_q=True) instead of reaching them from AttentionRequest (dnn-providers/hip-kernel-provider/rocke/library/tests/dispatch/attention/test_gfx1151_attention_dispatch_numeric.py:2165). |
| 11 | N/A | No new dispatcher candidates are registered or promoted in the Python dispatcher registries. |
| 12 | N/A | No dispatcher candidates, priorities, candidate names, or custom rankers are added or modified in the Python dispatcher registry. |
| 13 | N/A | No new dispatcher wrapper or entry point is added, and pinned request handling in CandidateRegistry.select is not modified. |
| 14 | PASS | Identity/hash preservation for off-by-default spec fields (dense_q, seqused_q) and differentiation when enabled are maintained in dnn-providers/hip-kernel-provider/rocke/library/dispatch/attention/common.py:990-1000 and verified in dnn-providers/hip-kernel-provider/rocke/library/tests/dispatch/attention/test_gfx1151_attention_selection.py:890-925, and C/Python ABI versions are bumped in… |
| 15 | N/A | No new dispatcher candidate or arch module is added to the Python dispatcher registry. |
| 16 | N/A | No new operator family is introduced in this PR. |
| 17 | PASS | Every new WmmaFmhaFwdV2Spec field (dense_q, seqused_q) is classified in WMMA_PROBLEM_FIELDS at dnn-providers/hip-kernel-provider/rocke/library/dispatch/attention/axes.py:346-347 and tested for spec/hash invariants in dnn-providers/hip-kernel-provider/rocke/library/tests/dispatch/attention/test_gfx1151_attention_selection.py:887-925. |
| 18 | N/A | Registry ratchets (require_build, require_binding, require_torch_binding) and exempt sets are not modified in this PR. |
| 19 | N/A | This PR does not touch run_unified_attention_torch or claim unified attention auto-routing registration. |
| 20 | PASS | Spec and builder changes in WmmaFmhaFwdV2Spec (dnn-providers/hip-kernel-provider/rocke/library/kernels/gfx1151/wmma_fmha_fwd_v2.py:123-124) are mirrored in C++ (dnn-providers/hip-kernel-provider/rocke/platform/cpp/include/rocke/instance_gfx1151_wmma_fmha_fwd_v2.h:63-64, dnn-providers/hip-kernel-provider/rocke/platform/cpp/instances/gfx1151/wmma_fmha_fwd_v2.cpp:144-154) and parity tests… |
| 21 | N/A | This PR adds kernel/spec features (dense_q, seqused_q, block_table_v, 64-bit page addressing) and hipDNN native matcher support rather than working around a selection bug in the emitter. |
§E graded 17/17
| # | Verdict | Evidence |
|---|---|---|
| 1 | PASS | dnn-providers/hip-kernel-provider/rocke/platform/python/rocke/helpers/rdna_attention.py:48 imports only from rocke.core.ir, while library kernels import from platform (dnn-providers/hip-kernel-provider/rocke/library/kernels/gfx1151/wmma_fmha_fwd_v2.py:460). |
| 2 | PASS | dnn-providers/hip-kernel-provider/rocke/library/tests/test_wmma_fmha_fwd_v2_features.py:699 uses rocke.runtime.hip_module.Runtime rather than importing torch. |
| 3 | PASS | dnn-providers/hip-kernel-provider/rocke/library/tests/test_wmma_fmha_fwd_v2_features.py:690 gates optional dependencies with pytest.importorskip and uses numpy/Runtime without requiring torch. |
| 4 | PASS | dnn-providers/hip-kernel-provider/rocke/library/kernels/gfx1151/wmma_fmha_fwd_v2.py:459-463 and dnn-providers/hip-kernel-provider/rocke/library/dispatch/attention/bindings.py:608 add no upward imports across library layers. |
| 5 | PASS | dnn-providers/hip-kernel-provider/rocke/platform/python/rocke/helpers/rdna_attention.py:213-220 and dnn-providers/hip-kernel-provider/rocke/platform/cpp/bindings/prove_parity_binding.py:809-852 introduce no absolute or repo-escaping paths or shell scripts. |
| 6 | PASS | dnn-providers/hip-kernel-provider/rocke/library/kernels/gfx1151/wmma_fmha_fwd_v2.py:556-564 and dnn-providers/hip-kernel-provider/rocke/platform/tests/instances/differential/golden/llvm_gfx_all.json:1016-1036 show intentional emitter edits paired with golden updates rather than lint autofix removals. |
| 7 | PASS | dnn-providers/hip-kernel-provider/rocke/platform/dsl_docs/instances/attention.md:38-41 and dnn-providers/hip-kernel-provider/rocke/library/dispatch/attention/README.md:492-527 update the instance and dispatch reference docs alongside the new dense_q, seqused_q, and block_table_v features. |
| 8 | PASS | dnn-providers/hip-kernel-provider/rocke/library/kernels/gfx1151/wmma_fmha_fwd_v2.py:33-57 and dnn-providers/hip-kernel-provider/src/engines/kernel_ingestor_engine/packs/Gfx1151WmmaAttentionNative.cpp:82-114 use descriptive spec/RFC references without Jira IDs, PR numbers, or user handles in code. |
| 9 | PASS | dnn-providers/hip-kernel-provider/rocke/library/kernels/gfx1151/wmma_fmha_fwd_v2.py:123-124 and dnn-providers/hip-kernel-provider/src/engines/kernel_ingestor_engine/packs/Gfx1151WmmaAttentionNative.cpp:761-764 keep all changes scoped to paged/ragged SDPA support in Gfx1151WmmaAttention and its rocKE kernel/test plumbing. |
| 10 | NEEDS-INFO | Merge-simulation against other open PRs touching the same files (e.g. abi.h, llvm_gfx_all.json, Gfx1151WmmaAttentionNative.cpp) cannot be verified from the PR diff alone. |
| 11 | PASS | dnn-providers/hip-kernel-provider/rocke/platform/dsl_docs/instances/attention.md:38-41 and dnn-providers/hip-kernel-provider/rocke/library/dispatch/attention/README.md:492-500 are required deliverable docs; no scratch notes or throwaway markdown files are committed. |
| 12 | PASS | dnn-providers/hip-kernel-provider/rocke/library/tests/parity/gfx1151_wmma_fmha_fwd_v2_emit.py:225-227 and dnn-providers/hip-kernel-provider/rocke/library/tests/test_wmma_fmha_fwd_v2_spec.py:263 cover both gfx11-generic and gfx12-generic RDNA families (CDNA is scoped out by kernel family). |
| 13 | PASS | dnn-providers/hip-kernel-provider/rocke/library/kernels/gfx1151/wmma_fmha_fwd_v2.py:548-560 and dnn-providers/hip-kernel-provider/rocke/platform/cpp/instances/gfx1151/wmma_fmha_fwd_v2.cpp:500-515 replace page_rsrc with page_head and update k_rsrc/v_rsrc paged usage consistently across Python, C++, headers, and docs. |
| 14 | PASS | dnn-providers/hip-kernel-provider/rocke/platform/cpp/include/rocke/abi.h:101-105 and dnn-providers/hip-kernel-provider/rocke/library/kernels/gfx1151/wmma_fmha_decode.py:748-774 preserve upstream history and comments while surgical additions/refactorings are applied. |
| 15 | PASS | dnn-providers/hip-kernel-provider/rocke/platform/python/rocke/helpers/rdna_attention.py:263-270 and 574-627, plus dnn-providers/hip-kernel-provider/rocke/platform/cpp/helpers/rdna_attention.cpp:713-815, use all newly added parameters (block_table_v, k_ptr, v_ptr, live_rows). |
| 16 | N/A | This item governs reviewer severity labeling and does not apply when there are no FAIL rows in this section. |
| 17 | PASS | dnn-providers/hip-kernel-provider/rocke/library/dispatch/attention/README.md:489-527 and the PR description contain no absolute or relative performance claims. |
§F graded 11/11
| # | Verdict | Evidence |
|---|---|---|
| 1 | PASS | branch users/dpeabody/hipdnn-rdna-paged-attn uses the users// prefix |
| 2 | PASS | title feat(hip-kernel-provider): paged/ragged SDPA in Gfx1151WmmaAttention matches type(scope): summary and is 68 characters |
| 3 | PASS | description is at least 30 characters and contains an issue reference |
| 4 | PASS | test file dnn-providers/hip-kernel-provider/rocke/library/tests/dispatch/attention/test_gfx1151_attention_dispatch_numeric.py is in the diff |
| 5 | PASS | The changed file list contains only source, test, documentation, and descriptor/golden files (e.g., dnn-providers/hip-kernel-provider/rocke/library/dispatch/attention/README.md:492) and no keys, certs, or secrets. |
| 6 | NEEDS-INFO | Local pre-commit execution status cannot be verified from the diff alone. |
| 7 | NEEDS-INFO | Bot, Copilot, and human review thread states are not included in the prompt. |
| 8 | N/A | This PR targets develop directly rather than being a stacked PR on a feature branch. |
| 9 | PASS | The PR keeps the full implementation, catalog, and test suite together in a single change (e.g., dnn-providers/hip-kernel-provider/src/integration_tests/kernel_ingestor_engine/IntegrationGpuGfx1151WmmaAttentionPaged.cpp:1) without artificial splitting. |
| 10 | PASS | Tests added in the diff exercise real spec validation, dispatch binding, parity, matcher, and GPU numeric/integration behavior (e.g., dnn-providers/hip-kernel-provider/rocke/library/tests/dispatch/attention/test_gfx1151_attention_dispatch_numeric.py:2130 and dnn-providers/hip-kernel-provider/src/tests/engines/kernel_ingestor_engine/packs/TestGfx1151WmmaAttentionMatchers.cpp:735). |
| 11 | NEEDS-INFO | While the provided title and body are populated in the prompt metadata, there is no diff path:line to cite for API field verification. |
§G graded 34/34
| # | Verdict | Evidence |
|---|---|---|
| 1 | PASS | Parity rows in dnn-providers/hip-kernel-provider/rocke/library/tests/parity/gfx1151_wmma_fmha_fwd_v2_emit.py:93 are paired with float64/CPU on-device numerical tests in dnn-providers/hip-kernel-provider/rocke/library/tests/test_wmma_fmha_fwd_v2_features.py:168 and dnn-providers/hip-kernel-provider/src/integration_tests/kernel_ingestor_engine/IntegrationGpuGfx1151WmmaAttentionPaged.cpp:324. |
| 2 | NEEDS-INFO | On-device numerical tests with fp64/fp32 references and bf16/fp16 tolerances are added (e.g. dnn-providers/hip-kernel-provider/rocke/library/tests/test_wmma_fmha_fwd_v2_features.py:643,685 and dnn-providers/hip-kernel-provider/src/integration_tests/kernel_ingestor_engine/IntegrationGpuGfx1151WmmaAttentionPaged.cpp:867), but the PR body is truncated before the test execution summary so on-silicon pass confirmation is missing. |
| 3 | NEEDS-INFO | Golden hashes are updated in dnn-providers/hip-kernel-provider/rocke/platform/tests/instances/differential/golden/llvm_gfx_all.json:1016, but the PR description is truncated in the Tests section so local gate sweep results (tools/run_checks.py, llvm23 byte-identity, check_ir_validity.py, pytest suites) cannot be verified from the diff alone. |
| 4 | PASS | Every new capability has targeted tests that fail without the change: dense_q/seqused_q/split block tables/clamped seqused_k (dnn-providers/hip-kernel-provider/rocke/library/tests/test_wmma_fmha_fwd_v2_features.py:168-225), >4 GiB paged cache (dnn-providers/hip-kernel-provider/rocke/library/tests/test_wmma_fmha_fwd_v2_features.py:685), binding page-extent I32 checks… |
| 5 | PASS | Tests exercise the shipped entry points (e.g. dispatch_attention in dnn-providers/hip-kernel-provider/rocke/library/tests/dispatch/attention/test_gfx1151_attention_selection.py:895, GraphMatchRegistry in dnn-providers/hip-kernel-provider/src/tests/engines/kernel_ingestor_engine/packs/Gfx1151WmmaAttentionTestSupport.hpp:149) and pin boundary gates directly with exact pass/fail thresholds… |
| 6 | PASS | Prose claims in PR body, docstrings, and code comments match the implementation and tests: 64-bit page/head addressing and page-extent I32 bound (dnn-providers/hip-kernel-provider/rocke/library/dispatch/attention/bindings.py:683, dnn-providers/hip-kernel-provider/src/engines/kernel_ingestor_engine/packs/Gfx1151WmmaAttentionNative.cpp:428), seqused_q padding writing O=0 and LSE=-inf… |
| 7 | N/A | This PR does not add or modify a design document defining a new fp32 parity reference oracle. |
| 8 | PASS | Test harnesses do not silently fall back on unsupported dtypes or frontend configurations; they explicitly skip via pytest.importorskip or GTEST_SKIP with a reason (dnn-providers/hip-kernel-provider/rocke/library/tests/test_wmma_fmha_fwd_v2_features.py:690, dnn-providers/hip-kernel-provider/src/integration_tests/kernel_ingestor_engine/IntegrationGpuGfx1151WmmaAttentionPaged.cpp:749-767). |
| 9 | N/A | This PR does not make new LDS or occupancy load-bearing numerical claims. |
| 10 | N/A | This PR is a functional feature addition (paged/ragged SDPA support) and makes no performance or latency claims. |
| 11 | N/A | This PR adds functional graph support for hipDNN paged SDPA rather than an optimization claim for a benchmarked shape. |
| 12 | PASS | Cross-engine parity is maintained across Python and C++ for both the helper (dnn-providers/hip-kernel-provider/rocke/platform/python/rocke/helpers/rdna_attention.py:213 and dnn-providers/hip-kernel-provider/rocke/platform/cpp/helpers/rdna_attention.cpp:214) and instance builder (dnn-providers/hip-kernel-provider/rocke/library/kernels/gfx1151/wmma_fmha_fwd_v2.py:123 and… |
| 13 | N/A | This PR does not introduce or recommend any new raw LLVM intrinsic mangled symbols. |
| 14 | N/A | This PR makes no hardware bottleneck classifications. |
| 15 | N/A | This is a review of the provided PR diff directly from the head commit, not a multi-worktree branch inspection task. |
| 16 | N/A | This PR implements a completed feature rather than proposing an unverified plan. |
| 17 | N/A | This PR does not add new standalone shipped CLI scripts. |
| 18 | PASS | New host-side guard _check_page_extent_i32 in dnn-providers/hip-kernel-provider/rocke/library/dispatch/attention/bindings.py:683 is wired into _rdna_validate_and_collect (bindings.py:908), documents its scope (page and head-dim axes fit I32 while block and head steps are 64-bit), and is tested on both passing and failing boundary shapes in… |
| 19 | PASS | Fixtures in dnn-providers/hip-kernel-provider/rocke/library/tests/test_wmma_fmha_fwd_v2_features.py:531 and dnn-providers/hip-kernel-provider/src/integration_tests/kernel_ingestor_engine/IntegrationGpuGfx1151WmmaAttentionPaged.cpp:163 allocate spare unreferenced pages (sum(pages) + 3 / blocks + 5) and extra table columns poisoned with out-of-range block IDs and NaN bytes so misplaced reads or writes are caught. |
| 20 | N/A | This PR does not introduce or modify a claim-ban detector or word/digit regex linter. |
| 21 | N/A | This PR does not add or modify performance measurement or benchmarking scripts. |
| 22 | NEEDS-INFO | dnn-providers/hip-kernel-provider/rocke/platform/tests/instances/differential/golden/llvm_gfx_all.json:1016 is truncated in the diff, so we cannot verify from the diff alone that all new golden entries (indices 170..179 from gfx1151_wmma_fmha_fwd_v2_emit.py:29) record bytes + sha256 without build errors across all flavors, nor can we see CI execution logs confirming the C++ parity and GPU integration harnesses ran without skipping. |
| 23 | N/A | This PR is a functional feature addition for paged/ragged SDPA and makes no performance speedup claims. |
| 24 | PASS | The re-blessed paged entries in dnn-providers/hip-kernel-provider/rocke/platform/tests/instances/differential/golden/llvm_gfx_all.json:1016 correspond to the 64-bit page/head addressing, seqused_k clamping, and block_table_v changes in dnn-providers/hip-kernel-provider/rocke/library/kernels/gfx1151/wmma_fmha_fwd_v2.py:510, which are GPU-validated across the affected shapes in… |
| 25 | N/A | This PR adds functional support for hipDNN paged/ragged SDPA graphs and 64-bit cache addressing rather than a codegen micro-optimization. |
| 26 | N/A | This PR does not include benchmark comparisons or performance claims. |
| 27 | N/A | This PR does not include performance A/B experiments. |
| 28 | N/A | This PR does not present performance delta tables or counterfactual benchmark arms. |
| 29 | N/A | This PR does not swap guarded math primitives for fast approximations. |
| 30 | N/A | This PR makes no register-spill or scratch-reduction claims. |
| 31 | N/A | This PR does not claim to fix a performance defect against a baseline measurement. |
| 32 | N/A | This PR does not report measured performance or resource numbers. |
| 33 | NEEDS-INFO | dnn-providers/hip-kernel-provider/src/engines/kernel_ingestor_engine/packs/Gfx1151WmmaAttentionNative.cpp:1339 and dnn-providers/hip-kernel-provider/src/tests/engines/kernel_ingestor_engine/packs/TestGfx1151WmmaAttentionPaged.cpp are truncated/omitted in the diff, so the changed routing decision logic between prefill and decode on dense vs paged graphs cannot be fully inspected or verified against a differential table. |
| 34 | PASS | In dnn-providers/hip-kernel-provider/rocke/platform/python/rocke/helpers/rdna_attention.py:1060 (and C++ mirror rdna_attention.cpp:1690), empty and o_row_off for O are computed independently of lse_rsrc, and both O and LSE are tested on edge rows (0 live queries, 0 keys, padding rows past seqused_q, with and without sinks) in dnn-providers/hip-kernel-provider/rocke/library/tests/test_wmma_fmha_fwd_v2_features.py:646 and… |
§H graded 5/5
| # | Verdict | Evidence |
|---|---|---|
| 1 | PASS | Buffer SRDs and 64-bit page bases are sized from runtime tensor shapes/strides and host-validated before launch via _check_page_extent_i32 / _check_byte_extent_i32 in dnn-providers/hip-kernel-provider/rocke/library/dispatch/attention/bindings.py:683-705 and pageFitsI32 / isWellFormedRaggedOperand in dnn-providers/hip-kernel-provider/src/engines/kernel_ingestor_engine/packs/Gfx1151WmmaAttentionNative.cpp:428-492. |
| 2 | PASS | Both block_table and block_table_v indirection values (phys_k, phys_v, phys) are explicitly range-checked on device by page_ok (0 <= phys < num_blocks) before 64-bit base pointer offsets are formed, clamping out-of-range page IDs to 0 and zeroing the loaded data in dnn-providers/hip-kernel-provider/rocke/platform/python/rocke/helpers/rdna_attention.py:554-616 and… |
| 3 | PASS | Power-of-two page_block_size is enforced in the spec (ps <= 0 or ps & (ps - 1) in dnn-providers/hip-kernel-provider/rocke/library/kernels/gfx1151/wmma_fmha_fwd_v2.py:164-165) and in the native matcher ((page & (page - 1)) != 0 in dnn-providers/hip-kernel-provider/src/engines/kernel_ingestor_engine/packs/Gfx1151WmmaAttentionNative.cpp:1151-1156), lowering hot-loop page div/mod to lshr and land in… |
| 4 | PASS | The kernel loops over runtime n_tiles = cdiv(smax(sk, 0), BN) where sk = smin(seqused_k[b], seqlen_k) is clamped in dnn-providers/hip-kernel-provider/rocke/library/kernels/gfx1151/wmma_fmha_fwd_v2.py:521 (and C++ twin at dnn-providers/hip-kernel-provider/rocke/platform/cpp/instances/gfx1151/wmma_fmha_fwd_v2.cpp:439-440), while live_rows = smin(seqused_q[b], span) masks padding query rows to skip the K loop and write O=0, LSE=-inf in… |
| 5 | PASS | Host validations for dense_q, seqused_q, paged K/V extents, and page_table_k/page_table_v/seq_len_q/seq_len_kv are in dnn-providers/hip-kernel-provider/rocke/library/dispatch/attention/bindings.py:897-915, 1047-1114 and dnn-providers/hip-kernel-provider/src/engines/kernel_ingestor_engine/packs/Gfx1151WmmaAttentionNative.cpp:1085-1305, paired with negative unit tests in… |
§I graded 6/6
| # | Verdict | Evidence |
|---|---|---|
| 1 | N/A | This PR adds paged/ragged SDPA kernel features and hipDNN native graph matching rather than introducing a tuning or routing search benchmark. |
| 2 | PASS | Invalid spec combinations and unsupported graph features are loudly rejected with descriptive errors or logged causes (e.g., dnn-providers/hip-kernel-provider/rocke/library/kernels/gfx1151/wmma_fmha_fwd_v2.py:168, dnn-providers/hip-kernel-provider/src/engines/kernel_ingestor_engine/packs/Gfx1151WmmaAttentionNative.cpp:836). |
| 3 | PASS | Parity and feature tests cover curated combinations of the new dense_q, seqused_q, and split block-table axes (dnn-providers/hip-kernel-provider/rocke/library/tests/parity/gfx1151_wmma_fmha_fwd_v2_emit.py:93, dnn-providers/hip-kernel-provider/rocke/library/tests/test_wmma_fmha_fwd_v2_features.py:168). |
| 4 | N/A | This PR does not add or modify benchmark reporting tables comparing shipped vs. achievable selection performance. |
| 5 | PASS | Generated catalog descriptors (kdp, kmd, ued) are updated from their YAML source config (dnn-providers/hip-kernel-provider/src/engines/kernel_ingestor_engine/descriptors/rocKE/gfx1151_wmma_attention/gfx1151_wmma_attention.kmd.json:42), and off-by-default spec fields preserve existing tuning cache hashes (dnn-providers/hip-kernel-provider/rocke/library/dispatch/attention/common.py:996). |
| 6 | PASS | The new helper rdna_attention_scaled_ptr exposes a general 64-bit scaled pointer arithmetic primitive rather than a hardware-instruction-specific op (dnn-providers/hip-kernel-provider/rocke/platform/python/rocke/helpers/rdna_attention.py:213, dnn-providers/hip-kernel-provider/rocke/platform/cpp/helpers/rdna_attention.cpp:214). |
§J graded 7/7
| # | Verdict | Evidence |
|---|---|---|
| 1 | N/A | The Markdown changes in README.md and attention.md update dispatch contract prose and instance bullet lists without introducing mathematical equations or matrix-multiplication orientation tables. |
| 2 | PASS | Symbols introduced in dnn-providers/hip-kernel-provider/rocke/library/dispatch/attention/README.md:492-521 and dnn-providers/hip-kernel-provider/rocke/platform/dsl_docs/instances/attention.md:38-41 (block_table_v, dense_q, seqused_q, use_sinks/sinks) are used with a single consistent meaning and match the code and spec definitions. |
| 3 | PASS | dnn-providers/hip-kernel-provider/rocke/platform/dsl_docs/instances/attention.md:38-41 and dnn-providers/hip-kernel-provider/rocke/library/dispatch/attention/README.md:492-526 state only the interface and catalog delta and reference platform/python/rocke/helpers/rdna_attention.py rather than restating the algorithm. |
| 4 | PASS | dnn-providers/hip-kernel-provider/rocke/library/dispatch/attention/README.md:492-526 and dnn-providers/hip-kernel-provider/rocke/platform/dsl_docs/instances/attention.md:38-40 accurately describe the newly added block_table_v, dense_q, seqused_q, and paged catalog sinks options introduced in this PR. |
| 5 | N/A | The PR makes no algorithmic scaling or asymptotic complexity claims in documentation or comments. |
| 6 | PASS | Catalog constants in dnn-providers/hip-kernel-provider/rocke/library/dispatch/attention/README.md:518-521 (buckets 64/96/128/256, page sizes 16/32/64) match SHIPPED_PAGE_SIZES in dnn-providers/hip-kernel-provider/src/engines/kernel_ingestor_engine/packs/Gfx1151WmmaAttentionNative.cpp:147 and the catalog descriptor. |
| 7 | N/A | No new Markdown design or algorithm document files are added in this PR; only existing README.md and attention.md files are updated. |
§K graded 13/13
| # | Verdict | Evidence |
|---|---|---|
| 1 | N/A | No tuned candidate definitions or make_tuned_candidate calls are added or modified in this PR. |
| 2 | PASS | New WmmaFmhaFwdV2Spec fields dense_q and seqused_q (dnn-providers/hip-kernel-provider/rocke/library/kernels/gfx1151/wmma_fmha_fwd_v2.py:123) are explicitly classified in WMMA_PROBLEM_FIELDS (dnn-providers/hip-kernel-provider/rocke/library/dispatch/attention/axes.py:346). |
| 3 | PASS | common.py omits dense_q and seqused_q when False so existing pins keep their spec_hash without a migration (dnn-providers/hip-kernel-provider/rocke/library/dispatch/attention/common.py:997), and test_gfx1151_attention_selection.py verifies both backward-compatible spec_hash preservation and distinct hashes when enabled… |
| 4 | N/A | KnobSpace.defaults() and recorded() are not touched; the new spec fields are problem-level layout options in WMMA_PROBLEM_FIELDS. |
| 5 | N/A | No knob drop, inertness, or refusal rules are modified in this PR. |
| 6 | N/A | KnobSpace.validate and tuning_specs.py are not modified in this PR. |
| 7 | N/A | KnobSpace.base(req) is not modified in this PR. |
| 8 | N/A | Knob value normalization and type coercion are not modified in this PR. |
| 9 | PASS | Replay verification via result.kernel_id.spec_hash is preserved for stored prefill pins (dnn-providers/hip-kernel-provider/rocke/library/tests/dispatch/attention/test_gfx1151_attention_selection.py:896). |
| 10 | N/A | KnobSpace.find and bare tuning_id resolution are not modified in this PR. |
| 11 | N/A | Sweep entry points and iteration helpers are not modified in this PR. |
| 12 | N/A | Production knob stacks, accept_default, and auto candidate routing are not modified in this PR. |
| 13 | N/A | No benchmark scripts or sweep harnesses are modified in this PR. |
§L graded 10/10
| # | Verdict | Evidence |
|---|---|---|
| 1 | N/A | The diff does not add or modify any gfx architecture target or ISA backend registry. |
| 2 | N/A | The diff does not add or modify any MMA or WMMA op_id or fragment layout map. |
| 3 | N/A | The diff does not change MMA atom selection, fragment lane maps, or accumulator layouts. |
| 4 | N/A | The diff does not add or change wave size handling or target wave-size queries. |
| 5 | N/A | The diff does not touch CDNA/gfx1250 arch-specific recipes, transpose-LDS reads, or AsyncTileLoader. |
| 6 | N/A | The diff does not add pipeline vocabulary to ArchTarget/core/arch or import CK Tile dispatcher modules. |
| 7 | N/A | The diff does not add or change logical dtypes, bit packing routines, or FP8 dialect rules. |
| 8 | N/A | The diff does not touch scaled-MMA contracts or scaled WMMA/MFMA emission. |
| 9 | N/A | The diff does not introduce a new architecture target. |
| 10 | N/A | The diff does not modify intrinsic arch-domain artifacts or LLVM flavor membership lists. |
§M graded 10/10
| # | Verdict | Evidence |
|---|---|---|
| 1 | PASS | Host booleans (spec.dense_q, spec.seqused_q, live_rows is not None) guard Python branches, while SSA predicates use b.cmp_* with scf_if and select (dnn-providers/hip-kernel-provider/rocke/library/kernels/gfx1151/wmma_fmha_fwd_v2.py:505, dnn-providers/hip-kernel-provider/rocke/platform/python/rocke/helpers/rdna_attention.py:571). |
| 2 | N/A | The diff does not touch legalize_for_target or any target legalization pass. |
| 3 | PASS | No wire format changes are made to ck.dsl.ir/v1, and kernel parameter order is identical between Python and C++ builders and verified by spec tests (dnn-providers/hip-kernel-provider/rocke/library/kernels/gfx1151/wmma_fmha_fwd_v2.py:416, dnn-providers/hip-kernel-provider/rocke/platform/cpp/instances/gfx1151/wmma_fmha_fwd_v2.cpp:291, dnn-providers/hip-kernel-provider/rocke/library/tests/test_wmma_fmha_fwd_v2_spec.py:224). |
| 4 | N/A | The diff only bumps BINARY_ABI in portable_ir/src/abi.py and does not touch portable IR schemas, importers, magic division, or recorder coverage. |
| 5 | N/A | The diff does not use or modify AsyncTileLoader or async DRAM-to-LDS operations. |
| 6 | PASS | The early return is CTA-uniform (q_start vs span), tile-paged loads use OOB buffer sentinels (at_tile), and 64-bit row-paged loads clamp invalid indices/offsets (blk, page, within) to 0 before issuing global_load and select (dnn-providers/hip-kernel-provider/rocke/library/kernels/gfx1151/wmma_fmha_fwd_v2.py:512, dnn-providers/hip-kernel-provider/rocke/platform/python/rocke/helpers/rdna_attention.py:607… |
| 7 | PASS | Buffer descriptors are constructed via b.buffer_rsrc / rocke_b_buffer_rsrc without hard-coding buffer-resource word 3 or using TemporalHint.STREAMING (dnn-providers/hip-kernel-provider/rocke/platform/python/rocke/helpers/rdna_attention.py:578, dnn-providers/hip-kernel-provider/rocke/platform/cpp/helpers/rdna_attention.cpp:732). |
| 8 | N/A | The diff does not add any new LLVM intrinsic declarations or modify backend selection logic. |
| 9 | PASS | Public C headers under cpp/include/rocke/ keep their names and extern "C" declarations, and ROCKE_ABI_VERSION / BINARY_ABI are bumped to 10 for the struct layout changes (dnn-providers/hip-kernel-provider/rocke/platform/cpp/include/rocke/abi.h:105, dnn-providers/hip-kernel-provider/rocke/platform/python/rocke/portable_ir/src/abi.py:78… |
| 10 | N/A | The diff does not touch !dbg metadata emission, Debug Info Version flags, or DWARF consumers. |
§N graded 18/18
| # | Verdict | Evidence |
|---|---|---|
| 1 | PASS | rdna_attention_scaled_ptr is added to dnn-providers/hip-kernel-provider/rocke/platform/python/rocke/helpers/rdna_attention.py:213 because both wmma_fmha_fwd_v2.py:462 and wmma_fmha_decode.py:607 share it, and it lands with its C++ twin rocke_rdna_attention_scaled_ptr in dnn-providers/hip-kernel-provider/rocke/platform/cpp/helpers/rdna_attention.cpp:214 and… |
| 2 | NEEDS-INFO | Both dnn-providers/hip-kernel-provider/rocke/library/kernels/gfx1151/wmma_fmha_decode.py:748 and dnn-providers/hip-kernel-provider/rocke/library/kernels/gfx1151/wmma_fmha_fwd_v2.py:552 (along with their C++ mirrors in wmma_fmha_decode.cpp:1097 and wmma_fmha_fwd_v2.cpp:503) call the shared rdna_attention_scaled_ptr / rocke_rdna_attention_scaled_ptr helper rather than duplicating 64-bit pointer arithmetic. — no diff line supports this verdict |
| 3 | N/A | The diff does not touch or use SoftwarePipeline. |
| 4 | N/A | The diff does not add or modify epilogue fusion or fused GEMM builders. |
| 5 | N/A | The diff does not touch rocke.helpers.tiling or add any tiling API. |
| 6 | N/A | The diff does not add or modify front-door/toolbox tiling helpers or LayoutStyle. |
| 7 | N/A | The diff does not use helpers/tiling/ layout overrides or TileMma. |
| 8 | N/A | The diff does not touch helpers/tiling/ LDS allocation or tiling memory verbs. |
| 9 | N/A | The diff does not use helpers/tiling/ N-D tensor axis roles or FlowStage. |
| 10 | N/A | The diff does not modify helpers/tiling/ emission, LDS staging, or MMA issue. |
| 11 | N/A | The diff does not touch helpers/tiling/ traits, lds_swizzle, or derive_sched_group_counts. |
| 12 | N/A | The PR makes no LDS bank-conflict or coalescing claims under helpers/tiling/. |
| 13 | N/A | The diff does not touch WarpDistributionEncoding, make_tile_desc, or RegisterMapper. |
| 14 | N/A | The diff does not use helpers/tiling/ verbs or add a tiling pipeline variant. |
| 15 | N/A | The diff does not modify helpers/tiling/ descriptor allocation or swizzled bounds checks. |
| 16 | N/A | The diff does not touch optimize_layout or evaluate_transform. |
| 17 | N/A | The diff does not touch tiling_recorder.py, round-trip analysis, or tiling visualization. |
| 18 | N/A | The diff does not touch InterleavedStyle or demo kernels under helpers/tiling/kernels/. |
§O graded 16/16
| # | Verdict | Evidence |
|---|---|---|
| 1 | N/A | This PR does not modify core/verify.py, cpp/core/verify.cpp, address spaces, or required IR attributes. |
| 2 | PASS | Early-exit scf.if guards properly terminate the then-region without emitting after the terminator in dnn-providers/hip-kernel-provider/rocke/library/kernels/gfx1151/wmma_fmha_fwd_v2.py:512 and dnn-providers/hip-kernel-provider/rocke/platform/cpp/instances/gfx1151/wmma_fmha_fwd_v2.cpp:424. |
| 3 | PASS | IR emission in dnn-providers/hip-kernel-provider/rocke/platform/python/rocke/helpers/rdna_attention.py:213 and dnn-providers/hip-kernel-provider/rocke/platform/cpp/helpers/rdna_attention.cpp:214 uses deterministic builder methods and ordered sequences without set iteration or id(). |
| 4 | PASS | Module singletons (I32, I64) and rocke_i32() / rocke_i64() are used in dnn-providers/hip-kernel-provider/rocke/platform/python/rocke/helpers/rdna_attention.py:55 and dnn-providers/hip-kernel-provider/rocke/platform/cpp/helpers/rdna_attention.cpp:220. |
| 5 | N/A | This PR does not touch core/passes.py, cpp/core/passes.cpp, or op purity tables. |
| 6 | N/A | This PR does not modify constant folding in either engine. |
| 7 | N/A | This PR does not alter kernel attributes, CodegenPolicy attribute contracts, or cluster_dims. |
| 8 | NEEDS-INFO | CodegenPolicy and apply_codegen_policy continue to be used in dnn-providers/hip-kernel-provider/rocke/library/kernels/gfx1151/wmma_fmha_decode.py:606 and dnn-providers/hip-kernel-provider/rocke/library/kernels/gfx1151/wmma_fmha_fwd_v2.py:459. — no diff line supports this verdict |
| 9 | N/A | This PR does not touch waitcnt encoding, barrier lowering, or optimization_barrier. |
| 10 | N/A | This PR does not modify lowerer attribute validation or _param_llvm_type. |
| 11 | N/A | This PR does not change ArchTarget queries or target ID resolution in rocke. |
| 12 | N/A | This PR does not modify target catalog width/size caps or add default arch or "gfx950" fallbacks. |
| 13 | PASS | Growing rocke_rdna_attn_params_t (dnn-providers/hip-kernel-provider/rocke/platform/cpp/include/rocke/helper_rocke.helpers.rdna_attention.h:136) and rocke_wmma_fmha_fwd_v2_spec_t (dnn-providers/hip-kernel-provider/rocke/platform/cpp/include/rocke/instance_gfx1151_wmma_fmha_fwd_v2.h:63) bumps ROCKE_ABI_VERSION to 10 in dnn-providers/hip-kernel-provider/rocke/platform/cpp/include/rocke/abi.h:105 and BINARY_ABI to 10 in… |
| 14 | PASS | C headers and sources follow rocke_* snake_case and correspondence prefaces in dnn-providers/hip-kernel-provider/rocke/platform/cpp/include/rocke/helper_rocke.helpers.rdna_attention.h:14, and additions to dnn-providers/hip-kernel-provider/rocke/platform/cpp/bindings/rocke_engine.cpp:4650 are minimal (2 lines). |
| 15 | N/A | This PR does not touch engine fallback mechanisms, _resolve_llvm_flavor, lower_hip.py, or lower_cktile.py. |
| 16 | PASS | Full Python and C++ instance and helper parity is maintained across dnn-providers/hip-kernel-provider/rocke/platform/cpp/instances/gfx1151/wmma_fmha_fwd_v2.cpp:144 and dnn-providers/hip-kernel-provider/rocke/platform/cpp/bindings/prove_parity_binding.py:813 without widening known_gaps.md. |
§P graded 14/14
| # | Verdict | Evidence |
|---|---|---|
| 1 | PASS | Builders keep the exact (spec, *, arch) signature (dnn-providers/hip-kernel-provider/rocke/library/kernels/gfx1151/wmma_fmha_fwd_v2.py:457, wmma_fmha_decode.py:604), and the new dense_q and seqused_q knobs are defaulted fields on WmmaFmhaFwdV2Spec (wmma_fmha_fwd_v2.py:123-124) and its C++ twin (dnn-providers/hip-kernel-provider/rocke/platform/cpp/include/rocke/instance_gfx1151_wmma_fmha_fwd_v2.h:63-64). |
| 2 | PASS | The kernarg signature order for seqused_q and block_table_v is updated identically in Python (dnn-providers/hip-kernel-provider/rocke/library/kernels/gfx1151/wmma_fmha_fwd_v2.py:416-425) and C++ (dnn-providers/hip-kernel-provider/rocke/platform/cpp/instances/gfx1151/wmma_fmha_fwd_v2.cpp:291-301), tested in dnn-providers/hip-kernel-provider/rocke/library/tests/test_wmma_fmha_fwd_v2_spec.py:205-243, and the ABI version is bumped to 10… |
| 3 | PASS | Test harnesses obtain launch geometry directly from wmma_fmha_fwd_v2_grid without recomputing it (dnn-providers/hip-kernel-provider/rocke/library/tests/test_wmma_fmha_fwd_v2_features.py:613-615, 826-828). |
| 4 | N/A | This PR does not modify LDS allocation math or LDS capacity gates. |
| 5 | N/A | This PR modifies RDNA attention kernels (wmma_fmha_fwd_v2 and wmma_fmha_decode), not WMMA/RDNA GEMM or GemmMultiD/GemmMultiAbd. |
| 6 | N/A | The paged prefill attention kernel requires zero workspace (dnn-providers/hip-kernel-provider/src/integration_tests/kernel_ingestor_engine/IntegrationGpuGfx1151WmmaAttentionPaged.cpp:855), and Stream-K, split-K, and… |
| 7 | N/A | This PR does not touch Stream-K, persistent GEMM, GroupedGemmLauncher, or SUPPORT_MATRIX.md. |
| 8 | N/A | This PR does not add or modify quantization helpers or MX GEMM scale decoding. |
| 9 | N/A | This PR does not introduce or change numeric emulation or data-type decode paths. |
| 10 | N/A | This PR does not add or modify example manifests (helpers/manifest.py or run_manifest). |
| 11 | PASS | GPU tests properly synchronize the stream via rt.sync() before reading device outputs and before freeing buffers (dnn-providers/hip-kernel-provider/rocke/library/tests/test_wmma_fmha_fwd_v2_features.py:835-844). |
| 12 | N/A | This PR does not touch COMGR/HIP library loading or fix a hipError launch failure. |
| 13 | PASS | Host-side changes to the attention spec cache key (dnn-providers/hip-kernel-provider/rocke/library/dispatch/attention/common.py:996-999) and paged K/V I32 extent guards (dnn-providers/hip-kernel-provider/rocke/library/dispatch/attention/bindings.py:683-704, 897-911) are covered by unit and GPU numeric tests (dnn-providers/hip-kernel-provider/rocke/library/tests/dispatch/attention/test_gfx1151_attention_selection.py:890-924… |
| 14 | PASS | No examples/ expected.json files are touched, and the shared 64-bit pointer offset helper rdna_attention_scaled_ptr is placed in dnn-providers/hip-kernel-provider/rocke/platform/python/rocke/helpers/rdna_attention.py:213-220 (with its C++ twin in dnn-providers/hip-kernel-provider/rocke/platform/cpp/helpers/rdna_attention.cpp:214-226) where both wmma_fmha_fwd_v2 and wmma_fmha_decode share it. |
§Q graded 7/7
| # | Verdict | Evidence |
|---|---|---|
| 1 | N/A | No Python test files were added or removed under rocke/platform/tests/ or rocke/library/tests/, so the installed rocke_pytest CMake lists and ROCKE_ENGINE_test_categories_external.yaml did not change. |
| 2 | PASS | New dense_q and seqused_q variants across gfx11-generic and gfx12-generic are added to the C and Python parity emitters in dnn-providers/hip-kernel-provider/rocke/library/tests/parity/gfx1151_wmma_fmha_fwd_v2_emit.c:101, dnn-providers/hip-kernel-provider/rocke/library/tests/parity/gfx1151_wmma_fmha_fwd_v2_emit.py:93, and dnn-providers/hip-kernel-provider/rocke/platform/cpp/bindings/prove_parity_binding.py:813. |
| 3 | PASS | Cross-engine IR/lowering parity is covered in dnn-providers/hip-kernel-provider/rocke/platform/cpp/bindings/prove_parity_binding.py:813 and dnn-providers/hip-kernel-provider/rocke/platform/tests/instances/differential/golden/llvm_gfx_all.json:1016, while numeric correctness is verified separately against float64 references in dnn-providers/hip-kernel-provider/rocke/library/tests/test_wmma_fmha_fwd_v2_features.py:168 and… |
| 4 | NEEDS-INFO | The diff for dnn-providers/hip-kernel-provider/rocke/platform/tests/instances/differential/golden/llvm_gfx_all.json is truncated after line 1040, so the remaining LLVM_FLAVORS sections (llvm20, llvm22, llvm23, llvm24, and canonical IR) and entries 170..179 cannot be verified from the provided diff. |
| 5 | PASS | GPU tests check the active device and skip cleanly when unavailable or not RDNA (dnn-providers/hip-kernel-provider/rocke/library/tests/test_wmma_fmha_fwd_v2_features.py:682, dnn-providers/hip-kernel-provider/src/integration_tests/kernel_ingestor_engine/IntegrationGpuGfx1151WmmaAttentionPaged.cpp:799), and CPU validation tests use fake tensors requiring neither torch nor a GPU… |
| 6 | N/A | This PR does not refactor any CI gate, differential runner, or parity harness execution logic. |
| 7 | N/A | This PR does not change backend resolution or ROCKE_BACKEND / ROCKE_CPP_STRICT fallback rejection contracts. |
§R graded 11/11
| # | Verdict | Evidence |
|---|---|---|
| 1 | N/A | This PR adds functional support for cuDNN paged/ragged SDPA semantics (dense_q, seqused_q, block_table_v, 64-bit page base addressing) rather than introducing a new kernel body, tiling strategy, pipeline variant, or… |
| 2 | N/A | The diff changes no performance levers or arch-specific optimization recipes; dense_q and seqused_q are problem-fixed fields (WMMA_PROBLEM_FIELDS). |
| 3 | N/A | No performance levers, tile sizes, LDS layouts, or prefetch pipelines are modified in this functional feature PR. |
| 4 | N/A | This PR makes no performance or primitive-selection claims and changes no performance levers. |
| 5 | N/A | The kernel does not use rocke.helpers.tiling and its per-lane 16-byte/8-byte vector load widths and striding patterns are unchanged (only the base pointer arithmetic is widened to 64-bit for >2 GiB caches and scalar… |
| 6 | N/A | Neither LDS layouts nor MMA layouts are modified in this diff. |
| 7 | N/A | The PR makes no bottleneck claims and does not target a performance bottleneck. |
| 8 | N/A | No waits, barriers, or scheduling hints are added, removed, or modified in this diff. |
| 9 | N/A | This PR is not a performance regression fix and does not trade performance across shapes via a tuning lever. |
| 10 | N/A | The PR does not change compiler/codegen policies, claim assembly parity, or report benchmark measurements. |
| 11 | N/A | No new optimization lessons, heuristics, case studies, or dsl_probes tools are introduced in this PR. |
§S graded 8/8
| # | Verdict | Evidence |
|---|---|---|
| 1 | PASS | New public symbols and spec fields (rdna_attention_scaled_ptr in dnn-providers/hip-kernel-provider/rocke/platform/python/rocke/helpers/rdna_attention.py:61, rocke_rdna_attention_scaled_ptr in dnn-providers/hip-kernel-provider/rocke/platform/cpp/include/rocke/helper_rocke.helpers.rdna_attention.h:86, and WmmaFmhaFwdV2Spec.dense_q / seqused_q in dnn-providers/hip-kernel-provider/rocke/library/kernels/gfx1151/wmma_fmha_fwd_v2.py:123)… |
| 2 | PASS | Existing callers and tests for wmma_fmha_fwd_v2_signature, rdna_attention_fwd_body, and bind_rdna_prefill_torch are updated in the same PR with new expectations (e.g., block_table_v added in dnn-providers/hip-kernel-provider/rocke/library/tests/test_wmma_fmha_fwd_v2_spec.py:205 and dnn-providers/hip-kernel-provider/rocke/library/tests/test_wmma_fmha_paged_guard.py:194, and >2 GiB prefill cache behavior updated in… |
| 3 | N/A | The diff does not remove or rename any public API symbol. |
| 4 | PASS | rdna_attention_scaled_ptr is added to __all__ in dnn-providers/hip-kernel-provider/rocke/platform/python/rocke/helpers/rdna_attention.py:65, resolves to the function defined at line 213, and is imported and exercised by dnn-providers/hip-kernel-provider/rocke/library/kernels/gfx1151/wmma_fmha_decode.py:607 and dnn-providers/hip-kernel-provider/rocke/library/kernels/gfx1151/wmma_fmha_fwd_v2.py:462. |
| 5 | PASS | Tests assert both positive outputs (IR serialization, signature lists, cache hashes, and float64 reference numerics) and exact negative ValueError exceptions on invalid parameters in dnn-providers/hip-kernel-provider/rocke/library/tests/test_wmma_fmha_fwd_v2_spec.py:40, dnn-providers/hip-kernel-provider/rocke/library/tests/dispatch/attention/test_gfx1151_attention_dispatch_numeric.py:2866, and… |
| 6 | PASS | CPU-checkable contracts (spec validation, kernel names, signature ordering, dispatch argument collection, I32 extent guards, and cache key hashing) are tested without a GPU or torch in dnn-providers/hip-kernel-provider/rocke/library/tests/test_wmma_fmha_fwd_v2_spec.py:37, dnn-providers/hip-kernel-provider/rocke/library/tests/dispatch/attention/test_gfx1151_attention_dispatch_numeric.py:2846… |
| 7 | PASS | The C/C++ twins (rocke_wmma_fmha_fwd_v2_spec_t, rocke_rdna_attn_params_t, rocke_rdna_attention_scaled_ptr, and the pybind11 bridge in dnn-providers/hip-kernel-provider/rocke/platform/cpp/bindings/rocke_engine.cpp:4650) are wired into the cross-engine parity harness in dnn-providers/hip-kernel-provider/rocke/platform/cpp/bindings/prove_parity_binding.py:813 and… |
| 8 | NEEDS-INFO | The PR body in the prompt is truncated mid-sentence under the - **Tests**: section, so it is not possible to verify whether the PR description states that tests/test_rocke.py and tests/run_all.py were run and names the GPU architecture tested. |
There was a problem hiding this comment.
🟡 Changes recommended
Unresolved critical findings permit unsafe accesses through invalid pages, empty caches, and unchecked sequence lengths, while the changed kernel argument layout lacks a dispatch ABI bump.
7 open issues
Reject seqlen_k values exceeding block-table capacity · New Bump the gfx1151 dispatch ABI for the changed kernel layout · New Clamp seqused_k in paged decode emitters · New Use a safe page ID before forming tile-paged pointers · New Suppress page-zero loads for empty row-paged caches · New Skip large-cache tests when device memory is insufficient · New Zero padded outputs explicitly when live values contain NaNs · New
What changed in this PR
Extends hipkernel:Gfx1151WmmaAttention to serve hipDNN’s paged K/V SDPA contract using rocKE’s RDNA prefill kernel, with dense or ragged queries. Paged graphs remain exclusive to the prefill engine.
Changes:
- Add query-length handling, separate K/V page tables, padded-row outputs, and 64-bit cache addressing in both kernel implementations.
- Expand prefill catalog metadata and variants, and update engine SDK requirements and the rocKE binary ABI.
- Extend matcher, binding, numerical, and parity coverage, with test registration and documentation updates.
| File | Description |
|---|---|
projects/hipdnn/tools/IngestorGenerator/tests/fixtures/unslotted_render_golden.json |
Refresh the generated catalog snapshot hash. |
projects/hipdnn/tools/IngestorGenerator/configs/gfx1151_wmma_attention.yaml |
Add paged query variants, sinks, and metadata. |
dnn-providers/hip-kernel-provider/src/tests/engines/kernel_ingestor_engine/packs/TestGfx1151WmmaAttentionMatchers.cpp |
Extend dense matcher checks and reuse shared fixtures. |
dnn-providers/hip-kernel-provider/src/tests/engines/kernel_ingestor_engine/packs/Gfx1151WmmaAttentionTestSupport.hpp |
Share fixtures, metadata, logs, and ABI helpers. |
dnn-providers/hip-kernel-provider/src/tests/engines/kernel_ingestor_engine/CMakeLists.txt |
Register paged matcher unit tests. |
dnn-providers/hip-kernel-provider/src/integration_tests/CMakeLists.txt |
Register supported paged GPU integration tests. |
dnn-providers/hip-kernel-provider/src/engines/kernel_ingestor_engine/descriptors/rocKE/gfx1151_wmma_attention/gfx1151_wmma_attention.ued.json |
Raise the prefill engine SDK requirement. |
dnn-providers/hip-kernel-provider/src/engines/kernel_ingestor_engine/descriptors/rocKE/gfx1151_wmma_attention/gfx1151_wmma_attention.kmd.json |
Define layout, page, query, and sink metadata. |
dnn-providers/hip-kernel-provider/rocke/platform/python/rocke/portable_ir/src/abi.py |
Update Python’s binary ABI version to 10. |
dnn-providers/hip-kernel-provider/rocke/platform/python/rocke/helpers/rdna_attention.py |
Implement 64-bit paging and live-query-row handling. |
dnn-providers/hip-kernel-provider/rocke/platform/dsl_docs/instances/attention.md |
Document the new paged query options. |
dnn-providers/hip-kernel-provider/rocke/platform/cpp/instances/gfx1151/wmma_fmha_fwd_v2.cpp |
Mirror prefill query forms and paged addressing. |
dnn-providers/hip-kernel-provider/rocke/platform/cpp/instances/gfx1151/wmma_fmha_decode.cpp |
Reuse the shared pointer helper without changing operations. |
dnn-providers/hip-kernel-provider/rocke/platform/cpp/include/rocke/instance_gfx1151_wmma_fmha_fwd_v2.h |
Add query-form flags to the C spec. |
dnn-providers/hip-kernel-provider/rocke/platform/cpp/include/rocke/helper_rocke.helpers.rdna_attention.h |
Declare the pointer helper and new operands. |
dnn-providers/hip-kernel-provider/rocke/platform/cpp/include/rocke/abi.h |
Version the changed public C struct layouts. |
dnn-providers/hip-kernel-provider/rocke/platform/cpp/helpers/rdna_attention.cpp |
Mirror paged loads and padded-row handling. |
dnn-providers/hip-kernel-provider/rocke/platform/cpp/bindings/rocke_engine.cpp |
Expose query-form flags through Python bindings. |
dnn-providers/hip-kernel-provider/rocke/platform/cpp/bindings/prove_parity_binding.py |
Add query-form binding parity configurations. |
dnn-providers/hip-kernel-provider/rocke/library/tests/test_wmma_fmha_paged_guard.py |
Pass the V page table in guard tests. |
dnn-providers/hip-kernel-provider/rocke/library/tests/test_wmma_fmha_fwd_v2_spec.py |
Test flag validation, names, signatures, and builds. |
dnn-providers/hip-kernel-provider/rocke/library/tests/test_wmma_fmha_fwd_v2_features.py |
Cover query padding, split tables, and large caches. |
dnn-providers/hip-kernel-provider/rocke/library/tests/parity/gfx1151_wmma_fmha_fwd_v2_emit.py |
Add Python query-form parity cases. |
dnn-providers/hip-kernel-provider/rocke/library/tests/parity/gfx1151_wmma_fmha_fwd_v2_emit.c |
Add matching C query-form parity cases. |
dnn-providers/hip-kernel-provider/rocke/library/tests/dispatch/attention/test_gfx1151_binding_i32_guard.py |
Test per-page bounds and large-cache acceptance. |
dnn-providers/hip-kernel-provider/rocke/library/tests/dispatch/attention/test_gfx1151_attention_selection.py |
Check cache identity for optional query flags. |
dnn-providers/hip-kernel-provider/rocke/library/tests/dispatch/attention/test_gfx1151_attention_dispatch_numeric.py |
Test dense paged query binding and padded outputs. |
dnn-providers/hip-kernel-provider/rocke/library/kernels/gfx1151/wmma_fmha_fwd_v2.py |
Add query flags, catalog accessors, and kernel arguments. |
dnn-providers/hip-kernel-provider/rocke/library/kernels/gfx1151/wmma_fmha_decode.py |
Reuse shared pointer arithmetic and clarify catalog comments. |
dnn-providers/hip-kernel-provider/rocke/library/dispatch/attention/README.md |
Document paged binding, addressing, and engine routing. |
dnn-providers/hip-kernel-provider/rocke/library/dispatch/attention/common.py |
Preserve cache keys while new flags are off. |
dnn-providers/hip-kernel-provider/rocke/library/dispatch/attention/bindings.py |
Collect query metadata and validate paged cache extents. |
dnn-providers/hip-kernel-provider/rocke/library/dispatch/attention/axes.py |
Classify query-form flags as non-tuning fields. |
Review effort: Max
…SdpaGraphMatch.hpp hipkernel:AttentionPagedUnified's graph_match carried its own copies of the SDPA match helpers that packs/SdpaGraphMatch.hpp now shares (findTensor, sdpaNode, hasSdpaForwardNode, describeOperand, the decline head and the axis constants). It now uses the header's, with sdpa_graph::DeclinedBy. The overflow-checked product it shares with hipkernel:Gfx942AttentionUnified3d moves into the header as sdpa_graph::checkedProduct: unlike pack_features::checkedProduct it takes zero factors, which every extent of 1 gives. FlydslSdpaShared.hpp, which already takes computesInFloat from the header, now takes checkedProduct and declaresAlignment from it too instead of keeping identical copies. The stricter operand checks it needs move into the header too (usesOverrideShapes, hasNonNegativeStrides, declaresAlignment, isInt32Vector), together with the cuDNN paged-K/V graph contract checks (SEM-009), so every pack serving a paged graph applies one set of rules with one cause vocabulary, logged with the calling engine's name: - pagesKAndV: one_sided_paging; - pagedContainersAgree: block_size, shape (one block size, one block count; head counts and head sizes stay the engine's to check); - pageTablesFor: page_table (INT32 [B, 1, M, 1], unit entry stride, not packed; separate K and V tables share M and the batch stride); - sequenceLengthsFor: seq_len_kv, seq_len_q (both required, INT32 [B,1,1,1]); - raggedQueryFor, countsOffsetsInTokens: ragged (Q and O through one [B+1,1,1,1] offset table counted in tokens; K/V never ragged); - declaresServedAlignment: alignment (vector operands at the engine's vector width, INT32 operands at 4 bytes); - sinkFor: sinks, and alignment below the engine's own minimum sink alignment (16 bytes for this engine's kernels); - takesNoAttentionMask: attn_mask. The reasons they log are the texts hipkernel:Gfx1151WmmaAttention's paged matcher (#14254) logs for the same rules, so that engine can call these checks without changing what it reports. The decline lines go through HIPDNN_PLUGIN_LOG_INFO((DeclinedBy{...}) << ...): the braces' comma would otherwise split the macro argument. The engine keeps its kernel limits (head and block sizes, GQA ratios, strides and address ranges) and the order of its declines; it checks itself that V has K's head count and head size. A malformed page-table tensor now logs page_table rather than operand. The header's file comment covers paged graphs.
… wmma_fmha_fwd_v2 wmma_fmha_fwd_v2 gains two query-side options for the packed layouts: dense_q (paged only) keeps Q / O / LSE batch-strided like the dense layout, so a uniform decode can read a [B, Sq, Hq, D] query against a paged cache without cu_seqlens_q; seqused_q (ragged or paged) takes each sequence's live query count from seqused_q[batch], capped at its span, with the causal diagonal aligned to it. A packed sequence's row base is now 64-bit, so a packed buffer may pass 2 GiB while each slice stays I32-addressable. The spec refuses a dense_q or seqused_q that is not a bool, since either one changes the argument layout. The paged layout also takes a V block table, block_table_v, next to block_table: K pages are looked up in block_table and V pages in block_table_v, at the same row offset (one block_table_stride). This serves cuDNN's separate page_table_k / page_table_v; a caller with one table (vLLM, the attention dispatch binding) passes it twice and gets the same results. A per-tile lookup now reads both tables; a page id outside [0, num_blocks) zeroes only that cache's rows. The prefill spec exposes catalog_layout, catalog_page_block_size, catalog_dense_q and catalog_seqused_q for the hipDNN catalog (only the prefill catalog has paged objects). The C++ mirror, engine binding, parity rows, spec tests (with num_blocks and block_table_v in the paged signatures), the paged-guard test and on-device feature cases (including separate K and V tables) change together. AttentionRequest gains dense_q and seqused_q fields (the dense_q / seqused_q features, which only the RDNA WMMA prefill declares). The prefill policy copies them into its spec; the decode refuses them, so such a request always runs the prefill. A request without them keeps its request_hash: the fields join normalized() only when set. They are semantic layout fields (WMMA_PROBLEM_FIELDS, no sweep varies them), and while they are off they stay out of the tuning spec's cache key, so every other RDNA WMMA spec_hash and stored pin is unchanged. The attention dispatch binding (bind_rdna_prefill_torch) reads both options: a paged dense_q spec takes a batch-strided [B, Sq, Hq, D] Q / O (and LSE), with batch strides from the tensors and no cu_seqlens_q, and a seqused_q spec takes an INT32 [B] seqused_q tensor. A GPU test dispatches one dense-Q paged prefill with seqused_q from the request and checks it against an FP64 reference; selection tests check the request fields reach the prefill spec, the decode and every other arch refuse them, bad values are rejected, and turning either option on changes the cache key. rocke_rdna_attn_params_t gains block_table_v and the paged pointer fields, and rocke_wmma_fmha_fwd_v2_spec_t gains dense_q and seqused_q, so ROCKE_ABI_VERSION and BINARY_ABI move to 13. block_table_v moves every later paged prefill argument, so the dispatch ABI string moves to rocke-attention-gfx1151/v8.
…nd wmma_fmha_decode A seqused_k entry past seqlen_k walked the block_table lookup off the end of its row, into the next sequence's pages (or past the table for the last one). The paged prefill and the paged split-KV decode now take min(seqused_k[b], seqlen_k), in the Python builders and the C++ mirrors, and the dispatch binding rejects a seqlen_k larger than the keys one table row maps (its width times the page size), for both kernels. The feature test pads table rows with out-of-range ids and adds a sequence whose seqused_k runs 100 keys past seqlen_k; the paged-guard test points a stale seqused_k's table padding at a NaN page, for the prefill and the decode; a binding test rejects a seqlen_k one key past the table.
The paged layout read K / V through one buffer resource over the whole cache at the head, so a page term past 2 GiB went out of range of the resource (and a cache past 4 GiB could not be addressed at all). The head and page terms are now 64-bit: - the kernel passes the body a 64-bit pointer to each cache at its head; - a page that holds whole key tiles is read through one resource per tile, based at that page (the page id is CTA-uniform); - a smaller page (one block-table lookup per cooperative row) is read by 64-bit addressed global loads per row; a dead chunk reads the head's first element and yields zeros. Within a page the offsets stay I32, so only one page of one head must span less than 2 GiB. Dense and ragged layouts emit unchanged IR. The Python builder and the C++ mirror change together. The feature test gains caches past 4 GiB (tile and row pages, BF16 and FP8 K/V) whose other pages hold NaN, so a truncated page term poisons the output; it skips on a device without the free memory for them. The 64-bit step from a base pointer by index * stride * element size is one rdna_attention helper (rdna_attention_scaled_ptr in Python, rocke_rdna_attention_scaled_ptr in C++), used by the paged head and page steps and by the decode kernel's batch, token and bias bases, with the ops in the order each copy had, so no golden changes. The dispatch binding checks a paged prefill K / V cache as the hipDNN pack's pageFitsI32 does (one page of one head I32-addressable in bytes) instead of capping the whole cache at 2 GiB. The decode kernel still forms its head and page offsets in I32, so its caches keep the whole-tensor cap. Since a dead row's load still reads page 0, the binding rejects a prefill cache with no pages (num_blocks is at least 1).
…with seqused_q in wmma_fmha_fwd_v2 With seqused_q, the rows of a sequence's span (its cu_seqlens_q gap, or seqlen_q for a dense_q object) past seqused_q[b] were neither read nor written, and a cu_seqlens_q sequence took its span from cu_seqlens_q[b + 1]. For a dense_q object, cuDNN and hipDNN's paged contract (SdpaSemanticsRegister SEM-008) give the rows past seqused_q[b], up to seqlen_q, O = 0 and LSE = -inf, with or without a sink, so the kernel now writes them. The O and LSE resources span seqlen_q; Q still covers only the live rows, so padding is never read. rdna_attention_fwd_body takes an optional live_rows: rows at or past it are treated as empty rows without a sink (O = 0, LSE = -inf), and a CTA whose tile is all padding skips the K loop. Empty and padding rows store O = 0 by select rather than by scaling with a zero reciprocal, since P = 0 times a non-finite V row still leaves NaN in their accumulators. For a cu_seqlens_q object, hipDNN's ragged contract (RFC 0014) says that with seq_len bound the origins may come in any order, ragged_offset[B] is neither used nor validated, and the rows of an offset block past seq_len_q belong to no batch. The sequence now owns only its live rows, min(seqused_q[b], seqlen_q): cu_seqlens_q[b + 1] is not read, and O / LSE cover only those rows, so no write lands past a footprint-sized buffer. The C mirror follows op for op. The feature test checks that dense-Q padding rows read O = 0 and LSE = -inf (also with a NaN in a V row every live row sees), and lays a ragged Q's sequences out in reverse order with cu_seqlens_q[B] poisoned, checking that the rows past seqused_q keep their sentinel.
hipkernel:Gfx1151WmmaAttention now binds the cuDNN paged-K/V SDPA graph: one INT32 page table shared by K and V over [num_blocks, H, block_size, D] containers of any strides (vLLM's interleaved K|V cache included), seq_len_kv, a build-time max_seq_len_kv, and a query that is either dense (uniform decode, with optional seq_len_q for its live rows) or ragged in token-unit offsets with seq_len_q. One launch of a wmma_fmha_fwd_v2 paged prefill object serves it; batch, lengths and block ids are read on the device, so one plan serves every step and replays under HIP-graph capture. The page table is passed as both of the kernel's tables, block_table (K) and block_table_v (V). The kernel receives num_blocks, so an out-of-range page id reads as zero rows; graphs whose K and V caches differ in num_blocks are declined. Pages are addressed in 64 bits, so a cache may pass 2 GiB (and 4 GiB) as long as one page of one head is I32-addressable. Sinks, ALiBi, FP8 descales and dropout decline with a logged cause. Paged graphs are the prefill engine's alone. hipkernel:Gfx1151WmmaAttentionDecode ships dense objects only: its graph matcher declines any graph with a page table and its kernel matcher admits only the dense layout. The prefill engine declines what the dense decode rule routes to decode only for dense graphs; a paged graph's key extent is its page size, so that rule never applies to it. Catalog: 72 paged prefill objects (head width 64/96/128/256 x page 16/32/64 x query form x dtype) in configs/gfx1151_wmma_attention.yaml, each object carrying layout, page_block_size, dense_q and seqused_q metadata (the dense objects state the dense defaults); the engine's sdk_version is 1.4.0 for the token-unit ragged offsets. The descriptors are regenerated with IngestorGenerator and spliced keeping every existing UUID. Tests: matcher, routing, score and ABI unit tests (TestGfx1151WmmaAttentionPaged), a dense routing pin in TestGfx1151WmmaAttentionMatchers, the regenerated census, and a GPU integration test against a gather-then-dense reference with HIP-graph capture and replay. The integration test skips when the frontend does not yet validate paged K/V SDPA graphs. Re-pin the gfx1151_wmma_attention.yaml digest in the IngestorGenerator fixture unslotted_render_golden.json (test_slots.py): this change adds the paged and ragged objects to that config, so its render changes by design. The other reference configs render as before.
… in wmma_fmha_fwd_v2 Two on-device feature cases run the paged prefill with sinks through the query forms hipDNN's paged graph binds: a batch-strided Q (gpt-oss GQA decode, a sequence with no keys writing O = 0 and LSE = the sink) and a batch-strided Q with seqused_q, padded rows and a causal window.
… and 64-bit paged addressing Re-blesses only the gfx1151_wmma_fmha_fwd_v2 and gfx1151_wmma_fmha_decode ll (llvm20, llvm22, llvm23, llvm24) and ir_canonical entries, with run_diff.py --python-only --record-golden --only gfx1151_wmma_fmha: for the prefill, the 64-bit packed row base, the seqused_k clamp, the separate V block table, 64-bit paged K/V pages, the padding rows past seqused_q (stored by select) and the new dense_q / seqused_q configs; for the decode, the paged seqused_k clamp. No other family changes.
…fx1151WmmaAttention The prefill catalog gains a sink variant of every paged prefill object (dtype x head-width bucket 64/96/128/256 x page size 16/32/64 x query form): wmma_fmha_fwd_v2 with use_sinks, named *_sinks, 72 objects (192 in all). A new KMD field 'sinks' (int, default 0) marks them; the wmma_fmha_fwd_v2 spec exposes it as catalog_sinks for the specialization contract, and every existing object states sinks 0. Retained descriptor ids are unchanged; the census lists the new objects for every shard. The decode catalog (hipkernel:Gfx1151WmmaAttentionDecode) is unchanged. The native matcher accepts sink_token on the cuDNN paged-K/V graph (dense Q, dense Q with seq_len_q, ragged Q): [1, Hq, 1, 1] in the Q data type with a unit head stride, bound as a sink uid that selects the paged objects whose catalog field sinks is 1. prepare() marshals sink_ptr after the stride arguments and before the query form's operands (wmma_fmha_fwd_v2._declare_params); launch passes the 2-byte-aligned sink buffer there. A FLOAT sink (the RDNA kernels load the sink in the I/O dtype), another dtype or shape, and sinks on dense-layout graphs are declined with cause 'sinks'. Paged graphs stay the prefill engine's: the decode engine declines them with or without sinks. Unit tests cover binding in every query form, the declines, sinks-object selection, the decode engine's decline and the kernel ABI; the GPU integration test adds three sinks cases (gpt-oss GQA decode with a window and a keyless sequence, dense Q with seq_len_q and LSE, a mixed ragged batch) against the gather-then-dense reference with the sink in max, denominator and LSE. Re-pin the gfx1151_wmma_attention.yaml digest in the IngestorGenerator fixture unslotted_render_golden.json (test_slots.py): the sink objects change that config's render by design.
…1151WmmaAttention hipDNN's paged K/V contract (SdpaSemanticsRegister SEM-008/009, #14358) differs from the one this engine was written against: - Every paged graph carries seq_len_q and seq_len_kv, also with a dense Q. The matcher now requires both and always binds seq_len_q. The dense-Q form without seq_len_q is gone: GraphLayout keeps PAGED_DENSE_Q (dense Q, live rows seq_len_q) and PAGED_RAGGED_Q, and an object without seqused_q serves no graph. - A dense Q's rows past seq_len_q, up to Sq, are padding with O = 0 and LSE = -inf. The kernel now writes them (an earlier commit); the GPU test expects that rather than untouched rows. - A ragged Q follows RFC 0014 (#15080): origins may come in any order, ragged_offset[B] is neither used nor validated, and the rows of an offset block past seq_len_q belong to no sequence. The kernel reads only each sequence's origin and writes only its live rows; the GPU test lays the sequences out in reverse order, poisons ragged_offset[B] and expects those rows untouched. - Separate K and V page tables (page_table_k, page_table_v) are valid graphs and are now served: the matcher binds both, and the launch passes them as the kernel's block_table and block_table_v (one table bound to both sides is passed twice). The two tables must have one width and, with more than one sequence, one row stride. - One-sided paging, per-side block sizes or page counts, K and V tables of different widths or row strides, and packed page tables are valid graphs the kernels cannot read, so each is declined with a logged cause. - An unset max_seq_len_kv is valid. The kernels use it only as the bound on seq_len_kv, so it is inferred as the table width times the block size, as cuDNN does. - A graph may declare a device-pointer alignment below the default 16 bytes (engine API 1.3.0, which the plugin's 1.4.0 admits). The kernels load Q / K / V / O rows as 16-byte vectors, INT32 operands as 4-byte words and the sink with 16-bit loads, so the matcher declines a graph declaring less for any of them (cause 'alignment', dense and paged) where hipDNN can still pick another engine, rather than the plan failing at execute. - left_bound follows cuDNN (L keys ending at the diagonal); maskFor already translates it to the kernel's window_left = L - 1. The GPU test's windows follow it (32-, 16- and 128-key windows). Catalog: the 48 paged objects with dense_q but without seqused_q (24 plain, 24 with sinks) could serve no hipDNN graph and are dropped; 96 paged objects remain (dense Q and ragged Q, each with and without sinks), 144 prefill objects in all. Retained descriptor ids are unchanged. The census is regenerated. Decline logs: the paged checks log the cause keys #14247 uses for the same contract rules, so the shared checks it moves into SdpaGraphMatch.hpp keep every key this engine logs: one_sided_paging; page_table (a packed table, K and V tables of different widths or row strides, a table that is not INT32 [B, 1, M, 1] with a unit entry stride); block_size (different K and V block sizes, a page size that is not a power of two, and one no paged object ships for, which used to find no object without a log); shape (different K and V page counts); seq_len_q, seq_len_kv and ragged (a missing or malformed sequence-length or offset tensor, declined silently before); attn_mask (additive bias on a paged graph). Each reason names the operand and values involved. The dense layout logs 'padding' for seq_len_q, seq_len_kv, max_seq_len_kv or padding_mask. Each line is headed by the declining engine's name (sdpa_graph::DeclinedBy): the decode engine runs the shared dense matcher too, and used to log under the prefill engine's name; it now also logs why it declines a paged graph. Tests: the matcher scaffolding both unit tests share (engine symbols, the graph fixture, object metadata, the decline-log reader, the ABI helpers) moves into Gfx1151WmmaAttentionTestSupport.hpp. The unit tests bind separate K and V tables (dense and ragged Q) and a V container with its own strides, and assert the engine, cause and reason of each decline: every paged check in isolation, the unshipped page sizes, alignment (sink included), FP8, dropout, a forced implementation, the other unserved features and ALiBi; the dense declines are asserted for both engines. The GPU test checks against test_sdk's CpuFpReferenceSdpa (gatherPagedKv per page table, sequence lengths, band, sink and LSE) instead of its own reference, adds two cases whose V pages sit in other blocks than their K pages, through a second page table, two whose V page, head and token strides all differ from K's, and runs the 96 and 256 buckets at pages 32 and 64 and the 128 bucket at page 64. Sinks: the engine still accepts a sink_token in the Q data type and declines a FLOAT one. hipDNN's frontend accepts only a FLOAT sink, so the GPU test's sink cases now skip with a message when the frontend rejects a Q-dtype sink; the paged-rules probe that skipped every case is removed, and the other cases run. Re-pin the gfx1151_wmma_attention.yaml digest in the IngestorGenerator fixture unslotted_render_golden.json (test_slots.py): the contract change alters that config's render by design.
…WmmaAttention hipDNN's frontend and references take sink_token only as FLOAT [1, Hq, 1, 1], which the paged sinks objects declined, so no graph that validates reached them. The sinks objects now read either form through one runtime argument, so no catalog object is added. - wmma_fmha_fwd_v2 (Python and the C mirror): a sinks object takes sink_is_f32 (i32) after sink_ptr. Nonzero reads sink_ptr[h] as f32, zero in the Q dtype. Two 16-bit loads stay inside either form's [Hq] extent and are combined into the f32 word or widened from the low half. Re-bless the gfx1151_wmma_fmha_fwd_v2 goldens. - The RDNA dispatch bindings pass sink_is_f32 to the prefill kernel and accept float32 sinks for it; the decode kernel still reads the Q dtype only. - Gfx1151WmmaAttention: sinkFor accepts FLOAT (declaring 4-byte alignment) or the Q data type (2 bytes); another dtype declines under `sinks`. The bound token gfx1151_wmma_attention.sink.f32 (0 or 1, checked when read) sets sink_is_f32 at launch, and execute checks the pointer against the dtype's alignment. - Tests: FLOAT sinks bind with the flag set, decline below 4 bytes, and a descriptor without sink_is_f32 is refused. The rocKE feature tests add f32-sink cases (dense Q, per-row and per-tile ragged Q). IntegrationGpuGfx1151WmmaAttentionPaged runs each sink case again with a FLOAT sink_token, which needs no frontend change. The Q-dtype sink cases still skip on a frontend that rejects them.
…on gfx11 in Gfx1151WmmaAttention At pages 32 and 64 the head-96 paged objects used 256 VGPRs on gfx11 and spilled inside the K loop. With block_m 64 (block_size 128) they use 212 and do not spill; gfx12 objects keep block_m 128. The 16 gfx11 objects get new ids; the descriptors, catalog census and render digest are regenerated.
…SdpaGraphMatch.hpp hipkernel:AttentionPagedUnified's graph_match carried its own copies of the SDPA match helpers that packs/SdpaGraphMatch.hpp now shares (findTensor, sdpaNode, hasSdpaForwardNode, describeOperand, the decline head and the axis constants). It now uses the header's, with sdpa_graph::DeclinedBy. The overflow-checked product it shares with hipkernel:Gfx942AttentionUnified3d moves into the header as sdpa_graph::checkedProduct: unlike pack_features::checkedProduct it takes zero factors, which every extent of 1 gives. FlydslSdpaShared.hpp, which already takes computesInFloat from the header, now takes checkedProduct and declaresAlignment from it too instead of keeping identical copies. The stricter operand checks it needs move into the header too (usesOverrideShapes, hasNonNegativeStrides, declaresAlignment, isInt32Vector), together with the cuDNN paged-K/V graph contract checks (SEM-009), so every pack serving a paged graph applies one set of rules with one cause vocabulary, logged with the calling engine's name: - pagesKAndV: one_sided_paging; - pagedContainersAgree: block_size, shape (one block size, one block count; head counts and head sizes stay the engine's to check); - pageTablesFor: page_table (INT32 [B, 1, M, 1], unit entry stride, not packed; separate K and V tables share M and the batch stride); - sequenceLengthsFor: seq_len_kv, seq_len_q (both required, INT32 [B,1,1,1]); - raggedQueryFor, countsOffsetsInTokens: ragged (Q and O through one [B+1,1,1,1] offset table counted in tokens; K/V never ragged); - declaresServedAlignment: alignment (vector operands at the engine's vector width, INT32 operands at 4 bytes); - sinkFor: sinks, and alignment below the engine's own minimum sink alignment (16 bytes for this engine's kernels); - takesNoAttentionMask: attn_mask. The reasons they log are the texts hipkernel:Gfx1151WmmaAttention's paged matcher (#14254) logs for the same rules, so that engine can call these checks without changing what it reports. The decline lines go through HIPDNN_PLUGIN_LOG_INFO((DeclinedBy{...}) << ...): the braces' comma would otherwise split the macro argument. The engine keeps its kernel limits (head and block sizes, GQA ratios, strides and address ranges) and the order of its declines; it checks itself that V has K's head count and head size. A malformed page-table tensor now logs page_table rather than operand. The header's file comment covers paged graphs.
89ee88a to
67a5479
Compare
|
Pushed 67a5479, rebased onto @cgmillette, follow-up review Reply as a top-level comment: Thanks for the follow-up review. On the open rows:
@cgmillette, follow-up review Reply as a top-level comment: Covered by the reply above; the remaining rows from this round:
Reply as a top-level comment: Thanks, Cong. All seven threads now have replies: six are addressed in the code and resolved, and the YAML |
|
Automated review: Model - qwen3.6:35b-a3b-coding-mtp-q4_K_M, effort: high PR #14254 — paged/ragged SDPA in Gfx1151WmmaAttention (dpeabody-amd, AICK-1847) — 🟡 Follow-up: fixes hold; CI running
Utility: 🟢 Positive — paged and ragged SDPA for RDNA prefill, now serving FLOAT sinks without spills. Legend: 🔴 blocking · 🟠 should-fix · 🟡 minor nit · 🟢 info/good · ⚪ retracted Follow-up review: first pass by qwen3.6 (local), verified by the orchestrator (Claude Opus 5.5). Checks: changes since |
yraparti
left a comment
There was a problem hiding this comment.
rocKE checklist re-review of the fixes since 89ee88af237d
ROCm/rocm-libraries#14254 · 67a5479ca128 · model gemini-4-argon, fallback gpt-5.5
PASS · graded 238/238 · BLOCKER 0, HIGH 0, MEDIUM 0, LOW 0, NEEDS-INFO 18
The diff was truncated before grading. Items that needed a missing hunk are NEEDS-INFO.
Since the last review (89ee88af237d)
Resolved 2 · still open 0 · new 0
- ✅ §B.4 was FAIL, now NEEDS-INFO — had asked: In
WmmaFmhaFwdV2Spec.__post_init__(dnn-providers/hip-kernel-provider/rocke/library/kernels/gfx1151/wmma_fmha_fwd_v2.py:168), validateif type(self.dense_q) is not bool or type(self.seqused_q) is not bool: raise TypeError(...)before checking layout compatibility. - ✅ §D.10 was FAIL, now PASS — had asked: Expose
dense_qandseqused_q(or the corresponding query layout/length presence) onAttentionRequestand wire the RDNA prefill spec builder to populate them from the request, or document a stacked PR deferral in the PR description.
Findings
- No FAIL rows.
Needs info
- §A.1 dnn-providers/hip-kernel-provider/rocke/library/kernels/gfx1151/wmma_fmha_fwd_v2.py:467 and dnn-providers/hip-kernel-provider/rocke/platform/cpp/instances/gfx1151/wmma_fmha_fwd_v2.cpp:400 keep the Python and C++ kernel-body emitters in sync (along with dnn-providers/hip-kernel-provider/rocke/library/kernels/gfx1151/wmma_fmha_decode.py:697 and dnn-providers/hip-kernel-provider/rocke/platform/cpp/instances/gfx1151/wmma_fmha_decode.cpp:974). — no diff line supports this verdict
- §A.5 The diff for dnn-providers/hip-kernel-provider/rocke/platform/tests/instances/differential/golden/llvm_gfx_all.json is omitted to fit the model budget (+1015 -935 lines), so verifying which golden entries were changed or left untouched requires the omitted hunk.
- §A.10 The diff for dnn-providers/hip-kernel-provider/rocke/platform/tests/instances/differential/golden/llvm_gfx_all.json is omitted (+1015 -935 lines), so the new golden hash entries for indices 170..185 (QUERY_LENGTHS and SHIPPED_PAGED flag-on cases across LLVM flavors and arches) are not visible in the diff.
- §B.4 Resolved since previous review:
WmmaFmhaFwdV2Spec.__post_init__indnn-providers/hip-kernel-provider/rocke/library/kernels/gfx1151/wmma_fmha_fwd_v2.py:173-176now enforcesif type(value) is not bool: raise TypeError(...)fordense_qandseqused_q(matchingdnn-providers/hip-kernel-provider/rocke/library/dispatch/attention/common.py:433-436), anddnn-providers/hip-kernel-provider/rocke/library/tests/test_wmma_fmha_fwd_v2_spec.py:48-61tests this rejection. — no diff line supports this verdict - §C.9 dnn-providers/hip-kernel-provider/rocke/library/kernels/gfx1151/wmma_fmha_fwd_v2.py:538, dnn-providers/hip-kernel-provider/rocke/platform/cpp/instances/gfx1151/wmma_fmha_fwd_v2.cpp:456, dnn-providers/hip-kernel-provider/rocke/library/kernels/gfx1151/wmma_fmha_decode.py:697, and dnn-providers/hip-kernel-provider/rocke/platform/cpp/instances/gfx1151/wmma_fmha_decode.cpp:974 apply the paged seqused_k clamp across both prefill and decode in both Python and C++ twins. — no diff line supports this verdict
- §E.10 Unresolved from previous review: merge-simulation against other open PRs touching the same files (e.g. abi.h, llvm_gfx_all.json, Gfx1151WmmaAttentionNative.cpp) still cannot be verified from the PR diff alone.
- §F.6 Local
pre-commitexecution status before push cannot be verified from the diff alone. - §F.7 Bot, Copilot, and human review thread states and deferred-item ticket IDs are not included in the prompt or diff.
Section notes
- §B: Re-review: Item 4 resolved by adding strict
type(value) is not boolchecks inWmmaFmhaFwdV2Spec.__post_init__alongside unit tests. - §G: Re-review of head 67a5479. Items 2 and 3 remain NEEDS-INFO because the PR body and diff are truncated before test/gate execution results.
- §G: Items 18–34 of 34
- §H: Trace of device-side bounds and masks: 1. Paged K/V key length
sk/Sk_b: extent source issmin(seqused_k[batch], seqlen_k)(wmma_fmha_fwd_v2.py:538,wmma_fmha_decode.py:697). The kernel loops overn_tiles = cdiv(smax(sk, 0), BN)(rdna_attention.py:826-827) and bounds per-tile valid rows viarem = select(valid, seqlen_k - tok0, 0)(rdna_attention.py:567). Host validatesseqlen_k <= block_table.shape[1] * page_block_size(bindings.py:1151-1157,Gfx1151WmmaAttentionNative.cpp:1282-1289), soblk = tok >> lpsnever reads past the sequence'sblock_table/block_table_vrow. 2. Physical page IDphys(phys_k,phys_v): extent source is runtime0 <= phys < num_blocksviapage_ok(phys)(rdna_attention.py:554-556). On the per-tile path (ps % BN == 0),rem_kandrem_vare zeroed andsafe_k/safe_vselect page 0 before forming the 64-bitpage_base(rdna_attention.py:578-588); on the per-row path (ps % BN != 0),okmasks bothpageto… - §J: Shapes: | Operand / Equation | Inner dims | Result vs Declared | |---|---|---| |
dense_qQ / O (README.md:501) | N/A (tensor layout annotation) |[B, Sq, Hq, D]| |seqused_q(README.md:502) | N/A (tensor layout annotation) |INT32 [B]| Symbols: | Symbol | Defined at | Domain | |---|---|---| |block_table/block_table_v|README.md:496,attention.md:38| K and V page table indices | |dense_q|README.md:500,attention.md:39| Boolean spec/request flag for batch-strided Q/O | |seqused_q|README.md:502,attention.md:40| Per-sequence live query row count (INT32 [B]) / bool spec flag | |use_sinks/sinks|README.md:528| Spec boolean / catalog integer flag for attention sinks | Novelty: | Claim | Symbol | Status vs develop | |---|---|---| | Separate V page table in v8 ABI |block_table_v,rocke-attention-gfx1151/v8| Added in this PR | | Paged prefill query options |dense_q,seqused_q| Added in this PR | | Shipped paged… - §K: Identity table: (1) WmmaFmhaFwdV2Spec.dense_q (default False, problem field in WMMA_PROBLEM_FIELDS) -> omitted from cache_key when False so config_key/spec_hash unchanged for existing pins, no TUNING_ID_VERSION bump needed, no stored pins refused; (2) WmmaFmhaFwdV2Spec.seqused_q (default False, problem field in WMMA_PROBLEM_FIELDS) -> omitted from cache_key when False so config_key/spec_hash unchanged for existing pins, no TUNING_ID_VERSION bump needed, no stored pins refused; (3) ATTENTION_GFX1151_ABI bumped from v7 to v8 for paged block_table_v kernel signature change.
Graded with the rocKE PR checklist, one model pass per section. FAIL rows that sit on a diff line are inline comments.
§A graded 14/14
| # | Verdict | Evidence |
|---|---|---|
| 1 | NEEDS-INFO | dnn-providers/hip-kernel-provider/rocke/library/kernels/gfx1151/wmma_fmha_fwd_v2.py:467 and dnn-providers/hip-kernel-provider/rocke/platform/cpp/instances/gfx1151/wmma_fmha_fwd_v2.cpp:400 keep the Python and C++ kernel-body emitters in sync (along with dnn-providers/hip-kernel-provider/rocke/library/kernels/gfx1151/wmma_fmha_decode.py:697 and… |
| 2 | N/A | No core lowering-op or op-handler files (core/lower_*, core/ir) are modified in this PR. |
| 3 | PASS | dnn-providers/hip-kernel-provider/rocke/library/kernels/gfx1151/wmma_fmha_fwd_v2.py:128, dnn-providers/hip-kernel-provider/rocke/platform/cpp/include/rocke/instance_gfx1151_wmma_fmha_fwd_v2.h:63… |
| 4 | PASS | dnn-providers/hip-kernel-provider/rocke/platform/cpp/helpers/rdna_attention.cpp:220 and dnn-providers/hip-kernel-provider/rocke/platform/cpp/instances/gfx1151/wmma_fmha_fwd_v2.cpp:489 bind operands left-to-right to temporaries to match the exact emit order in Python… |
| 5 | NEEDS-INFO | The diff for dnn-providers/hip-kernel-provider/rocke/platform/tests/instances/differential/golden/llvm_gfx_all.json is omitted to fit the model budget (+1015 -935 lines), so verifying which golden entries were changed or left untouched requires the omitted hunk. |
| 6 | PASS | dnn-providers/hip-kernel-provider/rocke/library/tests/parity/gfx1151_wmma_fmha_fwd_v2_emit.c:101, dnn-providers/hip-kernel-provider/rocke/library/tests/parity/gfx1151_wmma_fmha_fwd_v2_emit.py:97, and… |
| 7 | PASS | dnn-providers/hip-kernel-provider/rocke/library/kernels/gfx1151/wmma_fmha_fwd_v2.py:516 and dnn-providers/hip-kernel-provider/rocke/platform/python/rocke/helpers/rdna_attention.py:823 guard new ops behind layout != 'dense', spec.dense_q, spec.seqused_q, and live_rows… |
| 8 | PASS | dnn-providers/hip-kernel-provider/rocke/platform/cpp/helpers/rdna_attention.cpp:700 and dnn-providers/hip-kernel-provider/rocke/platform/cpp/instances/gfx1151/wmma_fmha_fwd_v2.cpp:412 match their Python twins and are wired into… |
| 9 | N/A | No shared core op-handler fix or lowerer-only arch divergence is introduced in this PR. |
| 10 | NEEDS-INFO | The diff for dnn-providers/hip-kernel-provider/rocke/platform/tests/instances/differential/golden/llvm_gfx_all.json is omitted (+1015 -935 lines), so the new golden hash entries for indices 170..185 (QUERY_LENGTHS and SHIPPED_PAGED flag-on cases across LLVM flavors and arches) are not visible in the diff. |
| 11 | N/A | No new IR opcodes are added in this PR. |
| 12 | N/A | core/backend.py and ROCKE_BACKEND lowering fallback logic are not touched in this PR. |
| 13 | N/A | LLVM .ll emission format and ROCKE_DEBUG_LOC handling are not modified in this PR. |
| 14 | PASS | dnn-providers/hip-kernel-provider/rocke/platform/python/rocke/helpers/rdna_attention.py:213, dnn-providers/hip-kernel-provider/rocke/platform/cpp/helpers/rdna_attention.cpp:214, and… |
§B graded 6/6
| # | Verdict | Evidence |
|---|---|---|
| 1 | PASS | New rejections in dnn-providers/hip-kernel-provider/src/engines/kernel_ingestor_engine/packs/Gfx1151WmmaAttentionNative.cpp:851-891 (unservedFeature) and Gfx1151WmmaAttentionNative.cpp:1110-1315 (gfx1151WmmaAttentionPagedGraphMatches), as well as… |
| 2 | PASS | WmmaFmhaFwdV2Spec.__post_init__ in dnn-providers/hip-kernel-provider/rocke/library/kernels/gfx1151/wmma_fmha_fwd_v2.py:173-182, C++ v2_post_init_ok in dnn-providers/hip-kernel-provider/rocke/platform/cpp/instances/gfx1151/wmma_fmha_fwd_v2.cpp:144-154, and… |
| 3 | PASS | No shapes are demoted to a slow fallback; paged/query-option requests are routed to the prefill kernel (dnn-providers/hip-kernel-provider/rocke/library/dispatch/attention/gfx1151.py:568-569) and unsupported hipDNN graph variants are loudly declined at INFO… |
| 4 | NEEDS-INFO | Resolved since previous review: WmmaFmhaFwdV2Spec.__post_init__ in dnn-providers/hip-kernel-provider/rocke/library/kernels/gfx1151/wmma_fmha_fwd_v2.py:173-176 now enforces if type(value) is not bool: raise TypeError(...) for dense_q and seqused_q (matching dnn-providers/hip-kernel-provider/rocke/library/dispatch/attention/common.py:433-436), and… |
| 5 | PASS | Newly opened gates (dense_q, seqused_q, separate block_table_v, FP32 sinks, and >4 GiB paged prefill K/V caches) are tested in dnn-providers/hip-kernel-provider/rocke/library/tests/test_wmma_fmha_fwd_v2_features.py:168-247… |
| 6 | PASS | Spec validation in dnn-providers/hip-kernel-provider/rocke/library/kernels/gfx1151/wmma_fmha_fwd_v2.py:173-182 and dnn-providers/hip-kernel-provider/rocke/platform/cpp/instances/gfx1151/wmma_fmha_fwd_v2.cpp:144-154 only rejects functionally invalid flag types and… |
§C graded 10/10
| # | Verdict | Evidence |
|---|---|---|
| 1 | PASS | dnn-providers/hip-kernel-provider/rocke/platform/python/rocke/helpers/rdna_attention.py:213 defines the shared rdna_attention_scaled_ptr helper, dnn-providers/hip-kernel-provider/rocke/library/kernels/gfx1151/wmma_fmha_decode.py:751 reuses it across batch/token/bias… |
| 2 | PASS | dnn-providers/hip-kernel-provider/rocke/library/kernels/gfx1151/wmma_fmha_fwd_v2.py:467 keeps the single public build_wmma_fmha_fwd_v2(spec) builder while adding the new behavior through spec fields declared at… |
| 3 | PASS | dnn-providers/hip-kernel-provider/rocke/library/kernels/gfx1151/wmma_fmha_fwd_v2.py:222 and dnn-providers/hip-kernel-provider/rocke/platform/cpp/instances/gfx1151/wmma_fmha_fwd_v2.cpp:804 add kernel-name suffixes for codegen/layout options (_denseq and _sequsedq), not… |
| 4 | PASS | dnn-providers/hip-kernel-provider/rocke/platform/python/rocke/helpers/rdna_attention.py:213 and dnn-providers/hip-kernel-provider/rocke/platform/cpp/helpers/rdna_attention.cpp:214 place the arch-independent 64-bit pointer-scaling logic in the shared RDNA attention… |
| 5 | PASS | dnn-providers/hip-kernel-provider/src/engines/kernel_ingestor_engine/packs/Gfx1151WmmaAttentionNative.cpp:62 and dnn-providers/hip-kernel-provider/rocke/library/dispatch/attention/README.md:525 confine the changes to the RDNA prefill/decode attention path and its… |
| 6 | PASS | dnn-providers/hip-kernel-provider/rocke/platform/python/rocke/helpers/rdna_attention.py:213 and dnn-providers/hip-kernel-provider/rocke/platform/cpp/helpers/rdna_attention.cpp:214 introduce the named rdna_attention_scaled_ptr helper (and its C++ twin) and refactor… |
| 7 | N/A | This PR is a feature addition (paged/ragged SDPA options, 64-bit cache addressing, and separate V block tables) that intentionally… |
| 8 | PASS | dnn-providers/hip-kernel-provider/rocke/platform/python/rocke/helpers/rdna_attention.py:213 and dnn-providers/hip-kernel-provider/rocke/platform/cpp/helpers/rdna_attention.cpp:214 factor the existing local batch_ptr/token_ptr/scaled_ptr lambdas from… |
| 9 | NEEDS-INFO | dnn-providers/hip-kernel-provider/rocke/library/kernels/gfx1151/wmma_fmha_fwd_v2.py:538, dnn-providers/hip-kernel-provider/rocke/platform/cpp/instances/gfx1151/wmma_fmha_fwd_v2.cpp:456, dnn-providers/hip-kernel-provider/rocke/library/kernels/gfx1151/wmma_fmha_decode.py:697, and dnn-providers/hip-kernel-provider/rocke/platform/cpp/instances/gfx1151/wmma_fmha_decode.cpp:974 apply the paged seqused_k clamp across both… |
| 10 | PASS | dnn-providers/hip-kernel-provider/rocke/platform/cpp/include/rocke/abi.h:112 and dnn-providers/hip-kernel-provider/rocke/platform/python/rocke/portable_ir/src/abi.py:84 bump the binary ABI to 13 for the widened C structs, while… |
§D graded 21/21
| # | Verdict | Evidence |
|---|---|---|
| 1 | PASS | New query layout options dense_q and seqused_q are declared as explicit WmmaFmhaFwdV2Spec fields with validation, name tags, selector wiring, and cache-key entries without reading os.environ… |
| 2 | PASS | The feature set update in gfx1151.py adds "dense_q" and "seqused_q" to the candidate's declared features without altering arches or transcribing kernel coverage numbers (dnn-providers/hip-kernel-provider/rocke/library/dispatch/attention/gfx1151.py:171). |
| 3 | PASS | _decode_refusal checks req.dense_q or req.seqused_q first and returns a specific refusal reason ("dense_q / seqused_q requests run on the prefill kernel"), and the selection test queries decode.admits(request) rather than _supports… |
| 4 | PASS | _prefill_spec and _decode_refusal are pure functions of req: AttentionRequest and do not introduce any host or device probes (dnn-providers/hip-kernel-provider/rocke/library/dispatch/attention/gfx1151.py:346… |
| 5 | PASS | dense_q and seqused_q are baked into kernel_name() (dnn-providers/hip-kernel-provider/rocke/library/kernels/gfx1151/wmma_fmha_fwd_v2.py:222) and included in the spec cache key / identity() when enabled while preserving existing hashes when off… |
| 6 | N/A | No dispatcher cohort routing is added or modified in this PR. |
| 7 | N/A | No heuristic tables or hand-pinned overrides in the Python dispatcher are modified in this PR. |
| 8 | N/A | No cohort definitions or spec override predicates are added or modified in this PR. |
| 9 | N/A | No new KernelCandidate or architecture is registered in the Python dispatcher registries. |
| 10 | PASS | Resolved since previous review: dense_q and seqused_q are now added to AttentionRequest (dnn-providers/hip-kernel-provider/rocke/library/dispatch/attention/common.py:213), wired into _prefill_spec… |
| 11 | N/A | No new dispatcher candidates are registered or promoted in the Python dispatcher registries. |
| 12 | N/A | No candidate priorities, candidate names, or custom rankers are added or modified in the Python dispatcher registry. |
| 13 | N/A | No new dispatcher wrapper or entry point is added, and pinned request handling in CandidateRegistry.select is not modified. |
| 14 | PASS | Adding block_table_v and sink_is_f32 to the prefill kernarg layout bumps ATTENTION_GFX1151_ABI to rocke-attention-gfx1151/v8 in dnn-providers/hip-kernel-provider/rocke/library/dispatch/attention/gfx1151.py:100 (tested in… |
| 15 | N/A | No new dispatcher candidate or arch module is added to the Python dispatcher registry. |
| 16 | N/A | No new operator family is introduced in this PR. |
| 17 | PASS | CPU routing and capability tests in dnn-providers/hip-kernel-provider/rocke/library/tests/dispatch/attention/test_gfx1151_attention_selection.py:798-873 verify prefill selection across RDNA arches, decode refusal via admits(), capability rejection on out-of-cohort… |
| 18 | N/A | Registry ratchets (require_build, require_binding, require_torch_binding) and exempt sets are not modified in this PR. |
| 19 | N/A | This PR does not touch run_unified_attention_torch or make claims about unified attention auto-routing registration. |
| 20 | PASS | Spec and builder changes in WmmaFmhaFwdV2Spec (dnn-providers/hip-kernel-provider/rocke/library/kernels/gfx1151/wmma_fmha_fwd_v2.py:128-129) and rdna_attention.py (dnn-providers/hip-kernel-provider/rocke/platform/python/rocke/helpers/rdna_attention.py:213-220)… |
| 21 | N/A | This PR adds new kernel/spec features (dense_q, seqused_q, block_table_v, sink_is_f32, 64-bit page addressing) and hipDNN… |
§E graded 17/17
| # | Verdict | Evidence |
|---|---|---|
| 1 | PASS | dnn-providers/hip-kernel-provider/rocke/platform/python/rocke/helpers/rdna_attention.py:48 imports only from rocke.core.ir, while library kernels import from platform (dnn-providers/hip-kernel-provider/rocke/library/kernels/gfx1151/wmma_fmha_fwd_v2.py:470). |
| 2 | PASS | dnn-providers/hip-kernel-provider/rocke/library/tests/test_wmma_fmha_fwd_v2_features.py:738 and 764 use rocke.runtime.hip_module (_resolve_hip and Runtime) rather than importing torch. |
| 3 | PASS | dnn-providers/hip-kernel-provider/rocke/library/tests/test_wmma_fmha_fwd_v2_features.py:755 gates optional dependencies with pytest.importorskip and uses numpy/Runtime without requiring torch. |
| 4 | PASS | dnn-providers/hip-kernel-provider/rocke/library/kernels/gfx1151/wmma_fmha_fwd_v2.py:469-473 and dnn-providers/hip-kernel-provider/rocke/library/dispatch/attention/bindings.py:611-612 add no upward imports across library layers. |
| 5 | PASS | dnn-providers/hip-kernel-provider/rocke/platform/python/rocke/helpers/rdna_attention.py:213-220 and dnn-providers/hip-kernel-provider/rocke/platform/cpp/bindings/prove_parity_binding.py:813-860 introduce no absolute or repo-escaping paths or shell scripts. |
| 6 | PASS | dnn-providers/hip-kernel-provider/rocke/library/kernels/gfx1151/wmma_fmha_fwd_v2.py:565-577 and dnn-providers/hip-kernel-provider/rocke/platform/python/rocke/helpers/rdna_attention.py:568-633 show intentional emitter edits matched with their C++ twins rather than lint… |
| 7 | PASS | dnn-providers/hip-kernel-provider/rocke/platform/dsl_docs/instances/attention.md:38-41 and dnn-providers/hip-kernel-provider/rocke/library/dispatch/attention/README.md:492-547 update the instance and dispatch reference docs alongside the new dense_q, seqused_q, and… |
| 8 | PASS | dnn-providers/hip-kernel-provider/rocke/library/kernels/gfx1151/wmma_fmha_fwd_v2.py:29-61 and dnn-providers/hip-kernel-provider/src/engines/kernel_ingestor_engine/packs/Gfx1151WmmaAttentionNative.cpp:83-119 use descriptive spec/RFC references without Jira IDs, PR… |
| 9 | PASS | dnn-providers/hip-kernel-provider/rocke/library/kernels/gfx1151/wmma_fmha_fwd_v2.py:128-129 and dnn-providers/hip-kernel-provider/src/engines/kernel_ingestor_engine/packs/Gfx1151WmmaAttentionNative.cpp:762-767 keep all changes scoped to paged/ragged SDPA support in… |
| 10 | NEEDS-INFO | Unresolved from previous review: merge-simulation against other open PRs touching the same files (e.g. abi.h, llvm_gfx_all.json, Gfx1151WmmaAttentionNative.cpp) still cannot be verified from the PR diff alone. |
| 11 | PASS | dnn-providers/hip-kernel-provider/rocke/platform/dsl_docs/instances/attention.md:38-41 and dnn-providers/hip-kernel-provider/rocke/library/dispatch/attention/README.md:492-507 are required deliverable docs; no scratch notes or throwaway markdown files are committed. |
| 12 | PASS | dnn-providers/hip-kernel-provider/rocke/library/tests/parity/gfx1151_wmma_fmha_fwd_v2_emit.py:235-240 and dnn-providers/hip-kernel-provider/rocke/library/tests/test_wmma_fmha_fwd_v2_spec.py:289 cover both gfx11-generic and gfx12-generic RDNA families (CDNA is scoped… |
| 13 | PASS | dnn-providers/hip-kernel-provider/rocke/library/kernels/gfx1151/wmma_fmha_fwd_v2.py:565-577 and dnn-providers/hip-kernel-provider/rocke/platform/cpp/instances/gfx1151/wmma_fmha_fwd_v2.cpp:516-531 replace page_rsrc with page_head and update k_rsrc/v_rsrc paged usage… |
| 14 | PASS | dnn-providers/hip-kernel-provider/rocke/platform/cpp/include/rocke/abi.h:111-116 and dnn-providers/hip-kernel-provider/rocke/library/kernels/gfx1151/wmma_fmha_decode.py:751-777 preserve upstream history and comments while surgical additions/refactorings are applied. |
| 15 | PASS | dnn-providers/hip-kernel-provider/rocke/platform/python/rocke/helpers/rdna_attention.py:263-270 and 577-633, plus dnn-providers/hip-kernel-provider/rocke/platform/cpp/helpers/rdna_attention.cpp:714-818, use all newly added parameters (block_table_v, k_ptr, v_ptr… |
| 16 | N/A | This item governs reviewer severity labeling and does not apply when there are no FAIL rows in this section. |
| 17 | PASS | dnn-providers/hip-kernel-provider/rocke/library/dispatch/attention/README.md:492-547 and the PR description contain no absolute or relative performance claims. |
§F graded 11/11
| # | Verdict | Evidence |
|---|---|---|
| 1 | PASS | Branch users/dpeabody/hipdnn-rdna-paged-attn matches users/<user>/... (PR metadata; diff context at dnn-providers/hip-kernel-provider/rocke/library/dispatch/attention/README.md:489). |
| 2 | PASS | title feat(hip-kernel-provider): paged/ragged SDPA in Gfx1151WmmaAttention matches type(scope): summary and is 68 characters |
| 3 | PASS | description is at least 30 characters and contains an issue reference |
| 4 | PASS | test file dnn-providers/hip-kernel-provider/rocke/library/tests/dispatch/attention/test_attention.py is in the diff |
| 5 | PASS | All changed files in the diff are source, test, documentation, or descriptor/golden files (e.g., dnn-providers/hip-kernel-provider/rocke/library/dispatch/attention/README.md:492) with no keys, certs, or secrets. |
| 6 | NEEDS-INFO | Local pre-commit execution status before push cannot be verified from the diff alone. |
| 7 | NEEDS-INFO | Bot, Copilot, and human review thread states and deferred-item ticket IDs are not included in the prompt or diff. |
| 8 | N/A | This PR targets develop directly (Base: develop) rather than being a stacked PR on a feature branch. |
| 9 | PASS | The PR keeps the full kernel, dispatch, native pack, catalog, and test changes together without artificial splitting (e.g., dnn-providers/hip-kernel-provider/src/integration_tests/kernel_ingestor_engine/IntegrationGpuGfx1151WmmaAttentionPaged.cpp:1). |
| 10 | PASS | Tests added in the diff exercise real spec validation, dispatch binding, I32 extent guards, catalog census, and GPU numeric/integration behavior (e.g., `dnn-providers/hip-kernel-provider/rocke/library/tests/dispatch/attention/test_gfx1151_attention_dispatch_numeric.py… |
| 11 | NEEDS-INFO | Although the title and body appear in the prompt metadata, live gh api / gh pr view verification of the GitHub API fields cannot be confirmed from the diff. |
§G graded 34/34
| # | Verdict | Evidence |
|---|---|---|
| 1 | PASS | Parity rows in dnn-providers/hip-kernel-provider/rocke/library/tests/parity/gfx1151_wmma_fmha_fwd_v2_emit.py:97-109 are paired with float64/CPU on-device numerical tests in dnn-providers/hip-kernel-provider/rocke/library/tests/test_wmma_fmha_fwd_v2_features.py:170-247… |
| 2 | NEEDS-INFO | On-device numerical tests with fp64/fp32 references and bf16/fp16 tolerances are present (e.g. dnn-providers/hip-kernel-provider/rocke/library/tests/test_wmma_fmha_fwd_v2_features.py:707,750 and dnn-providers/hip-kernel-provider/src/integration_tests/kernel_ingestor_engine/IntegrationGpuGfx1151WmmaAttentionPaged.cpp:324), but the PR body is truncated before the test execution summary so on-silicon pass confirmation… |
| 3 | NEEDS-INFO | Golden hashes are updated in dnn-providers/hip-kernel-provider/rocke/platform/tests/instances/differential/golden/llvm_gfx_all.json (diff omitted) and parity tables are updated in dnn-providers/hip-kernel-provider/rocke/platform/cpp/bindings/prove_parity_binding.py:813-825, but the PR description is truncated before the Tests section so local gate sweep results (tools/run_checks.py, llvm23 byte-identity… |
| 4 | PASS | Every new capability has targeted tests that fail without the change: dense_q/seqused_q/split block tables/f32 sinks/clamped seqused_k (dnn-providers/hip-kernel-provider/rocke/library/tests/test_wmma_fmha_fwd_v2_features.py:170-247), >4 GiB paged cache… |
| 5 | PASS | Tests exercise the shipped entry points (e.g. dispatch_attention in dnn-providers/hip-kernel-provider/rocke/library/tests/dispatch/attention/test_gfx1151_attention_selection.py:798-872, hipDNN frontend Graph in… |
| 6 | PASS | Prose claims in PR body, docstrings, and code comments match the implementation and tests on the new head: 64-bit page/head addressing and page-extent I32 bound (dnn-providers/hip-kernel-provider/rocke/library/dispatch/attention/bindings.py:683… |
| 7 | N/A | This PR does not add or modify a design document defining a new fp32 parity reference oracle. |
| 8 | PASS | Test harnesses do not silently fall back on unsupported dtypes or frontend configurations; they explicitly skip via pytest.importorskip or GTEST_SKIP with a reason (dnn-providers/hip-kernel-provider/rocke/library/tests/test_wmma_fmha_fwd_v2_features.py:755… |
| 9 | N/A | This PR does not make new LDS or occupancy load-bearing numerical claims. |
| 10 | N/A | This PR is a functional feature addition (paged/ragged SDPA support) and makes no performance or latency claims. |
| 11 | N/A | This PR adds functional graph support for hipDNN paged SDPA rather than an optimization claim for a benchmarked shape. |
| 12 | PASS | Cross-engine parity is maintained across Python and C++ for both the helper (dnn-providers/hip-kernel-provider/rocke/platform/python/rocke/helpers/rdna_attention.py:213 and dnn-providers/hip-kernel-provider/rocke/platform/cpp/helpers/rdna_attention.cpp:214) and… |
| 13 | N/A | This PR does not introduce or recommend any new raw LLVM intrinsic mangled symbols. |
| 14 | N/A | This PR makes no hardware bottleneck classifications. |
| 15 | N/A | This is a review of the provided PR diff directly from the head commit, not a multi-worktree branch inspection task. |
| 16 | N/A | This PR implements a completed feature rather than proposing an unverified plan. |
| 17 | N/A | This PR does not add new standalone shipped CLI scripts. |
| 18 | PASS | New host-side guard _check_page_extent_i32 in dnn-providers/hip-kernel-provider/rocke/library/dispatch/attention/bindings.py:683 is wired into _rdna_validate_and_collect (bindings.py:908), documents its scope (page and head-dim axes fit I32 while block and… |
| 19 | PASS | Fixtures in dnn-providers/hip-kernel-provider/rocke/library/tests/test_wmma_fmha_fwd_v2_features.py:574 and dnn-providers/hip-kernel-provider/src/integration_tests/kernel_ingestor_engine/IntegrationGpuGfx1151WmmaAttentionPaged.cpp:178 allocate spare unreferenced… |
| 20 | N/A | This PR does not introduce or modify a claim-ban detector or word/digit regex linter. |
| 21 | N/A | This PR does not add or modify performance measurement or benchmarking scripts. |
| 22 | NEEDS-INFO | dnn-providers/hip-kernel-provider/rocke/platform/tests/instances/differential/golden/llvm_gfx_all.json is omitted from the diff to fit the model budget, so we cannot verify from the diff alone that all new golden entries (indices 170..185 from dnn-providers/hip-kernel-provider/rocke/library/tests/parity/gfx1151_wmma_fmha_fwd_v2_emit.py:29) record bytes + sha256 without build errors across all flavors, nor… |
| 23 | N/A | This PR is a functional feature addition for paged/ragged SDPA and makes no performance speedup claims. |
| 24 | PASS | The re-blessed paged entries in dnn-providers/hip-kernel-provider/rocke/platform/tests/instances/differential/golden/llvm_gfx_all.json correspond to the 64-bit page/head addressing, seqused_k clamping, sink_is_f32, and block_table_v changes in… |
| 25 | N/A | This PR adds functional support for hipDNN paged/ragged SDPA graphs and 64-bit cache addressing rather than a codegen… |
| 26 | N/A | This PR does not include benchmark comparisons or performance claims. |
| 27 | N/A | This PR does not include performance A/B experiments. |
| 28 | N/A | This PR does not present performance delta tables or counterfactual benchmark arms. |
| 29 | N/A | This PR does not swap guarded math primitives for fast approximations. |
| 30 | N/A | This PR makes no register-spill or scratch-reduction claims. |
| 31 | N/A | This PR does not claim to fix a performance defect against a baseline measurement. |
| 32 | N/A | This PR does not report measured performance or resource numbers. |
| 33 | NEEDS-INFO | dnn-providers/hip-kernel-provider/src/engines/kernel_ingestor_engine/packs/Gfx1151WmmaAttentionNative.cpp:1320 and dnn-providers/hip-kernel-provider/src/tests/engines/kernel_ingestor_engine/packs/TestGfx1151WmmaAttentionPaged.cpp are truncated/omitted in the diff, so the changed routing decision logic between prefill and decode on dense vs paged graphs cannot be fully inspected or verified against a… |
| 34 | PASS | In dnn-providers/hip-kernel-provider/rocke/platform/python/rocke/helpers/rdna_attention.py:1065 (and C++ mirror dnn-providers/hip-kernel-provider/rocke/platform/cpp/helpers/rdna_attention.cpp:1693), empty and o_row_off for O are computed independently of… |
§H graded 5/5
| # | Verdict | Evidence |
|---|---|---|
| 1 | PASS | Buffer SRDs and 64-bit page bases are sized from runtime tensor shapes/strides and host-validated before launch via _check_page_extent_i32 / _check_byte_extent_i32 in dnn-providers/hip-kernel-provider/rocke/library/dispatch/attention/bindings.py:683-705 and… |
| 2 | PASS | Both block_table and block_table_v indirection values (phys_k, phys_v, phys) are explicitly range-checked on device by page_ok (0 <= phys < num_blocks) before 64-bit base pointer offsets are formed, clamping out-of-range page IDs to 0 and zeroing the… |
| 3 | PASS | Power-of-two page_block_size is enforced in the spec (ps <= 0 or ps & (ps - 1) in dnn-providers/hip-kernel-provider/rocke/library/kernels/gfx1151/wmma_fmha_fwd_v2.py:169-170) and in the native matcher ((page & (page - 1)) != 0 in… |
| 4 | PASS | The kernel loops over runtime n_tiles = cdiv(smax(sk, 0), BN) where sk = smin(seqused_k[b], seqlen_k) is clamped in dnn-providers/hip-kernel-provider/rocke/library/kernels/gfx1151/wmma_fmha_fwd_v2.py:538 (and C++ twin at… |
| 5 | PASS | Host validations for dense_q, seqused_q, paged K/V extents, table capacity (seqlen_k <= capacity), and non-empty caches (num_blocks >= 1) are in dnn-providers/hip-kernel-provider/rocke/library/dispatch/attention/bindings.py:900-910, 1047-1174 and… |
§I graded 6/6
| # | Verdict | Evidence |
|---|---|---|
| 1 | N/A | This PR adds paged/ragged SDPA kernel features and hipDNN native graph matching rather than introducing a tuning or routing search… |
| 2 | PASS | Invalid spec combinations and unsupported graph features are loudly rejected with descriptive errors or logged causes (e.g., dnn-providers/hip-kernel-provider/rocke/library/kernels/gfx1151/wmma_fmha_fwd_v2.py:173… |
| 3 | PASS | Parity, feature, and catalog census tests cover curated combinations and verify catalog invariants across the new dense_q, seqused_q, and split block-table axes (dnn-providers/hip-kernel-provider/rocke/library/tests/parity/gfx1151_wmma_fmha_fwd_v2_emit.py:97… |
| 4 | N/A | This PR does not add or modify benchmark reporting tables comparing shipped vs. achievable selection performance. |
| 5 | PASS | Generated catalog descriptors are updated alongside census tests checking page sizes and architectures against the loaded pack set (dnn-providers/hip-kernel-provider/src/engines/kernel_ingestor_engine/descriptors/rocKE/gfx1151_wmma_attention/gfx1151_wmma_attention.kmd.… |
| 6 | PASS | The new helper rdna_attention_scaled_ptr exposes a general 64-bit scaled pointer arithmetic primitive rather than a hardware-instruction-specific op (dnn-providers/hip-kernel-provider/rocke/platform/python/rocke/helpers/rdna_attention.py:213… |
§J graded 7/7
| # | Verdict | Evidence |
|---|---|---|
| 1 | N/A | The Markdown changes in dnn-providers/hip-kernel-provider/rocke/library/dispatch/attention/README.md and… |
| 2 | PASS | Symbols introduced in dnn-providers/hip-kernel-provider/rocke/library/dispatch/attention/README.md:492-547 and dnn-providers/hip-kernel-provider/rocke/platform/dsl_docs/instances/attention.md:38-41 (block_table_v, dense_q, seqused_q, use_sinks/sinks) are… |
| 3 | PASS | dnn-providers/hip-kernel-provider/rocke/platform/dsl_docs/instances/attention.md:38-41 and dnn-providers/hip-kernel-provider/rocke/library/dispatch/attention/README.md:492-547 state only the interface and catalog delta and link to… |
| 4 | PASS | dnn-providers/hip-kernel-provider/rocke/library/dispatch/attention/README.md:495-547 and dnn-providers/hip-kernel-provider/rocke/platform/dsl_docs/instances/attention.md:38-41 accurately attribute block_table_v (new in v8), dense_q, seqused_q, and paged… |
| 5 | N/A | The PR makes no algorithmic scaling or asymptotic complexity claims in documentation or comments. |
| 6 | PASS | Catalog constants in dnn-providers/hip-kernel-provider/rocke/library/dispatch/attention/README.md:525-528 (buckets 64/96/128/256, page sizes 16/32/64) match GFX1151_WMMA_ATTENTION_PAGE_SIZES in… |
| 7 | N/A | No new Markdown design or algorithm document files are added in this PR; only existing README.md and attention.md files are… |
§K graded 13/13
| # | Verdict | Evidence |
|---|---|---|
| 1 | N/A | No tuned candidate definitions or make_tuned_candidate calls are added or modified in this PR. |
| 2 | PASS | New WmmaFmhaFwdV2Spec fields dense_q and seqused_q (dnn-providers/hip-kernel-provider/rocke/library/kernels/gfx1151/wmma_fmha_fwd_v2.py:128) are explicitly classified in WMMA_PROBLEM_FIELDS… |
| 3 | PASS | common.py omits dense_q and seqused_q when False so existing pins keep their spec_hash without a migration (dnn-providers/hip-kernel-provider/rocke/library/dispatch/attention/common.py:1024), and test_gfx1151_attention_selection.py verifies both… |
| 4 | N/A | KnobSpace.defaults() and recorded() are not touched; the new spec fields are problem-level layout options in… |
| 5 | N/A | No knob drop, inertness, or refusal rules are modified in this PR. |
| 6 | N/A | KnobSpace.validate and tuning_specs.py are not modified in this PR. |
| 7 | N/A | KnobSpace.base(req) is not modified in this PR. |
| 8 | N/A | Knob value normalization and type coercion are not modified in this PR. |
| 9 | PASS | Replay verification via result.kernel_id.spec_hash is preserved for stored prefill pins (dnn-providers/hip-kernel-provider/rocke/library/tests/dispatch/attention/test_gfx1151_attention_selection.py:972). |
| 10 | N/A | KnobSpace.find and bare tuning_id resolution are not modified in this PR. |
| 11 | N/A | Sweep entry points and iteration helpers are not modified in this PR. |
| 12 | N/A | Production knob stacks, accept_default, and auto candidate routing are not modified in this PR. |
| 13 | N/A | No benchmark scripts or sweep harnesses are modified in this PR. |
§L graded 10/10
| # | Verdict | Evidence |
|---|---|---|
| 1 | N/A | The diff does not add or modify any gfx architecture target or ISA backend registry. |
| 2 | N/A | The diff does not add or modify any MMA or WMMA op_id or fragment layout map. |
| 3 | N/A | The diff does not change MMA atom selection, fragment lane maps, or accumulator layouts. |
| 4 | N/A | The diff does not add or change wave size handling or target wave-size queries. |
| 5 | N/A | The diff does not touch CDNA/gfx1250 arch-specific recipes, transpose-LDS reads, or AsyncTileLoader. |
| 6 | N/A | The diff does not add pipeline vocabulary to ArchTarget/core/arch or import CK Tile dispatcher modules. |
| 7 | N/A | The diff does not add or change logical dtypes, bit packing routines, or FP8 dialect rules. |
| 8 | N/A | The diff does not touch scaled-MMA contracts or scaled WMMA/MFMA emission. |
| 9 | N/A | The diff does not introduce a new architecture target. |
| 10 | N/A | The diff does not modify intrinsic arch-domain artifacts or LLVM flavor membership lists. |
§M graded 10/10
| # | Verdict | Evidence |
|---|---|---|
| 1 | PASS | Host booleans (spec.dense_q, spec.seqused_q, live_rows is not None, row_paged) guard Python branches, while runtime SSA predicates use b.cmp_* with scf_if and select… |
| 2 | N/A | The diff does not touch legalize_for_target or any target legalization pass. |
| 3 | PASS | No wire format changes are made to ck.dsl.ir/v1, and kernel parameter order is identical between Python and C++ builders and verified by spec tests (dnn-providers/hip-kernel-provider/rocke/library/kernels/gfx1151/wmma_fmha_fwd_v2.py:419… |
| 4 | N/A | The diff only bumps BINARY_ABI in portable_ir/src/abi.py and does not touch portable IR schemas, importers, magic division, or… |
| 5 | N/A | The diff does not use or modify AsyncTileLoader or async DRAM-to-LDS operations. |
| 6 | PASS | The early return is CTA-uniform (q_start vs span), tile-paged loads select page 0 (safe_k, safe_v) and use OOB buffer sentinels (at_tile), and 64-bit row-paged loads clamp invalid indices/offsets (blk, phys, within) to 0 before issuing global_load_vN… |
| 7 | PASS | Buffer descriptors are constructed via b.buffer_rsrc / rocke_b_buffer_rsrc without hard-coding buffer-resource word 3 or using TemporalHint.STREAMING (dnn-providers/hip-kernel-provider/rocke/platform/python/rocke/helpers/rdna_attention.py:582… |
| 8 | N/A | The diff does not add any new LLVM intrinsic declarations or modify backend selection logic. |
| 9 | PASS | Public C headers under cpp/include/rocke/ keep their names and extern "C" declarations, and ROCKE_ABI_VERSION / BINARY_ABI are bumped to 13 for the struct layout changes (dnn-providers/hip-kernel-provider/rocke/platform/cpp/include/rocke/abi.h:116… |
| 10 | N/A | The diff does not touch !dbg metadata emission, Debug Info Version flags, or DWARF consumers. |
§N graded 18/18
| # | Verdict | Evidence |
|---|---|---|
| 1 | PASS | rdna_attention_scaled_ptr is added to dnn-providers/hip-kernel-provider/rocke/platform/python/rocke/helpers/rdna_attention.py:213 because both wmma_fmha_fwd_v2.py:472 and wmma_fmha_decode.py:607 share it, and it lands with its C++ twin… |
| 2 | NEEDS-INFO | Resolved from previous NEEDS-INFO: both dnn-providers/hip-kernel-provider/rocke/library/kernels/gfx1151/wmma_fmha_decode.py:751 and dnn-providers/hip-kernel-provider/rocke/library/kernels/gfx1151/wmma_fmha_fwd_v2.py:569 (along with their C++ mirrors in dnn-providers/hip-kernel-provider/rocke/platform/cpp/instances/gfx1151/wmma_fmha_decode.cpp:1103 and… |
| 3 | N/A | The diff does not touch or use SoftwarePipeline. |
| 4 | N/A | The diff does not add or modify epilogue fusion or fused GEMM builders. |
| 5 | N/A | The diff does not touch rocke.helpers.tiling or add any tiling API. |
| 6 | N/A | The diff does not add or modify front-door/toolbox tiling helpers or LayoutStyle. |
| 7 | N/A | The diff does not use helpers/tiling/ layout overrides or TileMma. |
| 8 | N/A | The diff does not touch helpers/tiling/ LDS allocation or tiling memory verbs. |
| 9 | N/A | The diff does not use helpers/tiling/ N-D tensor axis roles or FlowStage. |
| 10 | N/A | The diff does not modify helpers/tiling/ emission, LDS staging, or MMA issue. |
| 11 | N/A | The diff does not touch helpers/tiling/ traits, lds_swizzle, or derive_sched_group_counts. |
| 12 | N/A | The PR makes no LDS bank-conflict or coalescing claims under helpers/tiling/. |
| 13 | N/A | The diff does not touch WarpDistributionEncoding, make_tile_desc, or RegisterMapper. |
| 14 | N/A | The diff does not use helpers/tiling/ verbs or add a tiling pipeline variant. |
| 15 | N/A | The diff does not modify helpers/tiling/ descriptor allocation or swizzled bounds checks. |
| 16 | N/A | The diff does not touch optimize_layout or evaluate_transform. |
| 17 | N/A | The diff does not touch tiling_recorder.py, round-trip analysis, or tiling visualization. |
| 18 | N/A | The diff does not touch InterleavedStyle or demo kernels under helpers/tiling/kernels/. |
§O graded 16/16
| # | Verdict | Evidence |
|---|---|---|
| 1 | N/A | This PR does not modify core/verify.py, cpp/core/verify.cpp, address spaces, or required IR attributes. |
| 2 | PASS | Early-exit scf.if guards properly terminate the then-region without emitting after the terminator in dnn-providers/hip-kernel-provider/rocke/library/kernels/gfx1151/wmma_fmha_fwd_v2.py:529 and… |
| 3 | PASS | IR emission in dnn-providers/hip-kernel-provider/rocke/platform/python/rocke/helpers/rdna_attention.py:213 and dnn-providers/hip-kernel-provider/rocke/platform/cpp/helpers/rdna_attention.cpp:214 uses deterministic builder methods and ordered sequences without set… |
| 4 | PASS | Module singletons (I16, I32, I64) and rocke_i16() / rocke_i32() / rocke_i64() are used in dnn-providers/hip-kernel-provider/rocke/platform/python/rocke/helpers/rdna_attention.py:55 and… |
| 5 | N/A | This PR does not touch core/passes.py, cpp/core/passes.cpp, or op purity tables. |
| 6 | N/A | This PR does not modify constant folding in either engine. |
| 7 | N/A | This PR does not alter kernel attributes, CodegenPolicy attribute contracts, or cluster_dims. |
| 8 | NEEDS-INFO | Resolved from previous review: CodegenPolicy and apply_codegen_policy are present in the current diff context at dnn-providers/hip-kernel-provider/rocke/library/kernels/gfx1151/wmma_fmha_decode.py:606 and dnn-providers/hip-kernel-provider/rocke/library/kernels/gfx1151/wmma_fmha_fwd_v2.py:469, and new spec variants receive distinct kernel names and cache entries… |
| 9 | N/A | This PR does not touch waitcnt encoding, barrier lowering, or optimization_barrier. |
| 10 | N/A | This PR does not modify lowerer attribute validation or _param_llvm_type. |
| 11 | N/A | This PR does not change ArchTarget queries or target ID resolution in rocke. |
| 12 | N/A | This PR does not modify target catalog width/size caps or add default arch or "gfx950" fallbacks. |
| 13 | PASS | Modifying rocke_rdna_attn_params_t (dnn-providers/hip-kernel-provider/rocke/platform/cpp/include/rocke/helper_rocke.helpers.rdna_attention.h:135) and rocke_wmma_fmha_fwd_v2_spec_t… |
| 14 | PASS | C headers and sources follow rocke_* snake_case and correspondence prefaces in dnn-providers/hip-kernel-provider/rocke/platform/cpp/include/rocke/helper_rocke.helpers.rdna_attention.h:14, and additions to… |
| 15 | N/A | This PR does not touch engine fallback mechanisms, _resolve_llvm_flavor, lower_hip.py, or lower_cktile.py. |
| 16 | PASS | Full Python and C++ instance and helper parity is maintained across dnn-providers/hip-kernel-provider/rocke/platform/cpp/instances/gfx1151/wmma_fmha_fwd_v2.cpp:144 and dnn-providers/hip-kernel-provider/rocke/platform/cpp/bindings/prove_parity_binding.py:813… |
§P graded 14/14
| # | Verdict | Evidence |
|---|---|---|
| 1 | PASS | Builders keep the exact (spec, *, arch) signature (dnn-providers/hip-kernel-provider/rocke/library/kernels/gfx1151/wmma_fmha_fwd_v2.py:465, wmma_fmha_decode.py:604), and the new dense_q and seqused_q knobs are defaulted fields on WmmaFmhaFwdV2Spec… |
| 2 | PASS | The kernarg signature order for sink_is_f32, seqused_q, and block_table_v is updated identically in Python (dnn-providers/hip-kernel-provider/rocke/library/kernels/gfx1151/wmma_fmha_fwd_v2.py:419-435) and C++… |
| 3 | PASS | Test harnesses obtain launch geometry directly from wmma_fmha_fwd_v2_grid without recomputing it (dnn-providers/hip-kernel-provider/rocke/library/tests/test_wmma_fmha_fwd_v2_features.py:656-658, 894-896). |
| 4 | N/A | This PR does not modify LDS allocation math or LDS capacity gates. |
| 5 | N/A | This PR modifies RDNA attention kernels (wmma_fmha_fwd_v2 and wmma_fmha_decode), not WMMA/RDNA GEMM or… |
| 6 | N/A | The paged prefill attention kernel requires zero workspace (`dnn-providers/hip-kernel-provider/src/integration_tests/kernel_ingesto… |
| 7 | N/A | This PR does not touch Stream-K, persistent GEMM, GroupedGemmLauncher, or SUPPORT_MATRIX.md. |
| 8 | N/A | This PR does not add or modify quantization helpers or MX GEMM scale decoding. |
| 9 | N/A | This PR does not introduce or change numeric emulation or data-type decode paths. |
| 10 | N/A | This PR does not add or modify example manifests (helpers/manifest.py or run_manifest). |
| 11 | NEEDS-INFO | GPU tests properly synchronize the stream via rt.sync() before reading device outputs and before freeing buffers (dnn-providers/hip-kernel-provider/rocke/library/tests/test_wmma_fmha_fwd_v2_features.py:665-672, 903-912). — no diff line supports this verdict |
| 12 | N/A | This PR does not touch COMGR/HIP library loading or fix a hipError launch failure. |
| 13 | PASS | Host-side changes to the attention spec cache key (dnn-providers/hip-kernel-provider/rocke/library/dispatch/attention/common.py:1017-1026), paged K/V I32 page-extent guards (dnn-providers/hip-kernel-provider/rocke/library/dispatch/attention/bindings.py:683-704… |
| 14 | PASS | No examples/ expected.json files are touched, and the shared 64-bit pointer offset helper rdna_attention_scaled_ptr is placed in dnn-providers/hip-kernel-provider/rocke/platform/python/rocke/helpers/rdna_attention.py:213-220 (with its C++ twin in… |
§Q graded 7/7
| # | Verdict | Evidence |
|---|---|---|
| 1 | N/A | No Python test files were added or removed under rocke/platform/tests/ or rocke/library/tests/, so the installed rocke_pytest… |
| 2 | PASS | New dense_q, seqused_q, and shipped paged catalog variants across gfx11-generic and gfx12-generic are added to the C and Python parity emitters and binding parity checker in… |
| 3 | PASS | Cross-engine parity is checked in dnn-providers/hip-kernel-provider/rocke/platform/cpp/bindings/prove_parity_binding.py:813 and dnn-providers/hip-kernel-provider/rocke/library/tests/parity/gfx1151_wmma_fmha_fwd_v2_emit.py:97, while numeric correctness is verified… |
| 4 | NEEDS-INFO | The diff for dnn-providers/hip-kernel-provider/rocke/platform/tests/instances/differential/golden/llvm_gfx_all.json is omitted from the provided diff, so re-blessing across every entry in LLVM_FLAVORS (llvm20, llvm22, llvm23, llvm24, and canonical IR) cannot be verified from the diff lines. |
| 5 | PASS | GPU tests check the active device and free memory and skip cleanly when unavailable or not RDNA (dnn-providers/hip-kernel-provider/rocke/library/tests/test_wmma_fmha_fwd_v2_features.py:748… |
| 6 | N/A | This PR does not refactor any CI gate, differential runner, or parity harness execution logic. |
| 7 | N/A | This PR does not change backend resolution or ROCKE_BACKEND / ROCKE_CPP_STRICT fallback rejection contracts. |
§R graded 11/11
| # | Verdict | Evidence |
|---|---|---|
| 1 | N/A | This PR adds functional support for cuDNN paged/ragged SDPA semantics (dense_q, seqused_q, block_table_v, 64-bit page base… |
| 2 | N/A | The diff changes no performance levers or arch-specific optimization recipes; dense_q and seqused_q are problem-fixed fields… |
| 3 | N/A | No performance levers, tile sizes, LDS layouts, or prefetch pipelines are modified in this functional feature PR. |
| 4 | N/A | This PR makes no performance or primitive-selection claims and changes no performance levers. |
| 5 | N/A | The kernel does not use rocke.helpers.tiling and its per-lane 16-byte/8-byte vector load widths and striding patterns are… |
| 6 | N/A | Neither LDS layouts nor MMA layouts are modified in this diff. |
| 7 | N/A | The PR makes no bottleneck claims and does not target a performance bottleneck. |
| 8 | N/A | No waits, barriers, or scheduling hints are added, removed, or modified in this diff. |
| 9 | N/A | This PR is not a performance regression fix and does not trade performance across shapes via a tuning lever. |
| 10 | N/A | The PR does not change compiler/codegen policies, claim assembly parity, or report benchmark measurements. |
| 11 | N/A | No new optimization lessons, heuristics, case studies, or dsl_probes tools are introduced in this PR. |
§S graded 8/8
| # | Verdict | Evidence |
|---|---|---|
| 1 | PASS | New public symbols and fields (rdna_attention_scaled_ptr in dnn-providers/hip-kernel-provider/rocke/platform/python/rocke/helpers/rdna_attention.py:61, rocke_rdna_attention_scaled_ptr in… |
| 2 | PASS | Existing callers and tests for wmma_fmha_fwd_v2_signature, rdna_attention_fwd_body, AttentionRequest, and bind_rdna_prefill_torch are updated in the same PR with new expectations (e.g., sink_is_f32 and block_table_v in… |
| 3 | N/A | The diff does not remove or rename any public API symbol. |
| 4 | PASS | rdna_attention_scaled_ptr is added to __all__ in dnn-providers/hip-kernel-provider/rocke/platform/python/rocke/helpers/rdna_attention.py:65, resolves to the function defined at line 213, and is imported and exercised by… |
| 5 | PASS | Tests assert both positive outputs (IR serialization, signature lists, cache hashes, and float64 reference numerics) and exact negative ValueError and TypeError exceptions on invalid parameters in… |
| 6 | PASS | CPU-checkable contracts (spec validation, kernel names, signature ordering, dispatch argument collection, I32 extent guards, candidate selection, and cache key hashing) are tested without a GPU or torch in… |
| 7 | PASS | The C/C++ twins (rocke_wmma_fmha_fwd_v2_spec_t, rocke_rdna_attn_params_t, rocke_rdna_attention_scaled_ptr, and the pybind11 bridge in dnn-providers/hip-kernel-provider/rocke/platform/cpp/bindings/rocke_engine.cpp:4663) are wired into the cross-engine parity… |
| 8 | NEEDS-INFO | The PR body in the prompt is still truncated mid-sentence under the - **Native matcher, prefill engine** section before the test summary, so it is not possible to verify from the provided text whether the PR description states that tests/test_rocke.py and tests/run_all.py were run and names the GPU architecture tested. |


ISSUE ID : AICK-1847
Based on
develop, which carries #14358 (hipDNN's paged K/V SDPA contract and references) and #13897 (the two-engine RDNA SDPA catalog).Summary
Teaches the RDNA prefill engine
hipkernel:Gfx1151WmmaAttentionto serve the paged K/V SDPA forward graph defined in #14358 (cuDNN semantics: vLLM-style block tables, with a dense or a ragged Q,seq_len_qandseq_len_kv). Each graph runs one launch of a rocKEwmma_fmha_fwd_v2paged object. Lengths, block ids and ragged offsets are read on the device, so one plan serves every step and can be captured into a HIP graph.The rocKE kernel gains the two query forms that hipDNN paged graphs carry:
dense_q, a batch-strided Q against a paged cache;seqused_q, per-sequence live query rows. For a dense Q the rows past them are padding and are written as O = 0 and LSE = -inf. A ragged Q owns only its live rows, as feat(hipdnn): ragged tensors with per-batch origins, lengths and strides (ALMIOPEN-3151) #15080 (RFC 0014) defines: origins may come in any order,ragged_offset[B]is not read, and the rows of an offset block pastseq_len_qare neither read nor written.It also clamps a paged
seqused_k, addresses paged K/V in 64 bits so caches can pass 2 GiB, and reads K and V through their own block tables (block_table_v), so graphs with separate K and V page tables are served. Attention sinks are served in hipDNN's FLOAT form as well as in the Q data type.The decode engine
hipkernel:Gfx1151WmmaAttentionDecodedoes not take paged graphs.What changes
wmma_fmha_fwd_v2:dense_q(paged only) andseqused_q(any packed layout), withcatalog_layout/catalog_page_block_size/catalog_dense_q/catalog_seqused_qaccessors for the hipDNN catalog. Adense_qorseqused_qthat is not aboolis refused withTypeError.dense_qandseqused_q, the rows pastseqused_q[b]up toseqlen_qare padding: they are not read, and they are written as O = 0 and LSE = -inf, with or without a sink (cuDNN's padded rows,SdpaSemanticsRegisterSEM-008). A CTA whose rows are all padding skips the K loop. Empty and padding rows store O = 0 by select rather than by scaling, so a NaN in a V row of their tile cannot reach them.cu_seqlens_qandseqused_q, a sequence owns only its live rows,min(seqused_q[b], seqlen_q): the kernel reads only its origincu_seqlens_q[b], nevercu_seqlens_q[b + 1], so origins may come in any order, and it writes no row past the live ones, so O and LSE may be exactly the addressed footprint (RFC 0014, feat(hipdnn): ragged tensors with per-batch origins, lengths and strides (ALMIOPEN-3151) #15080).seqused_kis clamped toseqlen_k, in the prefill and in the paged split-KV decode (wmma_fmha_decode, Python and C++ mirror).num_blocksbound is unchanged: ablock_tableentry outside[0, num_blocks)still reads as zero rows. A dead row on the per-row page path still loads from page 0, sonum_blocksmust be at least 1. The 64-bit pointer step is onerdna_attentionhelper (rdna_attention_scaled_ptr, with its C++ twin), which the decode kernel's batch, token and bias bases use too, with the ops in the same order.[0, num_blocks)before it forms the K and V page bases, as the per-row path already does, so no out-of-range page id reaches a pointer add.sink_is_f32(i32) aftersink_ptr: nonzero readssink_ptr[h]as f32, zero in the Q data type. The sink is read with two 16-bit loads, so neither form reads past its[Hq]extent.block_table_vargument: V pages are looked up in their own table, read at the same row offset as K's (block_table_strideandnum_blocksare shared). Passing one table twice gives the old behaviour.dense_q(a batch-strided[B, Sq, Hq, D]Q / O, nocu_seqlens_q) andseqused_q(an INT32[B]seqused_qtensor). It checks a paged prefill K/V cache as the pack does (one page of one head I32-addressable) rather than capping the whole cache at 2 GiB; decode caches keep the whole-tensor cap, because the decode kernel still forms those offsets in I32. It passessink_is_f32and accepts float32 sinks for prefill; the decode kernel still reads the sink in the Q data type. It rejects a pagedseqlen_klarger than the table maps (its width times the page size), for both kernels, and a prefill cache with no pages.AttentionRequestgainsdense_qandseqused_q, reported as features of the same names that only the RDNA WMMA prefill declares. The prefill policy copies them into its spec; the split-KV decode refuses them, so such a request always runs the prefill. A request that leaves them unset keeps itsrequest_hash(the fields joinnormalized()only when set). A non-bool,dense_qwithoutlayout="paged", orseqused_qwith a dense layout is rejected.ROCKE_ABI_VERSION/BINARY_ABImove to 13:rocke_rdna_attn_params_tchanges layout, androcke_wmma_fmha_fwd_v2_spec_tgainsdense_qandseqused_qin existing padding (its size and offsets are unchanged). The dispatch ABI string moves torocke-attention-gfx1151/v8, sinceblock_table_vmoves every later paged prefill argument.gfx1151_wmma_fmha_fwd_v2andgfx1151_wmma_fmha_decodegoldens are re-blessed for these IR changes (ll for llvm20, llvm22, llvm23 and llvm24, and canonical IR); no other family's goldens change. New parity rows cover sinks with a dense Q andseqused_qat page 64, sinks with a ragged Q andseqused_qon the per-tile path, and head 96 paged.Gfx1151WmmaAttentionNative.cpp):[num_blocks, H, page, D]of any row-aligned strides, including both halves of one interleaved cache, with one block size and equalnum_blocks;[B, 1, M, 1]for K and V, either one tensor for both or two separate tensors of the same widthM(and the same row stride whenB > 1);seq_len_qandseq_len_kv, both required, as feat(hipdnn): SDPA CPU and GPU references cover the full SDPA graph API #14358 requires them on every paged graph;max_seq_len_kv, or, when it is unset,M * page(cuDNN's inference; the kernel needs it only as the bound onseq_len_kv);[B, Hq, Sq, D]or ragged, sharing one token-unit offset tensor with O;left_boundwith cuDNN's meaning:Lkeys ending at the diagonal), GQA, bf16/fp16.one_sided_paging); a packed page table, K and V page tables of different width or row stride, or a table that is not INT32[B, 1, M, 1]with a unit entry stride (page_table); different K and V block sizes, a page size that is not a power of two, or one no paged object ships for (block_size); different K and V page counts (shape); a missing or malformedseq_len_q,seq_len_kvor ragged offset tensor (seq_len_q,seq_len_kv,ragged). Each check logs its own reason, naming the operand and values.ragged_offset_multiplieris not its token stride is declined with causeragged("ragged offsets must be in token units"), as is a Q and O that do not share one offset tensor. The objects take token offsets (vLLM'scu_seqlens_q); element-unit offsets (cuDNN's multiplier 1) would need a per-launch rescale, a new kernel argument and another ABI change, so they are declined rather than served.sink_token, SEM-022 semantics) on paged graphs: one logit per query head,[1, Hq, 1, 1], either FLOAT (hipDNN's form) or in the Q data type. Sinks are served by 48 dedicated paged prefill objects (use_sinks), which read either form: the matcher binds a token for the sink's data type and launch passes it assink_is_f32, so no catalog object is added. A sink of another data type is declined (sinks). Dense graphs with sinks are still declined.alibi), FP8 scales and amax (fp8), dropout (dropout), and block masks, max / sum_exp outputs or a forced implementation (feature) are declined, and the cause is logged at INFO. Additive bias on paged graphs (attn_mask) and stats with a ragged Q (stats) are also declined with a logged cause. A dense graph that carriesseq_len_q,seq_len_kv,max_seq_len_kvorpadding_maskis declined with causepadding.develop's provider reports plugin API 1.4.0, so these graphs reach this engine). The kernels load Q, K, V and O rows as 16-byte vectors, INT32 operands as 4-byte words, write the stats (LSE) as FP32 words, load an additive bias in its own element type and load the sink with 16-bit loads. Graphs are declined with causealignmentwhen they declare less than 16 bytes for Q, K, V or O; less than 4 bytes for a page table, sequence length or ragged offset; less than 4 bytes for the stats (dense and paged); less than the bias element (4 bytes for a FLOAT bias, 2 for one in the Q data type; dense, since paged graphs decline any bias); or less than the sink element (4 bytes for a FLOAT sink, 2 for one in the Q data type). hipDNN can then pick another engine at plan time, instead of the plan failing at execute.sdpa_graph::DeclinedBy). The decode engine runs the shared dense matcher too and now logs its declines underhipkernel:Gfx1151WmmaAttentionDecode, including why it declines a paged graph (page_table).configs/gfx1151_wmma_attention.yaml, regeneratedgfx1151_wmma_attention.{kdp,kmd,ued}.json):seqused_q), bf16/fp16, each with and without sinks. The prefill catalog has 144 objects in all.block_m64 (block_size128) on gfx11: withblock_m128 those 16 objects used 256 VGPRs and spilled inside the K loop, and withblock_m64 they use 212 and do not spill. The gfx12 objects keepblock_m128, which does not spill there.layout,page_block_size,dense_q,seqused_qandsinks. The 48 existing dense objects keep their UUIDs and gain the dense defaults.sdk_version1.4.0.resolvedentries dropseqused_q, which every arm sets.Gfx1151WmmaAttentionCatalog.hpp: the matcher's page-size check and the integration test's arch gate use them, and a census suite (TestGfx1151WmmaAttentionCatalog) checks both lists against the loaded objects.Gfx1151WmmaAttentionTestSupport.hpp, built on develop'sIngestorPackTestSupport.hpp(PackSymbols,GraphFixture,matchesGraph,matchesKernel,scoreKernel,dispatchHandler).TestGfx1151WmmaAttentionPaged): separate K and V page tables, a V container with its own strides, the inferredmax_seq_len_kv, FLOAT and Q-dtype sinks binding with the right flag, a descriptor withoutsink_is_f32refused, and routing tests (the prefill engine serves a paged decode-shaped graph, and the decode engine declines every paged graph under its own name). Each decline above is checked with the engine name, the cause and the reason: every paged check on its own, the unshipped page sizes, the ragged multiplier and unshared offsets, alignment (an 8-byte-aligned Q, K, V or O, a 2-byte-aligned INT32 operand, 1- or 2-byte stats, a sink below its element), the sink checks, FP8, dropout, a forced implementation, the other unserved features and ALiBi.TestGfx1151WmmaAttentionMatchers): the alignment declines (Q/K/V/O, stats, FLOAT and Q-dtype bias) and thepadding,sinks, FP8, dropout, feature (including a forced implementation) and ALiBi causes, each checked with its reason for both engines, and a test that both engines bind a dense graph alike.TestGfx1151WmmaAttentionPacks) and the catalog census (TestGfx1151WmmaAttentionCatalog).seqused_qdispatched from the request's fields on a GPU; selection tests for the request fields (they reach the prefill spec, the decode and other arches refuse them, bad values are rejected, an unset request keeps its hash); spec tests for the non-bool flags and the dispatch ABI string; binding tests for the table capacity and an empty prefill cache; a staleseqused_kpointed at a NaN page for both the prefill and the decode (paged guard); a dense-Q padding case with a NaN V row; the caches-past-4-GiB cases skip on a device without the free memory; f32-sink feature cases (dense Q, per-row and per-tile ragged Q), the binding's paged I32 checks, and the cache-key "on" side ofdense_q/seqused_q.IntegrationGpuGfx1151WmmaAttentionPaged, 21 cases): dense decode, spec-decode, dense Q with padding rows, ragged mixed batches, two cases with separate K and V page tables, two cases whose V page, head and token strides all differ from K's, three Q-dtype sink cases and the same three with a FLOATsink_token(float_sinks_*), and every shipped head-width bucket at every page size. Each runs against test_sdk'sCpuFpReferenceSdpa(gatherPagedKvper page table, sequence lengths, band, sink and LSE), with HIP-graph capture/replay using changed lengths and block ids. A dense Q's padding rows must read O = 0 and LSE = -inf. A ragged Q's sequences are laid out in reverse order withragged_offset[B]poisoned, and the rows of an offset block pastseq_len_qmust keep their NaN. The cases skip on an arch outside the shipped list.Behaviour and API changes
hipkernel:Gfx1151WmmaAttentionnow accepts paged graphs.hipkernel:Gfx1151WmmaAttentionDecodenever does. Both decline graphs that declare less alignment than the kernels load.developalready reports engine plugin API 1.4.0 (feat(hip-kernel-provider): Add wiring for ragged tensors to the ASM SDPA Engine #14202), which lets ragged and paged graphs reach this engine.Known integration work
sink_token, which this engine now serves; nothing here waits on hipDNN. vLLM passes its sinks in the model dtype, which these objects also read, but no hipDNN graph can carry that form yet, so the three Q-dtype sink cases ofIntegrationGpuGfx1151WmmaAttentionPageddetect the frontend's rejection andGTEST_SKIPwith an explicit message.Gfx1151WmmaAttentionNative.cppmatch feat(hip-kernel-provider): paged/ragged SDPA engine for gfx942/gfx950 #14247's gfx942/gfx950 engine rule for rule, and this PR logs feat(hip-kernel-provider): paged/ragged SDPA engine for gfx942/gfx950 #14247's cause keys. Whichever of the two lands second switches this matcher to the shared checks in develop'spacks/SdpaGraphMatch.hpp; this engine keeps only its kernel limits.ROCKE_ABI_VERSION13 (developis at 12); feat(hip-kernel-provider): paged/ragged SDPA engine for gfx942/gfx950 #14247 takes the next one.Validation (Linux, Slurm compute nodes)
Results below were taken on this head, 67a5479 (
develop1702865 as the base), unless a bullet says otherwise. Every hipDNN build had the HIP Flash2 and ASM SDPA engines on (-DENABLE_HIP_FLASH2_ENGINE=ON -DENABLE_ASM_SDPA_ENGINE=ON).rocke_arch_domain_reproducibilityskipped). Python-vs-C parity is GREEN under llvm20, llvm22, llvm23 and llvm24 (gfx1151_wmma_fmha_fwd_v2186 configs,gfx1151_wmma_fmha_decode128). Golden check: GOLDEN OK for ll llvm20/22/23/24 and canonical IR (314 blessed configs).prove_parity_binding: both gfx1151 families VALIDATED under all four flavours; each run also reports three selector mismatches ingfx950_attention_tiled_2dproblems, which this PR does not touch.test_abi.pyagainst a freshly builtlibrocke: 28 passed.test_slots.py) passes on every commit. IngestorGenerator pytest 1894 passed and descriptor-packaging pytest 1076 passed; their failures and errors (rocKE packing tests that needlibamd_comgr, which the compile node's interpreter cannot load) are the same set as ondevelop.tests/library/tests/dispatch) on gfx1201: head 2103 passed, 11 skipped, no failures or errors. Thedevelopbaseline stalled at about 95% in two runs on gfx1201 (no output after the first 12 minutes), so a head-vs-developcomparison is still pending.*Gfx1151*plus plugin-public and compiled-in engine tests) 94 passed; census (ctest -R 'Gfx1151WmmaAttention|census') 60/60;IntegrationGpuGfx1151WmmaAttentionPaged18 passed, including the threefloat_sinks_*cases, and the three Q-dtype sink cases skipped with the frontend's FLOAT-only message. rocKE pytest: spec 17, features 122 (including the caches past 4 GiB and the NaN-V padding case), paged guard 48 (including a staleseqused_kfor the prefill and the decode), dispatch numerics 327 passed with 7 skips (decode buckets that do not serve the head width), I32 guard 27.seqused_kclamp fails the four decode stale-seqused_kcases, reverting the prefill clamp fails the four prefill cases and bothpaged16_seqused_k_past_seqlen_kfeature cases, and the old epilogue (scaling by a zero reciprocal) fails bothdenseq_used_paged16_nan_vcases. On the CPU, removing the bool check, thev8ABI string, the table-capacity check, the empty-cache check, the request-to-spec wiring, the decode refusal or the request layout check each fails its test.hipkernel:Gfx1151WmmaAttentionon gfx1201: the five dense-Q paged bundles (quick bf16hd128_causal_paged/Decode, fp16hd64_causal_paged/SpecDecode, bf16 and fp16hd64_causal_paged_sinks/Decodewith FLOAT sinks, standard bf16hd128_causal_paged/LongDecode) pass against the GPU reference. Quick fp16hd64_window_ragged_paged/Mixedand bf16hd64_window_ragged_paged_sinks/Mixedpass with--validator cpu; under the GPU validator the harness compares ragged outputs on the host only, and they fail. Declined as expected with causestats(the paged kernels write no stats for a ragged Q): quick bf16hd128_causal_ragged_paged/Mixedand standard bf16hd64_window_ragged_paged_sinks/Long.IntegrationGpuGfx1151WmmaAttentionPaged8 passed, 3 sink cases skipped; the same paged bundle results as gfx1201 above (three dense-Q bundles pass against the GPU reference, ragged fp16Mixedpasses with--validator cpu, the sink and ragged-stats bundles are declined). The wholehip_kernel_provider_testsrun did not finish within the job's 12 h limit.ASIC coverage
Same families as #13897: gfx11-generic and gfx12-generic objects. GPU validation on this head: gfx1201; gfx1151 ran on the earlier head cbeece8c742.