Skip to content

feat(hip-kernel-provider): paged/ragged SDPA in Gfx1151WmmaAttention - #14254

Open
dpeabody-amd wants to merge 11 commits into
developfrom
users/dpeabody/hipdnn-rdna-paged-attn
Open

dpeabody-amd wants to merge 11 commits into
developfrom
users/dpeabody/hipdnn-rdna-paged-attn

Conversation

@dpeabody-amd

@dpeabody-amd dpeabody-amd commented Oct 23, 2026 •

Copy link
Copy Markdown
Contributor

ISSUE ID : AICK-1847

Based on develop, which carries #14358 (hipDNN's paged K/V SDPA contract and references) and #13897 (the two-engine RDNA SDPA catalog).

Summary

Teaches the RDNA prefill engine hipkernel:Gfx1151WmmaAttention to serve the paged K/V SDPA forward graph defined in #14358 (cuDNN semantics: vLLM-style block tables, with a dense or a ragged Q, seq_len_q and seq_len_kv). Each graph runs one launch of a rocKE wmma_fmha_fwd_v2 paged object. Lengths, block ids and ragged offsets are read on the device, so one plan serves every step and can be captured into a HIP graph.

The rocKE kernel gains the two query forms that hipDNN paged graphs carry:

It also clamps a paged seqused_k, addresses paged K/V in 64 bits so caches can pass 2 GiB, and reads K and V through their own block tables (block_table_v), so graphs with separate K and V page tables are served. Attention sinks are served in hipDNN's FLOAT form as well as in the Q data type.

The decode engine hipkernel:Gfx1151WmmaAttentionDecode does not take paged graphs.

What changes

  • rocKE wmma_fmha_fwd_v2:
    • New spec fields dense_q (paged only) and seqused_q (any packed layout), with catalog_layout / catalog_page_block_size / catalog_dense_q / catalog_seqused_q accessors for the hipDNN catalog. A dense_q or seqused_q that is not a bool is refused with TypeError.
    • With dense_q and seqused_q, the rows past seqused_q[b] up to seqlen_q are padding: they are not read, and they are written as O = 0 and LSE = -inf, with or without a sink (cuDNN's padded rows, SdpaSemanticsRegister SEM-008). A CTA whose rows are all padding skips the K loop. Empty and padding rows store O = 0 by select rather than by scaling, so a NaN in a V row of their tile cannot reach them.
    • With cu_seqlens_q and seqused_q, a sequence owns only its live rows, min(seqused_q[b], seqlen_q): the kernel reads only its origin cu_seqlens_q[b], never cu_seqlens_q[b + 1], so origins may come in any order, and it writes no row past the live ones, so O and LSE may be exactly the addressed footprint (RFC 0014, feat(hipdnn): ragged tensors with per-batch origins, lengths and strides (ALMIOPEN-3151) #15080).
    • The C mirror, parity emitters, binding parity rows, spec tests and on-device feature tests are updated to match.
    • A paged seqused_k is clamped to seqlen_k, in the prefill and in the paged split-KV decode (wmma_fmha_decode, Python and C++ mirror).
    • Paged K/V heads and pages are addressed in 64 bits, and the packed row base is 64-bit. Only one page of one head must be I32-addressable. The num_blocks bound is unchanged: a block_table entry outside [0, num_blocks) still reads as zero rows. A dead row on the per-row page path still loads from page 0, so num_blocks must be at least 1. The 64-bit pointer step is one rdna_attention helper (rdna_attention_scaled_ptr, with its C++ twin), which the decode kernel's batch, token and bias bases use too, with the ops in the same order.
    • The per-tile page path (pages of 32 and more) selects page 0 for a table entry outside [0, num_blocks) before it forms the K and V page bases, as the per-row path already does, so no out-of-range page id reaches a pointer add.
    • Paged sink objects take sink_is_f32 (i32) after sink_ptr: nonzero reads sink_ptr[h] as f32, zero in the Q data type. The sink is read with two 16-bit loads, so neither form reads past its [Hq] extent.
    • New block_table_v argument: V pages are looked up in their own table, read at the same row offset as K's (block_table_stride and num_blocks are shared). Passing one table twice gives the old behaviour.
    • The attention dispatch binding reads dense_q (a batch-strided [B, Sq, Hq, D] Q / O, no cu_seqlens_q) and seqused_q (an INT32 [B] seqused_q tensor). It checks a paged prefill K/V cache as the pack does (one page of one head I32-addressable) rather than capping the whole cache at 2 GiB; decode caches keep the whole-tensor cap, because the decode kernel still forms those offsets in I32. It passes sink_is_f32 and accepts float32 sinks for prefill; the decode kernel still reads the sink in the Q data type. It rejects a paged seqlen_k larger than the table maps (its width times the page size), for both kernels, and a prefill cache with no pages.
    • AttentionRequest gains dense_q and seqused_q, reported as features of the same names that only the RDNA WMMA prefill declares. The prefill policy copies them into its spec; the split-KV decode refuses them, so such a request always runs the prefill. A request that leaves them unset keeps its request_hash (the fields join normalized() only when set). A non-bool, dense_q without layout="paged", or seqused_q with a dense layout is rejected.
    • ROCKE_ABI_VERSION / BINARY_ABI move to 13: rocke_rdna_attn_params_t changes layout, and rocke_wmma_fmha_fwd_v2_spec_t gains dense_q and seqused_q in existing padding (its size and offsets are unchanged). The dispatch ABI string moves to rocke-attention-gfx1151/v8, since block_table_v moves every later paged prefill argument.
    • The gfx1151_wmma_fmha_fwd_v2 and gfx1151_wmma_fmha_decode goldens are re-blessed for these IR changes (ll for llvm20, llvm22, llvm23 and llvm24, and canonical IR); no other family's goldens change. New parity rows cover sinks with a dense Q and seqused_q at page 64, sinks with a ragged Q and seqused_q on the per-tile path, and head 96 paged.
  • Native matcher, prefill engine (Gfx1151WmmaAttentionNative.cpp):
    • It binds the cuDNN paged graph:
      • K/V page containers [num_blocks, H, page, D] of any row-aligned strides, including both halves of one interleaved cache, with one block size and equal num_blocks;
      • INT32 page tables [B, 1, M, 1] for K and V, either one tensor for both or two separate tensors of the same width M (and the same row stride when B > 1);
      • seq_len_q and seq_len_kv, both required, as feat(hipdnn): SDPA CPU and GPU references cover the full SDPA graph API #14358 requires them on every paged graph;
      • max_seq_len_kv, or, when it is unset, M * page (cuDNN's inference; the kernel needs it only as the bound on seq_len_kv);
      • a Q that is either dense [B, Hq, Sq, D] or ragged, sharing one token-unit offset tensor with O;
      • bottom-right causal plus window (left_bound with cuDNN's meaning: L keys ending at the diagonal), GQA, bf16/fp16.
    • The page size selects the object. A paged object serves exactly its page size and query form.
    • Paged graphs that are valid under feat(hipdnn): SDPA CPU and GPU references cover the full SDPA graph API #14358 but that the kernels cannot read are declined, and the cause is logged at INFO with the keys feat(hip-kernel-provider): paged/ragged SDPA engine for gfx942/gfx950 #14247 uses for the same rules: only K or only V paged (one_sided_paging); a packed page table, K and V page tables of different width or row stride, or a table that is not INT32 [B, 1, M, 1] with a unit entry stride (page_table); different K and V block sizes, a page size that is not a power of two, or one no paged object ships for (block_size); different K and V page counts (shape); a missing or malformed seq_len_q, seq_len_kv or ragged offset tensor (seq_len_q, seq_len_kv, ragged). Each check logs its own reason, naming the operand and values.
    • A ragged Q or O whose ragged_offset_multiplier is not its token stride is declined with cause ragged ("ragged offsets must be in token units"), as is a Q and O that do not share one offset tensor. The objects take token offsets (vLLM's cu_seqlens_q); element-unit offsets (cuDNN's multiplier 1) would need a per-launch rescale, a new kernel argument and another ABI change, so they are declined rather than served.
    • Attention sinks (sink_token, SEM-022 semantics) on paged graphs: one logit per query head, [1, Hq, 1, 1], either FLOAT (hipDNN's form) or in the Q data type. Sinks are served by 48 dedicated paged prefill objects (use_sinks), which read either form: the matcher binds a token for the sink's data type and launch passes it as sink_is_f32, so no catalog object is added. A sink of another data type is declined (sinks). Dense graphs with sinks are still declined.
    • ALiBi (alibi), FP8 scales and amax (fp8), dropout (dropout), and block masks, max / sum_exp outputs or a forced implementation (feature) are declined, and the cause is logged at INFO. Additive bias on paged graphs (attn_mask) and stats with a ragged Q (stats) are also declined with a logged cause. A dense graph that carries seq_len_q, seq_len_kv, max_seq_len_kv or padding_mask is declined with cause padding.
    • Declared alignment. A graph can declare a device-pointer alignment below the default 16 bytes (an engine API 1.3.0 feature; develop's provider reports plugin API 1.4.0, so these graphs reach this engine). The kernels load Q, K, V and O rows as 16-byte vectors, INT32 operands as 4-byte words, write the stats (LSE) as FP32 words, load an additive bias in its own element type and load the sink with 16-bit loads. Graphs are declined with cause alignment when they declare less than 16 bytes for Q, K, V or O; less than 4 bytes for a page table, sequence length or ragged offset; less than 4 bytes for the stats (dense and paged); less than the bias element (4 bytes for a FLOAT bias, 2 for one in the Q data type; dense, since paged graphs decline any bias); or less than the sink element (4 bytes for a FLOAT sink, 2 for one in the Q data type). hipDNN can then pick another engine at plan time, instead of the plan failing at execute.
    • Decline logs name the declining engine (develop's sdpa_graph::DeclinedBy). The decode engine runs the shared dense matcher too and now logs its declines under hipkernel:Gfx1151WmmaAttentionDecode, including why it declines a paged graph (page_table).
  • Routing between the two engines. Paged graphs belong to the prefill engine only:
    • The decode engine's graph matcher declines any graph with a page table (and logs it), and its kernel matcher requires the dense layout.
    • The prefill engine's "decline what decode would take" rule now applies only to dense graphs. On a paged graph the K length is the page size, so the dense decode rule would misfire.
    • Dense routing is unchanged.
  • Catalog (configs/gfx1151_wmma_attention.yaml, regenerated gfx1151_wmma_attention.{kdp,kmd,ued}.json):
    • 96 paged prefill objects: head widths 64/96/128/256, page 16/32/64, dense or ragged Q (both with seqused_q), bf16/fp16, each with and without sinks. The prefill catalog has 144 objects in all.
    • Head 96 at pages 32 and 64 (the per-tile page path) ships block_m 64 (block_size 128) on gfx11: with block_m 128 those 16 objects used 256 VGPRs and spilled inside the K loop, and with block_m 64 they use 212 and do not spill. The gfx12 objects keep block_m 128, which does not spill there.
    • New KMD fields layout, page_block_size, dense_q, seqused_q and sinks. The 48 existing dense objects keep their UUIDs and gain the dense defaults.
    • UED sdk_version 1.4.0.
    • The decode catalog is unchanged.
    • The paged groups' resolved entries drop seqused_q, which every arm sets.
    • The shipped page sizes and architectures are stated once, in Gfx1151WmmaAttentionCatalog.hpp: the matcher's page-size check and the integration test's arch gate use them, and a census suite (TestGfx1151WmmaAttentionCatalog) checks both lists against the loaded objects.
  • Tests:
    • The scaffolding both matcher unit tests share (object metadata, the decline-log reader, ABI helpers) is in Gfx1151WmmaAttentionTestSupport.hpp, built on develop's IngestorPackTestSupport.hpp (PackSymbols, GraphFixture, matchesGraph, matchesKernel, scoreKernel, dispatchHandler).
    • Paged matcher/binding unit tests (TestGfx1151WmmaAttentionPaged): separate K and V page tables, a V container with its own strides, the inferred max_seq_len_kv, FLOAT and Q-dtype sinks binding with the right flag, a descriptor without sink_is_f32 refused, and routing tests (the prefill engine serves a paged decode-shaped graph, and the decode engine declines every paged graph under its own name). Each decline above is checked with the engine name, the cause and the reason: every paged check on its own, the unshipped page sizes, the ragged multiplier and unshared offsets, alignment (an 8-byte-aligned Q, K, V or O, a 2-byte-aligned INT32 operand, 1- or 2-byte stats, a sink below its element), the sink checks, FP8, dropout, a forced implementation, the other unserved features and ALiBi.
    • Dense matcher unit tests (TestGfx1151WmmaAttentionMatchers): the alignment declines (Q/K/V/O, stats, FLOAT and Q-dtype bias) and the padding, sinks, FP8, dropout, feature (including a forced implementation) and ALiBi causes, each checked with its reason for both engines, and a test that both engines bind a dense graph alike.
    • The regenerated census (TestGfx1151WmmaAttentionPacks) and the catalog census (TestGfx1151WmmaAttentionCatalog).
    • rocKE: one dense-Q paged launch with seqused_q dispatched from the request's fields on a GPU; selection tests for the request fields (they reach the prefill spec, the decode and other arches refuse them, bad values are rejected, an unset request keeps its hash); spec tests for the non-bool flags and the dispatch ABI string; binding tests for the table capacity and an empty prefill cache; a stale seqused_k pointed at a NaN page for both the prefill and the decode (paged guard); a dense-Q padding case with a NaN V row; the caches-past-4-GiB cases skip on a device without the free memory; f32-sink feature cases (dense Q, per-row and per-tile ragged Q), the binding's paged I32 checks, and the cache-key "on" side of dense_q / seqused_q.
    • GPU integration tests (IntegrationGpuGfx1151WmmaAttentionPaged, 21 cases): dense decode, spec-decode, dense Q with padding rows, ragged mixed batches, two cases with separate K and V page tables, two cases whose V page, head and token strides all differ from K's, three Q-dtype sink cases and the same three with a FLOAT sink_token (float_sinks_*), and every shipped head-width bucket at every page size. Each runs against test_sdk's CpuFpReferenceSdpa (gatherPagedKv per page table, sequence lengths, band, sink and LSE), with HIP-graph capture/replay using changed lengths and block ids. A dense Q's padding rows must read O = 0 and LSE = -inf. A ragged Q's sequences are laid out in reverse order with ragged_offset[B] poisoned, and the rows of an offset block past seq_len_q must keep their NaN. The cases skip on an arch outside the shipped list.

Behaviour and API changes

Known integration work

  • Q-dtype sinks. hipDNN's frontend accepts only a FLOAT sink_token, which this engine now serves; nothing here waits on hipDNN. vLLM passes its sinks in the model dtype, which these objects also read, but no hipDNN graph can carry that form yet, so the three Q-dtype sink cases of IntegrationGpuGfx1151WmmaAttentionPaged detect the frontend's rejection and GTEST_SKIP with an explicit message.
  • Paged split-KV decode (decode plus merge) is not offered: paged decode-shaped graphs run on the prefill kernel. A paged decode program waits on kernel slots (RFC 0021).
  • Shared paged-graph checks. The cuDNN paged-graph checks in Gfx1151WmmaAttentionNative.cpp match feat(hip-kernel-provider): paged/ragged SDPA engine for gfx942/gfx950 #14247's gfx942/gfx950 engine rule for rule, and this PR logs feat(hip-kernel-provider): paged/ragged SDPA engine for gfx942/gfx950 #14247's cause keys. Whichever of the two lands second switches this matcher to the shared checks in develop's packs/SdpaGraphMatch.hpp; this engine keeps only its kernel limits.
  • Binary ABI. This PR takes ROCKE_ABI_VERSION 13 (develop is at 12); feat(hip-kernel-provider): paged/ragged SDPA engine for gfx942/gfx950 #14247 takes the next one.

Validation (Linux, Slurm compute nodes)

Results below were taken on this head, 67a5479 (develop 1702865 as the base), unless a bullet says otherwise. Every hipDNN build had the HIP Flash2 and ASM SDPA engines on (-DENABLE_HIP_FLASH2_ENGINE=ON -DENABLE_ASM_SDPA_ENGINE=ON).

  • rocKE gates (CPU compile node): rocKE ctest 60/60 passed (rocke_arch_domain_reproducibility skipped). Python-vs-C parity is GREEN under llvm20, llvm22, llvm23 and llvm24 (gfx1151_wmma_fmha_fwd_v2 186 configs, gfx1151_wmma_fmha_decode 128). Golden check: GOLDEN OK for ll llvm20/22/23/24 and canonical IR (314 blessed configs). prove_parity_binding: both gfx1151 families VALIDATED under all four flavours; each run also reports three selector mismatches in gfx950_attention_tiled_2d problems, which this PR does not touch. test_abi.py against a freshly built librocke: 28 passed.
  • Full tree in the CI configuration with hipDNN's clang-tidy (clang-tidy 20), all targets, compile node: no failed targets and no tidy diagnostics.
  • IngestorGenerator: the regenerated descriptors and census are byte-identical to the committed ones, and the render-digest test (test_slots.py) passes on every commit. IngestorGenerator pytest 1894 passed and descriptor-packaging pytest 1076 passed; their failures and errors (rocKE packing tests that need libamd_comgr, which the compile node's interpreter cannot load) are the same set as on develop.
  • Whole rocKE dispatch pytest (tests/library/tests/dispatch) on gfx1201: head 2103 passed, 11 skipped, no failures or errors. The develop baseline stalled at about 95% in two runs on gfx1201 (no output after the first 12 minutes), so a head-vs-develop comparison is still pending.
  • gfx1201 (one GPU): all targets built with no failed targets (tidy off). Unit (*Gfx1151* plus plugin-public and compiled-in engine tests) 94 passed; census (ctest -R 'Gfx1151WmmaAttention|census') 60/60; IntegrationGpuGfx1151WmmaAttentionPaged 18 passed, including the three float_sinks_* cases, and the three Q-dtype sink cases skipped with the frontend's FLOAT-only message. rocKE pytest: spec 17, features 122 (including the caches past 4 GiB and the NaN-V padding case), paged guard 48 (including a stale seqused_k for the prefill and the decode), dispatch numerics 327 passed with 7 skips (decode buckets that do not serve the head width), I32 guard 27.
  • Each round-4 fix was checked against its mutant. On gfx1201: reverting the decode seqused_k clamp fails the four decode stale-seqused_k cases, reverting the prefill clamp fails the four prefill cases and both paged16_seqused_k_past_seqlen_k feature cases, and the old epilogue (scaling by a zero reciprocal) fails both denseq_used_paged16_nan_v cases. On the CPU, removing the bool check, the v8 ABI string, the table-capacity check, the empty-cache check, the request-to-spec wiring, the decode refusal or the request layout check each fails its test.
  • feat(hipdnn): SDPA CPU and GPU references cover the full SDPA graph API #14358 paged bundles through hipkernel:Gfx1151WmmaAttention on gfx1201: the five dense-Q paged bundles (quick bf16 hd128_causal_paged/Decode, fp16 hd64_causal_paged/SpecDecode, bf16 and fp16 hd64_causal_paged_sinks/Decode with FLOAT sinks, standard bf16 hd128_causal_paged/LongDecode) pass against the GPU reference. Quick fp16 hd64_window_ragged_paged/Mixed and bf16 hd64_window_ragged_paged_sinks/Mixed pass with --validator cpu; under the GPU validator the harness compares ragged outputs on the host only, and they fail. Declined as expected with cause stats (the paged kernels write no stats for a ragged Q): quick bf16 hd128_causal_ragged_paged/Mixed and standard bf16 hd64_window_ragged_paged_sinks/Long.
  • gfx1151 (one GPU, earlier head cbeece8c742 of this branch, before this rebase): all targets built with no failed targets (tidy off). Unit 84 passed (1 ASM SDPA skip); census 15/15; IntegrationGpuGfx1151WmmaAttentionPaged 8 passed, 3 sink cases skipped; the same paged bundle results as gfx1201 above (three dense-Q bundles pass against the GPU reference, ragged fp16 Mixed passes with --validator cpu, the sink and ragged-stats bundles are declined). The whole hip_kernel_provider_tests run did not finish within the job's 12 h limit.

ASIC coverage

Same families as #13897: gfx11-generic and gfx12-generic objects. GPU validation on this head: gfx1201; gfx1151 ran on the earlier head cbeece8c742.

@codecov

codecov Bot commented Oct 23, 2026 •

Copy link
Copy Markdown

Codecov Report

✅ All modified and coverable lines are covered by tests.

Additional details and impacted files
@@           Coverage Diff            @@
##           develop   #14254   +/-   ##
========================================
  Coverage    78.16%   78.16%           
========================================
  Files         2977     2977           
  Lines       508843   508843           
  Branches     79012    79012           
========================================
  Hits        397712   397712           
  Misses       83900    83900           
  Partials     27231    27231           
Flag Coverage Δ *Carryforward flag
TensileLite-CPP 55.69% <ø> (ø) Carriedforward from 1702865
TensileLite-Unit 85.13% <ø> (ø) Carriedforward from 1702865
hipBLAS 90.62% <ø> (ø) Carriedforward from 1702865
hipBLASLt 70.05% <ø> (ø) Carriedforward from 1702865
hipCUB 82.68% <ø> (ø) Carriedforward from 1702865
hipDNN 87.34% <ø> (ø)
hipFFT 42.65% <ø> (ø) Carriedforward from 1702865
hipRAND 76.12% <ø> (ø) Carriedforward from 1702865
hipSOLVER 69.67% <ø> (ø) Carriedforward from 1702865
hipSPARSE 87.03% <ø> (ø) Carriedforward from 1702865
rocALUTION 86.47% <ø> (ø) Carriedforward from 1702865
rocBLAS 69.20% <ø> (ø) Carriedforward from 1702865
rocFFT 46.49% <ø> (ø) Carriedforward from 1702865
rocRAND 57.32% <ø> (ø) Carriedforward from 1702865
rocSOLVER 76.66% <ø> (ø) Carriedforward from 1702865
rocSPARSE 83.61% <ø> (ø) Carriedforward from 1702865
rocThrust 93.15% <ø> (ø) Carriedforward from 1702865

*This pull request uses carry forward flags. Click here to find out more.

🚀 New features to boost your workflow:
  • ❄️ Test Analytics: Detect flaky tests, report on failures, and find test suite problems.
  • 📦 JS Bundle Analysis: Save yourself from yourself by tracking and limiting bundle sizes in JS merges.

@dpeabody-amd
dpeabody-amd force-pushed the users/dpeabody/rdna-sdpa-hipdnn branch from ef6ca8e to 11a5c83 Compare October 23, 2026 15:00
Base automatically changed from users/dpeabody/rdna-sdpa-hipdnn to develop October 26, 2026 16:50
@therock-pr-bot

therock-pr-bot Bot commented Oct 26, 2026 •

Copy link
Copy Markdown

✅ All Checks Passed — Ready for Review

Check Status Details
📝 PR Description ✅ Pass —
⛔ Forbidden Files ✅ Pass —
🧪 Unit Test ✅ Pass —
🔎 pre-commit ✅ Pass —
🚫 Draft PR 🔜 To Be Enabled —
🚩 Feature Flag 🔜 To Be Enabled —
📊 Code Coverage 🔜 To Be Enabled —
🤖 therock-pr-bot ✅ Pass —

🎉 All checks passed! This PR is ready for review.

📖 Need help? See the Policy FAQ for details on every check and how to fix failures.

🙋 Wish to Override Policy?

@therock-pr-bot

therock-pr-bot Bot commented Oct 26, 2026 •

Copy link
Copy Markdown

🎉 All checks passed! This PR is ready for review.

adickin-amd pushed a commit that referenced this pull request Nov 2, 2026
…PI (#14358)

## Summary

This PR brings the hipDNN SDPA **CPU and GPU reference implementations**
up to everything the hipDNN SDPA graph API can express, forward and
backward. It also adds the tests and docs that let the references serve
as the oracle every attention engine is compared against.

cuDNN's semantics are the contract. Each place where hipDNN deliberately
differs from cuDNN is recorded in
`projects/hipdnn/docs/SdpaSemanticsRegister.md` (SEM-xxx). The design,
the full support table and the testing strategy are in
**`projects/hipdnn/docs/SdpaReferenceImplementations.md`**.

Builds on #14483 (cuDNN's `left_bound` meaning: L keeps the L keys
ending at the diagonal) and #14312 (SdpaBackward O and LSE derived from
the CPU forward), both merged.

JIRA ID : ALMIOPEN-2863
Related: ALMIOPEN-2928, AICK-1847

## What the references support

Both references (CPU `CpuFpReferenceSdpa`; GPU `GpuFpReferenceSdpa`,
hipRTC kernels) and both graph executors support:

**Forward (`SdpaAttributes`)**

| Feature | Notes |
|---|---|
| dtypes | FP32, FP16, BF16; FP8 E4M3 / E4M3 FNUZ / E5M2 / E5M2 FNUZ
inputs with BF16 or same-format fp8 output |
| Layout, GQA/MQA, `D_qk != D_v` | any strides; independent `H_k` /
`H_v` |
| Additive bias | FLOAT/HALF/BF16, rank 1-4 broadcast |
| Diagonal band | `left_bound` / `right_bound` / alignment and the
deprecated causal flags |
| Padding | `seq_len_q/kv`; padded rows give O = 0, LSE = -inf |
| Paged K/V | K, V or both, each with its own block size; extent
inferred in cuDNN's order when unset |
| Ragged (RFC 0014) | native packed forward (BF16, FP8 E4M3); every
other ragged graph, including backward and ragged Q over paged K/V,
through a ragged-as-dense rewrite |
| Attention sink | FLOAT `[1, H, 1, 1]` |
| ALiBi | cuDNN slopes, top-left causal |
| Block-sparse mask | cuDNN's UINT8 bitmask, one bit per 128x128 tile |
| Dropout | explicit mask; Philox through the engine's `rng_dump` |
| fp8 scaling | Q/K/V descales (scalar or per KV head); `scale_s` /
`descale_s` / `scale_o`; `amax_s` / `amax_o` |
| Outputs | LSE, Max, Sum_exp |

**Backward (`SdpaBackwardAttributes`)**: a new GPU backward reference.
Both references support FP32/FP16/BF16, GQA, `D_qk != D_v`, LSE or
recompute, bias + dBias, band, padding, ALiBi, explicit-mask dropout,
and ragged through the rewrite.

**Reference-only**: logit softcap, and an opt-in GPU P-rounding mode
that models provider kernels.

**Not supported because the graph API lacks it**: backward Philox (no
backward `rng_dump`), backward sinks / dSink, the fp8 backward node, and
`score_mod`.

**Structure**: each direction is one pipeline (score: scale, softcap,
bias, ALiBi, band, block mask; then softmax with sink; then P @ V),
configured by a named options struct. fp8, paging and ragged are thin
layers around that core. The CPU and GPU references share no code, only
the semantics. The graph-translation layer (`PlanUtils.hpp`, the ragged
rewrite) is shared by both executors, and the dense CPU and GPU plans
use one applicability gate per direction.

## Behavior changes

- **Frontend validation follows cuDNN (user-visible).** `SdpaFwdNode`
now rejects: `right_bound < -1`; `max_seq_len_kv` without paging; paging
without both `seq_len_q` and `seq_len_kv`; page tables that are not
INT32 `[B, 1, M, 1]`; M != `ceil(max_seq_len_kv / block_size)` when the
extent is set; `seq_len_*` not INT32 `[B, 1, 1, 1]`; `sink_token` not
FLOAT `[1, H_q, 1, 1]`. One-sided paging, per-side block sizes, an
inferred extent and packed page tables stay allowed. Each rule cites
cuDNN and has a decline test. In-repo callers that built such graphs are
fixed.
- **Breaking `test_sdk` change (deliberate).** The positional
`CpuFpReferenceSdpa::forward` / `backward` overloads are removed in
favor of `SdpaFwdOptions` / `SdpaBwdOptions` (`.setScale`, `.setMask`,
`.setBand(left, right, topLeft)`; causal is `.setBand(-1, 0, topLeft)`).
There is no forwarding shim on purpose: `left_bound` changed meaning
(SEM-001), so a shim would silently change old callers' results. Every
in-repo caller is migrated.
- **gfx1151 WMMA attention (`Gfx1151WmmaAttentionNative.cpp`
`maskFor`)**: the deprecated `causal_mask` / `causal_mask_bottom_right`
flags compose with explicit bounds and alignment, as the references do,
instead of overriding them. Graphs that set only the flags bind as
before.

## Absorbed PRs (authorship kept)

- **#14004** (EwanC, ALMIOPEN-2863): SDPA backward golden bundles and
generator, extended with GQA, padding, dBias, explicit dropout and an
FP64 autograd cross-check.
- **#13935** (scerzh, ALMIOPEN-2928): CPU ragged SDPA forward plan.
- **#14245** (dpeabody-amd, AICK-1847): the frontend paged / sink /
seq_len validation, the GROUP-mode ASM test fix, and the paged golden
generator. Its separate paged plans are not taken: paging goes through
the dense reference path, so every other feature composes with it.
Stacked work on #14245 (#14247, #14254) may need to follow the stricter
`seq_len_q` rule.

## Testing

How correctness is established (detail in
`SdpaReferenceImplementations.md` §3):

1. **Simple by construction**: FP32, no tiling, three-pass softmax, one
code path per feature.
2. **Closed-form oracles and properties**: uniform scores, single key,
dead rows; GQA = repeated heads, band and block mask = -inf bias,
padding = truncated batches, paged = dense, sink = extra zero-value key,
ragged = per-batch dense.
3. **Finite-difference gradient checks** of the backward.
4. **CPU vs GPU parity** (`hipdnn_gpu_ref_tests`, `TestGpuSdpa*Plan`):
every feature at small shapes with NaN-filled padding; Standard-tier
medium sweeps up to 1100 keys; fp8 device codecs bit-exact against the
host.
5. **PyTorch golden bundles** (`hipdnn_golden_data_tests` checks every
bundle against both references, with no engine involved). The backward
generator is FP64-autograd checked; the fp8-scaling and block-mask/ALiBi
paths reproduce cuDNN's own reference code. Golden data is 78 DVC
pointers (26.8 MiB, every bundle under 1 MiB, pushed to
`hipdnn/golden-data`).
6. **Mutation testing**: injected reference and gate bugs each fail a
test.
7. **Applicability**: every schema field is gated, with one decline test
per rule; a graph is declined, never accepted and then thrown on.

**Results on gfx942** (MI300X, `-Werror`; the current head is covered by
CI):

| Suite | Passed |
|---|---|
| test_sdk / frontend / public_frontend | 2609 (1 skipped, pre-existing)
/ 2247 / 487 |
| gpu_ref | 13836 |
| integration-tests unit | 1149 |
| golden (every bundle with golden data, both references) | 220 |
| ASM_SDPA_ENGINE SdpaBackward bundles under #14312's tolerances | 31
(41 graphs the engine does not accept) |
| hip-kernel-provider / its SDPA integration / ASM SDPA external quick
suite | 1393 / 45 / 26 |
| flatbuffers_sdk / plugin_sdk / generator pytest | 272 / 1293 / 146 |

**Devices**: gfx942 runs everything above. gfx950: no run. gfx1151 has
host matcher tests only (no usable device node; see Known issues).
Windows and the other families are covered by TheRock CI. GPU reference
kernels are hipRTC source, so there is no `GPU_TARGETS`-dependent host
configuration.

**Test time**: the new medium sweeps and long bundles are Standard tier
(~18 s). The GPU reference forward is faster than on `develop` (masked
keys skip the dot product), with byte-identical outputs.

## Known issues

- **fp8 scaling over 128 keys (SEM-024)**: cuDNN quantizes P per 128-key
tile against a running max; the references are exact (and bit-exact
against cuDNN's `fp8_ref`) up to 128 keys, and the bundles stay within
that.
- **gfx1151 (accepted risk)**: the `maskFor` composition change has host
matcher tests only; a 13-graph device check is prepared for when a
gfx1151 node is available.
- Once this lands, #14004, #13935 and #14245 are closed as superseded.

## Pre-merge

- [ ] CI green on the current head

🤖 Generated with [Claude Code](https://claude.com/claude-code)

---------

Co-authored-by: Claude Opus 5.5 <noreply@anthropic.com>
Co-authored-by: Ewan Crawford <ewan.crawford@streamhpc.com>
Co-authored-by: Sergei Grigoriev <102019268+scerzh@users.noreply.github.com>
Co-authored-by: dpeabody-amd <dpeabody-amd@users.noreply.github.com>
@dpeabody-amd
dpeabody-amd force-pushed the users/dpeabody/hipdnn-rdna-paged-attn branch 3 times, most recently from 43d0852 to 1c25c6d Compare November 3, 2026 16:05
@dpeabody-amd
dpeabody-amd marked this pull request as ready for review November 3, 2026 21:16
@dpeabody-amd
dpeabody-amd requested review from a team as code owners November 3, 2026 21:16

@BrianHarrisonAMD BrianHarrisonAMD left a comment •

Copy link
Copy Markdown
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

PR description: tests claim is broader than the tests

The description and commit message say every decline's logged reason is asserted. Only the paging cases check a reason; the others pass an empty reason, so only their cause is checked. Please assert the reasons.

- {dense_q: true, seqused_q: true, tag: denseq_used}
- {seqused_q: true, tag: raggedq}
shapes:
- {head_dim: 64, head_dim_v: 64, knobs: query, resolved: {kind: prefill, head_size: 64, head_size_v: 64, mask_mode: causal, bias: none, block_size: 256, block_m: 128, dv_split: 1, dense_q: 0, seqused_q: 0, sinks: 0}}

Copy link
Copy Markdown
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Paged resolved values don't match the output

The paged shapes list dense_q: 0, seqused_q: 0, but every arm overrides both, so the shipped objects carry other values. Please drop the two fields from the paged resolved entries (lines 206-209, 230-233), as the dense groups already do for bias.

Copy link
Copy Markdown
Contributor Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

I dropped seqused_q from the eight paged resolved entries, since every arm sets it. dense_q: 0 has to stay: the raggedq arms do not set dense_q, so resolved is where their shipped dense_q = 0 comes from, and the loader refuses the config without it. Setting dense_q: false on those arms instead would let both go, but it adds an explicit dense_q to 48 kernel records in the shipped KDP, so I kept the descriptors as they are. The descriptors regenerate byte for byte and the render digest does not move. Happy to switch to the arm form if you prefer it. (5f4c910)

Copy link
Copy Markdown
Contributor Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Nothing new here since my last reply: seqused_q is gone from every paged resolved entry, and dense_q: 0 stays because the raggedq arms take it from there. The paged head-96 objects are now split into their own groups (block_m 64 on gfx11, see the spill item on the PR), and those groups follow the same rule. I'm still happy to switch to setting dense_q: false on the arms if you prefer that form.

@cgmillette

Copy link
Copy Markdown
Contributor

Automated review: Model - qwen3.6:35b-a3b-coding-mtp-q4_K_M, effort: max

PR #14254 — paged/ragged SDPA in Gfx1151WmmaAttention (dpeabody-amd, AICK-1847) — 🟠 First-round: conflict, ABI bump, sink dtype

Prior item Status
BrianHarrisonAMD (11-04) ×5 matcher: decode log name, sink alignment, implementation test, silent page sizes, header 🟠 OPEN
• all five hold at 1c25c6d693 (read)
• sink: backend descriptors skip the dtype check, so reachable (read)
• decode log name: develop's DeclinedBy (14242) fits; no test declines a dense graph on decode (read)
BrianHarrisonAMD (11-04) ×5 tests: shared SDPA reference, shared scaffolding, cache-key "on" side, GPU buckets, decline reasons 🟠 OPEN
• all five hold (read); the quality lane found the same copies
• test_sdk CpuFpReferenceSdpa has band, seq lens, sink and LSE (read)
BrianHarrisonAMD (11-04) ×2 rocKE binding: paged K/V byte cap, dense_q/seqused_q unread 🟠 OPEN
• bindings.py:871 still caps whole paged K/V at I32 bytes; the kernel no longer needs it (read)
• neither field appears in bindings.py (read)
BrianHarrisonAMD (11-04) ×2 docs: README decode routing, YAML resolved values 🟡 OPEN
• both hold at head (read)
Bots / CI 🟢 Green, on 11-03's develop
• attempts 1–2: math-libs gfx94X, gfx125X never built: checkout hang, infra (read)
• attempt 3: 55 ✓ · 52 skip · 2 superseded; gfx110X ran padding-row, 64-bit paged-cache, paged-sink cases ✓ (read)
• merge: conflicts with develop, so no run since
Our checks 🟠 One conflict; gates clean
• merge vs 3ab1173644: 1 conflict (N1), resolved in scratch, CMake simulated (ran)
• native build + ctest 46 ✓ · 1 skip; byte identity GREEN ×4, decode 128 + fwd_v2 180 configs (ran)
• goldens: differential ×4 + canonical, representative ×4 OK; re-bless only fwd_v2: 23 paged, 10 ragged (64-bit row base), 10 new (ran)
• IR: 64-bit page and row terms, bad page ids read 0 (ran); padding rows O = 0, LSE = −inf (read)
• changed tests 54 ✓ · 154 GPU-only skip; generator tests as develop; descriptors regenerate identical; rules 1–3: 0 (ran)
• not run here: provider C++ build and tests (no hipDNN build); GPU cases; gfx12 objects (no CI runner)
Code quality 🟠 Should-fix
• paged-graph checks written twice with 14247, drifting: N4 (read)
• decline helper duplicates develop's new DeclinedBy, misnames decode: BrianHarrisonAMD's row (read)
• both rocKE engines carry every kernel change; 🟡 64-bit pointer step copied: N5 (read)
# Sev file:line Finding
N1 🟠 src/integration_tests/CMakeLists.txt:95 Conflicts with develop's Unified2D/3D knobs block (14242)
• impact: no CI against today's develop; git's keep-both does not parse (ran)
• fix: after develop's IntegrationGpuGfx950AttentionUnifiedKnobs.cpp line add ) and endif(), then your block unchanged; each suite stays under its own guard (ran)
N2 🟠 rocke/platform/cpp/include/rocke/helper_rocke.helpers.rdna_attention.h:126 Public struct layouts change without an ABI bump
• impact: rocke_rdna_attn_params_t grows 320 → 352 bytes, 4 offsets move; the fwd_v2 spec gains 2 fields; abi.h's rule needs a bump (ran)
• fix: bump ROCKE_ABI_VERSION (abi.h:100) and BINARY_ABI (abi.py:78) with a history line; develop is 8; takes develop + 1 when it lands
N3 🟠 src/engines/kernel_ingestor_engine/packs/Gfx1151WmmaAttentionNative.cpp:1005 48 paged sink objects serve a sink dtype the frontend rejects
• impact: the frontend admits only FLOAT (SdpaFwdNode.hpp:277), these read the Q dtype (ran); only backend-descriptor callers reach them, and CI skips all 3 sink cases (read)
• fix: serve a FLOAT sink (cuDNN's form), or hold the 48 until hipDNN decides, as 1c25c6d693 dropped unreachable dense-Q ones
N4 🟠 src/engines/kernel_ingestor_engine/packs/Gfx1151WmmaAttentionNative.cpp:1035 Paged-graph checks duplicate 14247's graphMatches and already drift
• impact: same contract rule for rule (14247 AttentionPagedUnifiedNative.cpp:477); cause keys differ; only 14247 checks the sink's alignment (read)
• fix: owner 14247 (wrote them first) moves them into packs/SdpaGraphMatch.hpp; drop yours and call those once it lands
N5 🟡 rocke/library/kernels/gfx1151/wmma_fmha_fwd_v2.py:545 The 64-bit pointer step gains new copies: page_head, page_base (rdna_attention.py:542), C++ twins
• impact: same ops as decode's batch_ptr/token_ptr; each copy mirrored in C++ (read)
• fix: one rdna_attention helper with its C++ twin (platform/AGENTS.md helpers rule 2); same op order, no golden moves

Utility: 🟢 Positive — paged and ragged SDPA for RDNA prefill; GPU cases pass on gfx110X CI


Legend: 🔴 blocking · 🟠 should-fix · 🟡 minor nit · 🟢 info/good · ⚪ retracted

First-round review: first pass by a local qwen3.6 model, verified by the orchestrator (Claude Opus 5.5). Checks: merge and conflict resolution, native build + ctest, byte identity, goldens and re-bless diff, IR hot spots, ABI layouts, changed and generator tests, descriptor regeneration, CI logs, reviewer items, hipDNN contract vs 14247, code quality. Flags whether a closer look is needed; not an approval or rejection.

@cgmillette

Copy link
Copy Markdown
Contributor

Automated overlap notice (rocKE merge monitor)

Hi @dpeabody-amd, this PR and #14247 (also yours) each write the same cuDNN paged-graph checks: gfx1151WmmaAttentionPagedGraphMatches here and graphMatches there apply them rule for rule, and the two copies already differ. Row N4 of our 4 November review has the evidence. The duplicate, here at 1c25c6d693 (Gfx1151WmmaAttentionNative.cpp) and in #14247 at 8e16db72e7 (AttentionPagedUnifiedNative.cpp), both in src/engines/kernel_ingestor_engine/packs/ (paths are under dnn-providers/hip-kernel-provider/):

  • The matchers: here :1035-1226, there :477-986.
  • Paging: K and V both paged (here :1040-1044, there :519-525); no packed page table (:1047-1054, :630-638); one block size (:1065-1071, :569-577) and one page count (:1072-1078, :578-591) for K and V.
  • Page tables: INT32 [B, 1, M, 1] with a unit entry stride (:1124-1135, :685-696); K and V tables of one width and one row stride (:1138-1151, :698-718).
  • Sequence lengths and ragged Q: seq_len_q and seq_len_kv required, as INT32 [B, 1, 1, 1] vectors (:1162-1176 with isDeviceI32Vector :488; :737-760 with isInt32Vector :337); a ragged Q's INT32 [B+1, 1, 1, 1] offsets, shared with O and counted in tokens (:1079-1087, :1171-1176 and isWellFormedRaggedOperand :456; :617-625, :639-650 and queryOperandOk :442-450).
  • Declared alignment: 16 bytes for Q, K, V and O, 4 for the INT32 operands (hasServedAlignment :873-897, called at :1177; :762-800), each file with its own declaresAlignment (:862, :355).
  • The sink: sink_token [1, Hq, 1, 1] in Q's data type with a unit head stride, FLOAT declined (sinkFor :985-1022; :859-898).

The copies already differ in two ways:

Owner: #14247. Neither PR puts the checks in a shared home and the merge plan orders neither, so the PR that wrote them first owns them: #14247's are dated 21 October, this PR's 22 October. Resolution: #14247 moves the paged-graph contract checks, with one cause vocabulary (its keys above), into develop's packs/SdpaGraphMatch.hpp (from #14242; N2 of our review of #14247 asks it to use that header). There they log through sdpa_graph::DeclinedBy with the calling engine's name, and the header's file comment, which today covers dense BSHD nodes only, widens to paged graphs. This PR keeps only its gfx1151 kernel limits and calls the shared checks. The shared version settles both differences: one set of keys, and a match-time alignment check that covers the sink, each kernel passing its own minimum (16 bytes for #14247's kernels, 2 for yours).

Suggested order: the merge plan (develop 07fcd51c3e) does not place either PR yet, and both conflict with develop: this PR in src/integration_tests/CMakeLists.txt, #14247 in six files. For this overlap, #14247 first is the shorter path: the shared checks land once, and this PR's rebase deletes its copy and calls them. In the other order this PR takes #14247's keys now, and #14247's rebase replaces the copy. Both PRs also bump the binary ABI (#14247 already, this PR per N2 of our review), so the second to land takes develop + 1.

Rebase, if #14247 lands first (checked on develop 07fcd51c3e merged with #14247 at 8e16db72e7, its six conflicts resolved as in our review of #14247, then this PR at 1c25c6d693: 3 files conflict, and Gfx1151WmmaAttentionNative.cpp merges cleanly), in your merge of develop:

  1. rocke/library/dispatch/attention/common.py, cache_key: one comment naming both sets of options, and one tuple: f.name in ("emit_lse", "runtime_layout", "dense_q", "seqused_q").
  2. src/tests/engines/kernel_ingestor_engine/CMakeLists.txt: keep both test sources.
  3. src/integration_tests/CMakeLists.txt: as N1 of our review, after develop's IntegrationGpuGfx950AttentionUnifiedKnobs.cpp line add ) and endif(), then your block unchanged. feat(hip-kernel-provider): paged/ragged SDPA engine for gfx942/gfx950 #14247's paged block stays above them; each suite keeps its own guard.
  4. src/engines/kernel_ingestor_engine/packs/Gfx1151WmmaAttentionNative.cpp: include SdpaGraphMatch.hpp, and in gfx1151WmmaAttentionPagedGraphMatches call the paged checks feat(hip-kernel-provider): paged/ragged SDPA engine for gfx942/gfx950 #14247 adds to sdpa_graph:: (their names are feat(hip-kernel-provider): paged/ragged SDPA engine for gfx942/gfx950 #14247's) in place of the rules listed above, passing this engine's name and its 2-byte sink minimum. Delete what that leaves without a caller (isDeviceI32Tensor :476, isDeviceI32Vector :488), and replace your declaresAlignment (:862) with the header's, which hasServedAlignment then calls for the dense matcher. Keep the gfx1151 kernel limits: power-of-two pages (:1097-1101), any row-aligned K/V strides with one page of one head I32-addressable (:1116-1122), the page tables' I32 limits (:1129-1130), the served head sizes and other I32 limits, the max_seq_len_kv bound (:1152-1161), which only these kernels read, no stats for a ragged Q (:1187-1191), and execute's pointer checks.
  5. src/tests/engines/kernel_ingestor_engine/packs/TestGfx1151WmmaAttentionPaged.cpp, LogsTheCauseOfEachDecline: map the expected causes, padding to seq_len_q (:758); paging to one_sided_paging (:759-760), page_table (:761, :771, :778), block_size (:762, :768) or shape (:765); bias to attn_mask (:784, :787); and the reason fragments to the shared texts. The dense matcher's padding (:1287, tested at TestGfx1151WmmaAttentionMatchers.cpp:916-923 in the same directory) is not a paged check and stays.

On those trees the resolved common.py parses, and both CMake files parse in CMake script mode, while git's plain keep-both of the integration file does not. Nothing was built: there is no hipDNN SDK here.

If this PR lands first, take the owner's keys now, so #14247's move keeps every key this engine logs:

  • In gfx1151WmmaAttentionPagedGraphMatches, log one_sided_paging (:1042), page_table (:1051, :1140, :1147), block_size (:1067, :1099), shape (:1074), seq_len_q or seq_len_kv, whichever is missing (:1166), and attn_mask (:1200), with the test expectations of step 5. BrianHarrisonAMD's :1097 comment asks to log unshipped page sizes as paging; under the shared keys that is block_size, feat(hip-kernel-provider): paged/ragged SDPA engine for gfx942/gfx950 #14247's key for its own unshipped block sizes (:600-604).
  • In sinkFor, decline a sink that does not declare a multiple of 2 bytes, cause alignment, with a case in DeclinesOperandsDeclaringLessThanTheKernelsLoad (TestGfx1151WmmaAttentionPaged.cpp:857).

#14247 then lands second and does the move in its rebase: it resolves common.py as in step 1 along with its six conflicts with develop (checked: #14247 merged onto develop with this PR, resolved as in N1, conflicts in exactly those 7 files, neither pack among them), puts the checks in SdpaGraphMatch.hpp, and switches this matcher to them as in step 4.

If you think this PR should own the checks instead, reply here and on #14247, and the reviews will follow the new decision.

Automated overlap check: one owner per shared piece of code, coordinated on the PRs. Nothing else changed.

@dpeabody-amd
dpeabody-amd force-pushed the users/dpeabody/hipdnn-rdna-paged-attn branch from 1c25c6d to 5f4c910 Compare November 4, 2026 23:43
@dpeabody-amd

Copy link
Copy Markdown
Contributor Author

Updated to 5f4c910, rebased onto develop. Replies to the inline threads are on each thread.

@BrianHarrisonAMD, review comment on the tests claim

Thanks. Every decline test now asserts the engine name, the cause and a part of the logged reason (expectDecline): TestGfx1151WmmaAttentionPaged.LogsTheCauseOfEachDecline, DeclinesOperandsDeclaringLessThanTheKernelsLoad and DeclinesSinksTheKernelsCannotRead, and TestGfx1151WmmaAttentionMatchers.LogsTheCauseOfEachDenseDecline and DeclinesOperandsDeclaringLessThanSixteenBytes. The reasons now name the operand and the values involved, for example "the K and V containers differ in block size (16 and 32)", and the tests check those. The description and the commit message now say what is asserted. (5f4c910)

@cgmillette, first-round review

  • N1: resolved as you describe: develop's Unified2D/3D knobs block is closed with its own ) and endif(), and the paged suite follows under its own guard. The CI-configuration build configures and builds the integration tests.
  • N2: ROCKE_ABI_VERSION and BINARY_ABI move to 9 (develop is 8), with a history line naming the rocke_rdna_attn_params_t and rocke_wmma_fmha_fwd_v2_spec_t changes. (93368d9)
  • N3: I am keeping the paged sink objects. The hipDNN team is deciding whether hipDNN accepts a sink in the Q data type, which is what vLLM passes. Until then the frontend's FLOAT-only rule means the GPU sink cases skip with an explicit message, and the engine declines a FLOAT sink with a logged cause.
  • N4: this PR is likely to land first, so it now logs feat(hip-kernel-provider): paged/ragged SDPA engine for gfx942/gfx950 #14247's cause keys (see the overlap notice reply). The move into SdpaGraphMatch.hpp stays with feat(hip-kernel-provider): paged/ragged SDPA engine for gfx942/gfx950 #14247.
  • N5: the step is now one helper, rdna_attention_scaled_ptr (C++ twin rocke_rdna_attention_scaled_ptr), used for page_base, page_head and the decode kernel's batch, token and bias bases, with the ops in the same order. The gfx1151 goldens are unchanged without re-blessing, and Python-vs-C parity is green. (467e119)

@cgmillette, overlap notice

Thanks. I have taken the "if this PR lands first" path. The paged matcher now logs #14247's keys: one_sided_paging; page_table for a packed table, K and V tables of different widths or row strides, and a malformed table; block_size for different K and V block sizes, a page size that is not a power of two, and an unshipped page size; shape for different K and V page counts; seq_len_q or seq_len_kv for whichever is missing or malformed; ragged for malformed offsets; and attn_mask for an additive bias. The tests map their expectations to these keys and assert the reasons. The sink alignment decline is in (cause alignment, 2 bytes). #14247 can then move the shared checks into SdpaGraphMatch.hpp and switch this matcher to them in its rebase. (5f4c910)

@cgmillette

Copy link
Copy Markdown
Contributor

Automated review: Model - qwen3.6:35b-a3b-coding-mtp-q4_K_M, effort: max

PR #14254 — paged/ragged SDPA in Gfx1151WmmaAttention (dpeabody-amd, AICK-1847) — 🟠 Follow-up: head-96 spills, sinks, open threads

Prior item Status
BrianHarrisonAMD (11-04) ×5 matcher: decode log name, sink alignment, implementation test, silent page sizes, header 🟢 FIXED 5f4c91012b
• each engine logs under its own name; sink under 2 bytes declined (read)
• forced-implementation rows; unshipped pages logged as block_size; header complete (read)
• these gtests pass in CI on gfx110X and gfx94X (read)
BrianHarrisonAMD (11-04) ×5 tests: shared SDPA reference, shared scaffolding, cache-key "on" side, GPU buckets, decline reasons 🟢 FIXED 5f4c91012b 93368d9c5c
• CpuFpReferenceSdpa with data_sdk types; one shared support header (read)
• "on"-side cache-key test passes here (ran); each decline asserts its reason (read)
• gfx110X CI: 15 paged GPU cases ✓, every bucket at every page; 3 sink cases skip (read)
BrianHarrisonAMD (11-04) ×2 rocKE binding: paged K/V byte cap, dense_q/seqused_q unread 🟢 FIXED 467e11912d 93368d9c5c
• paged prefill checks one page of one head; decode keeps its cap (read)
• both fields read; dense-Q launch and I32 tests pass on gfx1151 here (ran)
BrianHarrisonAMD (11-04) ×2 docs: README decode routing, YAML resolved values 🟢 FIXED 0600cad5de 5f4c91012b
• README scoped to dense graphs (read)
• seqused_q dropped; dense_q: 0 must stay: the generator refuses the config without it (ran)
• thread open: BrianHarrisonAMD resolves it or picks the arm form
AnaghaRaoAMD (11-04) ×3 matcher: LSE/bias alignment, FLOAT sink, ragged multiplier 🟠 OPEN
• no reply yet; sink under 2 bytes now declined, LSE and bias still unchecked (read)
• a ragged multiplier off the token stride still declines silently; 14247 logs ragged (read)
• FLOAT sink: Ours N3
AnaghaRaoAMD (11-04) ×3 kernel and tests: tile-path page id, arch gate, parity rows 🟠 OPEN
• tile path still feeds the raw page id to an inbounds GEP (read)
• the gfx12 prefix admits gfx1250, which no object lists (read)
• parity rows unchanged: no u row with sinks, page 64 or head 96 (read)
Ours (11-04) N1: conflict with develop's Unified2D/3D knobs block 🟢 FIXED 0600cad5de
• knobs block closed; paged suite under its own guard (ran: probe)
• merges cleanly with 027419e693 (ran)
Ours (11-04) N2: struct layouts change without an ABI bump 🟢 FIXED 93368d9c5c
• both counters at 9, with a history line (ran: probe); develop is 8; takes develop + 1 when it lands
• nit: the fwd_v2 spec keeps its size and offsets; that line says both change (ran)
Ours (11-04) N3: 48 paged sink objects, sink dtype the frontend rejects 🟠 DEFERRED
• author: "The hipDNN team is deciding" on Q-dtype sinks (read)
• no frontend graph reaches the 48 yet; FLOAT still declined (ran: probe)
• waiting is fair; hold the 48 until then, as AnaghaRaoAMD also asks
Ours (11-04) N4: paged checks duplicate 14247's and drift 🟢 FIXED 5f4c91012b
• 14247's cause keys; the sink's alignment decline added (read)
• the move into SdpaGraphMatch.hpp stays with 14247, as decided
• one drift left, the silent ragged multiplier: AnaghaRaoAMD's row
Ours (11-04) N5: 64-bit pointer step copied 🟢 FIXED 467e11912d
• one rdna_attention_scaled_ptr with its C++ twin; byte identity GREEN, goldens unchanged (ran)
Bots / CI 🟢 Green on re-run
• attempt 1: two Windows gfx110X test jobs lost their runner: infra; re-run ✓ (read)
• 51 ✓ · 51 skipped · 1 superseded; Multi-Arch, Component, pre-commit ✓ (read)
• pending: required Math CI Summary not posted yet; codecov (hipDNN) (read)
• merge: clean; CI's ref predates 14929 and 14834, both in our merge
Our checks 🟠 Gates clean; spills
• merge vs 027419e693: clean; touched differential goldens ✓, llvm20–24 + canonical (ran)
• with 14929: build + ctest 46 ✓ / 1 skip; byte identity GREEN ×4; library parity, both goldens ✓ (ran)
• changed tests 159 ✓ on CPU; installed 556 ✓ / 11 skip, GPU cases on gfx1151 (ran)
• descriptors regenerate byte-identical; probes: N1, N2 fixed, N3 reproduces (ran)
• catalog compile: 16 of 96 paged objects spill: N6; paged FP8 parity cfg 126 too (ran)
Code quality 🟡 Minor
• first-round copies merged: 64-bit step, SDK reference (read)
• paged checks follow the recorded 14247 decision (read)
• page-size list unguarded: N7; binding checks copy their siblings (read)
# Sev file:line Finding
N6 🟠 rocke/platform/python/rocke/helpers/rdna_attention.py:578 16 of the 96 new paged objects spill VGPRs on gfx11
• impact: every head-96 object at pages 32 and 64; 14 reload spills inside the K loop; gfx12 none. Since 467e119: the same spec used 242–243 VGPRs before, 256 after (ran)
• fix: win back the VGPRs the tile path's 64-bit page step added, or block_m 64 for head 96 on gfx11, which compiles without spills (ran)
N7 🟡 src/engines/kernel_ingestor_engine/packs/Gfx1151WmmaAttentionNative.cpp:147 SHIPPED_PAGE_SIZES restates the catalog's page sizes, and nothing ties the two
• impact: a page size added to the YAML ships objects this matcher declines; no test fails (read)
• fix: one pack-header list and a census case over the loaded objects, as develop's gfx950 catalog-tiles test does

Utility: 🟢 Positive — paged and ragged SDPA for RDNA prefill; most earlier items fixed


Legend: 🔴 blocking · 🟠 should-fix · 🟡 minor nit · 🟢 info/good · ⚪ retracted

Follow-up review: first pass by a local qwen3.6 model, verified by the orchestrator (Claude Opus 5.5). Checks: range-diff, merge with today's develop, build and ctest, byte identity, library parity, both goldens, changed and installed tests, catalog compile sweep, descriptor regeneration, ABI layouts, saved probes, CI logs, reviewer items, code quality. Flags whether a closer look is needed; not an approval or rejection.

dpeabody-amd added a commit that referenced this pull request Nov 5, 2026
…SdpaGraphMatch.hpp

hipkernel:AttentionPagedUnified's graph_match carried its own copies of the
SDPA match helpers that packs/SdpaGraphMatch.hpp now shares (findTensor,
sdpaNode, hasSdpaForwardNode, describeOperand, the decline head and the axis
constants). It now uses the header's, with sdpa_graph::DeclinedBy. Its local
overflow-checked product is renamed spanProduct: unlike
pack_features::checkedProduct it takes zero factors, which every extent of 1
gives.

The stricter operand checks it needs move into the header too
(usesOverrideShapes, hasNonNegativeStrides, declaresAlignment, isInt32Vector),
together with the cuDNN paged-K/V graph contract checks (SEM-009), so every
pack serving a paged graph applies one set of rules with one cause vocabulary,
logged with the calling engine's name:
- pagesKAndV: one_sided_paging;
- pagedContainersAgree: block_size, shape (one block size, one block count;
  head counts and head sizes stay the engine's to check);
- pageTablesFor: page_table (INT32 [B, 1, M, 1], unit entry stride, not
  packed; separate K and V tables share M and the batch stride);
- sequenceLengthsFor: seq_len_kv, seq_len_q (both required, INT32 [B,1,1,1]);
- raggedQueryFor, countsOffsetsInTokens: ragged (Q and O through one
  [B+1,1,1,1] offset table counted in tokens; K/V never ragged);
- declaresServedAlignment: alignment (vector operands at the engine's vector
  width, INT32 operands at 4 bytes);
- sinkFor: sinks, and alignment below the engine's own minimum sink
  alignment (16 bytes for this engine's kernels);
- takesNoAttentionMask: attn_mask.

The reasons they log are the texts hipkernel:Gfx1151WmmaAttention's paged
matcher (#14254) logs for the same rules, so that engine can call these checks
without changing what it reports. The decline lines go through
HIPDNN_PLUGIN_LOG_INFO((DeclinedBy{...}) << ...): the braces' comma would
otherwise split the macro argument.

The engine keeps its kernel limits (head and block sizes, GQA ratios, strides
and address ranges) and the order of its declines; it checks itself that V has
K's head count and head size. A malformed page-table tensor now logs page_table
rather than operand. The header's file comment covers paged graphs.
@dpeabody-amd
dpeabody-amd force-pushed the users/dpeabody/hipdnn-rdna-paged-attn branch from 5f4c910 to 89ee88a Compare November 5, 2026 21:11
@cgmillette

Copy link
Copy Markdown
Contributor

Automated review: Model - qwen3.6:35b-a3b-coding-mtp-q4_K_M, effort: max

PR #14254 — paged/ragged SDPA in Gfx1151WmmaAttention (dpeabody-amd, AICK-1847) — 🟠 Follow-up: rebase; threads, spills, sinks open

Prior item Status
BrianHarrisonAMD (11-04) ×14: matcher, tests, binding, docs 🟢 FIXED 89ee88af23, f8e1d5dc77, 45a53a41b1, a12b606330
• rebase only since 5f4c91012b: every fix carried unchanged (ran)
• YAML thread open: seqused_q gone from the 8 paged entries; dense_q: 0 stays, the loader needs it (read)
• BrianHarrisonAMD resolves it or picks the arm form
AnaghaRaoAMD (11-04) ×3 matcher: LSE/bias alignment, FLOAT sink, ragged multiplier 🟠 OPEN
• no reply; LSE and bias still unchecked (Gfx1151WmmaAttentionNative.cpp:903) (read)
• multiplier still declines silently (:1136); 14247's shared check logs it, so the N4 swap closes it (read)
• FLOAT sink: Ours N3
AnaghaRaoAMD (11-04) ×3 kernel and tests: tile-path page id, arch gate, parity rows 🟠 OPEN
• no reply; raw page id still feeds the page base (rdna_attention.py:578, rdna_attention.cpp:734) (read)
• gfx11/gfx12 prefix gate unchanged (IntegrationGpuGfx1151WmmaAttentionPaged.cpp:850) (read)
• parity rows: still no u row with sinks, page 64 or head 96 (read)
Ours (11-04) N2: struct layouts change without an ABI bump 🟢 FIXED f8e1d5dc77
• re-cut as entry 10 above develop's 9; both counters 10 (ran: probe)
• develop is 9; takes develop + 1 when it lands
• nit unchanged: the fwd_v2 spec keeps 120 bytes and its offsets; the entry says both change (ran)
Ours (11-04) N3: 48 paged sink objects, sink dtype the frontend rejects 🟠 DEFERRED
• author: "The hipDNN team is deciding" on Q-dtype sinks (read)
• still 48 Q-dtype sink objects; the frontend admits only FLOAT (ran: probe)
• waiting is fair; hold the 48 until then, as AnaghaRaoAMD also asks
Ours (11-04) N4 + overlap notice: paged contract checks, owner 14247 🟡 OPEN
• drift fixed: 14247's cause keys (89ee88af23); own copies remain (Gfx1151WmmaAttentionNative.cpp:1081-1304), as decided (read)
• 14247 now shares them in SdpaGraphMatch.hpp (19f309b9f7) with these reasons; not mergeable yet (read)
• left: if this lands first, 14247's rebase switches this matcher (author's plan); else this PR swaps at its rebase
Ours (11-05) N6: 16 of the 96 paged objects spill VGPRs on gfx11 🟠 OPEN
• no reply; the same 16 objects: head 96, pages 32 and 64; gfx12 none (ran: probe)
• fix as posted: win back the tile path's VGPRs, or block_m 64 for head 96 on gfx11
Ours (11-05) N7: SHIPPED_PAGE_SIZES restates the catalog's page sizes 🟡 OPEN
• no reply; the set is spelled 6 times: :147, the reason at :1163, YAML :20/:200/:224, README :520 (read)
• fix: one YAML anchor, which loads identically (144 objects) (ran); the C++ list in a pack header, pinned by a census case over loadedSet()
Ours (11-04) N1, N5: integration CMake block; 64-bit pointer step 🟢 FIXED a12b606330, 45a53a41b1
• N1 probe passes at head (ran)
• N5: one helper per engine; byte identity GREEN ×4 (ran)
Bots / CI 🟡 Running
• 43 ✓ · 51 skipped · 1 cancelled (superseded PR Bot run) (read)
• gfx94X rocKE shards ✓: paged and dense matcher gtests pass; GPU cases skip, need RDNA (read)
• owed: Windows gfx110X rocKE shards (the 18 paged GPU cases); sanity queued since 22:42Z
• merge: clean; blocked: review required, 7 threads open (read)
Our checks 🟠 Gates clean; spills
• merge vs 8c19a6a975 (has 15101 and 14831, which CI's ref lacks): clean; develop's next commit misses every PR path (ran)
• build + ctest 48 ✓ / 1 skip; byte identity GREEN ×4, every family; parity ×4; both goldens ✓ (ran)
• changed tests 159 ✓ on CPU; installed 556 ✓ / 11 skip, GPU cases on gfx1151 (ran)
• descriptors regenerate byte-identical; generator tests 554 ✓ with 15101 (ran)
• probes: N1, N2 fixed; N3, N6 reproduce; compile adds parity cfg 126's spill (ran)
Code quality 🟡 Minor
• N7 open: the YAML is the page sizes' home (ran)
• paged checks now shared in 14247; the swap is owed: N4 row (read)
• test support repeats develop's GraphFixture, matchesGraph, dispatchHandler (Gfx1151WmmaAttentionTestSupport.hpp:125): reuse IngestorPackTestSupport.hpp (read)
• unchanged: binding I32 checks copy their siblings (bindings.py:683); ABI history stated once (read)

No net-new findings. The push re-cut only the ABI bump; the quality lane's points sit in the N4, N7 and Code quality rows.

Utility: 🟢 Positive — paged and ragged SDPA for RDNA prefill; the rebase kept every fix.


Legend: 🔴 blocking · 🟠 should-fix · 🟡 minor nit · 🟢 info/good · ⚪ retracted

Follow-up review: first pass by a local qwen3.6 model, verified by the orchestrator (Claude Opus 5.5). Checks: changes since 5f4c91012b (range-diff, rebase replay), merge with today's develop (build + ctest, byte identity, parity, goldens, differential golden, CPU and installed tests, compile), struct layouts, saved probes, the paged catalog compile, descriptor regeneration and generator tests on the merge, CI at head with shard logs, the 14247 overlap, reviewer threads, code quality. Flags whether a closer look is needed; not an approval or rejection.

dpeabody-amd added a commit that referenced this pull request Nov 5, 2026
…SdpaGraphMatch.hpp

hipkernel:AttentionPagedUnified's graph_match carried its own copies of the
SDPA match helpers that packs/SdpaGraphMatch.hpp now shares (findTensor,
sdpaNode, hasSdpaForwardNode, describeOperand, the decline head and the axis
constants). It now uses the header's, with sdpa_graph::DeclinedBy. Its local
overflow-checked product is renamed spanProduct: unlike
pack_features::checkedProduct it takes zero factors, which every extent of 1
gives.

The stricter operand checks it needs move into the header too
(usesOverrideShapes, hasNonNegativeStrides, declaresAlignment, isInt32Vector),
together with the cuDNN paged-K/V graph contract checks (SEM-009), so every
pack serving a paged graph applies one set of rules with one cause vocabulary,
logged with the calling engine's name:
- pagesKAndV: one_sided_paging;
- pagedContainersAgree: block_size, shape (one block size, one block count;
  head counts and head sizes stay the engine's to check);
- pageTablesFor: page_table (INT32 [B, 1, M, 1], unit entry stride, not
  packed; separate K and V tables share M and the batch stride);
- sequenceLengthsFor: seq_len_kv, seq_len_q (both required, INT32 [B,1,1,1]);
- raggedQueryFor, countsOffsetsInTokens: ragged (Q and O through one
  [B+1,1,1,1] offset table counted in tokens; K/V never ragged);
- declaresServedAlignment: alignment (vector operands at the engine's vector
  width, INT32 operands at 4 bytes);
- sinkFor: sinks, and alignment below the engine's own minimum sink
  alignment (16 bytes for this engine's kernels);
- takesNoAttentionMask: attn_mask.

The reasons they log are the texts hipkernel:Gfx1151WmmaAttention's paged
matcher (#14254) logs for the same rules, so that engine can call these checks
without changing what it reports. The decline lines go through
HIPDNN_PLUGIN_LOG_INFO((DeclinedBy{...}) << ...): the braces' comma would
otherwise split the macro argument.

The engine keeps its kernel limits (head and block sizes, GQA ratios, strides
and address ranges) and the order of its declines; it checks itself that V has
K's head count and head size. A malformed page-table tensor now logs page_table
rather than operand. The header's file comment covers paged graphs.
dpeabody-amd added a commit that referenced this pull request Nov 6, 2026
…SdpaGraphMatch.hpp

hipkernel:AttentionPagedUnified's graph_match carried its own copies of the
SDPA match helpers that packs/SdpaGraphMatch.hpp now shares (findTensor,
sdpaNode, hasSdpaForwardNode, describeOperand, the decline head and the axis
constants). It now uses the header's, with sdpa_graph::DeclinedBy. Its local
overflow-checked product is renamed spanProduct: unlike
pack_features::checkedProduct it takes zero factors, which every extent of 1
gives.

The stricter operand checks it needs move into the header too
(usesOverrideShapes, hasNonNegativeStrides, declaresAlignment, isInt32Vector),
together with the cuDNN paged-K/V graph contract checks (SEM-009), so every
pack serving a paged graph applies one set of rules with one cause vocabulary,
logged with the calling engine's name:
- pagesKAndV: one_sided_paging;
- pagedContainersAgree: block_size, shape (one block size, one block count;
  head counts and head sizes stay the engine's to check);
- pageTablesFor: page_table (INT32 [B, 1, M, 1], unit entry stride, not
  packed; separate K and V tables share M and the batch stride);
- sequenceLengthsFor: seq_len_kv, seq_len_q (both required, INT32 [B,1,1,1]);
- raggedQueryFor, countsOffsetsInTokens: ragged (Q and O through one
  [B+1,1,1,1] offset table counted in tokens; K/V never ragged);
- declaresServedAlignment: alignment (vector operands at the engine's vector
  width, INT32 operands at 4 bytes);
- sinkFor: sinks, and alignment below the engine's own minimum sink
  alignment (16 bytes for this engine's kernels);
- takesNoAttentionMask: attn_mask.

The reasons they log are the texts hipkernel:Gfx1151WmmaAttention's paged
matcher (#14254) logs for the same rules, so that engine can call these checks
without changing what it reports. The decline lines go through
HIPDNN_PLUGIN_LOG_INFO((DeclinedBy{...}) << ...): the braces' comma would
otherwise split the macro argument.

The engine keeps its kernel limits (head and block sizes, GQA ratios, strides
and address ranges) and the order of its declines; it checks itself that V has
K's head count and head size. A malformed page-table tensor now logs page_table
rather than operand. The header's file comment covers paged graphs.

@yraparti yraparti left a comment

Copy link
Copy Markdown
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

rocKE checklist review

ROCm/rocm-libraries#14254 · 89ee88af237d · model gemini-4-argon, fallback gpt-5.5

PASS · graded 238/238 · BLOCKER 0, HIGH 2, MEDIUM 0, LOW 0, NEEDS-INFO 15

The diff was truncated before grading. Items that needed a missing hunk are NEEDS-INFO.

Findings

  • HIGH §B · dnn-providers/hip-kernel-provider/rocke/library/kernels/gfx1151/wmma_fmha_fwd_v2.py:123 · In WmmaFmhaFwdV2Spec.__post_init__ (dnn-providers/hip-kernel-provider/rocke/library/kernels/gfx1151/wmma_fmha_fwd_v2.py:168), validate if type(self.dense_q) is not bool or type(self.seqused_q) is not bool: raise TypeError(...) before checking layout compatibility.
  • HIGH §D · dnn-providers/hip-kernel-provider/rocke/library/kernels/gfx1151/wmma_fmha_fwd_v2.py:123 · Expose dense_q and seqused_q (or the corresponding query layout/length presence) on AttentionRequest and wire the RDNA prefill spec builder to populate them from the request, or document a stacked PR deferral in the PR description.

Needs info

  • §A.10 dnn-providers/hip-kernel-provider/rocke/platform/tests/instances/differential/golden/llvm_gfx_all.json is truncated at line 1040, so the new golden hash entries for indices 170..179 (QUERY_LENGTHS flag-on cases across LLVM flavors and arches) are not visible in the diff.
  • §C.7 The diff is truncated and does not show enough of the large omitted generated/test/config hunks to verify whether any cleanup-only portion was proven as a no-op.
  • §C.8 The diff adds a new helper/primitive, but it does not include evidence that current develop was grepped for an existing equivalent before adding it.
  • §E.10 Merge-simulation against other open PRs touching the same files (e.g. abi.h, llvm_gfx_all.json, Gfx1151WmmaAttentionNative.cpp) cannot be verified from the PR diff alone.
  • §F.6 Local pre-commit execution status cannot be verified from the diff alone.
  • §F.7 Bot, Copilot, and human review thread states are not included in the prompt.
  • §F.11 While the provided title and body are populated in the prompt metadata, there is no diff path:line to cite for API field verification.
  • §G.2 On-device numerical tests with fp64/fp32 references and bf16/fp16 tolerances are added (e.g. dnn-providers/hip-kernel-provider/rocke/library/tests/test_wmma_fmha_fwd_v2_features.py:643,685 and dnn-providers/hip-kernel-provider/src/integration_tests/kernel_ingestor_engine/IntegrationGpuGfx1151WmmaAttentionPaged.cpp:867), but the PR body is truncated before the test execution summary so on-silicon pass confirmation is missing.

Section notes

  • §A: Two-engine parity and byte-identity verified across Python and C++ kernels, helpers, parity emitters, bindings, and goldens.
  • §C: Diff is truncated; items that require omitted hunks or non-diff evidence are marked NEEDS-INFO.
  • §D: Items 1–11 of 21 in Section D.
  • §D: Items 12-21 of Section D
  • §G: Items 1-17 of 34 in section G.
  • §G: Items 18-34 of 34.

Graded with the rocKE PR checklist, one model pass per section. FAIL rows that sit on a diff line are inline comments.

§A graded 14/14
# Verdict Evidence
1 PASS dnn-providers/hip-kernel-provider/rocke/library/kernels/gfx1151/wmma_fmha_fwd_v2.py:457 and dnn-providers/hip-kernel-provider/rocke/platform/cpp/instances/gfx1151/wmma_fmha_fwd_v2.cpp:408 keep the Python and C++ kernel-body emitters in sync (along with wmma_fmha_decode.py:748 and wmma_fmha_decode.cpp:1096).
2 N/A No core lowering-op or op-handler files (core/lower_*, core/ir) are modified in this PR.
3 PASS dnn-providers/hip-kernel-provider/rocke/library/kernels/gfx1151/wmma_fmha_fwd_v2.py:123, dnn-providers/hip-kernel-provider/rocke/platform/cpp/include/rocke/instance_gfx1151_wmma_fmha_fwd_v2.h:63, dnn-providers/hip-kernel-provider/rocke/platform/cpp/bindings/rocke_engine.cpp:4650, and dnn-providers/hip-kernel-provider/rocke/platform/cpp/bindings/prove_parity_binding.py:813 mirror the new spec fields and parity binding cases.
4 PASS dnn-providers/hip-kernel-provider/rocke/platform/cpp/helpers/rdna_attention.cpp:220 and dnn-providers/hip-kernel-provider/rocke/platform/cpp/instances/gfx1151/wmma_fmha_fwd_v2.cpp:473 bind operands left-to-right to temporaries to match the exact emit order in Python (rdna_attention.py:219, wmma_fmha_fwd_v2.py:536).
5 PASS dnn-providers/hip-kernel-provider/rocke/platform/tests/instances/differential/golden/llvm_gfx_all.json:1016 re-blesses only the packed/paged gfx1151_wmma_fmha_fwd_v2 entries affected by the intentional IR changes while leaving dense/unrelated entries untouched.
6 PASS dnn-providers/hip-kernel-provider/rocke/library/tests/parity/gfx1151_wmma_fmha_fwd_v2_emit.c:101, dnn-providers/hip-kernel-provider/rocke/library/tests/parity/gfx1151_wmma_fmha_fwd_v2_emit.py:93, and dnn-providers/hip-kernel-provider/rocke/platform/cpp/bindings/prove_parity_binding.py:813 add cross-engine parity cases alongside the golden updates.
7 PASS dnn-providers/hip-kernel-provider/rocke/library/kernels/gfx1151/wmma_fmha_fwd_v2.py:504 and dnn-providers/hip-kernel-provider/rocke/platform/python/rocke/helpers/rdna_attention.py:818 guard new ops behind layout != 'dense', spec.dense_q, spec.seqused_q, and live_rows is not None so covered dense shapes emit identical IR.
8 PASS dnn-providers/hip-kernel-provider/rocke/platform/cpp/helpers/rdna_attention.cpp:702 and dnn-providers/hip-kernel-provider/rocke/platform/cpp/instances/gfx1151/wmma_fmha_fwd_v2.cpp:410 were reviewed against their Python twins and wired into gfx1151_wmma_fmha_fwd_v2_emit.c:235.
9 N/A No shared core op-handler fix or lowerer-only arch divergence is introduced in this PR.
10 NEEDS-INFO dnn-providers/hip-kernel-provider/rocke/platform/tests/instances/differential/golden/llvm_gfx_all.json is truncated at line 1040, so the new golden hash entries for indices 170..179 (QUERY_LENGTHS flag-on cases across LLVM flavors and arches) are not visible in the diff.
11 N/A No new IR opcodes are added in this PR.
12 N/A core/backend.py and ROCKE_BACKEND lowering fallback logic are not touched in this PR.
13 N/A LLVM .ll emission format and ROCKE_DEBUG_LOC handling are not modified in this PR.
14 PASS dnn-providers/hip-kernel-provider/rocke/platform/python/rocke/helpers/rdna_attention.py:213, dnn-providers/hip-kernel-provider/rocke/platform/cpp/helpers/rdna_attention.cpp:214, and dnn-providers/hip-kernel-provider/rocke/platform/cpp/include/rocke/helper_rocke.helpers.rdna_attention.h:86 update the Python helper and its C++ twin and header in the same PR.
§B graded 6/6
# Verdict Evidence
1 PASS New rejections in dnn-providers/hip-kernel-provider/src/engines/kernel_ingestor_engine/packs/Gfx1151WmmaAttentionNative.cpp:848-888 (unservedFeature) and Gfx1151WmmaAttentionNative.cpp:1080-1324 (gfx1151WmmaAttentionPagedGraphMatches), as well as dnn-providers/hip-kernel-provider/rocke/library/dispatch/attention/bindings.py:683-705 (_check_page_extent_i32), clearly classify and log hardware/kernel limits vs unserved graph contracts.
2 PASS WmmaFmhaFwdV2Spec.__post_init__ in dnn-providers/hip-kernel-provider/rocke/library/kernels/gfx1151/wmma_fmha_fwd_v2.py:168-173 and C++ v2_post_init_ok in dnn-providers/hip-kernel-provider/rocke/platform/cpp/instances/gfx1151/wmma_fmha_fwd_v2.cpp:144-154 reject the exact same dense_q and seqused_q layout combinations.
3 PASS No shapes are demoted to a slow fallback; paged graphs are routed to the prefill engine (dnn-providers/hip-kernel-provider/src/engines/kernel_ingestor_engine/packs/Gfx1151WmmaAttentionNative.cpp:1339-1342) and unsupported graph variants are loudly declined at INFO (Gfx1151WmmaAttentionNative.cpp:836-839).
4 FAIL dnn-providers/hip-kernel-provider/rocke/library/kernels/gfx1151/wmma_fmha_fwd_v2.py:123-124 and 168-173 add public spec flags dense_q: bool = False and seqused_q: bool = False that change the kernel parameter ABI (wmma_fmha_fwd_v2.py:416-425), but WmmaFmhaFwdV2Spec.__post_init__ does not enforce type(self.dense_q) is bool or type(self.seqused_q) is bool, allowing truthy non-bools (e.g. "false") to silently enable the ABI.
5 PASS Newly opened gates (dense_q, seqused_q, separate block_table_v, and >2 GiB paged prefill K/V caches) are tested in dnn-providers/hip-kernel-provider/rocke/library/tests/test_wmma_fmha_fwd_v2_features.py:168-225, test_wmma_fmha_fwd_v2_features.py:685-855, dnn-providers/hip-kernel-provider/rocke/library/tests/dispatch/attention/test_gfx1151_binding_i32_guard.py:119-175, and…
6 PASS Spec validation in dnn-providers/hip-kernel-provider/rocke/library/kernels/gfx1151/wmma_fmha_fwd_v2.py:168-173 and dnn-providers/hip-kernel-provider/rocke/platform/cpp/instances/gfx1151/wmma_fmha_fwd_v2.cpp:144-154 only rejects functionally invalid layout combinations (dense_q outside paged, seqused_q on dense).
§C graded 10/10
# Verdict Evidence
1 PASS dnn-providers/hip-kernel-provider/rocke/platform/python/rocke/helpers/rdna_attention.py:210 defines the shared rdna_attention_scaled_ptr helper, and dnn-providers/hip-kernel-provider/rocke/library/kernels/gfx1151/wmma_fmha_decode.py:745 uses it instead of repeated local pointer-scaling logic.
2 PASS dnn-providers/hip-kernel-provider/rocke/library/kernels/gfx1151/wmma_fmha_fwd_v2.py:457 keeps the single public build_wmma_fmha_fwd_v2(spec) builder while adding the new behavior through spec fields declared at dnn-providers/hip-kernel-provider/rocke/library/kernels/gfx1151/wmma_fmha_fwd_v2.py:120.
3 PASS dnn-providers/hip-kernel-provider/rocke/library/kernels/gfx1151/wmma_fmha_fwd_v2.py:210 adds kernel-name suffixes for codegen/layout options (_denseq and _sequsedq), not workload sizes.
4 PASS dnn-providers/hip-kernel-provider/rocke/platform/python/rocke/helpers/rdna_attention.py:210 and dnn-providers/hip-kernel-provider/rocke/platform/cpp/helpers/rdna_attention.cpp:211 put the arch-independent pointer-scaling operation in shared helpers rather than duplicating it per arch instance.
5 PASS dnn-providers/hip-kernel-provider/src/engines/kernel_ingestor_engine/packs/Gfx1151WmmaAttentionNative.cpp:58 documents the scope as the existing Gfx1151WmmaAttention/Gfx1151WmmaAttentionDecode engines, and dnn-providers/hip-kernel-provider/rocke/library/dispatch/attention/README.md:515 confines the documented routing changes to paged prefill versus dense decode.
6 PASS dnn-providers/hip-kernel-provider/rocke/platform/python/rocke/helpers/rdna_attention.py:210 adds a named helper first, with the C++ twin at dnn-providers/hip-kernel-provider/rocke/platform/cpp/helpers/rdna_attention.cpp:211, instead of ad-hoc pointer math at each call site.
7 NEEDS-INFO The diff is truncated and does not show enough of the large omitted generated/test/config hunks to verify whether any cleanup-only portion was proven as a no-op.
8 NEEDS-INFO The diff adds a new helper/primitive, but it does not include evidence that current develop was grepped for an existing equivalent before adding it.
9 PASS dnn-providers/hip-kernel-provider/rocke/library/kernels/gfx1151/wmma_fmha_decode.py:604 updates the decode sibling to use the shared helper, and dnn-providers/hip-kernel-provider/rocke/platform/cpp/instances/gfx1151/wmma_fmha_decode.cpp:1094 updates the C++ twin as well.
10 PASS dnn-providers/hip-kernel-provider/rocke/platform/cpp/include/rocke/abi.h:98 documents the widened ABI structs and bumps ROCKE_ABI_VERSION, while dnn-providers/hip-kernel-provider/rocke/platform/python/rocke/portable_ir/src/abi.py:75 bumps the Python binary ABI mirror.
§D graded 21/21
# Verdict Evidence
1 PASS New query layout options dense_q and seqused_q are declared as explicit WmmaFmhaFwdV2Spec fields with validation, name tags, and cache-key entries without reading os.environ (dnn-providers/hip-kernel-provider/rocke/library/kernels/gfx1151/wmma_fmha_fwd_v2.py:123, dnn-providers/hip-kernel-provider/rocke/library/kernels/gfx1151/wmma_fmha_fwd_v2.py:213, dnn-providers/hip-kernel-provider/rocke/library/dispatch/attention/common.py:997).
2 N/A No Capability declarations or candidate arches lists are added or modified in this PR.
3 N/A No _supports predicates or admits call sites are added or modified in this PR.
4 N/A No Python dispatcher selector or spec-builder routing logic on the selection path is modified in this PR.
5 PASS dense_q and seqused_q are baked into kernel_name() (dnn-providers/hip-kernel-provider/rocke/library/kernels/gfx1151/wmma_fmha_fwd_v2.py:213) and included in the spec cache key / identity() when enabled while preserving existing hashes when off (dnn-providers/hip-kernel-provider/rocke/library/dispatch/attention/common.py:997, dnn-providers/hip-kernel-provider/rocke/library/tests/dispatch/attention/test_gfx1151_attention_selection.py:900).
6 N/A No dispatcher cohort routing is added or modified in this PR.
7 N/A No heuristic tables or hand-pinned overrides in the Python dispatcher are modified in this PR.
8 N/A No cohort definitions or spec override predicates are added or modified in this PR.
9 N/A No new KernelCandidate or architecture is registered in the Python dispatcher registries.
10 FAIL dense_q and seqused_q are added to WmmaFmhaFwdV2Spec (dnn-providers/hip-kernel-provider/rocke/library/kernels/gfx1151/wmma_fmha_fwd_v2.py:123) and classified under WMMA_PROBLEM_FIELDS as coming from the request (dnn-providers/hip-kernel-provider/rocke/library/dispatch/attention/axes.py:346), but common.py explicitly notes dispatch never sets them (dnn-providers/hip-kernel-provider/rocke/library/dispatch/attention/common.py:991) and tests must manually construct WmmaFmhaFwdV2Spec(dense_q=True, seqused_q=True) instead of reaching them from AttentionRequest (dnn-providers/hip-kernel-provider/rocke/library/tests/dispatch/attention/test_gfx1151_attention_dispatch_numeric.py:2165).
11 N/A No new dispatcher candidates are registered or promoted in the Python dispatcher registries.
12 N/A No dispatcher candidates, priorities, candidate names, or custom rankers are added or modified in the Python dispatcher registry.
13 N/A No new dispatcher wrapper or entry point is added, and pinned request handling in CandidateRegistry.select is not modified.
14 PASS Identity/hash preservation for off-by-default spec fields (dense_q, seqused_q) and differentiation when enabled are maintained in dnn-providers/hip-kernel-provider/rocke/library/dispatch/attention/common.py:990-1000 and verified in dnn-providers/hip-kernel-provider/rocke/library/tests/dispatch/attention/test_gfx1151_attention_selection.py:890-925, and C/Python ABI versions are bumped in…
15 N/A No new dispatcher candidate or arch module is added to the Python dispatcher registry.
16 N/A No new operator family is introduced in this PR.
17 PASS Every new WmmaFmhaFwdV2Spec field (dense_q, seqused_q) is classified in WMMA_PROBLEM_FIELDS at dnn-providers/hip-kernel-provider/rocke/library/dispatch/attention/axes.py:346-347 and tested for spec/hash invariants in dnn-providers/hip-kernel-provider/rocke/library/tests/dispatch/attention/test_gfx1151_attention_selection.py:887-925.
18 N/A Registry ratchets (require_build, require_binding, require_torch_binding) and exempt sets are not modified in this PR.
19 N/A This PR does not touch run_unified_attention_torch or claim unified attention auto-routing registration.
20 PASS Spec and builder changes in WmmaFmhaFwdV2Spec (dnn-providers/hip-kernel-provider/rocke/library/kernels/gfx1151/wmma_fmha_fwd_v2.py:123-124) are mirrored in C++ (dnn-providers/hip-kernel-provider/rocke/platform/cpp/include/rocke/instance_gfx1151_wmma_fmha_fwd_v2.h:63-64, dnn-providers/hip-kernel-provider/rocke/platform/cpp/instances/gfx1151/wmma_fmha_fwd_v2.cpp:144-154) and parity tests…
21 N/A This PR adds kernel/spec features (dense_q, seqused_q, block_table_v, 64-bit page addressing) and hipDNN native matcher support rather than working around a selection bug in the emitter.
§E graded 17/17
# Verdict Evidence
1 PASS dnn-providers/hip-kernel-provider/rocke/platform/python/rocke/helpers/rdna_attention.py:48 imports only from rocke.core.ir, while library kernels import from platform (dnn-providers/hip-kernel-provider/rocke/library/kernels/gfx1151/wmma_fmha_fwd_v2.py:460).
2 PASS dnn-providers/hip-kernel-provider/rocke/library/tests/test_wmma_fmha_fwd_v2_features.py:699 uses rocke.runtime.hip_module.Runtime rather than importing torch.
3 PASS dnn-providers/hip-kernel-provider/rocke/library/tests/test_wmma_fmha_fwd_v2_features.py:690 gates optional dependencies with pytest.importorskip and uses numpy/Runtime without requiring torch.
4 PASS dnn-providers/hip-kernel-provider/rocke/library/kernels/gfx1151/wmma_fmha_fwd_v2.py:459-463 and dnn-providers/hip-kernel-provider/rocke/library/dispatch/attention/bindings.py:608 add no upward imports across library layers.
5 PASS dnn-providers/hip-kernel-provider/rocke/platform/python/rocke/helpers/rdna_attention.py:213-220 and dnn-providers/hip-kernel-provider/rocke/platform/cpp/bindings/prove_parity_binding.py:809-852 introduce no absolute or repo-escaping paths or shell scripts.
6 PASS dnn-providers/hip-kernel-provider/rocke/library/kernels/gfx1151/wmma_fmha_fwd_v2.py:556-564 and dnn-providers/hip-kernel-provider/rocke/platform/tests/instances/differential/golden/llvm_gfx_all.json:1016-1036 show intentional emitter edits paired with golden updates rather than lint autofix removals.
7 PASS dnn-providers/hip-kernel-provider/rocke/platform/dsl_docs/instances/attention.md:38-41 and dnn-providers/hip-kernel-provider/rocke/library/dispatch/attention/README.md:492-527 update the instance and dispatch reference docs alongside the new dense_q, seqused_q, and block_table_v features.
8 PASS dnn-providers/hip-kernel-provider/rocke/library/kernels/gfx1151/wmma_fmha_fwd_v2.py:33-57 and dnn-providers/hip-kernel-provider/src/engines/kernel_ingestor_engine/packs/Gfx1151WmmaAttentionNative.cpp:82-114 use descriptive spec/RFC references without Jira IDs, PR numbers, or user handles in code.
9 PASS dnn-providers/hip-kernel-provider/rocke/library/kernels/gfx1151/wmma_fmha_fwd_v2.py:123-124 and dnn-providers/hip-kernel-provider/src/engines/kernel_ingestor_engine/packs/Gfx1151WmmaAttentionNative.cpp:761-764 keep all changes scoped to paged/ragged SDPA support in Gfx1151WmmaAttention and its rocKE kernel/test plumbing.
10 NEEDS-INFO Merge-simulation against other open PRs touching the same files (e.g. abi.h, llvm_gfx_all.json, Gfx1151WmmaAttentionNative.cpp) cannot be verified from the PR diff alone.
11 PASS dnn-providers/hip-kernel-provider/rocke/platform/dsl_docs/instances/attention.md:38-41 and dnn-providers/hip-kernel-provider/rocke/library/dispatch/attention/README.md:492-500 are required deliverable docs; no scratch notes or throwaway markdown files are committed.
12 PASS dnn-providers/hip-kernel-provider/rocke/library/tests/parity/gfx1151_wmma_fmha_fwd_v2_emit.py:225-227 and dnn-providers/hip-kernel-provider/rocke/library/tests/test_wmma_fmha_fwd_v2_spec.py:263 cover both gfx11-generic and gfx12-generic RDNA families (CDNA is scoped out by kernel family).
13 PASS dnn-providers/hip-kernel-provider/rocke/library/kernels/gfx1151/wmma_fmha_fwd_v2.py:548-560 and dnn-providers/hip-kernel-provider/rocke/platform/cpp/instances/gfx1151/wmma_fmha_fwd_v2.cpp:500-515 replace page_rsrc with page_head and update k_rsrc/v_rsrc paged usage consistently across Python, C++, headers, and docs.
14 PASS dnn-providers/hip-kernel-provider/rocke/platform/cpp/include/rocke/abi.h:101-105 and dnn-providers/hip-kernel-provider/rocke/library/kernels/gfx1151/wmma_fmha_decode.py:748-774 preserve upstream history and comments while surgical additions/refactorings are applied.
15 PASS dnn-providers/hip-kernel-provider/rocke/platform/python/rocke/helpers/rdna_attention.py:263-270 and 574-627, plus dnn-providers/hip-kernel-provider/rocke/platform/cpp/helpers/rdna_attention.cpp:713-815, use all newly added parameters (block_table_v, k_ptr, v_ptr, live_rows).
16 N/A This item governs reviewer severity labeling and does not apply when there are no FAIL rows in this section.
17 PASS dnn-providers/hip-kernel-provider/rocke/library/dispatch/attention/README.md:489-527 and the PR description contain no absolute or relative performance claims.
§F graded 11/11
# Verdict Evidence
1 PASS branch users/dpeabody/hipdnn-rdna-paged-attn uses the users// prefix
2 PASS title feat(hip-kernel-provider): paged/ragged SDPA in Gfx1151WmmaAttention matches type(scope): summary and is 68 characters
3 PASS description is at least 30 characters and contains an issue reference
4 PASS test file dnn-providers/hip-kernel-provider/rocke/library/tests/dispatch/attention/test_gfx1151_attention_dispatch_numeric.py is in the diff
5 PASS The changed file list contains only source, test, documentation, and descriptor/golden files (e.g., dnn-providers/hip-kernel-provider/rocke/library/dispatch/attention/README.md:492) and no keys, certs, or secrets.
6 NEEDS-INFO Local pre-commit execution status cannot be verified from the diff alone.
7 NEEDS-INFO Bot, Copilot, and human review thread states are not included in the prompt.
8 N/A This PR targets develop directly rather than being a stacked PR on a feature branch.
9 PASS The PR keeps the full implementation, catalog, and test suite together in a single change (e.g., dnn-providers/hip-kernel-provider/src/integration_tests/kernel_ingestor_engine/IntegrationGpuGfx1151WmmaAttentionPaged.cpp:1) without artificial splitting.
10 PASS Tests added in the diff exercise real spec validation, dispatch binding, parity, matcher, and GPU numeric/integration behavior (e.g., dnn-providers/hip-kernel-provider/rocke/library/tests/dispatch/attention/test_gfx1151_attention_dispatch_numeric.py:2130 and dnn-providers/hip-kernel-provider/src/tests/engines/kernel_ingestor_engine/packs/TestGfx1151WmmaAttentionMatchers.cpp:735).
11 NEEDS-INFO While the provided title and body are populated in the prompt metadata, there is no diff path:line to cite for API field verification.
§G graded 34/34
# Verdict Evidence
1 PASS Parity rows in dnn-providers/hip-kernel-provider/rocke/library/tests/parity/gfx1151_wmma_fmha_fwd_v2_emit.py:93 are paired with float64/CPU on-device numerical tests in dnn-providers/hip-kernel-provider/rocke/library/tests/test_wmma_fmha_fwd_v2_features.py:168 and dnn-providers/hip-kernel-provider/src/integration_tests/kernel_ingestor_engine/IntegrationGpuGfx1151WmmaAttentionPaged.cpp:324.
2 NEEDS-INFO On-device numerical tests with fp64/fp32 references and bf16/fp16 tolerances are added (e.g. dnn-providers/hip-kernel-provider/rocke/library/tests/test_wmma_fmha_fwd_v2_features.py:643,685 and dnn-providers/hip-kernel-provider/src/integration_tests/kernel_ingestor_engine/IntegrationGpuGfx1151WmmaAttentionPaged.cpp:867), but the PR body is truncated before the test execution summary so on-silicon pass confirmation is missing.
3 NEEDS-INFO Golden hashes are updated in dnn-providers/hip-kernel-provider/rocke/platform/tests/instances/differential/golden/llvm_gfx_all.json:1016, but the PR description is truncated in the Tests section so local gate sweep results (tools/run_checks.py, llvm23 byte-identity, check_ir_validity.py, pytest suites) cannot be verified from the diff alone.
4 PASS Every new capability has targeted tests that fail without the change: dense_q/seqused_q/split block tables/clamped seqused_k (dnn-providers/hip-kernel-provider/rocke/library/tests/test_wmma_fmha_fwd_v2_features.py:168-225), >4 GiB paged cache (dnn-providers/hip-kernel-provider/rocke/library/tests/test_wmma_fmha_fwd_v2_features.py:685), binding page-extent I32 checks…
5 PASS Tests exercise the shipped entry points (e.g. dispatch_attention in dnn-providers/hip-kernel-provider/rocke/library/tests/dispatch/attention/test_gfx1151_attention_selection.py:895, GraphMatchRegistry in dnn-providers/hip-kernel-provider/src/tests/engines/kernel_ingestor_engine/packs/Gfx1151WmmaAttentionTestSupport.hpp:149) and pin boundary gates directly with exact pass/fail thresholds…
6 PASS Prose claims in PR body, docstrings, and code comments match the implementation and tests: 64-bit page/head addressing and page-extent I32 bound (dnn-providers/hip-kernel-provider/rocke/library/dispatch/attention/bindings.py:683, dnn-providers/hip-kernel-provider/src/engines/kernel_ingestor_engine/packs/Gfx1151WmmaAttentionNative.cpp:428), seqused_q padding writing O=0 and LSE=-inf…
7 N/A This PR does not add or modify a design document defining a new fp32 parity reference oracle.
8 PASS Test harnesses do not silently fall back on unsupported dtypes or frontend configurations; they explicitly skip via pytest.importorskip or GTEST_SKIP with a reason (dnn-providers/hip-kernel-provider/rocke/library/tests/test_wmma_fmha_fwd_v2_features.py:690, dnn-providers/hip-kernel-provider/src/integration_tests/kernel_ingestor_engine/IntegrationGpuGfx1151WmmaAttentionPaged.cpp:749-767).
9 N/A This PR does not make new LDS or occupancy load-bearing numerical claims.
10 N/A This PR is a functional feature addition (paged/ragged SDPA support) and makes no performance or latency claims.
11 N/A This PR adds functional graph support for hipDNN paged SDPA rather than an optimization claim for a benchmarked shape.
12 PASS Cross-engine parity is maintained across Python and C++ for both the helper (dnn-providers/hip-kernel-provider/rocke/platform/python/rocke/helpers/rdna_attention.py:213 and dnn-providers/hip-kernel-provider/rocke/platform/cpp/helpers/rdna_attention.cpp:214) and instance builder (dnn-providers/hip-kernel-provider/rocke/library/kernels/gfx1151/wmma_fmha_fwd_v2.py:123 and…
13 N/A This PR does not introduce or recommend any new raw LLVM intrinsic mangled symbols.
14 N/A This PR makes no hardware bottleneck classifications.
15 N/A This is a review of the provided PR diff directly from the head commit, not a multi-worktree branch inspection task.
16 N/A This PR implements a completed feature rather than proposing an unverified plan.
17 N/A This PR does not add new standalone shipped CLI scripts.
18 PASS New host-side guard _check_page_extent_i32 in dnn-providers/hip-kernel-provider/rocke/library/dispatch/attention/bindings.py:683 is wired into _rdna_validate_and_collect (bindings.py:908), documents its scope (page and head-dim axes fit I32 while block and head steps are 64-bit), and is tested on both passing and failing boundary shapes in…
19 PASS Fixtures in dnn-providers/hip-kernel-provider/rocke/library/tests/test_wmma_fmha_fwd_v2_features.py:531 and dnn-providers/hip-kernel-provider/src/integration_tests/kernel_ingestor_engine/IntegrationGpuGfx1151WmmaAttentionPaged.cpp:163 allocate spare unreferenced pages (sum(pages) + 3 / blocks + 5) and extra table columns poisoned with out-of-range block IDs and NaN bytes so misplaced reads or writes are caught.
20 N/A This PR does not introduce or modify a claim-ban detector or word/digit regex linter.
21 N/A This PR does not add or modify performance measurement or benchmarking scripts.
22 NEEDS-INFO dnn-providers/hip-kernel-provider/rocke/platform/tests/instances/differential/golden/llvm_gfx_all.json:1016 is truncated in the diff, so we cannot verify from the diff alone that all new golden entries (indices 170..179 from gfx1151_wmma_fmha_fwd_v2_emit.py:29) record bytes + sha256 without build errors across all flavors, nor can we see CI execution logs confirming the C++ parity and GPU integration harnesses ran without skipping.
23 N/A This PR is a functional feature addition for paged/ragged SDPA and makes no performance speedup claims.
24 PASS The re-blessed paged entries in dnn-providers/hip-kernel-provider/rocke/platform/tests/instances/differential/golden/llvm_gfx_all.json:1016 correspond to the 64-bit page/head addressing, seqused_k clamping, and block_table_v changes in dnn-providers/hip-kernel-provider/rocke/library/kernels/gfx1151/wmma_fmha_fwd_v2.py:510, which are GPU-validated across the affected shapes in…
25 N/A This PR adds functional support for hipDNN paged/ragged SDPA graphs and 64-bit cache addressing rather than a codegen micro-optimization.
26 N/A This PR does not include benchmark comparisons or performance claims.
27 N/A This PR does not include performance A/B experiments.
28 N/A This PR does not present performance delta tables or counterfactual benchmark arms.
29 N/A This PR does not swap guarded math primitives for fast approximations.
30 N/A This PR makes no register-spill or scratch-reduction claims.
31 N/A This PR does not claim to fix a performance defect against a baseline measurement.
32 N/A This PR does not report measured performance or resource numbers.
33 NEEDS-INFO dnn-providers/hip-kernel-provider/src/engines/kernel_ingestor_engine/packs/Gfx1151WmmaAttentionNative.cpp:1339 and dnn-providers/hip-kernel-provider/src/tests/engines/kernel_ingestor_engine/packs/TestGfx1151WmmaAttentionPaged.cpp are truncated/omitted in the diff, so the changed routing decision logic between prefill and decode on dense vs paged graphs cannot be fully inspected or verified against a differential table.
34 PASS In dnn-providers/hip-kernel-provider/rocke/platform/python/rocke/helpers/rdna_attention.py:1060 (and C++ mirror rdna_attention.cpp:1690), empty and o_row_off for O are computed independently of lse_rsrc, and both O and LSE are tested on edge rows (0 live queries, 0 keys, padding rows past seqused_q, with and without sinks) in dnn-providers/hip-kernel-provider/rocke/library/tests/test_wmma_fmha_fwd_v2_features.py:646 and…
§H graded 5/5
# Verdict Evidence
1 PASS Buffer SRDs and 64-bit page bases are sized from runtime tensor shapes/strides and host-validated before launch via _check_page_extent_i32 / _check_byte_extent_i32 in dnn-providers/hip-kernel-provider/rocke/library/dispatch/attention/bindings.py:683-705 and pageFitsI32 / isWellFormedRaggedOperand in dnn-providers/hip-kernel-provider/src/engines/kernel_ingestor_engine/packs/Gfx1151WmmaAttentionNative.cpp:428-492.
2 PASS Both block_table and block_table_v indirection values (phys_k, phys_v, phys) are explicitly range-checked on device by page_ok (0 <= phys < num_blocks) before 64-bit base pointer offsets are formed, clamping out-of-range page IDs to 0 and zeroing the loaded data in dnn-providers/hip-kernel-provider/rocke/platform/python/rocke/helpers/rdna_attention.py:554-616 and…
3 PASS Power-of-two page_block_size is enforced in the spec (ps <= 0 or ps & (ps - 1) in dnn-providers/hip-kernel-provider/rocke/library/kernels/gfx1151/wmma_fmha_fwd_v2.py:164-165) and in the native matcher ((page & (page - 1)) != 0 in dnn-providers/hip-kernel-provider/src/engines/kernel_ingestor_engine/packs/Gfx1151WmmaAttentionNative.cpp:1151-1156), lowering hot-loop page div/mod to lshr and land in…
4 PASS The kernel loops over runtime n_tiles = cdiv(smax(sk, 0), BN) where sk = smin(seqused_k[b], seqlen_k) is clamped in dnn-providers/hip-kernel-provider/rocke/library/kernels/gfx1151/wmma_fmha_fwd_v2.py:521 (and C++ twin at dnn-providers/hip-kernel-provider/rocke/platform/cpp/instances/gfx1151/wmma_fmha_fwd_v2.cpp:439-440), while live_rows = smin(seqused_q[b], span) masks padding query rows to skip the K loop and write O=0, LSE=-inf in…
5 PASS Host validations for dense_q, seqused_q, paged K/V extents, and page_table_k/page_table_v/seq_len_q/seq_len_kv are in dnn-providers/hip-kernel-provider/rocke/library/dispatch/attention/bindings.py:897-915, 1047-1114 and dnn-providers/hip-kernel-provider/src/engines/kernel_ingestor_engine/packs/Gfx1151WmmaAttentionNative.cpp:1085-1305, paired with negative unit tests in…
§I graded 6/6
# Verdict Evidence
1 N/A This PR adds paged/ragged SDPA kernel features and hipDNN native graph matching rather than introducing a tuning or routing search benchmark.
2 PASS Invalid spec combinations and unsupported graph features are loudly rejected with descriptive errors or logged causes (e.g., dnn-providers/hip-kernel-provider/rocke/library/kernels/gfx1151/wmma_fmha_fwd_v2.py:168, dnn-providers/hip-kernel-provider/src/engines/kernel_ingestor_engine/packs/Gfx1151WmmaAttentionNative.cpp:836).
3 PASS Parity and feature tests cover curated combinations of the new dense_q, seqused_q, and split block-table axes (dnn-providers/hip-kernel-provider/rocke/library/tests/parity/gfx1151_wmma_fmha_fwd_v2_emit.py:93, dnn-providers/hip-kernel-provider/rocke/library/tests/test_wmma_fmha_fwd_v2_features.py:168).
4 N/A This PR does not add or modify benchmark reporting tables comparing shipped vs. achievable selection performance.
5 PASS Generated catalog descriptors (kdp, kmd, ued) are updated from their YAML source config (dnn-providers/hip-kernel-provider/src/engines/kernel_ingestor_engine/descriptors/rocKE/gfx1151_wmma_attention/gfx1151_wmma_attention.kmd.json:42), and off-by-default spec fields preserve existing tuning cache hashes (dnn-providers/hip-kernel-provider/rocke/library/dispatch/attention/common.py:996).
6 PASS The new helper rdna_attention_scaled_ptr exposes a general 64-bit scaled pointer arithmetic primitive rather than a hardware-instruction-specific op (dnn-providers/hip-kernel-provider/rocke/platform/python/rocke/helpers/rdna_attention.py:213, dnn-providers/hip-kernel-provider/rocke/platform/cpp/helpers/rdna_attention.cpp:214).
§J graded 7/7
# Verdict Evidence
1 N/A The Markdown changes in README.md and attention.md update dispatch contract prose and instance bullet lists without introducing mathematical equations or matrix-multiplication orientation tables.
2 PASS Symbols introduced in dnn-providers/hip-kernel-provider/rocke/library/dispatch/attention/README.md:492-521 and dnn-providers/hip-kernel-provider/rocke/platform/dsl_docs/instances/attention.md:38-41 (block_table_v, dense_q, seqused_q, use_sinks/sinks) are used with a single consistent meaning and match the code and spec definitions.
3 PASS dnn-providers/hip-kernel-provider/rocke/platform/dsl_docs/instances/attention.md:38-41 and dnn-providers/hip-kernel-provider/rocke/library/dispatch/attention/README.md:492-526 state only the interface and catalog delta and reference platform/python/rocke/helpers/rdna_attention.py rather than restating the algorithm.
4 PASS dnn-providers/hip-kernel-provider/rocke/library/dispatch/attention/README.md:492-526 and dnn-providers/hip-kernel-provider/rocke/platform/dsl_docs/instances/attention.md:38-40 accurately describe the newly added block_table_v, dense_q, seqused_q, and paged catalog sinks options introduced in this PR.
5 N/A The PR makes no algorithmic scaling or asymptotic complexity claims in documentation or comments.
6 PASS Catalog constants in dnn-providers/hip-kernel-provider/rocke/library/dispatch/attention/README.md:518-521 (buckets 64/96/128/256, page sizes 16/32/64) match SHIPPED_PAGE_SIZES in dnn-providers/hip-kernel-provider/src/engines/kernel_ingestor_engine/packs/Gfx1151WmmaAttentionNative.cpp:147 and the catalog descriptor.
7 N/A No new Markdown design or algorithm document files are added in this PR; only existing README.md and attention.md files are updated.
§K graded 13/13
# Verdict Evidence
1 N/A No tuned candidate definitions or make_tuned_candidate calls are added or modified in this PR.
2 PASS New WmmaFmhaFwdV2Spec fields dense_q and seqused_q (dnn-providers/hip-kernel-provider/rocke/library/kernels/gfx1151/wmma_fmha_fwd_v2.py:123) are explicitly classified in WMMA_PROBLEM_FIELDS (dnn-providers/hip-kernel-provider/rocke/library/dispatch/attention/axes.py:346).
3 PASS common.py omits dense_q and seqused_q when False so existing pins keep their spec_hash without a migration (dnn-providers/hip-kernel-provider/rocke/library/dispatch/attention/common.py:997), and test_gfx1151_attention_selection.py verifies both backward-compatible spec_hash preservation and distinct hashes when enabled…
4 N/A KnobSpace.defaults() and recorded() are not touched; the new spec fields are problem-level layout options in WMMA_PROBLEM_FIELDS.
5 N/A No knob drop, inertness, or refusal rules are modified in this PR.
6 N/A KnobSpace.validate and tuning_specs.py are not modified in this PR.
7 N/A KnobSpace.base(req) is not modified in this PR.
8 N/A Knob value normalization and type coercion are not modified in this PR.
9 PASS Replay verification via result.kernel_id.spec_hash is preserved for stored prefill pins (dnn-providers/hip-kernel-provider/rocke/library/tests/dispatch/attention/test_gfx1151_attention_selection.py:896).
10 N/A KnobSpace.find and bare tuning_id resolution are not modified in this PR.
11 N/A Sweep entry points and iteration helpers are not modified in this PR.
12 N/A Production knob stacks, accept_default, and auto candidate routing are not modified in this PR.
13 N/A No benchmark scripts or sweep harnesses are modified in this PR.
§L graded 10/10
# Verdict Evidence
1 N/A The diff does not add or modify any gfx architecture target or ISA backend registry.
2 N/A The diff does not add or modify any MMA or WMMA op_id or fragment layout map.
3 N/A The diff does not change MMA atom selection, fragment lane maps, or accumulator layouts.
4 N/A The diff does not add or change wave size handling or target wave-size queries.
5 N/A The diff does not touch CDNA/gfx1250 arch-specific recipes, transpose-LDS reads, or AsyncTileLoader.
6 N/A The diff does not add pipeline vocabulary to ArchTarget/core/arch or import CK Tile dispatcher modules.
7 N/A The diff does not add or change logical dtypes, bit packing routines, or FP8 dialect rules.
8 N/A The diff does not touch scaled-MMA contracts or scaled WMMA/MFMA emission.
9 N/A The diff does not introduce a new architecture target.
10 N/A The diff does not modify intrinsic arch-domain artifacts or LLVM flavor membership lists.
§M graded 10/10
# Verdict Evidence
1 PASS Host booleans (spec.dense_q, spec.seqused_q, live_rows is not None) guard Python branches, while SSA predicates use b.cmp_* with scf_if and select (dnn-providers/hip-kernel-provider/rocke/library/kernels/gfx1151/wmma_fmha_fwd_v2.py:505, dnn-providers/hip-kernel-provider/rocke/platform/python/rocke/helpers/rdna_attention.py:571).
2 N/A The diff does not touch legalize_for_target or any target legalization pass.
3 PASS No wire format changes are made to ck.dsl.ir/v1, and kernel parameter order is identical between Python and C++ builders and verified by spec tests (dnn-providers/hip-kernel-provider/rocke/library/kernels/gfx1151/wmma_fmha_fwd_v2.py:416, dnn-providers/hip-kernel-provider/rocke/platform/cpp/instances/gfx1151/wmma_fmha_fwd_v2.cpp:291, dnn-providers/hip-kernel-provider/rocke/library/tests/test_wmma_fmha_fwd_v2_spec.py:224).
4 N/A The diff only bumps BINARY_ABI in portable_ir/src/abi.py and does not touch portable IR schemas, importers, magic division, or recorder coverage.
5 N/A The diff does not use or modify AsyncTileLoader or async DRAM-to-LDS operations.
6 PASS The early return is CTA-uniform (q_start vs span), tile-paged loads use OOB buffer sentinels (at_tile), and 64-bit row-paged loads clamp invalid indices/offsets (blk, page, within) to 0 before issuing global_load and select (dnn-providers/hip-kernel-provider/rocke/library/kernels/gfx1151/wmma_fmha_fwd_v2.py:512, dnn-providers/hip-kernel-provider/rocke/platform/python/rocke/helpers/rdna_attention.py:607…
7 PASS Buffer descriptors are constructed via b.buffer_rsrc / rocke_b_buffer_rsrc without hard-coding buffer-resource word 3 or using TemporalHint.STREAMING (dnn-providers/hip-kernel-provider/rocke/platform/python/rocke/helpers/rdna_attention.py:578, dnn-providers/hip-kernel-provider/rocke/platform/cpp/helpers/rdna_attention.cpp:732).
8 N/A The diff does not add any new LLVM intrinsic declarations or modify backend selection logic.
9 PASS Public C headers under cpp/include/rocke/ keep their names and extern "C" declarations, and ROCKE_ABI_VERSION / BINARY_ABI are bumped to 10 for the struct layout changes (dnn-providers/hip-kernel-provider/rocke/platform/cpp/include/rocke/abi.h:105, dnn-providers/hip-kernel-provider/rocke/platform/python/rocke/portable_ir/src/abi.py:78…
10 N/A The diff does not touch !dbg metadata emission, Debug Info Version flags, or DWARF consumers.
§N graded 18/18
# Verdict Evidence
1 PASS rdna_attention_scaled_ptr is added to dnn-providers/hip-kernel-provider/rocke/platform/python/rocke/helpers/rdna_attention.py:213 because both wmma_fmha_fwd_v2.py:462 and wmma_fmha_decode.py:607 share it, and it lands with its C++ twin rocke_rdna_attention_scaled_ptr in dnn-providers/hip-kernel-provider/rocke/platform/cpp/helpers/rdna_attention.cpp:214 and…
2 NEEDS-INFO Both dnn-providers/hip-kernel-provider/rocke/library/kernels/gfx1151/wmma_fmha_decode.py:748 and dnn-providers/hip-kernel-provider/rocke/library/kernels/gfx1151/wmma_fmha_fwd_v2.py:552 (along with their C++ mirrors in wmma_fmha_decode.cpp:1097 and wmma_fmha_fwd_v2.cpp:503) call the shared rdna_attention_scaled_ptr / rocke_rdna_attention_scaled_ptr helper rather than duplicating 64-bit pointer arithmetic. — no diff line supports this verdict
3 N/A The diff does not touch or use SoftwarePipeline.
4 N/A The diff does not add or modify epilogue fusion or fused GEMM builders.
5 N/A The diff does not touch rocke.helpers.tiling or add any tiling API.
6 N/A The diff does not add or modify front-door/toolbox tiling helpers or LayoutStyle.
7 N/A The diff does not use helpers/tiling/ layout overrides or TileMma.
8 N/A The diff does not touch helpers/tiling/ LDS allocation or tiling memory verbs.
9 N/A The diff does not use helpers/tiling/ N-D tensor axis roles or FlowStage.
10 N/A The diff does not modify helpers/tiling/ emission, LDS staging, or MMA issue.
11 N/A The diff does not touch helpers/tiling/ traits, lds_swizzle, or derive_sched_group_counts.
12 N/A The PR makes no LDS bank-conflict or coalescing claims under helpers/tiling/.
13 N/A The diff does not touch WarpDistributionEncoding, make_tile_desc, or RegisterMapper.
14 N/A The diff does not use helpers/tiling/ verbs or add a tiling pipeline variant.
15 N/A The diff does not modify helpers/tiling/ descriptor allocation or swizzled bounds checks.
16 N/A The diff does not touch optimize_layout or evaluate_transform.
17 N/A The diff does not touch tiling_recorder.py, round-trip analysis, or tiling visualization.
18 N/A The diff does not touch InterleavedStyle or demo kernels under helpers/tiling/kernels/.
§O graded 16/16
# Verdict Evidence
1 N/A This PR does not modify core/verify.py, cpp/core/verify.cpp, address spaces, or required IR attributes.
2 PASS Early-exit scf.if guards properly terminate the then-region without emitting after the terminator in dnn-providers/hip-kernel-provider/rocke/library/kernels/gfx1151/wmma_fmha_fwd_v2.py:512 and dnn-providers/hip-kernel-provider/rocke/platform/cpp/instances/gfx1151/wmma_fmha_fwd_v2.cpp:424.
3 PASS IR emission in dnn-providers/hip-kernel-provider/rocke/platform/python/rocke/helpers/rdna_attention.py:213 and dnn-providers/hip-kernel-provider/rocke/platform/cpp/helpers/rdna_attention.cpp:214 uses deterministic builder methods and ordered sequences without set iteration or id().
4 PASS Module singletons (I32, I64) and rocke_i32() / rocke_i64() are used in dnn-providers/hip-kernel-provider/rocke/platform/python/rocke/helpers/rdna_attention.py:55 and dnn-providers/hip-kernel-provider/rocke/platform/cpp/helpers/rdna_attention.cpp:220.
5 N/A This PR does not touch core/passes.py, cpp/core/passes.cpp, or op purity tables.
6 N/A This PR does not modify constant folding in either engine.
7 N/A This PR does not alter kernel attributes, CodegenPolicy attribute contracts, or cluster_dims.
8 NEEDS-INFO CodegenPolicy and apply_codegen_policy continue to be used in dnn-providers/hip-kernel-provider/rocke/library/kernels/gfx1151/wmma_fmha_decode.py:606 and dnn-providers/hip-kernel-provider/rocke/library/kernels/gfx1151/wmma_fmha_fwd_v2.py:459. — no diff line supports this verdict
9 N/A This PR does not touch waitcnt encoding, barrier lowering, or optimization_barrier.
10 N/A This PR does not modify lowerer attribute validation or _param_llvm_type.
11 N/A This PR does not change ArchTarget queries or target ID resolution in rocke.
12 N/A This PR does not modify target catalog width/size caps or add default arch or "gfx950" fallbacks.
13 PASS Growing rocke_rdna_attn_params_t (dnn-providers/hip-kernel-provider/rocke/platform/cpp/include/rocke/helper_rocke.helpers.rdna_attention.h:136) and rocke_wmma_fmha_fwd_v2_spec_t (dnn-providers/hip-kernel-provider/rocke/platform/cpp/include/rocke/instance_gfx1151_wmma_fmha_fwd_v2.h:63) bumps ROCKE_ABI_VERSION to 10 in dnn-providers/hip-kernel-provider/rocke/platform/cpp/include/rocke/abi.h:105 and BINARY_ABI to 10 in…
14 PASS C headers and sources follow rocke_* snake_case and correspondence prefaces in dnn-providers/hip-kernel-provider/rocke/platform/cpp/include/rocke/helper_rocke.helpers.rdna_attention.h:14, and additions to dnn-providers/hip-kernel-provider/rocke/platform/cpp/bindings/rocke_engine.cpp:4650 are minimal (2 lines).
15 N/A This PR does not touch engine fallback mechanisms, _resolve_llvm_flavor, lower_hip.py, or lower_cktile.py.
16 PASS Full Python and C++ instance and helper parity is maintained across dnn-providers/hip-kernel-provider/rocke/platform/cpp/instances/gfx1151/wmma_fmha_fwd_v2.cpp:144 and dnn-providers/hip-kernel-provider/rocke/platform/cpp/bindings/prove_parity_binding.py:813 without widening known_gaps.md.
§P graded 14/14
# Verdict Evidence
1 PASS Builders keep the exact (spec, *, arch) signature (dnn-providers/hip-kernel-provider/rocke/library/kernels/gfx1151/wmma_fmha_fwd_v2.py:457, wmma_fmha_decode.py:604), and the new dense_q and seqused_q knobs are defaulted fields on WmmaFmhaFwdV2Spec (wmma_fmha_fwd_v2.py:123-124) and its C++ twin (dnn-providers/hip-kernel-provider/rocke/platform/cpp/include/rocke/instance_gfx1151_wmma_fmha_fwd_v2.h:63-64).
2 PASS The kernarg signature order for seqused_q and block_table_v is updated identically in Python (dnn-providers/hip-kernel-provider/rocke/library/kernels/gfx1151/wmma_fmha_fwd_v2.py:416-425) and C++ (dnn-providers/hip-kernel-provider/rocke/platform/cpp/instances/gfx1151/wmma_fmha_fwd_v2.cpp:291-301), tested in dnn-providers/hip-kernel-provider/rocke/library/tests/test_wmma_fmha_fwd_v2_spec.py:205-243, and the ABI version is bumped to 10…
3 PASS Test harnesses obtain launch geometry directly from wmma_fmha_fwd_v2_grid without recomputing it (dnn-providers/hip-kernel-provider/rocke/library/tests/test_wmma_fmha_fwd_v2_features.py:613-615, 826-828).
4 N/A This PR does not modify LDS allocation math or LDS capacity gates.
5 N/A This PR modifies RDNA attention kernels (wmma_fmha_fwd_v2 and wmma_fmha_decode), not WMMA/RDNA GEMM or GemmMultiD/GemmMultiAbd.
6 N/A The paged prefill attention kernel requires zero workspace (dnn-providers/hip-kernel-provider/src/integration_tests/kernel_ingestor_engine/IntegrationGpuGfx1151WmmaAttentionPaged.cpp:855), and Stream-K, split-K, and…
7 N/A This PR does not touch Stream-K, persistent GEMM, GroupedGemmLauncher, or SUPPORT_MATRIX.md.
8 N/A This PR does not add or modify quantization helpers or MX GEMM scale decoding.
9 N/A This PR does not introduce or change numeric emulation or data-type decode paths.
10 N/A This PR does not add or modify example manifests (helpers/manifest.py or run_manifest).
11 PASS GPU tests properly synchronize the stream via rt.sync() before reading device outputs and before freeing buffers (dnn-providers/hip-kernel-provider/rocke/library/tests/test_wmma_fmha_fwd_v2_features.py:835-844).
12 N/A This PR does not touch COMGR/HIP library loading or fix a hipError launch failure.
13 PASS Host-side changes to the attention spec cache key (dnn-providers/hip-kernel-provider/rocke/library/dispatch/attention/common.py:996-999) and paged K/V I32 extent guards (dnn-providers/hip-kernel-provider/rocke/library/dispatch/attention/bindings.py:683-704, 897-911) are covered by unit and GPU numeric tests (dnn-providers/hip-kernel-provider/rocke/library/tests/dispatch/attention/test_gfx1151_attention_selection.py:890-924…
14 PASS No examples/ expected.json files are touched, and the shared 64-bit pointer offset helper rdna_attention_scaled_ptr is placed in dnn-providers/hip-kernel-provider/rocke/platform/python/rocke/helpers/rdna_attention.py:213-220 (with its C++ twin in dnn-providers/hip-kernel-provider/rocke/platform/cpp/helpers/rdna_attention.cpp:214-226) where both wmma_fmha_fwd_v2 and wmma_fmha_decode share it.
§Q graded 7/7
# Verdict Evidence
1 N/A No Python test files were added or removed under rocke/platform/tests/ or rocke/library/tests/, so the installed rocke_pytest CMake lists and ROCKE_ENGINE_test_categories_external.yaml did not change.
2 PASS New dense_q and seqused_q variants across gfx11-generic and gfx12-generic are added to the C and Python parity emitters in dnn-providers/hip-kernel-provider/rocke/library/tests/parity/gfx1151_wmma_fmha_fwd_v2_emit.c:101, dnn-providers/hip-kernel-provider/rocke/library/tests/parity/gfx1151_wmma_fmha_fwd_v2_emit.py:93, and dnn-providers/hip-kernel-provider/rocke/platform/cpp/bindings/prove_parity_binding.py:813.
3 PASS Cross-engine IR/lowering parity is covered in dnn-providers/hip-kernel-provider/rocke/platform/cpp/bindings/prove_parity_binding.py:813 and dnn-providers/hip-kernel-provider/rocke/platform/tests/instances/differential/golden/llvm_gfx_all.json:1016, while numeric correctness is verified separately against float64 references in dnn-providers/hip-kernel-provider/rocke/library/tests/test_wmma_fmha_fwd_v2_features.py:168 and…
4 NEEDS-INFO The diff for dnn-providers/hip-kernel-provider/rocke/platform/tests/instances/differential/golden/llvm_gfx_all.json is truncated after line 1040, so the remaining LLVM_FLAVORS sections (llvm20, llvm22, llvm23, llvm24, and canonical IR) and entries 170..179 cannot be verified from the provided diff.
5 PASS GPU tests check the active device and skip cleanly when unavailable or not RDNA (dnn-providers/hip-kernel-provider/rocke/library/tests/test_wmma_fmha_fwd_v2_features.py:682, dnn-providers/hip-kernel-provider/src/integration_tests/kernel_ingestor_engine/IntegrationGpuGfx1151WmmaAttentionPaged.cpp:799), and CPU validation tests use fake tensors requiring neither torch nor a GPU…
6 N/A This PR does not refactor any CI gate, differential runner, or parity harness execution logic.
7 N/A This PR does not change backend resolution or ROCKE_BACKEND / ROCKE_CPP_STRICT fallback rejection contracts.
§R graded 11/11
# Verdict Evidence
1 N/A This PR adds functional support for cuDNN paged/ragged SDPA semantics (dense_q, seqused_q, block_table_v, 64-bit page base addressing) rather than introducing a new kernel body, tiling strategy, pipeline variant, or…
2 N/A The diff changes no performance levers or arch-specific optimization recipes; dense_q and seqused_q are problem-fixed fields (WMMA_PROBLEM_FIELDS).
3 N/A No performance levers, tile sizes, LDS layouts, or prefetch pipelines are modified in this functional feature PR.
4 N/A This PR makes no performance or primitive-selection claims and changes no performance levers.
5 N/A The kernel does not use rocke.helpers.tiling and its per-lane 16-byte/8-byte vector load widths and striding patterns are unchanged (only the base pointer arithmetic is widened to 64-bit for >2 GiB caches and scalar…
6 N/A Neither LDS layouts nor MMA layouts are modified in this diff.
7 N/A The PR makes no bottleneck claims and does not target a performance bottleneck.
8 N/A No waits, barriers, or scheduling hints are added, removed, or modified in this diff.
9 N/A This PR is not a performance regression fix and does not trade performance across shapes via a tuning lever.
10 N/A The PR does not change compiler/codegen policies, claim assembly parity, or report benchmark measurements.
11 N/A No new optimization lessons, heuristics, case studies, or dsl_probes tools are introduced in this PR.
§S graded 8/8
# Verdict Evidence
1 PASS New public symbols and spec fields (rdna_attention_scaled_ptr in dnn-providers/hip-kernel-provider/rocke/platform/python/rocke/helpers/rdna_attention.py:61, rocke_rdna_attention_scaled_ptr in dnn-providers/hip-kernel-provider/rocke/platform/cpp/include/rocke/helper_rocke.helpers.rdna_attention.h:86, and WmmaFmhaFwdV2Spec.dense_q / seqused_q in dnn-providers/hip-kernel-provider/rocke/library/kernels/gfx1151/wmma_fmha_fwd_v2.py:123)…
2 PASS Existing callers and tests for wmma_fmha_fwd_v2_signature, rdna_attention_fwd_body, and bind_rdna_prefill_torch are updated in the same PR with new expectations (e.g., block_table_v added in dnn-providers/hip-kernel-provider/rocke/library/tests/test_wmma_fmha_fwd_v2_spec.py:205 and dnn-providers/hip-kernel-provider/rocke/library/tests/test_wmma_fmha_paged_guard.py:194, and >2 GiB prefill cache behavior updated in…
3 N/A The diff does not remove or rename any public API symbol.
4 PASS rdna_attention_scaled_ptr is added to __all__ in dnn-providers/hip-kernel-provider/rocke/platform/python/rocke/helpers/rdna_attention.py:65, resolves to the function defined at line 213, and is imported and exercised by dnn-providers/hip-kernel-provider/rocke/library/kernels/gfx1151/wmma_fmha_decode.py:607 and dnn-providers/hip-kernel-provider/rocke/library/kernels/gfx1151/wmma_fmha_fwd_v2.py:462.
5 PASS Tests assert both positive outputs (IR serialization, signature lists, cache hashes, and float64 reference numerics) and exact negative ValueError exceptions on invalid parameters in dnn-providers/hip-kernel-provider/rocke/library/tests/test_wmma_fmha_fwd_v2_spec.py:40, dnn-providers/hip-kernel-provider/rocke/library/tests/dispatch/attention/test_gfx1151_attention_dispatch_numeric.py:2866, and…
6 PASS CPU-checkable contracts (spec validation, kernel names, signature ordering, dispatch argument collection, I32 extent guards, and cache key hashing) are tested without a GPU or torch in dnn-providers/hip-kernel-provider/rocke/library/tests/test_wmma_fmha_fwd_v2_spec.py:37, dnn-providers/hip-kernel-provider/rocke/library/tests/dispatch/attention/test_gfx1151_attention_dispatch_numeric.py:2846…
7 PASS The C/C++ twins (rocke_wmma_fmha_fwd_v2_spec_t, rocke_rdna_attn_params_t, rocke_rdna_attention_scaled_ptr, and the pybind11 bridge in dnn-providers/hip-kernel-provider/rocke/platform/cpp/bindings/rocke_engine.cpp:4650) are wired into the cross-engine parity harness in dnn-providers/hip-kernel-provider/rocke/platform/cpp/bindings/prove_parity_binding.py:813 and…
8 NEEDS-INFO The PR body in the prompt is truncated mid-sentence under the - **Tests**: section, so it is not possible to verify whether the PR description states that tests/test_rocke.py and tests/run_all.py were run and names the GPU architecture tested.

Copilot AI left a comment

Copy link
Copy Markdown
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

🟡 Changes recommended

Unresolved critical findings permit unsafe accesses through invalid pages, empty caches, and unchecked sequence lengths, while the changed kernel argument layout lacks a dispatch ABI bump.

7 open issues
What changed in this PR

Extends hipkernel:Gfx1151WmmaAttention to serve hipDNN’s paged K/V SDPA contract using rocKE’s RDNA prefill kernel, with dense or ragged queries. Paged graphs remain exclusive to the prefill engine.

Changes:

  • Add query-length handling, separate K/V page tables, padded-row outputs, and 64-bit cache addressing in both kernel implementations.
  • Expand prefill catalog metadata and variants, and update engine SDK requirements and the rocKE binary ABI.
  • Extend matcher, binding, numerical, and parity coverage, with test registration and documentation updates.
File Description
projects/​hipdnn/​tools/​IngestorGenerator/​tests/​fixtures/​unslotted_render_golden.json Refresh the generated catalog snapshot hash.
projects/​hipdnn/​tools/​IngestorGenerator/​configs/​gfx1151_wmma_attention.yaml Add paged query variants, sinks, and metadata.
dnn-providers/​hip-kernel-provider/​src/​tests/​engines/​kernel_ingestor_engine/​packs/​TestGfx1151WmmaAttentionMatchers.cpp Extend dense matcher checks and reuse shared fixtures.
dnn-providers/​hip-kernel-provider/​src/​tests/​engines/​kernel_ingestor_engine/​packs/​Gfx1151WmmaAttentionTestSupport.hpp Share fixtures, metadata, logs, and ABI helpers.
dnn-providers/​hip-kernel-provider/​src/​tests/​engines/​kernel_ingestor_engine/​CMakeLists.txt Register paged matcher unit tests.
dnn-providers/​hip-kernel-provider/​src/​integration_tests/​CMakeLists.txt Register supported paged GPU integration tests.
dnn-providers/​hip-kernel-provider/​src/​engines/​kernel_ingestor_engine/​descriptors/​rocKE/​gfx1151_wmma_attention/​gfx1151_wmma_attention.ued.json Raise the prefill engine SDK requirement.
dnn-providers/​hip-kernel-provider/​src/​engines/​kernel_ingestor_engine/​descriptors/​rocKE/​gfx1151_wmma_attention/​gfx1151_wmma_attention.kmd.json Define layout, page, query, and sink metadata.
dnn-providers/​hip-kernel-provider/​rocke/​platform/​python/​rocke/​portable_ir/​src/​abi.py Update Python’s binary ABI version to 10.
dnn-providers/​hip-kernel-provider/​rocke/​platform/​python/​rocke/​helpers/​rdna_attention.py Implement 64-bit paging and live-query-row handling.
dnn-providers/​hip-kernel-provider/​rocke/​platform/​dsl_docs/​instances/​attention.md Document the new paged query options.
dnn-providers/​hip-kernel-provider/​rocke/​platform/​cpp/​instances/​gfx1151/​wmma_fmha_fwd_v2.cpp Mirror prefill query forms and paged addressing.
dnn-providers/​hip-kernel-provider/​rocke/​platform/​cpp/​instances/​gfx1151/​wmma_fmha_decode.cpp Reuse the shared pointer helper without changing operations.
dnn-providers/​hip-kernel-provider/​rocke/​platform/​cpp/​include/​rocke/​instance_gfx1151_wmma_fmha_fwd_v2.h Add query-form flags to the C spec.
dnn-providers/​hip-kernel-provider/​rocke/​platform/​cpp/​include/​rocke/​helper_rocke.helpers.rdna_attention.h Declare the pointer helper and new operands.
dnn-providers/​hip-kernel-provider/​rocke/​platform/​cpp/​include/​rocke/​abi.h Version the changed public C struct layouts.
dnn-providers/​hip-kernel-provider/​rocke/​platform/​cpp/​helpers/​rdna_attention.cpp Mirror paged loads and padded-row handling.
dnn-providers/​hip-kernel-provider/​rocke/​platform/​cpp/​bindings/​rocke_engine.cpp Expose query-form flags through Python bindings.
dnn-providers/​hip-kernel-provider/​rocke/​platform/​cpp/​bindings/​prove_parity_binding.py Add query-form binding parity configurations.
dnn-providers/​hip-kernel-provider/​rocke/​library/​tests/​test_wmma_fmha_paged_guard.py Pass the V page table in guard tests.
dnn-providers/​hip-kernel-provider/​rocke/​library/​tests/​test_wmma_fmha_fwd_v2_spec.py Test flag validation, names, signatures, and builds.
dnn-providers/​hip-kernel-provider/​rocke/​library/​tests/​test_wmma_fmha_fwd_v2_features.py Cover query padding, split tables, and large caches.
dnn-providers/​hip-kernel-provider/​rocke/​library/​tests/​parity/​gfx1151_wmma_fmha_fwd_v2_emit.py Add Python query-form parity cases.
dnn-providers/​hip-kernel-provider/​rocke/​library/​tests/​parity/​gfx1151_wmma_fmha_fwd_v2_emit.c Add matching C query-form parity cases.
dnn-providers/​hip-kernel-provider/​rocke/​library/​tests/​dispatch/​attention/​test_gfx1151_binding_i32_guard.py Test per-page bounds and large-cache acceptance.
dnn-providers/​hip-kernel-provider/​rocke/​library/​tests/​dispatch/​attention/​test_gfx1151_attention_selection.py Check cache identity for optional query flags.
dnn-providers/​hip-kernel-provider/​rocke/​library/​tests/​dispatch/​attention/​test_gfx1151_attention_dispatch_numeric.py Test dense paged query binding and padded outputs.
dnn-providers/​hip-kernel-provider/​rocke/​library/​kernels/​gfx1151/​wmma_fmha_fwd_v2.py Add query flags, catalog accessors, and kernel arguments.
dnn-providers/​hip-kernel-provider/​rocke/​library/​kernels/​gfx1151/​wmma_fmha_decode.py Reuse shared pointer arithmetic and clarify catalog comments.
dnn-providers/​hip-kernel-provider/​rocke/​library/​dispatch/​attention/​README.md Document paged binding, addressing, and engine routing.
dnn-providers/​hip-kernel-provider/​rocke/​library/​dispatch/​attention/​common.py Preserve cache keys while new flags are off.
dnn-providers/​hip-kernel-provider/​rocke/​library/​dispatch/​attention/​bindings.py Collect query metadata and validate paged cache extents.
dnn-providers/​hip-kernel-provider/​rocke/​library/​dispatch/​attention/​axes.py Classify query-form flags as non-tuning fields.

Review effort: Max

dpeabody-amd added a commit that referenced this pull request Nov 7, 2026
…SdpaGraphMatch.hpp

hipkernel:AttentionPagedUnified's graph_match carried its own copies of the
SDPA match helpers that packs/SdpaGraphMatch.hpp now shares (findTensor,
sdpaNode, hasSdpaForwardNode, describeOperand, the decline head and the axis
constants). It now uses the header's, with sdpa_graph::DeclinedBy. The
overflow-checked product it shares with hipkernel:Gfx942AttentionUnified3d
moves into the header as sdpa_graph::checkedProduct: unlike
pack_features::checkedProduct it takes zero factors, which every extent of 1
gives. FlydslSdpaShared.hpp, which
already takes computesInFloat from the header, now takes checkedProduct and
declaresAlignment from it too instead of keeping identical copies.

The stricter operand checks it needs move into the header too
(usesOverrideShapes, hasNonNegativeStrides, declaresAlignment, isInt32Vector),
together with the cuDNN paged-K/V graph contract checks (SEM-009), so every
pack serving a paged graph applies one set of rules with one cause vocabulary,
logged with the calling engine's name:
- pagesKAndV: one_sided_paging;
- pagedContainersAgree: block_size, shape (one block size, one block count;
  head counts and head sizes stay the engine's to check);
- pageTablesFor: page_table (INT32 [B, 1, M, 1], unit entry stride, not
  packed; separate K and V tables share M and the batch stride);
- sequenceLengthsFor: seq_len_kv, seq_len_q (both required, INT32 [B,1,1,1]);
- raggedQueryFor, countsOffsetsInTokens: ragged (Q and O through one
  [B+1,1,1,1] offset table counted in tokens; K/V never ragged);
- declaresServedAlignment: alignment (vector operands at the engine's vector
  width, INT32 operands at 4 bytes);
- sinkFor: sinks, and alignment below the engine's own minimum sink
  alignment (16 bytes for this engine's kernels);
- takesNoAttentionMask: attn_mask.

The reasons they log are the texts hipkernel:Gfx1151WmmaAttention's paged
matcher (#14254) logs for the same rules, so that engine can call these checks
without changing what it reports. The decline lines go through
HIPDNN_PLUGIN_LOG_INFO((DeclinedBy{...}) << ...): the braces' comma would
otherwise split the macro argument.

The engine keeps its kernel limits (head and block sizes, GQA ratios, strides
and address ranges) and the order of its declines; it checks itself that V has
K's head count and head size. A malformed page-table tensor now logs page_table
rather than operand. The header's file comment covers paged graphs.
… wmma_fmha_fwd_v2

wmma_fmha_fwd_v2 gains two query-side options for the packed layouts:
dense_q (paged only) keeps Q / O / LSE batch-strided like the dense layout,
so a uniform decode can read a [B, Sq, Hq, D] query against a paged cache
without cu_seqlens_q; seqused_q (ragged or paged) takes each sequence's live
query count from seqused_q[batch], capped at its span, with the causal
diagonal aligned to it. A packed sequence's row base is now 64-bit, so a
packed buffer may pass 2 GiB while each slice stays I32-addressable. The
spec refuses a dense_q or seqused_q that is not a bool, since either one
changes the argument layout.

The paged layout also takes a V block table, block_table_v, next to
block_table: K pages are looked up in block_table and V pages in
block_table_v, at the same row offset (one block_table_stride). This serves
cuDNN's separate page_table_k / page_table_v; a caller with one table (vLLM,
the attention dispatch binding) passes it twice and gets the same results.
A per-tile lookup now reads both tables; a page id outside [0, num_blocks)
zeroes only that cache's rows.

The prefill spec exposes catalog_layout, catalog_page_block_size,
catalog_dense_q and catalog_seqused_q for the hipDNN catalog (only the
prefill catalog has paged objects). The C++ mirror, engine binding, parity
rows, spec tests (with num_blocks and block_table_v in the paged
signatures), the paged-guard test and on-device feature cases (including
separate K and V tables) change together.

AttentionRequest gains dense_q and seqused_q fields (the dense_q /
seqused_q features, which only the RDNA WMMA prefill declares). The prefill
policy copies them into its spec; the decode refuses them, so such a request
always runs the prefill. A request without them keeps its request_hash: the
fields join normalized() only when set. They are semantic layout fields
(WMMA_PROBLEM_FIELDS, no sweep varies them), and while they are off they stay
out of the tuning spec's cache key, so every other RDNA WMMA spec_hash and
stored pin is unchanged.

The attention dispatch binding (bind_rdna_prefill_torch) reads both options:
a paged dense_q spec takes a batch-strided [B, Sq, Hq, D] Q / O (and LSE),
with batch strides from the tensors and no cu_seqlens_q, and a seqused_q spec
takes an INT32 [B] seqused_q tensor. A GPU test dispatches one dense-Q paged
prefill with seqused_q from the request and checks it against an FP64
reference; selection tests check the request fields reach the prefill spec,
the decode and every other arch refuse them, bad values are rejected, and
turning either option on changes the cache key.

rocke_rdna_attn_params_t gains block_table_v and the paged pointer fields,
and rocke_wmma_fmha_fwd_v2_spec_t gains dense_q and seqused_q, so
ROCKE_ABI_VERSION and BINARY_ABI move to 13. block_table_v moves every later
paged prefill argument, so the dispatch ABI string moves to
rocke-attention-gfx1151/v8.
…nd wmma_fmha_decode

A seqused_k entry past seqlen_k walked the block_table lookup off the end of
its row, into the next sequence's pages (or past the table for the last one).
The paged prefill and the paged split-KV decode now take
min(seqused_k[b], seqlen_k), in the Python builders and the C++ mirrors, and
the dispatch binding rejects a seqlen_k larger than the keys one table row
maps (its width times the page size), for both kernels. The feature test pads
table rows with out-of-range ids and adds a sequence whose seqused_k runs 100
keys past seqlen_k; the paged-guard test points a stale seqused_k's table
padding at a NaN page, for the prefill and the decode; a binding test rejects
a seqlen_k one key past the table.
The paged layout read K / V through one buffer resource over the whole
cache at the head, so a page term past 2 GiB went out of range of the
resource (and a cache past 4 GiB could not be addressed at all). The head
and page terms are now 64-bit:

- the kernel passes the body a 64-bit pointer to each cache at its head;
- a page that holds whole key tiles is read through one resource per tile,
  based at that page (the page id is CTA-uniform);
- a smaller page (one block-table lookup per cooperative row) is read by
  64-bit addressed global loads per row; a dead chunk reads the head's
  first element and yields zeros.

Within a page the offsets stay I32, so only one page of one head must span
less than 2 GiB. Dense and ragged layouts emit unchanged IR. The Python
builder and the C++ mirror change together. The feature test gains caches
past 4 GiB (tile and row pages, BF16 and FP8 K/V) whose other pages hold
NaN, so a truncated page term poisons the output; it skips on a device
without the free memory for them.

The 64-bit step from a base pointer by index * stride * element size is one
rdna_attention helper (rdna_attention_scaled_ptr in Python,
rocke_rdna_attention_scaled_ptr in C++), used by the paged head and page
steps and by the decode kernel's batch, token and bias bases, with the ops in
the order each copy had, so no golden changes.

The dispatch binding checks a paged prefill K / V cache as the hipDNN pack's
pageFitsI32 does (one page of one head I32-addressable in bytes) instead of
capping the whole cache at 2 GiB. The decode kernel still forms its head and
page offsets in I32, so its caches keep the whole-tensor cap. Since a dead
row's load still reads page 0, the binding rejects a prefill cache with no
pages (num_blocks is at least 1).
…with seqused_q in wmma_fmha_fwd_v2

With seqused_q, the rows of a sequence's span (its cu_seqlens_q gap, or
seqlen_q for a dense_q object) past seqused_q[b] were neither read nor
written, and a cu_seqlens_q sequence took its span from cu_seqlens_q[b + 1].

For a dense_q object, cuDNN and hipDNN's paged contract
(SdpaSemanticsRegister SEM-008) give the rows past seqused_q[b], up to
seqlen_q, O = 0 and LSE = -inf, with or without a sink, so the kernel now
writes them. The O and LSE resources span seqlen_q; Q still covers only the
live rows, so padding is never read. rdna_attention_fwd_body takes an
optional live_rows: rows at or past it are treated as empty rows without a
sink (O = 0, LSE = -inf), and a CTA whose tile is all padding skips the K
loop. Empty and padding rows store O = 0 by select rather than by scaling
with a zero reciprocal, since P = 0 times a non-finite V row still leaves
NaN in their accumulators.

For a cu_seqlens_q object, hipDNN's ragged contract (RFC 0014) says that
with seq_len bound the origins may come in any order, ragged_offset[B] is
neither used nor validated, and the rows of an offset block past seq_len_q
belong to no batch. The sequence now owns only its live rows,
min(seqused_q[b], seqlen_q): cu_seqlens_q[b + 1] is not read, and O / LSE
cover only those rows, so no write lands past a footprint-sized buffer.

The C mirror follows op for op.

The feature test checks that dense-Q padding rows read O = 0 and LSE = -inf
(also with a NaN in a V row every live row sees), and lays a ragged Q's
sequences out in reverse order with cu_seqlens_q[B] poisoned, checking that
the rows past seqused_q keep their sentinel.
hipkernel:Gfx1151WmmaAttention now binds the cuDNN paged-K/V SDPA graph: one
INT32 page table shared by K and V over [num_blocks, H, block_size, D]
containers of any strides (vLLM's interleaved K|V cache included), seq_len_kv,
a build-time max_seq_len_kv, and a query that is either dense (uniform decode,
with optional seq_len_q for its live rows) or ragged in token-unit offsets
with seq_len_q. One launch of a wmma_fmha_fwd_v2 paged prefill object serves
it; batch, lengths and block ids are read on the device, so one plan serves
every step and replays under HIP-graph capture. The page table is passed as
both of the kernel's tables, block_table (K) and block_table_v (V). The kernel
receives num_blocks, so an out-of-range page id reads as zero rows; graphs
whose K and V caches differ in num_blocks are declined. Pages are addressed in
64 bits, so a cache may pass 2 GiB (and 4 GiB) as long as one page of one head
is I32-addressable. Sinks, ALiBi, FP8 descales and dropout decline with a
logged cause.

Paged graphs are the prefill engine's alone. hipkernel:Gfx1151WmmaAttentionDecode
ships dense objects only: its graph matcher declines any graph with a page
table and its kernel matcher admits only the dense layout. The prefill engine
declines what the dense decode rule routes to decode only for dense graphs; a
paged graph's key extent is its page size, so that rule never applies to it.

Catalog: 72 paged prefill objects (head width 64/96/128/256 x page 16/32/64 x
query form x dtype) in configs/gfx1151_wmma_attention.yaml, each object
carrying layout, page_block_size, dense_q and seqused_q metadata (the dense
objects state the dense defaults); the engine's sdk_version is 1.4.0 for the
token-unit ragged offsets. The descriptors are regenerated with
IngestorGenerator and spliced keeping every existing UUID.

Tests: matcher, routing, score and ABI unit tests
(TestGfx1151WmmaAttentionPaged), a dense routing pin in
TestGfx1151WmmaAttentionMatchers, the regenerated census, and a GPU
integration test against a gather-then-dense reference with HIP-graph
capture and replay. The integration test skips when the frontend does not
yet validate paged K/V SDPA graphs.

Re-pin the gfx1151_wmma_attention.yaml digest in the IngestorGenerator
fixture unslotted_render_golden.json (test_slots.py): this change adds the
paged and ragged objects to that config, so its render changes by design.
The other reference configs render as before.
… in wmma_fmha_fwd_v2

Two on-device feature cases run the paged prefill with sinks through the
query forms hipDNN's paged graph binds: a batch-strided Q (gpt-oss GQA
decode, a sequence with no keys writing O = 0 and LSE = the sink) and a
batch-strided Q with seqused_q, padded rows and a causal window.
… and 64-bit paged addressing

Re-blesses only the gfx1151_wmma_fmha_fwd_v2 and gfx1151_wmma_fmha_decode ll
(llvm20, llvm22, llvm23, llvm24) and ir_canonical entries, with run_diff.py
--python-only --record-golden --only gfx1151_wmma_fmha: for the prefill, the
64-bit packed row base, the seqused_k clamp, the separate V block table,
64-bit paged K/V pages, the padding rows past seqused_q (stored by select)
and the new dense_q / seqused_q configs; for the decode, the paged seqused_k
clamp. No other family changes.
…fx1151WmmaAttention

The prefill catalog gains a sink variant of every paged prefill object (dtype
x head-width bucket 64/96/128/256 x page size 16/32/64 x query form):
wmma_fmha_fwd_v2 with use_sinks, named *_sinks, 72 objects (192 in all).
A new KMD field 'sinks' (int, default 0) marks them; the wmma_fmha_fwd_v2
spec exposes it as catalog_sinks for the specialization contract, and every
existing object states sinks 0. Retained descriptor ids are unchanged; the
census lists the new objects for every shard. The decode catalog
(hipkernel:Gfx1151WmmaAttentionDecode) is unchanged.

The native matcher accepts sink_token on the cuDNN paged-K/V graph (dense
Q, dense Q with seq_len_q, ragged Q): [1, Hq, 1, 1] in the Q data type with
a unit head stride, bound as a sink uid that selects the paged objects
whose catalog field sinks is 1. prepare() marshals sink_ptr after the
stride arguments and before the query form's operands
(wmma_fmha_fwd_v2._declare_params); launch passes the 2-byte-aligned sink
buffer there. A FLOAT sink (the RDNA kernels load the sink in the I/O
dtype), another dtype or shape, and sinks on dense-layout graphs are
declined with cause 'sinks'. Paged graphs stay the prefill engine's: the
decode engine declines them with or without sinks.

Unit tests cover binding in every query form, the declines, sinks-object
selection, the decode engine's decline and the kernel ABI; the GPU
integration test adds three sinks cases (gpt-oss GQA decode with a window
and a keyless sequence, dense Q with seq_len_q and LSE, a mixed ragged
batch) against the gather-then-dense reference with the sink in max,
denominator and LSE.

Re-pin the gfx1151_wmma_attention.yaml digest in the IngestorGenerator
fixture unslotted_render_golden.json (test_slots.py): the sink objects
change that config's render by design.
…1151WmmaAttention

hipDNN's paged K/V contract (SdpaSemanticsRegister SEM-008/009, #14358)
differs from the one this engine was written against:

- Every paged graph carries seq_len_q and seq_len_kv, also with a dense Q.
  The matcher now requires both and always binds seq_len_q. The dense-Q
  form without seq_len_q is gone: GraphLayout keeps PAGED_DENSE_Q (dense Q,
  live rows seq_len_q) and PAGED_RAGGED_Q, and an object without seqused_q
  serves no graph.
- A dense Q's rows past seq_len_q, up to Sq, are padding with O = 0 and
  LSE = -inf. The kernel now writes them (an earlier commit); the GPU test
  expects that rather than untouched rows.
- A ragged Q follows RFC 0014 (#15080): origins may come in any order,
  ragged_offset[B] is neither used nor validated, and the rows of an offset
  block past seq_len_q belong to no sequence. The kernel reads only each
  sequence's origin and writes only its live rows; the GPU test lays the
  sequences out in reverse order, poisons ragged_offset[B] and expects
  those rows untouched.
- Separate K and V page tables (page_table_k, page_table_v) are valid
  graphs and are now served: the matcher binds both, and the launch passes
  them as the kernel's block_table and block_table_v (one table bound to both
  sides is passed twice). The two tables must have one width and, with more
  than one sequence, one row stride.
- One-sided paging, per-side block sizes or page counts, K and V tables of
  different widths or row strides, and packed page tables are valid graphs
  the kernels cannot read, so each is declined with a logged cause.
- An unset max_seq_len_kv is valid. The kernels use it only as the bound on
  seq_len_kv, so it is inferred as the table width times the block size, as
  cuDNN does.
- A graph may declare a device-pointer alignment below the default 16 bytes
  (engine API 1.3.0, which the plugin's 1.4.0 admits). The kernels load
  Q / K / V / O rows as 16-byte vectors, INT32 operands as 4-byte words and
  the sink with 16-bit loads, so the matcher declines a graph declaring less
  for any of them (cause 'alignment', dense and paged) where hipDNN can
  still pick another engine, rather than the plan failing at execute.
- left_bound follows cuDNN (L keys ending at the diagonal); maskFor already
  translates it to the kernel's window_left = L - 1. The GPU test's windows
  follow it (32-, 16- and 128-key windows).

Catalog: the 48 paged objects with dense_q but without seqused_q (24 plain,
24 with sinks) could serve no hipDNN graph and are dropped; 96 paged
objects remain (dense Q and ragged Q, each with and without sinks), 144
prefill objects in all. Retained descriptor ids are unchanged. The census
is regenerated.

Decline logs: the paged checks log the cause keys #14247 uses for the same
contract rules, so the shared checks it moves into SdpaGraphMatch.hpp keep
every key this engine logs: one_sided_paging; page_table (a packed table,
K and V tables of different widths or row strides, a table that is not
INT32 [B, 1, M, 1] with a unit entry stride); block_size (different K and V
block sizes, a page size that is not a power of two, and one no paged object
ships for, which used to find no object without a log); shape (different K
and V page counts); seq_len_q, seq_len_kv and ragged (a missing or malformed
sequence-length or offset tensor, declined silently before); attn_mask
(additive bias on a paged graph). Each reason names the operand and values
involved. The dense layout logs 'padding' for seq_len_q, seq_len_kv,
max_seq_len_kv or padding_mask. Each line is headed by the declining
engine's name (sdpa_graph::DeclinedBy): the decode engine runs the shared
dense matcher too, and used to log under the prefill engine's name; it now
also logs why it declines a paged graph.

Tests: the matcher scaffolding both unit tests share (engine symbols, the
graph fixture, object metadata, the decline-log reader, the ABI helpers)
moves into Gfx1151WmmaAttentionTestSupport.hpp. The unit tests bind
separate K and V tables (dense and ragged Q) and a V container with its own
strides, and assert the engine, cause and reason of each decline: every
paged check in isolation, the unshipped page sizes, alignment (sink
included), FP8, dropout, a forced implementation, the other unserved
features and ALiBi; the dense declines are asserted for both engines. The
GPU test checks against test_sdk's CpuFpReferenceSdpa (gatherPagedKv per
page table, sequence lengths, band, sink and LSE) instead of its own
reference, adds two cases whose V pages sit in other blocks than their K
pages, through a second page table, two whose V page, head and token strides
all differ from K's, and runs the 96 and 256 buckets at pages 32 and 64 and
the 128 bucket at page 64.

Sinks: the engine still accepts a sink_token in the Q data type and
declines a FLOAT one. hipDNN's frontend accepts only a FLOAT sink, so the
GPU test's sink cases now skip with a message when the frontend rejects a
Q-dtype sink; the paged-rules probe that skipped every case is removed, and
the other cases run.

Re-pin the gfx1151_wmma_attention.yaml digest in the IngestorGenerator
fixture unslotted_render_golden.json (test_slots.py): the contract
change alters that config's render by design.
…WmmaAttention

hipDNN's frontend and references take sink_token only as FLOAT [1, Hq, 1, 1], which the
paged sinks objects declined, so no graph that validates reached them. The sinks
objects now read either form through one runtime argument, so no catalog object is
added.

- wmma_fmha_fwd_v2 (Python and the C mirror): a sinks object takes sink_is_f32 (i32)
  after sink_ptr. Nonzero reads sink_ptr[h] as f32, zero in the Q dtype. Two 16-bit loads
  stay inside either form's [Hq] extent and are combined into the f32 word or widened
  from the low half. Re-bless the gfx1151_wmma_fmha_fwd_v2 goldens.
- The RDNA dispatch bindings pass sink_is_f32 to the prefill kernel and accept float32
  sinks for it; the decode kernel still reads the Q dtype only.
- Gfx1151WmmaAttention: sinkFor accepts FLOAT (declaring 4-byte alignment) or the Q data
  type (2 bytes); another dtype declines under `sinks`. The bound token
  gfx1151_wmma_attention.sink.f32 (0 or 1, checked when read) sets sink_is_f32 at launch,
  and execute checks the pointer against the dtype's alignment.
- Tests: FLOAT sinks bind with the flag set, decline below 4 bytes, and a descriptor
  without sink_is_f32 is refused. The rocKE feature tests add f32-sink cases (dense Q,
  per-row and per-tile ragged Q). IntegrationGpuGfx1151WmmaAttentionPaged runs each sink
  case again with a FLOAT sink_token, which needs no frontend change. The Q-dtype sink
  cases still skip on a frontend that rejects them.
…on gfx11 in Gfx1151WmmaAttention

At pages 32 and 64 the head-96 paged objects used 256 VGPRs on gfx11 and spilled inside the K loop. With block_m 64 (block_size 128) they use 212 and do not spill; gfx12 objects keep block_m 128. The 16 gfx11 objects get new ids; the descriptors, catalog census and render digest are regenerated.
dpeabody-amd added a commit that referenced this pull request Nov 7, 2026
…SdpaGraphMatch.hpp


hipkernel:AttentionPagedUnified's graph_match carried its own copies of the
SDPA match helpers that packs/SdpaGraphMatch.hpp now shares (findTensor,
sdpaNode, hasSdpaForwardNode, describeOperand, the decline head and the axis
constants). It now uses the header's, with sdpa_graph::DeclinedBy. The
overflow-checked product it shares with hipkernel:Gfx942AttentionUnified3d
moves into the header as sdpa_graph::checkedProduct: unlike
pack_features::checkedProduct it takes zero factors, which every extent of 1
gives. FlydslSdpaShared.hpp, which
already takes computesInFloat from the header, now takes checkedProduct and
declaresAlignment from it too instead of keeping identical copies.

The stricter operand checks it needs move into the header too
(usesOverrideShapes, hasNonNegativeStrides, declaresAlignment, isInt32Vector),
together with the cuDNN paged-K/V graph contract checks (SEM-009), so every
pack serving a paged graph applies one set of rules with one cause vocabulary,
logged with the calling engine's name:
- pagesKAndV: one_sided_paging;
- pagedContainersAgree: block_size, shape (one block size, one block count;
  head counts and head sizes stay the engine's to check);
- pageTablesFor: page_table (INT32 [B, 1, M, 1], unit entry stride, not
  packed; separate K and V tables share M and the batch stride);
- sequenceLengthsFor: seq_len_kv, seq_len_q (both required, INT32 [B,1,1,1]);
- raggedQueryFor, countsOffsetsInTokens: ragged (Q and O through one
  [B+1,1,1,1] offset table counted in tokens; K/V never ragged);
- declaresServedAlignment: alignment (vector operands at the engine's vector
  width, INT32 operands at 4 bytes);
- sinkFor: sinks, and alignment below the engine's own minimum sink
  alignment (16 bytes for this engine's kernels);
- takesNoAttentionMask: attn_mask.

The reasons they log are the texts hipkernel:Gfx1151WmmaAttention's paged
matcher (#14254) logs for the same rules, so that engine can call these checks
without changing what it reports. The decline lines go through
HIPDNN_PLUGIN_LOG_INFO((DeclinedBy{...}) << ...): the braces' comma would
otherwise split the macro argument.

The engine keeps its kernel limits (head and block sizes, GQA ratios, strides
and address ranges) and the order of its declines; it checks itself that V has
K's head count and head size. A malformed page-table tensor now logs page_table
rather than operand. The header's file comment covers paged graphs.
@dpeabody-amd
dpeabody-amd force-pushed the users/dpeabody/hipdnn-rdna-paged-attn branch from 89ee88a to 67a5479 Compare November 7, 2026 10:49
@dpeabody-amd

Copy link
Copy Markdown
Contributor Author

Pushed 67a5479, rebased onto develop 1702865. Replies to the inline threads are on each thread; all but Brian's YAML thread are resolved.

@cgmillette, follow-up review

Reply as a top-level comment:

Thanks for the follow-up review. On the open rows:

  • LSE and bias alignment, FLOAT sink, ragged multiplier, tile-path page id, arch gate and parity rows: all addressed; see the replies on each thread.
  • N3 (48 paged sink objects that no frontend graph could reach): closed by the FLOAT-sink change. A FLOAT sink_token now selects those objects, and the hipDNN-level GPU tests and the FLOAT-sink bundles exercise them.
  • N6 (head-96 spills on gfx11): at pages 32 and 64 those objects used 256 VGPRs and spilled. They now ship with block_m 64 (block_size 128) on gfx11, which uses 212 VGPRs and doesn't spill; gfx12 keeps block_m 128, which doesn't spill there. Those 16 objects get new ids.
  • N7 (SHIPPED_PAGE_SIZES): the page sizes and archs are now stated once in Gfx1151WmmaAttentionCatalog.hpp, and the census case checks both against the loaded objects.
  • The ABI history nit is fixed: the entry now says the fwd_v2 spec keeps its size and offsets. After rebasing onto today's develop the bump is entry 13 (develop is at 12).

@cgmillette, follow-up review

Reply as a top-level comment:

Covered by the reply above; the remaining rows from this round:

  • N4: the paged checks stay in this matcher for now, with feat(hip-kernel-provider): paged/ragged SDPA engine for gfx942/gfx950 #14247's cause keys, as agreed. Whichever PR lands second switches this matcher to the shared SdpaGraphMatch.hpp checks.
  • Test support: it now reuses develop's PackSymbols, GraphFixture, matchesGraph, matchesKernel, scoreKernel and dispatchHandler from IngestorPackTestSupport.hpp, rather than its own copies.
  • The binding's I32 checks still follow their sibling bindings' form; I left those as they are.

@CongMa13

Reply as a top-level comment:

Thanks, Cong. All seven threads now have replies: six are addressed in the code and resolved, and the YAML resolved thread is left for Brian to resolve or pick the arm form. Both automated reviews have a reply above.

@cgmillette

Copy link
Copy Markdown
Contributor

Automated review: Model - qwen3.6:35b-a3b-coding-mtp-q4_K_M, effort: high

PR #14254 — paged/ragged SDPA in Gfx1151WmmaAttention (dpeabody-amd, AICK-1847) — 🟡 Follow-up: fixes hold; CI running

Prior item Status
BrianHarrisonAMD (11-04) ×14: matcher, tests, binding, docs 🟢 FIXED 67a5479ca1
• rebase re-cut every commit; changed tests pass at head (ran)
• YAML thread open: dense_q: 0 stays for the raggedq arms (read)
• BrianHarrisonAMD resolves it or picks the arm form
AnaghaRaoAMD (11-04) ×3 matcher: LSE/bias alignment, FLOAT sink, ragged multiplier 🟢 FIXED 68988aa7c0, 1f335c0e59
• stats, bias and sink alignment declines added (Gfx1151WmmaAttentionNative.cpp:1091) (read)
• FLOAT sink served via sink_is_f32; f32-sink GPU cases pass on gfx1151 (ran)
• multiplier declines with cause ragged (:1181) (read)
AnaghaRaoAMD (11-04) ×3 kernel and tests: tile-path page id, arch gate, parity rows 🟢 FIXED 80cd54e96f, 68988aa7c0
• tile path selects page 0 first (rdna_attention.py:572, rdna_attention.cpp:705); byte identity GREEN ×4 (ran)
• GPU cases gate on the shipped arch list (IntegrationGpuGfx1151WmmaAttentionPaged.cpp:891) (read)
• parity rows 180–185 added; head 96 only at page 16: N8
yraparti (11-07) ×2 HIGH: bool layout flags, request wiring 🟢 FIXED 0b4bb759cf
• non-bool flags raise TypeError; request fields reach the prefill spec (read)
• spec and selection tests pass in the installed tree (ran)
Copilot (11-07) ×7: capacity, ABI string, clamps, page id, empty cache, memory skip, NaN padding 🟢 FIXED 67610ff7fe, 0b4bb759cf, 80cd54e96f, 1ec3fd74e4
• changed tests pass installed, GPU cases on gfx1151 (ran)
• all 7 threads answered and resolved (read)
CongMa13 (11-06) threads and posts without replies 🟢 ANSWERED
• author replied on every thread and to both automated posts (read)
• CongMa13 reassesses
Ours (11-04, 11-05) N1, N2, N5: CMake block, ABI bump, 64-bit step 🟢 FIXED 0b4bb759cf
• history nit fixed: entry 13 says the spec keeps its layout (read)
• develop is 12; takes develop + 1 when it lands; 14247 and 14707 also take 13 (ran)
• one pointer-step helper per engine; byte identity GREEN ×4 (ran)
Ours (11-04) N3: 48 paged sink objects, sink dtype the frontend rejects 🟢 FIXED 1f335c0e59
• the 56 sink objects read a FLOAT or a Q-dtype sink (sink_is_f32) (read)
• f32-sink feature cases pass on gfx1151 (ran)
Ours (11-04) N4 + overlap notice: paged contract checks, owner 14247 🟡 OPEN
• own copies remain (Gfx1151WmmaAttentionNative.cpp:1119-1256), with 14247's cause keys (read)
• two more copied this round: sinkFor (:1076), token-unit offsets (:1171) (read)
• 14247 (3bed02a4f1) shares all in SdpaGraphMatch.hpp, not landed; whichever lands second switches (ran)
Ours (11-05) N6: 16 of the 96 paged objects spill VGPRs on gfx11 🟢 FIXED 67a5479ca1
• 0 of 192 paged compiles spill, each on its own targets; gfx1151 too (ran)
• gfx11 head-96 pages 32/64 at block_m 64: 212 VGPRs; 8 GPU cases pass on gfx1151 (ran)
Ours (11-05) N7: SHIPPED_PAGE_SIZES restates the catalog's page sizes 🟢 FIXED 68988aa7c0
• page sizes and archs stated once, in Gfx1151WmmaAttentionCatalog.hpp (read)
• census pins the page sizes both ways (read)
• nit: archs pinned one way only, a dropped arch passes (TestGfx1151WmmaAttentionCatalog.cpp:74) (read)
Bots / CI 🟡 Running
• 0 ✗ so far; hipDNN Linux superbuild ✓ (read)
• running: 3 math-libs stages (gfx94X, gfx110X, gfx1250); rocKE shards wait on them
• merge: clean; blocked: review required, 1 thread open (read)
Our checks 🟢 Gates clean
• merge vs 1702865a2f (today's develop): clean, 0 behind (ran)
• build + ctest 60 ✓ / 1 skip; byte identity GREEN ×4; parity ×4; both goldens ✓ (ran)
• changed tests 212 ✓ on CPU; installed 625 ✓ / 11 skip, GPU cases on gfx1151 (ran)
• compile: one feature-combo parity config still spills, no shipped object (ran)
• body: says 144 prefill and 48 sink objects; the KDP ships 160 and 56 (ran)
Code quality 🟡 Minor
• N4 open, two more shared checks copied; N7 fixed (read)
• decode tile path still forms the out-of-cache page base (wmma_fmha_decode.py:833): develop's code, the prefill fix's class (read)
• dense_q/seqused_q listed in five places (common.py:234, :433, :1024; gfx1151.py): one constant (read)
# Sev file:line Finding
N8 🟡 gfx1151_wmma_fmha_fwd_v2_emit.py:105 No parity row for the 16 shipped paged block_m 64 objects
• impact: byte identity never sees paged + block_m 64; C++ and Python match today (ran)
• fix: a head-96 page-32 block_m 64 gfx11 row in both emitters and prove_parity_binding

Utility: 🟢 Positive — paged and ragged SDPA for RDNA prefill, now serving FLOAT sinks without spills.


Legend: 🔴 blocking · 🟠 should-fix · 🟡 minor nit · 🟢 info/good · ⚪ retracted

Follow-up review: first pass by qwen3.6 (local), verified by the orchestrator (Claude Opus 5.5). Checks: changes since 89ee88af23, merge with today's develop (build + ctest, byte identity, parity, goldens, differential golden, CPU and installed tests, compile), struct layouts, saved probes, the paged catalog compiled per target, binding IR and gfx1151 GPU runs of the block_m 64 objects, CI at head, the 14247 overlap, reviewer threads, code quality. Flags whether a closer look is needed; not an approval or rejection.

@yraparti yraparti left a comment

Copy link
Copy Markdown
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

rocKE checklist re-review of the fixes since 89ee88af237d

ROCm/rocm-libraries#14254 · 67a5479ca128 · model gemini-4-argon, fallback gpt-5.5

PASS · graded 238/238 · BLOCKER 0, HIGH 0, MEDIUM 0, LOW 0, NEEDS-INFO 18

The diff was truncated before grading. Items that needed a missing hunk are NEEDS-INFO.

Since the last review (89ee88af237d)

Resolved 2 · still open 0 · new 0

  • ✅ §B.4 was FAIL, now NEEDS-INFO — had asked: In WmmaFmhaFwdV2Spec.__post_init__ (dnn-providers/hip-kernel-provider/rocke/library/kernels/gfx1151/wmma_fmha_fwd_v2.py:168), validate if type(self.dense_q) is not bool or type(self.seqused_q) is not bool: raise TypeError(...) before checking layout compatibility.
  • ✅ §D.10 was FAIL, now PASS — had asked: Expose dense_q and seqused_q (or the corresponding query layout/length presence) on AttentionRequest and wire the RDNA prefill spec builder to populate them from the request, or document a stacked PR deferral in the PR description.

Findings

  • No FAIL rows.

Needs info

  • §A.1 dnn-providers/hip-kernel-provider/rocke/library/kernels/gfx1151/wmma_fmha_fwd_v2.py:467 and dnn-providers/hip-kernel-provider/rocke/platform/cpp/instances/gfx1151/wmma_fmha_fwd_v2.cpp:400 keep the Python and C++ kernel-body emitters in sync (along with dnn-providers/hip-kernel-provider/rocke/library/kernels/gfx1151/wmma_fmha_decode.py:697 and dnn-providers/hip-kernel-provider/rocke/platform/cpp/instances/gfx1151/wmma_fmha_decode.cpp:974). — no diff line supports this verdict
  • §A.5 The diff for dnn-providers/hip-kernel-provider/rocke/platform/tests/instances/differential/golden/llvm_gfx_all.json is omitted to fit the model budget (+1015 -935 lines), so verifying which golden entries were changed or left untouched requires the omitted hunk.
  • §A.10 The diff for dnn-providers/hip-kernel-provider/rocke/platform/tests/instances/differential/golden/llvm_gfx_all.json is omitted (+1015 -935 lines), so the new golden hash entries for indices 170..185 (QUERY_LENGTHS and SHIPPED_PAGED flag-on cases across LLVM flavors and arches) are not visible in the diff.
  • §B.4 Resolved since previous review: WmmaFmhaFwdV2Spec.__post_init__ in dnn-providers/hip-kernel-provider/rocke/library/kernels/gfx1151/wmma_fmha_fwd_v2.py:173-176 now enforces if type(value) is not bool: raise TypeError(...) for dense_q and seqused_q (matching dnn-providers/hip-kernel-provider/rocke/library/dispatch/attention/common.py:433-436), and dnn-providers/hip-kernel-provider/rocke/library/tests/test_wmma_fmha_fwd_v2_spec.py:48-61 tests this rejection. — no diff line supports this verdict
  • §C.9 dnn-providers/hip-kernel-provider/rocke/library/kernels/gfx1151/wmma_fmha_fwd_v2.py:538, dnn-providers/hip-kernel-provider/rocke/platform/cpp/instances/gfx1151/wmma_fmha_fwd_v2.cpp:456, dnn-providers/hip-kernel-provider/rocke/library/kernels/gfx1151/wmma_fmha_decode.py:697, and dnn-providers/hip-kernel-provider/rocke/platform/cpp/instances/gfx1151/wmma_fmha_decode.cpp:974 apply the paged seqused_k clamp across both prefill and decode in both Python and C++ twins. — no diff line supports this verdict
  • §E.10 Unresolved from previous review: merge-simulation against other open PRs touching the same files (e.g. abi.h, llvm_gfx_all.json, Gfx1151WmmaAttentionNative.cpp) still cannot be verified from the PR diff alone.
  • §F.6 Local pre-commit execution status before push cannot be verified from the diff alone.
  • §F.7 Bot, Copilot, and human review thread states and deferred-item ticket IDs are not included in the prompt or diff.

Section notes

  • §B: Re-review: Item 4 resolved by adding strict type(value) is not bool checks in WmmaFmhaFwdV2Spec.__post_init__ alongside unit tests.
  • §G: Re-review of head 67a5479. Items 2 and 3 remain NEEDS-INFO because the PR body and diff are truncated before test/gate execution results.
  • §G: Items 18–34 of 34
  • §H: Trace of device-side bounds and masks: 1. Paged K/V key length sk / Sk_b: extent source is smin(seqused_k[batch], seqlen_k) (wmma_fmha_fwd_v2.py:538, wmma_fmha_decode.py:697). The kernel loops over n_tiles = cdiv(smax(sk, 0), BN) (rdna_attention.py:826-827) and bounds per-tile valid rows via rem = select(valid, seqlen_k - tok0, 0) (rdna_attention.py:567). Host validates seqlen_k <= block_table.shape[1] * page_block_size (bindings.py:1151-1157, Gfx1151WmmaAttentionNative.cpp:1282-1289), so blk = tok >> lps never reads past the sequence's block_table / block_table_v row. 2. Physical page ID phys (phys_k, phys_v): extent source is runtime 0 <= phys < num_blocks via page_ok(phys) (rdna_attention.py:554-556). On the per-tile path (ps % BN == 0), rem_k and rem_v are zeroed and safe_k/safe_v select page 0 before forming the 64-bit page_base (rdna_attention.py:578-588); on the per-row path (ps % BN != 0), ok masks both page to…
  • §J: Shapes: | Operand / Equation | Inner dims | Result vs Declared | |---|---|---| | dense_q Q / O (README.md:501) | N/A (tensor layout annotation) | [B, Sq, Hq, D] | | seqused_q (README.md:502) | N/A (tensor layout annotation) | INT32 [B] | Symbols: | Symbol | Defined at | Domain | |---|---|---| | block_table / block_table_v | README.md:496, attention.md:38 | K and V page table indices | | dense_q | README.md:500, attention.md:39 | Boolean spec/request flag for batch-strided Q/O | | seqused_q | README.md:502, attention.md:40 | Per-sequence live query row count (INT32 [B]) / bool spec flag | | use_sinks / sinks | README.md:528 | Spec boolean / catalog integer flag for attention sinks | Novelty: | Claim | Symbol | Status vs develop | |---|---|---| | Separate V page table in v8 ABI | block_table_v, rocke-attention-gfx1151/v8 | Added in this PR | | Paged prefill query options | dense_q, seqused_q | Added in this PR | | Shipped paged…
  • §K: Identity table: (1) WmmaFmhaFwdV2Spec.dense_q (default False, problem field in WMMA_PROBLEM_FIELDS) -> omitted from cache_key when False so config_key/spec_hash unchanged for existing pins, no TUNING_ID_VERSION bump needed, no stored pins refused; (2) WmmaFmhaFwdV2Spec.seqused_q (default False, problem field in WMMA_PROBLEM_FIELDS) -> omitted from cache_key when False so config_key/spec_hash unchanged for existing pins, no TUNING_ID_VERSION bump needed, no stored pins refused; (3) ATTENTION_GFX1151_ABI bumped from v7 to v8 for paged block_table_v kernel signature change.

Graded with the rocKE PR checklist, one model pass per section. FAIL rows that sit on a diff line are inline comments.

§A graded 14/14
# Verdict Evidence
1 NEEDS-INFO dnn-providers/hip-kernel-provider/rocke/library/kernels/gfx1151/wmma_fmha_fwd_v2.py:467 and dnn-providers/hip-kernel-provider/rocke/platform/cpp/instances/gfx1151/wmma_fmha_fwd_v2.cpp:400 keep the Python and C++ kernel-body emitters in sync (along with dnn-providers/hip-kernel-provider/rocke/library/kernels/gfx1151/wmma_fmha_decode.py:697 and…
2 N/A No core lowering-op or op-handler files (core/lower_*, core/ir) are modified in this PR.
3 PASS dnn-providers/hip-kernel-provider/rocke/library/kernels/gfx1151/wmma_fmha_fwd_v2.py:128, dnn-providers/hip-kernel-provider/rocke/platform/cpp/include/rocke/instance_gfx1151_wmma_fmha_fwd_v2.h:63…
4 PASS dnn-providers/hip-kernel-provider/rocke/platform/cpp/helpers/rdna_attention.cpp:220 and dnn-providers/hip-kernel-provider/rocke/platform/cpp/instances/gfx1151/wmma_fmha_fwd_v2.cpp:489 bind operands left-to-right to temporaries to match the exact emit order in Python…
5 NEEDS-INFO The diff for dnn-providers/hip-kernel-provider/rocke/platform/tests/instances/differential/golden/llvm_gfx_all.json is omitted to fit the model budget (+1015 -935 lines), so verifying which golden entries were changed or left untouched requires the omitted hunk.
6 PASS dnn-providers/hip-kernel-provider/rocke/library/tests/parity/gfx1151_wmma_fmha_fwd_v2_emit.c:101, dnn-providers/hip-kernel-provider/rocke/library/tests/parity/gfx1151_wmma_fmha_fwd_v2_emit.py:97, and…
7 PASS dnn-providers/hip-kernel-provider/rocke/library/kernels/gfx1151/wmma_fmha_fwd_v2.py:516 and dnn-providers/hip-kernel-provider/rocke/platform/python/rocke/helpers/rdna_attention.py:823 guard new ops behind layout != 'dense', spec.dense_q, spec.seqused_q, and live_rows…
8 PASS dnn-providers/hip-kernel-provider/rocke/platform/cpp/helpers/rdna_attention.cpp:700 and dnn-providers/hip-kernel-provider/rocke/platform/cpp/instances/gfx1151/wmma_fmha_fwd_v2.cpp:412 match their Python twins and are wired into…
9 N/A No shared core op-handler fix or lowerer-only arch divergence is introduced in this PR.
10 NEEDS-INFO The diff for dnn-providers/hip-kernel-provider/rocke/platform/tests/instances/differential/golden/llvm_gfx_all.json is omitted (+1015 -935 lines), so the new golden hash entries for indices 170..185 (QUERY_LENGTHS and SHIPPED_PAGED flag-on cases across LLVM flavors and arches) are not visible in the diff.
11 N/A No new IR opcodes are added in this PR.
12 N/A core/backend.py and ROCKE_BACKEND lowering fallback logic are not touched in this PR.
13 N/A LLVM .ll emission format and ROCKE_DEBUG_LOC handling are not modified in this PR.
14 PASS dnn-providers/hip-kernel-provider/rocke/platform/python/rocke/helpers/rdna_attention.py:213, dnn-providers/hip-kernel-provider/rocke/platform/cpp/helpers/rdna_attention.cpp:214, and…
§B graded 6/6
# Verdict Evidence
1 PASS New rejections in dnn-providers/hip-kernel-provider/src/engines/kernel_ingestor_engine/packs/Gfx1151WmmaAttentionNative.cpp:851-891 (unservedFeature) and Gfx1151WmmaAttentionNative.cpp:1110-1315 (gfx1151WmmaAttentionPagedGraphMatches), as well as…
2 PASS WmmaFmhaFwdV2Spec.__post_init__ in dnn-providers/hip-kernel-provider/rocke/library/kernels/gfx1151/wmma_fmha_fwd_v2.py:173-182, C++ v2_post_init_ok in dnn-providers/hip-kernel-provider/rocke/platform/cpp/instances/gfx1151/wmma_fmha_fwd_v2.cpp:144-154, and…
3 PASS No shapes are demoted to a slow fallback; paged/query-option requests are routed to the prefill kernel (dnn-providers/hip-kernel-provider/rocke/library/dispatch/attention/gfx1151.py:568-569) and unsupported hipDNN graph variants are loudly declined at INFO…
4 NEEDS-INFO Resolved since previous review: WmmaFmhaFwdV2Spec.__post_init__ in dnn-providers/hip-kernel-provider/rocke/library/kernels/gfx1151/wmma_fmha_fwd_v2.py:173-176 now enforces if type(value) is not bool: raise TypeError(...) for dense_q and seqused_q (matching dnn-providers/hip-kernel-provider/rocke/library/dispatch/attention/common.py:433-436), and…
5 PASS Newly opened gates (dense_q, seqused_q, separate block_table_v, FP32 sinks, and >4 GiB paged prefill K/V caches) are tested in dnn-providers/hip-kernel-provider/rocke/library/tests/test_wmma_fmha_fwd_v2_features.py:168-247…
6 PASS Spec validation in dnn-providers/hip-kernel-provider/rocke/library/kernels/gfx1151/wmma_fmha_fwd_v2.py:173-182 and dnn-providers/hip-kernel-provider/rocke/platform/cpp/instances/gfx1151/wmma_fmha_fwd_v2.cpp:144-154 only rejects functionally invalid flag types and…
§C graded 10/10
# Verdict Evidence
1 PASS dnn-providers/hip-kernel-provider/rocke/platform/python/rocke/helpers/rdna_attention.py:213 defines the shared rdna_attention_scaled_ptr helper, dnn-providers/hip-kernel-provider/rocke/library/kernels/gfx1151/wmma_fmha_decode.py:751 reuses it across batch/token/bias…
2 PASS dnn-providers/hip-kernel-provider/rocke/library/kernels/gfx1151/wmma_fmha_fwd_v2.py:467 keeps the single public build_wmma_fmha_fwd_v2(spec) builder while adding the new behavior through spec fields declared at…
3 PASS dnn-providers/hip-kernel-provider/rocke/library/kernels/gfx1151/wmma_fmha_fwd_v2.py:222 and dnn-providers/hip-kernel-provider/rocke/platform/cpp/instances/gfx1151/wmma_fmha_fwd_v2.cpp:804 add kernel-name suffixes for codegen/layout options (_denseq and _sequsedq), not…
4 PASS dnn-providers/hip-kernel-provider/rocke/platform/python/rocke/helpers/rdna_attention.py:213 and dnn-providers/hip-kernel-provider/rocke/platform/cpp/helpers/rdna_attention.cpp:214 place the arch-independent 64-bit pointer-scaling logic in the shared RDNA attention…
5 PASS dnn-providers/hip-kernel-provider/src/engines/kernel_ingestor_engine/packs/Gfx1151WmmaAttentionNative.cpp:62 and dnn-providers/hip-kernel-provider/rocke/library/dispatch/attention/README.md:525 confine the changes to the RDNA prefill/decode attention path and its…
6 PASS dnn-providers/hip-kernel-provider/rocke/platform/python/rocke/helpers/rdna_attention.py:213 and dnn-providers/hip-kernel-provider/rocke/platform/cpp/helpers/rdna_attention.cpp:214 introduce the named rdna_attention_scaled_ptr helper (and its C++ twin) and refactor…
7 N/A This PR is a feature addition (paged/ragged SDPA options, 64-bit cache addressing, and separate V block tables) that intentionally…
8 PASS dnn-providers/hip-kernel-provider/rocke/platform/python/rocke/helpers/rdna_attention.py:213 and dnn-providers/hip-kernel-provider/rocke/platform/cpp/helpers/rdna_attention.cpp:214 factor the existing local batch_ptr/token_ptr/scaled_ptr lambdas from…
9 NEEDS-INFO dnn-providers/hip-kernel-provider/rocke/library/kernels/gfx1151/wmma_fmha_fwd_v2.py:538, dnn-providers/hip-kernel-provider/rocke/platform/cpp/instances/gfx1151/wmma_fmha_fwd_v2.cpp:456, dnn-providers/hip-kernel-provider/rocke/library/kernels/gfx1151/wmma_fmha_decode.py:697, and dnn-providers/hip-kernel-provider/rocke/platform/cpp/instances/gfx1151/wmma_fmha_decode.cpp:974 apply the paged seqused_k clamp across both…
10 PASS dnn-providers/hip-kernel-provider/rocke/platform/cpp/include/rocke/abi.h:112 and dnn-providers/hip-kernel-provider/rocke/platform/python/rocke/portable_ir/src/abi.py:84 bump the binary ABI to 13 for the widened C structs, while…
§D graded 21/21
# Verdict Evidence
1 PASS New query layout options dense_q and seqused_q are declared as explicit WmmaFmhaFwdV2Spec fields with validation, name tags, selector wiring, and cache-key entries without reading os.environ…
2 PASS The feature set update in gfx1151.py adds "dense_q" and "seqused_q" to the candidate's declared features without altering arches or transcribing kernel coverage numbers (dnn-providers/hip-kernel-provider/rocke/library/dispatch/attention/gfx1151.py:171).
3 PASS _decode_refusal checks req.dense_q or req.seqused_q first and returns a specific refusal reason ("dense_q / seqused_q requests run on the prefill kernel"), and the selection test queries decode.admits(request) rather than _supports…
4 PASS _prefill_spec and _decode_refusal are pure functions of req: AttentionRequest and do not introduce any host or device probes (dnn-providers/hip-kernel-provider/rocke/library/dispatch/attention/gfx1151.py:346…
5 PASS dense_q and seqused_q are baked into kernel_name() (dnn-providers/hip-kernel-provider/rocke/library/kernels/gfx1151/wmma_fmha_fwd_v2.py:222) and included in the spec cache key / identity() when enabled while preserving existing hashes when off…
6 N/A No dispatcher cohort routing is added or modified in this PR.
7 N/A No heuristic tables or hand-pinned overrides in the Python dispatcher are modified in this PR.
8 N/A No cohort definitions or spec override predicates are added or modified in this PR.
9 N/A No new KernelCandidate or architecture is registered in the Python dispatcher registries.
10 PASS Resolved since previous review: dense_q and seqused_q are now added to AttentionRequest (dnn-providers/hip-kernel-provider/rocke/library/dispatch/attention/common.py:213), wired into _prefill_spec…
11 N/A No new dispatcher candidates are registered or promoted in the Python dispatcher registries.
12 N/A No candidate priorities, candidate names, or custom rankers are added or modified in the Python dispatcher registry.
13 N/A No new dispatcher wrapper or entry point is added, and pinned request handling in CandidateRegistry.select is not modified.
14 PASS Adding block_table_v and sink_is_f32 to the prefill kernarg layout bumps ATTENTION_GFX1151_ABI to rocke-attention-gfx1151/v8 in dnn-providers/hip-kernel-provider/rocke/library/dispatch/attention/gfx1151.py:100 (tested in…
15 N/A No new dispatcher candidate or arch module is added to the Python dispatcher registry.
16 N/A No new operator family is introduced in this PR.
17 PASS CPU routing and capability tests in dnn-providers/hip-kernel-provider/rocke/library/tests/dispatch/attention/test_gfx1151_attention_selection.py:798-873 verify prefill selection across RDNA arches, decode refusal via admits(), capability rejection on out-of-cohort…
18 N/A Registry ratchets (require_build, require_binding, require_torch_binding) and exempt sets are not modified in this PR.
19 N/A This PR does not touch run_unified_attention_torch or make claims about unified attention auto-routing registration.
20 PASS Spec and builder changes in WmmaFmhaFwdV2Spec (dnn-providers/hip-kernel-provider/rocke/library/kernels/gfx1151/wmma_fmha_fwd_v2.py:128-129) and rdna_attention.py (dnn-providers/hip-kernel-provider/rocke/platform/python/rocke/helpers/rdna_attention.py:213-220)…
21 N/A This PR adds new kernel/spec features (dense_q, seqused_q, block_table_v, sink_is_f32, 64-bit page addressing) and hipDNN…
§E graded 17/17
# Verdict Evidence
1 PASS dnn-providers/hip-kernel-provider/rocke/platform/python/rocke/helpers/rdna_attention.py:48 imports only from rocke.core.ir, while library kernels import from platform (dnn-providers/hip-kernel-provider/rocke/library/kernels/gfx1151/wmma_fmha_fwd_v2.py:470).
2 PASS dnn-providers/hip-kernel-provider/rocke/library/tests/test_wmma_fmha_fwd_v2_features.py:738 and 764 use rocke.runtime.hip_module (_resolve_hip and Runtime) rather than importing torch.
3 PASS dnn-providers/hip-kernel-provider/rocke/library/tests/test_wmma_fmha_fwd_v2_features.py:755 gates optional dependencies with pytest.importorskip and uses numpy/Runtime without requiring torch.
4 PASS dnn-providers/hip-kernel-provider/rocke/library/kernels/gfx1151/wmma_fmha_fwd_v2.py:469-473 and dnn-providers/hip-kernel-provider/rocke/library/dispatch/attention/bindings.py:611-612 add no upward imports across library layers.
5 PASS dnn-providers/hip-kernel-provider/rocke/platform/python/rocke/helpers/rdna_attention.py:213-220 and dnn-providers/hip-kernel-provider/rocke/platform/cpp/bindings/prove_parity_binding.py:813-860 introduce no absolute or repo-escaping paths or shell scripts.
6 PASS dnn-providers/hip-kernel-provider/rocke/library/kernels/gfx1151/wmma_fmha_fwd_v2.py:565-577 and dnn-providers/hip-kernel-provider/rocke/platform/python/rocke/helpers/rdna_attention.py:568-633 show intentional emitter edits matched with their C++ twins rather than lint…
7 PASS dnn-providers/hip-kernel-provider/rocke/platform/dsl_docs/instances/attention.md:38-41 and dnn-providers/hip-kernel-provider/rocke/library/dispatch/attention/README.md:492-547 update the instance and dispatch reference docs alongside the new dense_q, seqused_q, and…
8 PASS dnn-providers/hip-kernel-provider/rocke/library/kernels/gfx1151/wmma_fmha_fwd_v2.py:29-61 and dnn-providers/hip-kernel-provider/src/engines/kernel_ingestor_engine/packs/Gfx1151WmmaAttentionNative.cpp:83-119 use descriptive spec/RFC references without Jira IDs, PR…
9 PASS dnn-providers/hip-kernel-provider/rocke/library/kernels/gfx1151/wmma_fmha_fwd_v2.py:128-129 and dnn-providers/hip-kernel-provider/src/engines/kernel_ingestor_engine/packs/Gfx1151WmmaAttentionNative.cpp:762-767 keep all changes scoped to paged/ragged SDPA support in…
10 NEEDS-INFO Unresolved from previous review: merge-simulation against other open PRs touching the same files (e.g. abi.h, llvm_gfx_all.json, Gfx1151WmmaAttentionNative.cpp) still cannot be verified from the PR diff alone.
11 PASS dnn-providers/hip-kernel-provider/rocke/platform/dsl_docs/instances/attention.md:38-41 and dnn-providers/hip-kernel-provider/rocke/library/dispatch/attention/README.md:492-507 are required deliverable docs; no scratch notes or throwaway markdown files are committed.
12 PASS dnn-providers/hip-kernel-provider/rocke/library/tests/parity/gfx1151_wmma_fmha_fwd_v2_emit.py:235-240 and dnn-providers/hip-kernel-provider/rocke/library/tests/test_wmma_fmha_fwd_v2_spec.py:289 cover both gfx11-generic and gfx12-generic RDNA families (CDNA is scoped…
13 PASS dnn-providers/hip-kernel-provider/rocke/library/kernels/gfx1151/wmma_fmha_fwd_v2.py:565-577 and dnn-providers/hip-kernel-provider/rocke/platform/cpp/instances/gfx1151/wmma_fmha_fwd_v2.cpp:516-531 replace page_rsrc with page_head and update k_rsrc/v_rsrc paged usage…
14 PASS dnn-providers/hip-kernel-provider/rocke/platform/cpp/include/rocke/abi.h:111-116 and dnn-providers/hip-kernel-provider/rocke/library/kernels/gfx1151/wmma_fmha_decode.py:751-777 preserve upstream history and comments while surgical additions/refactorings are applied.
15 PASS dnn-providers/hip-kernel-provider/rocke/platform/python/rocke/helpers/rdna_attention.py:263-270 and 577-633, plus dnn-providers/hip-kernel-provider/rocke/platform/cpp/helpers/rdna_attention.cpp:714-818, use all newly added parameters (block_table_v, k_ptr, v_ptr…
16 N/A This item governs reviewer severity labeling and does not apply when there are no FAIL rows in this section.
17 PASS dnn-providers/hip-kernel-provider/rocke/library/dispatch/attention/README.md:492-547 and the PR description contain no absolute or relative performance claims.
§F graded 11/11
# Verdict Evidence
1 PASS Branch users/dpeabody/hipdnn-rdna-paged-attn matches users/<user>/... (PR metadata; diff context at dnn-providers/hip-kernel-provider/rocke/library/dispatch/attention/README.md:489).
2 PASS title feat(hip-kernel-provider): paged/ragged SDPA in Gfx1151WmmaAttention matches type(scope): summary and is 68 characters
3 PASS description is at least 30 characters and contains an issue reference
4 PASS test file dnn-providers/hip-kernel-provider/rocke/library/tests/dispatch/attention/test_attention.py is in the diff
5 PASS All changed files in the diff are source, test, documentation, or descriptor/golden files (e.g., dnn-providers/hip-kernel-provider/rocke/library/dispatch/attention/README.md:492) with no keys, certs, or secrets.
6 NEEDS-INFO Local pre-commit execution status before push cannot be verified from the diff alone.
7 NEEDS-INFO Bot, Copilot, and human review thread states and deferred-item ticket IDs are not included in the prompt or diff.
8 N/A This PR targets develop directly (Base: develop) rather than being a stacked PR on a feature branch.
9 PASS The PR keeps the full kernel, dispatch, native pack, catalog, and test changes together without artificial splitting (e.g., dnn-providers/hip-kernel-provider/src/integration_tests/kernel_ingestor_engine/IntegrationGpuGfx1151WmmaAttentionPaged.cpp:1).
10 PASS Tests added in the diff exercise real spec validation, dispatch binding, I32 extent guards, catalog census, and GPU numeric/integration behavior (e.g., `dnn-providers/hip-kernel-provider/rocke/library/tests/dispatch/attention/test_gfx1151_attention_dispatch_numeric.py…
11 NEEDS-INFO Although the title and body appear in the prompt metadata, live gh api / gh pr view verification of the GitHub API fields cannot be confirmed from the diff.
§G graded 34/34
# Verdict Evidence
1 PASS Parity rows in dnn-providers/hip-kernel-provider/rocke/library/tests/parity/gfx1151_wmma_fmha_fwd_v2_emit.py:97-109 are paired with float64/CPU on-device numerical tests in dnn-providers/hip-kernel-provider/rocke/library/tests/test_wmma_fmha_fwd_v2_features.py:170-247…
2 NEEDS-INFO On-device numerical tests with fp64/fp32 references and bf16/fp16 tolerances are present (e.g. dnn-providers/hip-kernel-provider/rocke/library/tests/test_wmma_fmha_fwd_v2_features.py:707,750 and dnn-providers/hip-kernel-provider/src/integration_tests/kernel_ingestor_engine/IntegrationGpuGfx1151WmmaAttentionPaged.cpp:324), but the PR body is truncated before the test execution summary so on-silicon pass confirmation…
3 NEEDS-INFO Golden hashes are updated in dnn-providers/hip-kernel-provider/rocke/platform/tests/instances/differential/golden/llvm_gfx_all.json (diff omitted) and parity tables are updated in dnn-providers/hip-kernel-provider/rocke/platform/cpp/bindings/prove_parity_binding.py:813-825, but the PR description is truncated before the Tests section so local gate sweep results (tools/run_checks.py, llvm23 byte-identity…
4 PASS Every new capability has targeted tests that fail without the change: dense_q/seqused_q/split block tables/f32 sinks/clamped seqused_k (dnn-providers/hip-kernel-provider/rocke/library/tests/test_wmma_fmha_fwd_v2_features.py:170-247), >4 GiB paged cache…
5 PASS Tests exercise the shipped entry points (e.g. dispatch_attention in dnn-providers/hip-kernel-provider/rocke/library/tests/dispatch/attention/test_gfx1151_attention_selection.py:798-872, hipDNN frontend Graph in…
6 PASS Prose claims in PR body, docstrings, and code comments match the implementation and tests on the new head: 64-bit page/head addressing and page-extent I32 bound (dnn-providers/hip-kernel-provider/rocke/library/dispatch/attention/bindings.py:683…
7 N/A This PR does not add or modify a design document defining a new fp32 parity reference oracle.
8 PASS Test harnesses do not silently fall back on unsupported dtypes or frontend configurations; they explicitly skip via pytest.importorskip or GTEST_SKIP with a reason (dnn-providers/hip-kernel-provider/rocke/library/tests/test_wmma_fmha_fwd_v2_features.py:755…
9 N/A This PR does not make new LDS or occupancy load-bearing numerical claims.
10 N/A This PR is a functional feature addition (paged/ragged SDPA support) and makes no performance or latency claims.
11 N/A This PR adds functional graph support for hipDNN paged SDPA rather than an optimization claim for a benchmarked shape.
12 PASS Cross-engine parity is maintained across Python and C++ for both the helper (dnn-providers/hip-kernel-provider/rocke/platform/python/rocke/helpers/rdna_attention.py:213 and dnn-providers/hip-kernel-provider/rocke/platform/cpp/helpers/rdna_attention.cpp:214) and…
13 N/A This PR does not introduce or recommend any new raw LLVM intrinsic mangled symbols.
14 N/A This PR makes no hardware bottleneck classifications.
15 N/A This is a review of the provided PR diff directly from the head commit, not a multi-worktree branch inspection task.
16 N/A This PR implements a completed feature rather than proposing an unverified plan.
17 N/A This PR does not add new standalone shipped CLI scripts.
18 PASS New host-side guard _check_page_extent_i32 in dnn-providers/hip-kernel-provider/rocke/library/dispatch/attention/bindings.py:683 is wired into _rdna_validate_and_collect (bindings.py:908), documents its scope (page and head-dim axes fit I32 while block and…
19 PASS Fixtures in dnn-providers/hip-kernel-provider/rocke/library/tests/test_wmma_fmha_fwd_v2_features.py:574 and dnn-providers/hip-kernel-provider/src/integration_tests/kernel_ingestor_engine/IntegrationGpuGfx1151WmmaAttentionPaged.cpp:178 allocate spare unreferenced…
20 N/A This PR does not introduce or modify a claim-ban detector or word/digit regex linter.
21 N/A This PR does not add or modify performance measurement or benchmarking scripts.
22 NEEDS-INFO dnn-providers/hip-kernel-provider/rocke/platform/tests/instances/differential/golden/llvm_gfx_all.json is omitted from the diff to fit the model budget, so we cannot verify from the diff alone that all new golden entries (indices 170..185 from dnn-providers/hip-kernel-provider/rocke/library/tests/parity/gfx1151_wmma_fmha_fwd_v2_emit.py:29) record bytes + sha256 without build errors across all flavors, nor…
23 N/A This PR is a functional feature addition for paged/ragged SDPA and makes no performance speedup claims.
24 PASS The re-blessed paged entries in dnn-providers/hip-kernel-provider/rocke/platform/tests/instances/differential/golden/llvm_gfx_all.json correspond to the 64-bit page/head addressing, seqused_k clamping, sink_is_f32, and block_table_v changes in…
25 N/A This PR adds functional support for hipDNN paged/ragged SDPA graphs and 64-bit cache addressing rather than a codegen…
26 N/A This PR does not include benchmark comparisons or performance claims.
27 N/A This PR does not include performance A/B experiments.
28 N/A This PR does not present performance delta tables or counterfactual benchmark arms.
29 N/A This PR does not swap guarded math primitives for fast approximations.
30 N/A This PR makes no register-spill or scratch-reduction claims.
31 N/A This PR does not claim to fix a performance defect against a baseline measurement.
32 N/A This PR does not report measured performance or resource numbers.
33 NEEDS-INFO dnn-providers/hip-kernel-provider/src/engines/kernel_ingestor_engine/packs/Gfx1151WmmaAttentionNative.cpp:1320 and dnn-providers/hip-kernel-provider/src/tests/engines/kernel_ingestor_engine/packs/TestGfx1151WmmaAttentionPaged.cpp are truncated/omitted in the diff, so the changed routing decision logic between prefill and decode on dense vs paged graphs cannot be fully inspected or verified against a…
34 PASS In dnn-providers/hip-kernel-provider/rocke/platform/python/rocke/helpers/rdna_attention.py:1065 (and C++ mirror dnn-providers/hip-kernel-provider/rocke/platform/cpp/helpers/rdna_attention.cpp:1693), empty and o_row_off for O are computed independently of…
§H graded 5/5
# Verdict Evidence
1 PASS Buffer SRDs and 64-bit page bases are sized from runtime tensor shapes/strides and host-validated before launch via _check_page_extent_i32 / _check_byte_extent_i32 in dnn-providers/hip-kernel-provider/rocke/library/dispatch/attention/bindings.py:683-705 and…
2 PASS Both block_table and block_table_v indirection values (phys_k, phys_v, phys) are explicitly range-checked on device by page_ok (0 <= phys < num_blocks) before 64-bit base pointer offsets are formed, clamping out-of-range page IDs to 0 and zeroing the…
3 PASS Power-of-two page_block_size is enforced in the spec (ps <= 0 or ps & (ps - 1) in dnn-providers/hip-kernel-provider/rocke/library/kernels/gfx1151/wmma_fmha_fwd_v2.py:169-170) and in the native matcher ((page & (page - 1)) != 0 in…
4 PASS The kernel loops over runtime n_tiles = cdiv(smax(sk, 0), BN) where sk = smin(seqused_k[b], seqlen_k) is clamped in dnn-providers/hip-kernel-provider/rocke/library/kernels/gfx1151/wmma_fmha_fwd_v2.py:538 (and C++ twin at…
5 PASS Host validations for dense_q, seqused_q, paged K/V extents, table capacity (seqlen_k <= capacity), and non-empty caches (num_blocks >= 1) are in dnn-providers/hip-kernel-provider/rocke/library/dispatch/attention/bindings.py:900-910, 1047-1174 and…
§I graded 6/6
# Verdict Evidence
1 N/A This PR adds paged/ragged SDPA kernel features and hipDNN native graph matching rather than introducing a tuning or routing search…
2 PASS Invalid spec combinations and unsupported graph features are loudly rejected with descriptive errors or logged causes (e.g., dnn-providers/hip-kernel-provider/rocke/library/kernels/gfx1151/wmma_fmha_fwd_v2.py:173…
3 PASS Parity, feature, and catalog census tests cover curated combinations and verify catalog invariants across the new dense_q, seqused_q, and split block-table axes (dnn-providers/hip-kernel-provider/rocke/library/tests/parity/gfx1151_wmma_fmha_fwd_v2_emit.py:97…
4 N/A This PR does not add or modify benchmark reporting tables comparing shipped vs. achievable selection performance.
5 PASS Generated catalog descriptors are updated alongside census tests checking page sizes and architectures against the loaded pack set (dnn-providers/hip-kernel-provider/src/engines/kernel_ingestor_engine/descriptors/rocKE/gfx1151_wmma_attention/gfx1151_wmma_attention.kmd.…
6 PASS The new helper rdna_attention_scaled_ptr exposes a general 64-bit scaled pointer arithmetic primitive rather than a hardware-instruction-specific op (dnn-providers/hip-kernel-provider/rocke/platform/python/rocke/helpers/rdna_attention.py:213…
§J graded 7/7
# Verdict Evidence
1 N/A The Markdown changes in dnn-providers/hip-kernel-provider/rocke/library/dispatch/attention/README.md and…
2 PASS Symbols introduced in dnn-providers/hip-kernel-provider/rocke/library/dispatch/attention/README.md:492-547 and dnn-providers/hip-kernel-provider/rocke/platform/dsl_docs/instances/attention.md:38-41 (block_table_v, dense_q, seqused_q, use_sinks/sinks) are…
3 PASS dnn-providers/hip-kernel-provider/rocke/platform/dsl_docs/instances/attention.md:38-41 and dnn-providers/hip-kernel-provider/rocke/library/dispatch/attention/README.md:492-547 state only the interface and catalog delta and link to…
4 PASS dnn-providers/hip-kernel-provider/rocke/library/dispatch/attention/README.md:495-547 and dnn-providers/hip-kernel-provider/rocke/platform/dsl_docs/instances/attention.md:38-41 accurately attribute block_table_v (new in v8), dense_q, seqused_q, and paged…
5 N/A The PR makes no algorithmic scaling or asymptotic complexity claims in documentation or comments.
6 PASS Catalog constants in dnn-providers/hip-kernel-provider/rocke/library/dispatch/attention/README.md:525-528 (buckets 64/96/128/256, page sizes 16/32/64) match GFX1151_WMMA_ATTENTION_PAGE_SIZES in…
7 N/A No new Markdown design or algorithm document files are added in this PR; only existing README.md and attention.md files are…
§K graded 13/13
# Verdict Evidence
1 N/A No tuned candidate definitions or make_tuned_candidate calls are added or modified in this PR.
2 PASS New WmmaFmhaFwdV2Spec fields dense_q and seqused_q (dnn-providers/hip-kernel-provider/rocke/library/kernels/gfx1151/wmma_fmha_fwd_v2.py:128) are explicitly classified in WMMA_PROBLEM_FIELDS…
3 PASS common.py omits dense_q and seqused_q when False so existing pins keep their spec_hash without a migration (dnn-providers/hip-kernel-provider/rocke/library/dispatch/attention/common.py:1024), and test_gfx1151_attention_selection.py verifies both…
4 N/A KnobSpace.defaults() and recorded() are not touched; the new spec fields are problem-level layout options in…
5 N/A No knob drop, inertness, or refusal rules are modified in this PR.
6 N/A KnobSpace.validate and tuning_specs.py are not modified in this PR.
7 N/A KnobSpace.base(req) is not modified in this PR.
8 N/A Knob value normalization and type coercion are not modified in this PR.
9 PASS Replay verification via result.kernel_id.spec_hash is preserved for stored prefill pins (dnn-providers/hip-kernel-provider/rocke/library/tests/dispatch/attention/test_gfx1151_attention_selection.py:972).
10 N/A KnobSpace.find and bare tuning_id resolution are not modified in this PR.
11 N/A Sweep entry points and iteration helpers are not modified in this PR.
12 N/A Production knob stacks, accept_default, and auto candidate routing are not modified in this PR.
13 N/A No benchmark scripts or sweep harnesses are modified in this PR.
§L graded 10/10
# Verdict Evidence
1 N/A The diff does not add or modify any gfx architecture target or ISA backend registry.
2 N/A The diff does not add or modify any MMA or WMMA op_id or fragment layout map.
3 N/A The diff does not change MMA atom selection, fragment lane maps, or accumulator layouts.
4 N/A The diff does not add or change wave size handling or target wave-size queries.
5 N/A The diff does not touch CDNA/gfx1250 arch-specific recipes, transpose-LDS reads, or AsyncTileLoader.
6 N/A The diff does not add pipeline vocabulary to ArchTarget/core/arch or import CK Tile dispatcher modules.
7 N/A The diff does not add or change logical dtypes, bit packing routines, or FP8 dialect rules.
8 N/A The diff does not touch scaled-MMA contracts or scaled WMMA/MFMA emission.
9 N/A The diff does not introduce a new architecture target.
10 N/A The diff does not modify intrinsic arch-domain artifacts or LLVM flavor membership lists.
§M graded 10/10
# Verdict Evidence
1 PASS Host booleans (spec.dense_q, spec.seqused_q, live_rows is not None, row_paged) guard Python branches, while runtime SSA predicates use b.cmp_* with scf_if and select…
2 N/A The diff does not touch legalize_for_target or any target legalization pass.
3 PASS No wire format changes are made to ck.dsl.ir/v1, and kernel parameter order is identical between Python and C++ builders and verified by spec tests (dnn-providers/hip-kernel-provider/rocke/library/kernels/gfx1151/wmma_fmha_fwd_v2.py:419…
4 N/A The diff only bumps BINARY_ABI in portable_ir/src/abi.py and does not touch portable IR schemas, importers, magic division, or…
5 N/A The diff does not use or modify AsyncTileLoader or async DRAM-to-LDS operations.
6 PASS The early return is CTA-uniform (q_start vs span), tile-paged loads select page 0 (safe_k, safe_v) and use OOB buffer sentinels (at_tile), and 64-bit row-paged loads clamp invalid indices/offsets (blk, phys, within) to 0 before issuing global_load_vN…
7 PASS Buffer descriptors are constructed via b.buffer_rsrc / rocke_b_buffer_rsrc without hard-coding buffer-resource word 3 or using TemporalHint.STREAMING (dnn-providers/hip-kernel-provider/rocke/platform/python/rocke/helpers/rdna_attention.py:582…
8 N/A The diff does not add any new LLVM intrinsic declarations or modify backend selection logic.
9 PASS Public C headers under cpp/include/rocke/ keep their names and extern "C" declarations, and ROCKE_ABI_VERSION / BINARY_ABI are bumped to 13 for the struct layout changes (dnn-providers/hip-kernel-provider/rocke/platform/cpp/include/rocke/abi.h:116…
10 N/A The diff does not touch !dbg metadata emission, Debug Info Version flags, or DWARF consumers.
§N graded 18/18
# Verdict Evidence
1 PASS rdna_attention_scaled_ptr is added to dnn-providers/hip-kernel-provider/rocke/platform/python/rocke/helpers/rdna_attention.py:213 because both wmma_fmha_fwd_v2.py:472 and wmma_fmha_decode.py:607 share it, and it lands with its C++ twin…
2 NEEDS-INFO Resolved from previous NEEDS-INFO: both dnn-providers/hip-kernel-provider/rocke/library/kernels/gfx1151/wmma_fmha_decode.py:751 and dnn-providers/hip-kernel-provider/rocke/library/kernels/gfx1151/wmma_fmha_fwd_v2.py:569 (along with their C++ mirrors in dnn-providers/hip-kernel-provider/rocke/platform/cpp/instances/gfx1151/wmma_fmha_decode.cpp:1103 and…
3 N/A The diff does not touch or use SoftwarePipeline.
4 N/A The diff does not add or modify epilogue fusion or fused GEMM builders.
5 N/A The diff does not touch rocke.helpers.tiling or add any tiling API.
6 N/A The diff does not add or modify front-door/toolbox tiling helpers or LayoutStyle.
7 N/A The diff does not use helpers/tiling/ layout overrides or TileMma.
8 N/A The diff does not touch helpers/tiling/ LDS allocation or tiling memory verbs.
9 N/A The diff does not use helpers/tiling/ N-D tensor axis roles or FlowStage.
10 N/A The diff does not modify helpers/tiling/ emission, LDS staging, or MMA issue.
11 N/A The diff does not touch helpers/tiling/ traits, lds_swizzle, or derive_sched_group_counts.
12 N/A The PR makes no LDS bank-conflict or coalescing claims under helpers/tiling/.
13 N/A The diff does not touch WarpDistributionEncoding, make_tile_desc, or RegisterMapper.
14 N/A The diff does not use helpers/tiling/ verbs or add a tiling pipeline variant.
15 N/A The diff does not modify helpers/tiling/ descriptor allocation or swizzled bounds checks.
16 N/A The diff does not touch optimize_layout or evaluate_transform.
17 N/A The diff does not touch tiling_recorder.py, round-trip analysis, or tiling visualization.
18 N/A The diff does not touch InterleavedStyle or demo kernels under helpers/tiling/kernels/.
§O graded 16/16
# Verdict Evidence
1 N/A This PR does not modify core/verify.py, cpp/core/verify.cpp, address spaces, or required IR attributes.
2 PASS Early-exit scf.if guards properly terminate the then-region without emitting after the terminator in dnn-providers/hip-kernel-provider/rocke/library/kernels/gfx1151/wmma_fmha_fwd_v2.py:529 and…
3 PASS IR emission in dnn-providers/hip-kernel-provider/rocke/platform/python/rocke/helpers/rdna_attention.py:213 and dnn-providers/hip-kernel-provider/rocke/platform/cpp/helpers/rdna_attention.cpp:214 uses deterministic builder methods and ordered sequences without set…
4 PASS Module singletons (I16, I32, I64) and rocke_i16() / rocke_i32() / rocke_i64() are used in dnn-providers/hip-kernel-provider/rocke/platform/python/rocke/helpers/rdna_attention.py:55 and…
5 N/A This PR does not touch core/passes.py, cpp/core/passes.cpp, or op purity tables.
6 N/A This PR does not modify constant folding in either engine.
7 N/A This PR does not alter kernel attributes, CodegenPolicy attribute contracts, or cluster_dims.
8 NEEDS-INFO Resolved from previous review: CodegenPolicy and apply_codegen_policy are present in the current diff context at dnn-providers/hip-kernel-provider/rocke/library/kernels/gfx1151/wmma_fmha_decode.py:606 and dnn-providers/hip-kernel-provider/rocke/library/kernels/gfx1151/wmma_fmha_fwd_v2.py:469, and new spec variants receive distinct kernel names and cache entries…
9 N/A This PR does not touch waitcnt encoding, barrier lowering, or optimization_barrier.
10 N/A This PR does not modify lowerer attribute validation or _param_llvm_type.
11 N/A This PR does not change ArchTarget queries or target ID resolution in rocke.
12 N/A This PR does not modify target catalog width/size caps or add default arch or "gfx950" fallbacks.
13 PASS Modifying rocke_rdna_attn_params_t (dnn-providers/hip-kernel-provider/rocke/platform/cpp/include/rocke/helper_rocke.helpers.rdna_attention.h:135) and rocke_wmma_fmha_fwd_v2_spec_t…
14 PASS C headers and sources follow rocke_* snake_case and correspondence prefaces in dnn-providers/hip-kernel-provider/rocke/platform/cpp/include/rocke/helper_rocke.helpers.rdna_attention.h:14, and additions to…
15 N/A This PR does not touch engine fallback mechanisms, _resolve_llvm_flavor, lower_hip.py, or lower_cktile.py.
16 PASS Full Python and C++ instance and helper parity is maintained across dnn-providers/hip-kernel-provider/rocke/platform/cpp/instances/gfx1151/wmma_fmha_fwd_v2.cpp:144 and dnn-providers/hip-kernel-provider/rocke/platform/cpp/bindings/prove_parity_binding.py:813…
§P graded 14/14
# Verdict Evidence
1 PASS Builders keep the exact (spec, *, arch) signature (dnn-providers/hip-kernel-provider/rocke/library/kernels/gfx1151/wmma_fmha_fwd_v2.py:465, wmma_fmha_decode.py:604), and the new dense_q and seqused_q knobs are defaulted fields on WmmaFmhaFwdV2Spec…
2 PASS The kernarg signature order for sink_is_f32, seqused_q, and block_table_v is updated identically in Python (dnn-providers/hip-kernel-provider/rocke/library/kernels/gfx1151/wmma_fmha_fwd_v2.py:419-435) and C++…
3 PASS Test harnesses obtain launch geometry directly from wmma_fmha_fwd_v2_grid without recomputing it (dnn-providers/hip-kernel-provider/rocke/library/tests/test_wmma_fmha_fwd_v2_features.py:656-658, 894-896).
4 N/A This PR does not modify LDS allocation math or LDS capacity gates.
5 N/A This PR modifies RDNA attention kernels (wmma_fmha_fwd_v2 and wmma_fmha_decode), not WMMA/RDNA GEMM or…
6 N/A The paged prefill attention kernel requires zero workspace (`dnn-providers/hip-kernel-provider/src/integration_tests/kernel_ingesto…
7 N/A This PR does not touch Stream-K, persistent GEMM, GroupedGemmLauncher, or SUPPORT_MATRIX.md.
8 N/A This PR does not add or modify quantization helpers or MX GEMM scale decoding.
9 N/A This PR does not introduce or change numeric emulation or data-type decode paths.
10 N/A This PR does not add or modify example manifests (helpers/manifest.py or run_manifest).
11 NEEDS-INFO GPU tests properly synchronize the stream via rt.sync() before reading device outputs and before freeing buffers (dnn-providers/hip-kernel-provider/rocke/library/tests/test_wmma_fmha_fwd_v2_features.py:665-672, 903-912). — no diff line supports this verdict
12 N/A This PR does not touch COMGR/HIP library loading or fix a hipError launch failure.
13 PASS Host-side changes to the attention spec cache key (dnn-providers/hip-kernel-provider/rocke/library/dispatch/attention/common.py:1017-1026), paged K/V I32 page-extent guards (dnn-providers/hip-kernel-provider/rocke/library/dispatch/attention/bindings.py:683-704…
14 PASS No examples/ expected.json files are touched, and the shared 64-bit pointer offset helper rdna_attention_scaled_ptr is placed in dnn-providers/hip-kernel-provider/rocke/platform/python/rocke/helpers/rdna_attention.py:213-220 (with its C++ twin in…
§Q graded 7/7
# Verdict Evidence
1 N/A No Python test files were added or removed under rocke/platform/tests/ or rocke/library/tests/, so the installed rocke_pytest…
2 PASS New dense_q, seqused_q, and shipped paged catalog variants across gfx11-generic and gfx12-generic are added to the C and Python parity emitters and binding parity checker in…
3 PASS Cross-engine parity is checked in dnn-providers/hip-kernel-provider/rocke/platform/cpp/bindings/prove_parity_binding.py:813 and dnn-providers/hip-kernel-provider/rocke/library/tests/parity/gfx1151_wmma_fmha_fwd_v2_emit.py:97, while numeric correctness is verified…
4 NEEDS-INFO The diff for dnn-providers/hip-kernel-provider/rocke/platform/tests/instances/differential/golden/llvm_gfx_all.json is omitted from the provided diff, so re-blessing across every entry in LLVM_FLAVORS (llvm20, llvm22, llvm23, llvm24, and canonical IR) cannot be verified from the diff lines.
5 PASS GPU tests check the active device and free memory and skip cleanly when unavailable or not RDNA (dnn-providers/hip-kernel-provider/rocke/library/tests/test_wmma_fmha_fwd_v2_features.py:748…
6 N/A This PR does not refactor any CI gate, differential runner, or parity harness execution logic.
7 N/A This PR does not change backend resolution or ROCKE_BACKEND / ROCKE_CPP_STRICT fallback rejection contracts.
§R graded 11/11
# Verdict Evidence
1 N/A This PR adds functional support for cuDNN paged/ragged SDPA semantics (dense_q, seqused_q, block_table_v, 64-bit page base…
2 N/A The diff changes no performance levers or arch-specific optimization recipes; dense_q and seqused_q are problem-fixed fields…
3 N/A No performance levers, tile sizes, LDS layouts, or prefetch pipelines are modified in this functional feature PR.
4 N/A This PR makes no performance or primitive-selection claims and changes no performance levers.
5 N/A The kernel does not use rocke.helpers.tiling and its per-lane 16-byte/8-byte vector load widths and striding patterns are…
6 N/A Neither LDS layouts nor MMA layouts are modified in this diff.
7 N/A The PR makes no bottleneck claims and does not target a performance bottleneck.
8 N/A No waits, barriers, or scheduling hints are added, removed, or modified in this diff.
9 N/A This PR is not a performance regression fix and does not trade performance across shapes via a tuning lever.
10 N/A The PR does not change compiler/codegen policies, claim assembly parity, or report benchmark measurements.
11 N/A No new optimization lessons, heuristics, case studies, or dsl_probes tools are introduced in this PR.
§S graded 8/8
# Verdict Evidence
1 PASS New public symbols and fields (rdna_attention_scaled_ptr in dnn-providers/hip-kernel-provider/rocke/platform/python/rocke/helpers/rdna_attention.py:61, rocke_rdna_attention_scaled_ptr in…
2 PASS Existing callers and tests for wmma_fmha_fwd_v2_signature, rdna_attention_fwd_body, AttentionRequest, and bind_rdna_prefill_torch are updated in the same PR with new expectations (e.g., sink_is_f32 and block_table_v in…
3 N/A The diff does not remove or rename any public API symbol.
4 PASS rdna_attention_scaled_ptr is added to __all__ in dnn-providers/hip-kernel-provider/rocke/platform/python/rocke/helpers/rdna_attention.py:65, resolves to the function defined at line 213, and is imported and exercised by…
5 PASS Tests assert both positive outputs (IR serialization, signature lists, cache hashes, and float64 reference numerics) and exact negative ValueError and TypeError exceptions on invalid parameters in…
6 PASS CPU-checkable contracts (spec validation, kernel names, signature ordering, dispatch argument collection, I32 extent guards, candidate selection, and cache key hashing) are tested without a GPU or torch in…
7 PASS The C/C++ twins (rocke_wmma_fmha_fwd_v2_spec_t, rocke_rdna_attn_params_t, rocke_rdna_attention_scaled_ptr, and the pybind11 bridge in dnn-providers/hip-kernel-provider/rocke/platform/cpp/bindings/rocke_engine.cpp:4663) are wired into the cross-engine parity…
8 NEEDS-INFO The PR body in the prompt is still truncated mid-sentence under the - **Native matcher, prefill engine** section before the test summary, so it is not possible to verify from the provided text whether the PR description states that tests/test_rocke.py and tests/run_all.py were run and names the GPU architecture tested.

This branch has not been deployed

No deployments
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Projects

None yet

Development

Successfully merging this pull request may close these issues.

7 participants