Repository navigation
fix(hipdnn): follow cuDNN semantics for the SDPA left diagonal-band bound - #14483
Merged
Merged
Conversation
✅ All Checks Passed — Ready for Review
📖 Need help? See the Policy FAQ for details on every check and how to fix failures. |
|
🎉 All checks passed! This PR is ready for review. |
1 of 6 tasks
Codecov Report✅ All modified and coverable lines are covered by tests. Additional details and impacted files@@ Coverage Diff @@
## develop #14483 +/- ##
===========================================
+ Coverage 77.25% 77.29% +0.04%
===========================================
Files 2889 2889
Lines 485084 484855 -229
Branches 73234 73231 -3
===========================================
- Hits 374739 374732 -7
+ Misses 85612 85389 -223
- Partials 24733 24734 +1
*This pull request uses carry forward flags. Click here to find out more.
🚀 New features to boost your workflow:
|
BradPepersAMD
force-pushed
the
users/brpepers/sdpa-left-bound-cudnn
branch
from
October 28, 2026 02:02
27e0c9e to
0526693
Compare
BradPepersAMD
force-pushed
the
users/brpepers/sdpa-left-bound-cudnn
branch
from
October 28, 2026 12:52
14897cc to
0526693
Compare
BradPepersAMD
added a commit
that referenced
this pull request
Oct 28, 2026
…r restacking on #14483 The reference PR and the left_bound fix (#14483) both move every component to cuDNN's left_bound (L keeps the L keys ending at the diagonal; kernels take window_left = L - 1; 0 and values below -1 are rejected). After restacking onto the fix, keep one implementation of each piece: - Frontend: one band check, detail::validateSdpaDiagonalBand (the fix's helper, extended to right_bound), called by SdpaFwdNode and SdpaBwdNode where the reference PR checked inline. Drop the fix's PreValidateChecksLeftBound tests; PreValidateChecksDiagonalBandBounds covers both bounds. - CPU reference: drop the duplicated isMasked comment. Keep the fix's LeftBoundKeepsLKeysIncludingDiagonalLikeCudnn (every row, two bounds) on the named-options API and drop LeftBoundCountsTheDiagonalKey; SEM-001 cites it. Fold the fix's unset-left_bound case into TestSdpaFwdPlanBand.LeftBoundZeroIsRejected and drop TestSdpaDiagonalBandParams.RejectsZeroLeftBound. - gfx1151 pack: BindsCudnnLeftBoundAsFlashAttentionWindowLeft already declines 0 and binds 1, so drop LeftBoundZeroIsDeclined. - Golden generators: --left-bound is the graph's (cuDNN) value and --window-left is FlashAttention's window_size_left (written as W + 1), mutually exclusive, in the forward and paged generators (shared helpers in sdpa_common.py); internal bounds are named left_bound and the metadata records config.left_bound. The backward generator already took --left-bound. generate_golden_data.sh passes the same graph values as --left-bound, so graphs and tensors are unchanged. The fix's CLI and cuDNN-oracle tests follow; README merges both sections. - Comments: left_bound = 0 is described by SEM-005 (hipDNN's domain) rather than "cuDNN rejects it / keeps no key". Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
1 task
adickin-amd
pushed a commit
that referenced
this pull request
Oct 28, 2026
…tor root (#14553) ## Summary The hip-kernel-provider census now passes the descriptor loader an absolute, normalized root. Before, it passed a path that climbs out of the test tree and back down. On Windows, that climb pushed the gfx1151 WMMA attention decode pack (#14412) over the 259-character `MAX_PATH` limit. The pack's descriptors never loaded, and `TestGfx1151WmmaAttentionDecodePacks` failed on the Windows gfx110X lane. JIRA ID : ALMIOPEN-3093 ## Problem On develop, TheRock Multi-Arch CI's Windows gfx110X hip-kernel-provider shards fail these four census cases: - `hkp-census-gfx1100-TestGfx1151WmmaAttentionDecodePacks` - `hkp-census-gfx1101-TestGfx1151WmmaAttentionDecodePacks` - `hkp-census-gfx1102-TestGfx1151WmmaAttentionDecodePacks` - `hkp-census-gfx1103-TestGfx1151WmmaAttentionDecodePacks` The error in each is: ``` descriptor set 'hipkernel:Gfx1151WmmaAttentionDecode' did not load ``` Because these cases fail, the required **Multi-Arch CI Summary** check is red on every PR that runs the hip-kernel-provider tests. Develop's own post-merge run [40611307596](https://github.com/ROCm/rocm-libraries/actions/runs/40611307596) fails the same cases. **Cause.** The census entry names its shard relative to its own directory: ``` census/<arch>/../../../../arch_content/... ``` Windows resolves every descriptor file as that directory plus the climb, so the climb counts against `MAX_PATH` even though the shard itself sits well within it. The decode pack has the longest file names in the tree. Its paths come to about 264-267 characters. The other packs come to about 253-257. ## Change `dnn-providers/hip-kernel-provider/src/tests/main.cpp` (census mode only): - After the existing check that the root is a directory, the census computes `std::filesystem::absolute(root).lexically_normal()`. - It sets `HIPDNN_DESCRIPTOR_DIR` to that value before any suite loads. - If `absolute()` fails, the census keeps the original root, so the old behavior is the fallback. This is the same hunk that #14111 already carries. That PR found it and validated it on a TheRock Multi-Arch run that passed on Windows gfx110X. It is split out here so it can land ahead of #14111. When #14111 rebases, its copy of the hunk becomes a no-op. ## Risks and mitigations | Risk | Mitigation | |---|---| | A census now loads a different tree | The normalized path names the same directory, so the set of files loaded is identical. Only the spelling of the path changes. | | Something other than the census is affected | The change is inside census mode in the test binary's `main()`. The plugin, the product loader and the non-census test runs are untouched. | ## Testing - The hunk is identical to the one in #14111, whose TheRock Multi-Arch run passed the Windows gfx110X census. - This PR's own TheRock Multi-Arch run should turn the four `TestGfx1151WmmaAttentionDecodePacks` census cases green on Windows gfx110X. ## Landing order This PR unblocks the Windows census for: - #14483 (ALMIOPEN-3024) - #14358 and #14111, which land after #14483 The separate rocKE golden mismatch on the same shards is fixed by #14501. 🤖 Generated with [Claude Code](https://claude.com/claude-code) Co-authored-by: Claude Opus 5.5 <noreply@anthropic.com>
…ound cuDNN's left_bound = L keeps the L keys ending at the diagonal, diagonal included (cudnn-frontend sliding_window_mask keeps col + L > row). hipDNN kept L + 1, because its references, provider window mappings and golden generator all used FlashAttention's window_size_left convention. Code written against the cuDNN API (set_sliding_window_length or set_diagonal_band_left_bound) therefore attended to one extra key per query. - References: the CPU dense/ragged and GPU dense/ragged SDPA references now mask skv <= sq + off - L. - Frontend: SDPA forward/backward validation rejects left_bound = 0 (and < -1), as cuDNN does; -1 stays "unbounded". The setters document the cuDNN meaning. - Providers: resolveDiagonalBand converts the graph's left_bound to the kernels' FlashAttention window (L - 1) for asm/AITER, hip_flash2 and gfx950; the gfx1151 pack converts the same way. Kernels and rocKE keep their FlashAttention convention. - Golden generator: --window-left stays FlashAttention-style, but the graph now carries left_bound = W + 1. - Shape miner: left_bound passes through as rocKE's sliding_window. - Tests: a cuDNN-rule oracle (from cudnn-frontend's fp16_ref.py) checks the generator's mask; the reference, provider, frontend and miner tests cover the new meaning and the rejection of 0. Behavior change: callers that passed FlashAttention-style values into left_bound must now pass W + 1. Co-Authored-By: Claude <noreply@anthropic.com>
- The gfx1151 decode-engine tests (from #13897) ask for a 1024-key window with left_bound = 1023, the old convention; under cuDNN's meaning that window is 1024. - The heuristics test's windowed graph used left_bound = 0, which is now invalid; use 1. - The GPU ragged plan's mask-attribute test now also rejects left_bound = 0. JIRA ID: ALMIOPEN-3024 Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
…erator --left-bound takes the graph's (cuDNN) left_bound and writes it unchanged; --window-left keeps FlashAttention's window_size_left (written as left_bound = W + 1) for --validate against AITER. The two are mutually exclusive, and each is named after the convention it uses. The README records the conversion. JIRA ID: ALMIOPEN-3024 Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
Cover the rejection in extractDiagonalBandParams directly: the frontend rejects the value first, so no graph built through it reaches the plan. JIRA ID: ALMIOPEN-3024 Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
left_bound = 0 keeps no key at or before the diagonal (it can still keep keys to its right), and cuDNN rejects it only in the backward pass and, before backend 9.10, the forward pass. hipDNN rejects it in both because a provider's L - 1 conversion would turn it into -1, which kernels read as unbounded. Comments and messages no longer say "keeps none" or "as cuDNN does". JIRA ID: ALMIOPEN-3024 Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
…eftBound clang-tidy bugprone-optional-value-conversion (warnings as errors in the Linux superbuild) flagged unwrapping and rewrapping the optional. Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
BradPepersAMD
force-pushed
the
users/brpepers/sdpa-left-bound-cudnn
branch
from
October 28, 2026 14:47
0526693 to
4555603
Compare
iq136boy
reviewed
Oct 28, 2026
iq136boy
reviewed
Oct 28, 2026
BradPepersAMD
added a commit
that referenced
this pull request
Oct 28, 2026
…r restacking on #14483 The reference PR and the left_bound fix (#14483) both move every component to cuDNN's left_bound (L keeps the L keys ending at the diagonal; kernels take window_left = L - 1; 0 and values below -1 are rejected). After restacking onto the fix, keep one implementation of each piece: - Frontend: one band check, detail::validateSdpaDiagonalBand (the fix's helper, extended to right_bound), called by SdpaFwdNode and SdpaBwdNode where the reference PR checked inline. Drop the fix's PreValidateChecksLeftBound tests; PreValidateChecksDiagonalBandBounds covers both bounds. - CPU reference: drop the duplicated isMasked comment. Keep the fix's LeftBoundKeepsLKeysIncludingDiagonalLikeCudnn (every row, two bounds) on the named-options API and drop LeftBoundCountsTheDiagonalKey; SEM-001 cites it. Fold the fix's unset-left_bound case into TestSdpaFwdPlanBand.LeftBoundZeroIsRejected and drop TestSdpaDiagonalBandParams.RejectsZeroLeftBound. - gfx1151 pack: BindsCudnnLeftBoundAsFlashAttentionWindowLeft already declines 0 and binds 1, so drop LeftBoundZeroIsDeclined. - Golden generators: --left-bound is the graph's (cuDNN) value and --window-left is FlashAttention's window_size_left (written as W + 1), mutually exclusive, in the forward and paged generators (shared helpers in sdpa_common.py); internal bounds are named left_bound and the metadata records config.left_bound. The backward generator already took --left-bound. generate_golden_data.sh passes the same graph values as --left-bound, so graphs and tensors are unchanged. The fix's CLI and cuDNN-oracle tests follow; README merges both sections. - Comments: left_bound = 0 is described by SEM-005 (hipDNN's domain) rather than "cuDNN rejects it / keeps no key". Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
… window literally The left_bound Doxygen on SdpaAttributes and SdpaBackwardAttributes said only "Must be >= 1, or -1 for unbounded" under "Follows cuDNN". It now names the two values that differ: -1 is unbounded here where cuDNN leaves the bound unset, and 0 is rejected in both directions where cuDNN 9.10+ accepts it in the forward pass, because a FlashAttention-window provider's L - 1 would read it as unbounded. ConvertsCudnnLeftBoundToWindowLeft restated the conversion it checks; it now compares against a literal table. Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
BradPepersAMD
added a commit
that referenced
this pull request
Oct 28, 2026
…r restacking on #14483 The reference PR and the left_bound fix (#14483) both move every component to cuDNN's left_bound (L keeps the L keys ending at the diagonal; kernels take window_left = L - 1; 0 and values below -1 are rejected). After restacking onto the fix, keep one implementation of each piece: - Frontend: one band check, detail::validateSdpaDiagonalBand (the fix's helper, extended to right_bound), called by SdpaFwdNode and SdpaBwdNode where the reference PR checked inline. Drop the fix's PreValidateChecksLeftBound tests; PreValidateChecksDiagonalBandBounds covers both bounds. - CPU reference: drop the duplicated isMasked comment. Keep the fix's LeftBoundKeepsLKeysIncludingDiagonalLikeCudnn (every row, two bounds) on the named-options API and drop LeftBoundCountsTheDiagonalKey; SEM-001 cites it. Fold the fix's unset-left_bound case into TestSdpaFwdPlanBand.LeftBoundZeroIsRejected and drop TestSdpaDiagonalBandParams.RejectsZeroLeftBound. - gfx1151 pack: BindsCudnnLeftBoundAsFlashAttentionWindowLeft already declines 0 and binds 1, so drop LeftBoundZeroIsDeclined. - Golden generators: --left-bound is the graph's (cuDNN) value and --window-left is FlashAttention's window_size_left (written as W + 1), mutually exclusive, in the forward and paged generators (shared helpers in sdpa_common.py); internal bounds are named left_bound and the metadata records config.left_bound. The backward generator already took --left-bound. generate_golden_data.sh passes the same graph values as --left-bound, so graphs and tensors are unchanged. The fix's CLI and cuDNN-oracle tests follow; README merges both sections. - Comments: left_bound = 0 is described by SEM-005 (hipDNN's domain) rather than "cuDNN rejects it / keeps no key". Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
BradPepersAMD
added a commit
that referenced
this pull request
Oct 29, 2026
#14483 makes the graph's left_bound mean what it means in cuDNN: L keeps the L keys ending at the diagonal. resolveMaskBounds, the one place every FlyDSL SDPA engine (RDNA prefill and decode, gfx1201 aiw, CDNA) reads it, now hands the kernels the FlashAttention window L - 1 and declines left_bound 0, as resolveDiagonalBand does for the asm engines. The device tests' host masks, the binding expectations and COVERAGE follow the same convention; reverting the conversion fails the RDNA window device tests. Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
brockhargreaves-amd
added a commit
that referenced
this pull request
Oct 29, 2026
…undles #14483 gave the SDPA left_bound the cuDNN meaning on develop (SEM-001): left_bound = L keeps the L keys ending at the diagonal, and 0 is illegal. The frontend, both references and ASM_SDPA_ENGINE moved with it. The generator still wrote develop's old form, left_bound = W - 1, so every window case described a window one key narrower than its id says, and the W = 1 cases wrote the now-illegal 0. WINDOW_SEMANTICS is now "after_sem001": the 117 window cases are regenerated with left_bound = W. Case ids, levels, tolerance classes and the TOML block are unchanged, and the claims still describe the same windows. The plan and the generator now cite #14483, which replaced the closed #14287, including in regression_of; the window-semantics test sets both meanings explicitly instead of assuming the default. Co-Authored-By: Claude Opus 5.5 (1M context) <noreply@anthropic.com>
15 of 16 tasks
BradPepersAMD
added a commit
that referenced
this pull request
Oct 29, 2026
…r restacking on #14483 The reference PR and the left_bound fix (#14483) both move every component to cuDNN's left_bound (L keeps the L keys ending at the diagonal; kernels take window_left = L - 1; 0 and values below -1 are rejected). After restacking onto the fix, keep one implementation of each piece: - Frontend: one band check, detail::validateSdpaDiagonalBand (the fix's helper, extended to right_bound), called by SdpaFwdNode and SdpaBwdNode where the reference PR checked inline. Drop the fix's PreValidateChecksLeftBound tests; PreValidateChecksDiagonalBandBounds covers both bounds. - CPU reference: drop the duplicated isMasked comment. Keep the fix's LeftBoundKeepsLKeysIncludingDiagonalLikeCudnn (every row, two bounds) on the named-options API and drop LeftBoundCountsTheDiagonalKey; SEM-001 cites it. Fold the fix's unset-left_bound case into TestSdpaFwdPlanBand.LeftBoundZeroIsRejected and drop TestSdpaDiagonalBandParams.RejectsZeroLeftBound. - gfx1151 pack: BindsCudnnLeftBoundAsFlashAttentionWindowLeft already declines 0 and binds 1, so drop LeftBoundZeroIsDeclined. - Golden generators: --left-bound is the graph's (cuDNN) value and --window-left is FlashAttention's window_size_left (written as W + 1), mutually exclusive, in the forward and paged generators (shared helpers in sdpa_common.py); internal bounds are named left_bound and the metadata records config.left_bound. The backward generator already took --left-bound. generate_golden_data.sh passes the same graph values as --left-bound, so graphs and tensors are unchanged. The fix's CLI and cuDNN-oracle tests follow; README merges both sections. - Comments: left_bound = 0 is described by SEM-005 (hipDNN's domain) rather than "cuDNN rejects it / keeps no key". Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
BradPepersAMD
added a commit
that referenced
this pull request
Oct 29, 2026
…r restacking on #14483 The reference PR and the left_bound fix (#14483) both move every component to cuDNN's left_bound (L keeps the L keys ending at the diagonal; kernels take window_left = L - 1; 0 and values below -1 are rejected). After restacking onto the fix, keep one implementation of each piece: - Frontend: one band check, detail::validateSdpaDiagonalBand (the fix's helper, extended to right_bound), called by SdpaFwdNode and SdpaBwdNode where the reference PR checked inline. Drop the fix's PreValidateChecksLeftBound tests; PreValidateChecksDiagonalBandBounds covers both bounds. - CPU reference: drop the duplicated isMasked comment. Keep the fix's LeftBoundKeepsLKeysIncludingDiagonalLikeCudnn (every row, two bounds) on the named-options API and drop LeftBoundCountsTheDiagonalKey; SEM-001 cites it. Fold the fix's unset-left_bound case into TestSdpaFwdPlanBand.LeftBoundZeroIsRejected and drop TestSdpaDiagonalBandParams.RejectsZeroLeftBound. - gfx1151 pack: BindsCudnnLeftBoundAsFlashAttentionWindowLeft already declines 0 and binds 1, so drop LeftBoundZeroIsDeclined. - Golden generators: --left-bound is the graph's (cuDNN) value and --window-left is FlashAttention's window_size_left (written as W + 1), mutually exclusive, in the forward and paged generators (shared helpers in sdpa_common.py); internal bounds are named left_bound and the metadata records config.left_bound. The backward generator already took --left-bound. generate_golden_data.sh passes the same graph values as --left-bound, so graphs and tensors are unchanged. The fix's CLI and cuDNN-oracle tests follow; README merges both sections. - Comments: left_bound = 0 is described by SEM-005 (hipDNN's domain) rather than "cuDNN rejects it / keeps no key". Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
BradPepersAMD
added a commit
that referenced
this pull request
Oct 29, 2026
…e options API DeprecatedCausalMaskWithLeftBoundIsSlidingWindow called the positional GpuFpReferenceSdpa::fprop that this PR replaces with GpuSdpaFwdOptions. Its comment also predates #14483: left_bound=2 now keeps 2 keys per row, as cuDNN does. Regenerate SUPPORT_MATRIX_OVERVIEW.md after the rebase. Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
Contributor
TheRock Submodule Bump ActivityNewest first
|
adickin-amd
added a commit
that referenced
this pull request
Oct 30, 2026
Brings in the SDPA backward O/LSE derivation (#14312), the cuDNN left-bound semantics (#14483), the SDPA kernarg range checks (#14075) and --benchmark (#13915). Conflicts: the outcome publishing moved into runBody() on this branch, so the benchmark report now sits there; TestFillInputs and the docs keep both sides.
1 task done
adickin-amd
pushed a commit
that referenced
this pull request
Oct 30, 2026
…ward (RDNA, CDNA) (#14111) JIRA ID: ALMIOPEN-2953, ALMIOPEN-2954, ALMIOPEN-2955 Supersedes #13712, #13713 and #13714. ## Summary Adds **FlyDSL** as a second kernel source for the hip-kernel-provider's kernel ingestor, beside rocKE: a pipeline that ships prebuilt FlyDSL code objects, and eight engines on it. | Area | Engines | Targets | |---|---|---| | RMSNorm | `hipkernel:flydsl_rmsnorm` (inference and training forward), `hipkernel:flydsl_rmsnorm_bwd` (dx, dgamma) | gfx11-generic, gfx12-generic, gfx90a, gfx942, gfx950 | | SDPA forward, RDNA | `hipkernel:flydsl_sdpa_rdna`, `hipkernel:flydsl_sdpa_rdna_decode`, `hipkernel:flydsl_sdpa_aiw` (gfx12 only) | gfx11-generic, gfx12-generic | | SDPA forward, CDNA | `hipkernel:flydsl_sdpa_cdna`, `hipkernel:flydsl_sdpa_cdna_decode`, `hipkernel:flydsl_sdpa_cdna_dualwave` (gfx950 only) | gfx90a, gfx942, gfx950 | ## How it works - **Prebuilt objects, no compiler at build or run time.** FlyDSL kernels are compiled ahead of time into code objects (`.hsaco`). `flydsl/gen_descriptors.py` generates their descriptors as a `FlyDSL/` bundle, and the provider's existing packer puts them in each arch's kpack shard. Neither the build nor the runtime needs FlyDSL, a kernel compiler or Python at run time. - **Objects live in DVC.** Each `FlyDSL/<op>/<target>/` directory (1,453 objects, about 29 MB) is a DVC output on the `ingestor` remote that rocKE's descriptors use; git holds the 22 `.dvc` sidecars, the op-level descriptors, the generators and docs. Fetch with `dvc pull -r ingestor dnn-providers/hip-kernel-provider/src/engines/kernel_ingestor_engine/descriptors/FlyDSL/*/*.dvc`. The provider's configure-time DVC check covers FlyDSL: a missing directory warns and that target ships no FlyDSL kernels; `HIPKERNELPROVIDER_REQUIRE_DVC_DESCRIPTORS=ON` (CI, release) makes it an error. - **Integrity.** Each `manifest.json` records every object's SHA256 and a digest of that op's own sources. `gen_descriptors.py --check` runs before every pack and fails on any mismatch, so a source edit without regeneration fails the build; `check_shards.py` verifies every shard after the pack. Regeneration is byte-reproducible with the pinned toolchain (`REGEN.md`). - **Generic targets.** RDNA objects are built once for `gfx11-generic` / `gfx12-generic` and copied into every member arch's shard. - **Runtime arguments.** Shapes, strides, head counts, scale, epsilon, mask bounds, bias strides and the LSE output are kernel arguments, so one object serves every shape of its class. Only tiling, head-dim tiers and measured RMSNorm widths are baked. - **Declines.** Each matcher gates every graph field it reads and declines everything else, so another engine gets the plan. `flydsl/COVERAGE.md` lists every accept and decline rule with its test. ## What each engine serves - **RMSNorm** (153 objects per target): bf16, f16, f32, and bf16/f16 with an f32 scale; training forward writes a per-row f32 inv_rms; any finite epsilon, fixed or pass-by-value; the frontend's normalized-axes rule (including cuDNN's `x {B*S, D, 1, 1}` layout). Backward is FlyDSL's deterministic two-stage kernel (dgamma bitwise reproducible). Baked widths were kept only where they measured at least 15% over the generic tier on two archs. - **SDPA RDNA:** bf16/f16 at head_dim 64/96/128/256 plus generic tiers for any multiple of 8 up to 256; causal at either corner, bands, sliding windows; additive bias (f32 or the query dtype, any broadcast); attention sinks; MLA 192/128 and wide single-KV-head decode (576/512, 512/512); MHA/GQA/MQA. - **SDPA CDNA:** bf16/f16 at head_dim 64/128/256 with bucket objects between (gfx90a also 96/160/192/224); MLA 192/128 and absorbed-MLA decode (576/512); bias and sinks; BSHD, BHSD and packed QKV strides. Each class ships a default tile plus measured companion tiles. - **Same answer on every family.** RDNA, CDNA and aiw give the same result at the edge cases (keyless rows, −inf and lowest-finite bias rows, NaN rows, sinks, decode splits), matching PyTorch and hipDNN's reference (`COVERAGE.md` §5.7). - **`left_bound`** follows hipDNN's cuDNN meaning (#14483): `L` keeps the L keys ending at the diagonal; the kernels receive `L - 1`, and 0 declines. ## Build flag and CI - `HIPKERNELPROVIDER_ENABLE_FLYDSL` is **OFF by default**, like rocKE, and takes effect only with `HIPDNN_ENABLE_KERNEL_INGESTOR=ON`. A default rocm-libraries build is unchanged and needs no `dvc pull`. TheRock turns FlyDSL on through its `HIPKERNELPROVIDER_ENABLE_FLYDSL` flag (default ON, ROCm/TheRock#9615) together with the ingestor (TheRock #9375), and can turn it off there; the hipDNN superbuild CI passes `-DHIPKERNELPROVIDER_ENABLE_FLYDSL=ON`. - FlyDSL engines and tests compile whenever FlyDSL content exists, independent of `GPU_TARGETS` (TheRock's multi-arch build ships one shard's host half to every arch); device tests decide at run time whether the device's shard is staged. Packing, the shard check and the census follow the requested archs. - `THIRD_PARTY_LICENSES.md` (FlyDSL Apache-2.0, AITER and AOTriton MIT) installs under `share/doc/hip-kernel-provider/flydsl/`; TheRock packages it through the `doc` component added by ROCm/TheRock#9476. - This PR's TheRock Multi-Arch CI builds FlyDSL, runs its device tests and runs the FlyDSL integration entries on Linux gfx94X and Windows gfx110X once its TheRock pin includes ROCm/TheRock#9615. ## Changes outside `flydsl/` - Provider `CMakeLists.txt`: the option, `add_subdirectory(flydsl)` under the ingestor gate, and the DVC check extended to FlyDSL. - `HkpPackaging.cmake`: `FlyDSL` in `HKP_DESCRIPTOR_FAMILIES`. - Ingestor: one pack-table entry and module cache per engine; natives under `kernel_ingestor_engine/packs/Flydsl*`, with shared headers for RMSNorm, RDNA SDPA, CDNA SDPA and both SDPA families. - `descriptors/FlyDSL/`: op-level descriptors, `.dvc` sidecars and per-target `SOURCE.md`. - Tests: `main.cpp` adds the product shard as an additive descriptor root when FlyDSL is active; one engine suite and one census suite per engine, plus a per-object SDPA suite; `TestContainer`'s engine bound raised from 8 to 32. - Integration suite: one external integration entry per FlyDSL engine and architecture (55), pinned with `--test-engine`, checked by the GPU reference executor against the shared RMSNorm, RMSNormBackward and SdpaFwd bundles, routed into the architecture's census shard and gated on FlyDSL content. Each entry's generated TOML skips every other FlyDSL architecture, so a family package (gfx1100–gfx1103) runs each suite once. Category YAMLs `hipkernel_flydsl_*_test_categories_integration.yaml`; the two 4096-token fp16 causal bhsd SdpaFwd cases run from the comprehensive tier up. Support claims recorded for gfx1151. - rocKE test fixes for two Windows CI failures: the GPU-reference worker's temporary directory removal retries while Windows releases a killed worker's handles, and `test_lowers_and_compiles_without_scratch` gets its own 180 s limit (it ran ~54 s of 60 s). `flydsl/` holds the vendored kernel sources at pinned commits (FlyDSL, AITER, AOTriton; every local modification listed in each file's header and in the attention READMEs), the generators, `gen_descriptors.py`, `tools/check_shards.py`, the drift-guard tests, and `README.md`, `REGEN.md` and `COVERAGE.md`. ## Risks | Risk | Mitigation | |---|---| | TheRock ships FlyDSL on merge | TheRock's `HIPKERNELPROVIDER_ENABLE_FLYDSL` flag switches it off; this PR's TheRock CI built and tested FlyDSL, and a run pinned to TheRock main passed on gfx942, gfx950 and gfx110X | | Prebuilt objects drift from their sources | Per-object SHA256 and per-op source digest checked before every pack; drift-guard tests prove the checks refuse tampered trees | | An engine runs a graph it cannot compute | Every read field is gated, with one decline test per rule, mutation-checked | | One object set per generic family | Objects verified as `gfx11-generic` / `gfx12-generic`; every member shard verified after pack; gfx1150 and gfx1153 have no device run (same objects as gfx1151 and gfx1100) | | A device suite passes with no objects staged | Device cases fail rather than skip on an arch that ships their family; shard censuses fail on an empty shard | | Selection between overlapping engines | Static per-object priority today; autotune measures every applicable engine and object; UHD heuristics are a follow-up | **Known deviation:** RMSNorm backward and the two split-KV decode engines launch a second kernel compiled into the same object, named in KMD metadata (`reduce_symbol`, `merge_symbol`), instead of RFC 0017 §15.1 `sources` entries. RFC 0021 §3 names this form as an accepted alternative (`COVERAGE.md` §8). ## Testing - **Strategy:** host tests for every matcher, score and decline rule, checked against compiled mutants; GPU launches against a double-precision host reference; a per-object suite that launches every shipped object (every shape class in its `Full/` tier); shard censuses; `gen_descriptors --check` before pack and `check_shards` after. - **CI:** all required checks green at `fcb674c8766`, with FlyDSL device tests on Linux gfx94X and Windows gfx110X (FlyDSL ON by default then). - **Device runs on the shipped objects:** gfx950, gfx942, gfx90a, gfx1201, gfx1200 and gfx1100 ran the default and `Full/` tiers, launching every object of their families, with 0 failures; gfx1151 runs the default tier. With an empty descriptor root every device case of the arch's families fails. - **Local, this head (gfx1151):** 841 FlyDSL, container and ingestor gtests passed (62 skipped need other GPUs); census, FlyDSL and name-validation CTests pass; the FlyDSL integration entries pass the quick tier (rmsnorm 12, rmsnorm_bwd 5, sdpa_rdna 14, sdpa_rdna_decode 13 cases verified, 0 failed). - **After the next TheRock pin move:** a Multi-Arch run of this head with the TheRock pin moved to TheRock main plus #9615 and ROCm/TheRock#9619 (test branch, not merged) built FlyDSL and passed every hipkernelprovider, hipDNN and integration job on gfx950, gfx942 and Windows gfx110X. The FlyDSL integration entries verified 9–12 cases per SDPA engine and 12/5 per RMSNorm engine, 0 failed; only the runner's own architecture's entries ran. - **Regeneration:** every op on every target is byte-identical to the shipped objects; `diff_upstream.py` shows only recorded modifications. - **Tiers:** quick, standard and comprehensive run everything but `Full/`; the full tier adds every object on every shape class. ## Follow-ups - Support claims for the architectures beyond gfx1151 (harvest from CI once the TheRock pin includes #9615), and UHD heuristics for the FlyDSL engines. - Move the two-launch objects to `hsaco` multi-launch `sources` entries now that #14215 has landed. - RMSNorm: bias/dbias, strided rows, fusions (residual add, FP8 output, zero-centered gamma), baked widths ≤ 2048, backward above 16384. - SDPA: padding/varlen, exact head-dim 96/160/192/224 objects on gfx942/gfx950, a d512 prefill object, the deferred performance items in `COVERAGE.md` §7.5, and upstreaming the kernel modifications. 🤖 Generated with [Claude Code](https://claude.com/claude-code) --------- Co-authored-by: Claude Opus 5.5 <noreply@anthropic.com>
BradPepersAMD
added a commit
that referenced
this pull request
Oct 30, 2026
…r restacking on #14483 The reference PR and the left_bound fix (#14483) both move every component to cuDNN's left_bound (L keeps the L keys ending at the diagonal; kernels take window_left = L - 1; 0 and values below -1 are rejected). After restacking onto the fix, keep one implementation of each piece: - Frontend: one band check, detail::validateSdpaDiagonalBand (the fix's helper, extended to right_bound), called by SdpaFwdNode and SdpaBwdNode where the reference PR checked inline. Drop the fix's PreValidateChecksLeftBound tests; PreValidateChecksDiagonalBandBounds covers both bounds. - CPU reference: drop the duplicated isMasked comment. Keep the fix's LeftBoundKeepsLKeysIncludingDiagonalLikeCudnn (every row, two bounds) on the named-options API and drop LeftBoundCountsTheDiagonalKey; SEM-001 cites it. Fold the fix's unset-left_bound case into TestSdpaFwdPlanBand.LeftBoundZeroIsRejected and drop TestSdpaDiagonalBandParams.RejectsZeroLeftBound. - gfx1151 pack: BindsCudnnLeftBoundAsFlashAttentionWindowLeft already declines 0 and binds 1, so drop LeftBoundZeroIsDeclined. - Golden generators: --left-bound is the graph's (cuDNN) value and --window-left is FlashAttention's window_size_left (written as W + 1), mutually exclusive, in the forward and paged generators (shared helpers in sdpa_common.py); internal bounds are named left_bound and the metadata records config.left_bound. The backward generator already took --left-bound. generate_golden_data.sh passes the same graph values as --left-bound, so graphs and tensors are unchanged. The fix's CLI and cuDNN-oracle tests follow; README merges both sections. - Comments: left_bound = 0 is described by SEM-005 (hipDNN's domain) rather than "cuDNN rejects it / keeps no key". Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
BradPepersAMD
added a commit
that referenced
this pull request
Oct 30, 2026
…e options API DeprecatedCausalMaskWithLeftBoundIsSlidingWindow called the positional GpuFpReferenceSdpa::fprop that this PR replaces with GpuSdpaFwdOptions. Its comment also predates #14483: left_bound=2 now keeps 2 keys per row, as cuDNN does. Regenerate SUPPORT_MATRIX_OVERVIEW.md after the rebase. Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
BradPepersAMD
added a commit
that referenced
this pull request
Oct 31, 2026
…r restacking on #14483 The reference PR and the left_bound fix (#14483) both move every component to cuDNN's left_bound (L keeps the L keys ending at the diagonal; kernels take window_left = L - 1; 0 and values below -1 are rejected). After restacking onto the fix, keep one implementation of each piece: - Frontend: one band check, detail::validateSdpaDiagonalBand (the fix's helper, extended to right_bound), called by SdpaFwdNode and SdpaBwdNode where the reference PR checked inline. Drop the fix's PreValidateChecksLeftBound tests; PreValidateChecksDiagonalBandBounds covers both bounds. - CPU reference: drop the duplicated isMasked comment. Keep the fix's LeftBoundKeepsLKeysIncludingDiagonalLikeCudnn (every row, two bounds) on the named-options API and drop LeftBoundCountsTheDiagonalKey; SEM-001 cites it. Fold the fix's unset-left_bound case into TestSdpaFwdPlanBand.LeftBoundZeroIsRejected and drop TestSdpaDiagonalBandParams.RejectsZeroLeftBound. - gfx1151 pack: BindsCudnnLeftBoundAsFlashAttentionWindowLeft already declines 0 and binds 1, so drop LeftBoundZeroIsDeclined. - Golden generators: --left-bound is the graph's (cuDNN) value and --window-left is FlashAttention's window_size_left (written as W + 1), mutually exclusive, in the forward and paged generators (shared helpers in sdpa_common.py); internal bounds are named left_bound and the metadata records config.left_bound. The backward generator already took --left-bound. generate_golden_data.sh passes the same graph values as --left-bound, so graphs and tensors are unchanged. The fix's CLI and cuDNN-oracle tests follow; README merges both sections. - Comments: left_bound = 0 is described by SEM-005 (hipDNN's domain) rather than "cuDNN rejects it / keeps no key". Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
BradPepersAMD
added a commit
that referenced
this pull request
Oct 31, 2026
…e options API DeprecatedCausalMaskWithLeftBoundIsSlidingWindow called the positional GpuFpReferenceSdpa::fprop that this PR replaces with GpuSdpaFwdOptions. Its comment also predates #14483: left_bound=2 now keeps 2 keys per row, as cuDNN does. Regenerate SUPPORT_MATRIX_OVERVIEW.md after the rebase. Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
BradPepersAMD
added a commit
that referenced
this pull request
Oct 31, 2026
…r restacking on #14483 The reference PR and the left_bound fix (#14483) both move every component to cuDNN's left_bound (L keeps the L keys ending at the diagonal; kernels take window_left = L - 1; 0 and values below -1 are rejected). After restacking onto the fix, keep one implementation of each piece: - Frontend: one band check, detail::validateSdpaDiagonalBand (the fix's helper, extended to right_bound), called by SdpaFwdNode and SdpaBwdNode where the reference PR checked inline. Drop the fix's PreValidateChecksLeftBound tests; PreValidateChecksDiagonalBandBounds covers both bounds. - CPU reference: drop the duplicated isMasked comment. Keep the fix's LeftBoundKeepsLKeysIncludingDiagonalLikeCudnn (every row, two bounds) on the named-options API and drop LeftBoundCountsTheDiagonalKey; SEM-001 cites it. Fold the fix's unset-left_bound case into TestSdpaFwdPlanBand.LeftBoundZeroIsRejected and drop TestSdpaDiagonalBandParams.RejectsZeroLeftBound. - gfx1151 pack: BindsCudnnLeftBoundAsFlashAttentionWindowLeft already declines 0 and binds 1, so drop LeftBoundZeroIsDeclined. - Golden generators: --left-bound is the graph's (cuDNN) value and --window-left is FlashAttention's window_size_left (written as W + 1), mutually exclusive, in the forward and paged generators (shared helpers in sdpa_common.py); internal bounds are named left_bound and the metadata records config.left_bound. The backward generator already took --left-bound. generate_golden_data.sh passes the same graph values as --left-bound, so graphs and tensors are unchanged. The fix's CLI and cuDNN-oracle tests follow; README merges both sections. - Comments: left_bound = 0 is described by SEM-005 (hipDNN's domain) rather than "cuDNN rejects it / keeps no key". Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
BradPepersAMD
added a commit
that referenced
this pull request
Oct 31, 2026
…e options API DeprecatedCausalMaskWithLeftBoundIsSlidingWindow called the positional GpuFpReferenceSdpa::fprop that this PR replaces with GpuSdpaFwdOptions. Its comment also predates #14483: left_bound=2 now keeps 2 keys per row, as cuDNN does. Regenerate SUPPORT_MATRIX_OVERVIEW.md after the rebase. Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
adickin-amd
pushed a commit
that referenced
this pull request
Nov 2, 2026
…PI (#14358) ## Summary This PR brings the hipDNN SDPA **CPU and GPU reference implementations** up to everything the hipDNN SDPA graph API can express, forward and backward. It also adds the tests and docs that let the references serve as the oracle every attention engine is compared against. cuDNN's semantics are the contract. Each place where hipDNN deliberately differs from cuDNN is recorded in `projects/hipdnn/docs/SdpaSemanticsRegister.md` (SEM-xxx). The design, the full support table and the testing strategy are in **`projects/hipdnn/docs/SdpaReferenceImplementations.md`**. Builds on #14483 (cuDNN's `left_bound` meaning: L keeps the L keys ending at the diagonal) and #14312 (SdpaBackward O and LSE derived from the CPU forward), both merged. JIRA ID : ALMIOPEN-2863 Related: ALMIOPEN-2928, AICK-1847 ## What the references support Both references (CPU `CpuFpReferenceSdpa`; GPU `GpuFpReferenceSdpa`, hipRTC kernels) and both graph executors support: **Forward (`SdpaAttributes`)** | Feature | Notes | |---|---| | dtypes | FP32, FP16, BF16; FP8 E4M3 / E4M3 FNUZ / E5M2 / E5M2 FNUZ inputs with BF16 or same-format fp8 output | | Layout, GQA/MQA, `D_qk != D_v` | any strides; independent `H_k` / `H_v` | | Additive bias | FLOAT/HALF/BF16, rank 1-4 broadcast | | Diagonal band | `left_bound` / `right_bound` / alignment and the deprecated causal flags | | Padding | `seq_len_q/kv`; padded rows give O = 0, LSE = -inf | | Paged K/V | K, V or both, each with its own block size; extent inferred in cuDNN's order when unset | | Ragged (RFC 0014) | native packed forward (BF16, FP8 E4M3); every other ragged graph, including backward and ragged Q over paged K/V, through a ragged-as-dense rewrite | | Attention sink | FLOAT `[1, H, 1, 1]` | | ALiBi | cuDNN slopes, top-left causal | | Block-sparse mask | cuDNN's UINT8 bitmask, one bit per 128x128 tile | | Dropout | explicit mask; Philox through the engine's `rng_dump` | | fp8 scaling | Q/K/V descales (scalar or per KV head); `scale_s` / `descale_s` / `scale_o`; `amax_s` / `amax_o` | | Outputs | LSE, Max, Sum_exp | **Backward (`SdpaBackwardAttributes`)**: a new GPU backward reference. Both references support FP32/FP16/BF16, GQA, `D_qk != D_v`, LSE or recompute, bias + dBias, band, padding, ALiBi, explicit-mask dropout, and ragged through the rewrite. **Reference-only**: logit softcap, and an opt-in GPU P-rounding mode that models provider kernels. **Not supported because the graph API lacks it**: backward Philox (no backward `rng_dump`), backward sinks / dSink, the fp8 backward node, and `score_mod`. **Structure**: each direction is one pipeline (score: scale, softcap, bias, ALiBi, band, block mask; then softmax with sink; then P @ V), configured by a named options struct. fp8, paging and ragged are thin layers around that core. The CPU and GPU references share no code, only the semantics. The graph-translation layer (`PlanUtils.hpp`, the ragged rewrite) is shared by both executors, and the dense CPU and GPU plans use one applicability gate per direction. ## Behavior changes - **Frontend validation follows cuDNN (user-visible).** `SdpaFwdNode` now rejects: `right_bound < -1`; `max_seq_len_kv` without paging; paging without both `seq_len_q` and `seq_len_kv`; page tables that are not INT32 `[B, 1, M, 1]`; M != `ceil(max_seq_len_kv / block_size)` when the extent is set; `seq_len_*` not INT32 `[B, 1, 1, 1]`; `sink_token` not FLOAT `[1, H_q, 1, 1]`. One-sided paging, per-side block sizes, an inferred extent and packed page tables stay allowed. Each rule cites cuDNN and has a decline test. In-repo callers that built such graphs are fixed. - **Breaking `test_sdk` change (deliberate).** The positional `CpuFpReferenceSdpa::forward` / `backward` overloads are removed in favor of `SdpaFwdOptions` / `SdpaBwdOptions` (`.setScale`, `.setMask`, `.setBand(left, right, topLeft)`; causal is `.setBand(-1, 0, topLeft)`). There is no forwarding shim on purpose: `left_bound` changed meaning (SEM-001), so a shim would silently change old callers' results. Every in-repo caller is migrated. - **gfx1151 WMMA attention (`Gfx1151WmmaAttentionNative.cpp` `maskFor`)**: the deprecated `causal_mask` / `causal_mask_bottom_right` flags compose with explicit bounds and alignment, as the references do, instead of overriding them. Graphs that set only the flags bind as before. ## Absorbed PRs (authorship kept) - **#14004** (EwanC, ALMIOPEN-2863): SDPA backward golden bundles and generator, extended with GQA, padding, dBias, explicit dropout and an FP64 autograd cross-check. - **#13935** (scerzh, ALMIOPEN-2928): CPU ragged SDPA forward plan. - **#14245** (dpeabody-amd, AICK-1847): the frontend paged / sink / seq_len validation, the GROUP-mode ASM test fix, and the paged golden generator. Its separate paged plans are not taken: paging goes through the dense reference path, so every other feature composes with it. Stacked work on #14245 (#14247, #14254) may need to follow the stricter `seq_len_q` rule. ## Testing How correctness is established (detail in `SdpaReferenceImplementations.md` §3): 1. **Simple by construction**: FP32, no tiling, three-pass softmax, one code path per feature. 2. **Closed-form oracles and properties**: uniform scores, single key, dead rows; GQA = repeated heads, band and block mask = -inf bias, padding = truncated batches, paged = dense, sink = extra zero-value key, ragged = per-batch dense. 3. **Finite-difference gradient checks** of the backward. 4. **CPU vs GPU parity** (`hipdnn_gpu_ref_tests`, `TestGpuSdpa*Plan`): every feature at small shapes with NaN-filled padding; Standard-tier medium sweeps up to 1100 keys; fp8 device codecs bit-exact against the host. 5. **PyTorch golden bundles** (`hipdnn_golden_data_tests` checks every bundle against both references, with no engine involved). The backward generator is FP64-autograd checked; the fp8-scaling and block-mask/ALiBi paths reproduce cuDNN's own reference code. Golden data is 78 DVC pointers (26.8 MiB, every bundle under 1 MiB, pushed to `hipdnn/golden-data`). 6. **Mutation testing**: injected reference and gate bugs each fail a test. 7. **Applicability**: every schema field is gated, with one decline test per rule; a graph is declined, never accepted and then thrown on. **Results on gfx942** (MI300X, `-Werror`; the current head is covered by CI): | Suite | Passed | |---|---| | test_sdk / frontend / public_frontend | 2609 (1 skipped, pre-existing) / 2247 / 487 | | gpu_ref | 13836 | | integration-tests unit | 1149 | | golden (every bundle with golden data, both references) | 220 | | ASM_SDPA_ENGINE SdpaBackward bundles under #14312's tolerances | 31 (41 graphs the engine does not accept) | | hip-kernel-provider / its SDPA integration / ASM SDPA external quick suite | 1393 / 45 / 26 | | flatbuffers_sdk / plugin_sdk / generator pytest | 272 / 1293 / 146 | **Devices**: gfx942 runs everything above. gfx950: no run. gfx1151 has host matcher tests only (no usable device node; see Known issues). Windows and the other families are covered by TheRock CI. GPU reference kernels are hipRTC source, so there is no `GPU_TARGETS`-dependent host configuration. **Test time**: the new medium sweeps and long bundles are Standard tier (~18 s). The GPU reference forward is faster than on `develop` (masked keys skip the dot product), with byte-identical outputs. ## Known issues - **fp8 scaling over 128 keys (SEM-024)**: cuDNN quantizes P per 128-key tile against a running max; the references are exact (and bit-exact against cuDNN's `fp8_ref`) up to 128 keys, and the bundles stay within that. - **gfx1151 (accepted risk)**: the `maskFor` composition change has host matcher tests only; a 13-graph device check is prepared for when a gfx1151 node is available. - Once this lands, #14004, #13935 and #14245 are closed as superseded. ## Pre-merge - [ ] CI green on the current head 🤖 Generated with [Claude Code](https://claude.com/claude-code) --------- Co-authored-by: Claude Opus 5.5 <noreply@anthropic.com> Co-authored-by: Ewan Crawford <ewan.crawford@streamhpc.com> Co-authored-by: Sergei Grigoriev <102019268+scerzh@users.noreply.github.com> Co-authored-by: dpeabody-amd <dpeabody-amd@users.noreply.github.com>
brockhargreaves-amd
added a commit
that referenced
this pull request
Nov 2, 2026
…undles #14483 gave the SDPA left_bound the cuDNN meaning on develop (SEM-001): left_bound = L keeps the L keys ending at the diagonal, and 0 is illegal. The frontend, both references and ASM_SDPA_ENGINE moved with it. The generator still wrote develop's old form, left_bound = W - 1, so every window case described a window one key narrower than its id says, and the W = 1 cases wrote the now-illegal 0. WINDOW_SEMANTICS is now "after_sem001": the 117 window cases are regenerated with left_bound = W. Case ids, levels, tolerance classes and the TOML block are unchanged, and the claims still describe the same windows. The plan and the generator now cite #14483, which replaced the closed #14287, including in regression_of; the window-semantics test sets both meanings explicitly instead of assuming the default. Co-Authored-By: Claude Opus 5.5 (1M context) <noreply@anthropic.com>
ThruptiRajLakshmanaGowda
added a commit
that referenced
this pull request
Nov 2, 2026
…A contract Addresses the rocKE merge monitor review on #13596 (N1-N3, §D.16). - N1: hipDNN's left_bound = L keeps the L keys ending at the diagonal since #14483. SdpaRequest and resolve_hipdnn_band reject left_bound = 0 as SdpaFwdNode does; the request oracle converts L to the array oracle's FlashAttention window once, through the new hipdnn_left_to_window. Tests now mask with a transliteration of hipDNN's isMasked instead of the shared keep(), plus #14483's hand-checked cases. The lowering note now says L passes to AttentionBand unchanged. - N2: _validate skips the batch check per paged side and the K/V seq_kv check when either side is paged, as SdpaFwdNode does, and applies validatePagedKv's page-table rules. s_kv is max_seq_len_kv (or cuDNN's inferred length) when paged. All 9 paged golden graphs from #14358 now translate. - N3: the request oracle moves from platform to tests/sdpa/request_reference.py and gates on req.features(); SdpaRequest uses canonical_arch; the window_right feature is renamed noncausal_window so it no longer collides with AttentionRequest.window_right. - §D.16: registered_sdpa_combos / sdpa_sweep_space / dispatch_sdpa_all are norm.py's wrappers over an empty SDPA_REGISTRY (CandidateRegistry), so dispatch_sdpa_all returns DispatchResults; SDPA_ABI_VERSION is dropped until the first candidate carries one. - from_hipdnn leaves empty paged/fp8 groups to SdpaRequest's normalization. Co-Authored-By: Claude <noreply@anthropic.com>
8 of 9 tasks
4 tasks done
ThruptiRajLakshmanaGowda
added a commit
that referenced
this pull request
Nov 3, 2026
…A bindings hipDNN's left_bound = L now keeps the L keys ending at the diagonal (#14483), the same count rocKE's sliding_window uses, so to_attention_request and asked_band pass it through instead of adding 1. Tests follow the contract #13596 landed with: - request_bridge lifts the case table's FlashAttention window W to left_bound = W + 1, so every case keeps the same keys (and the same kernels) as before; - the binding and case-table oracles convert back with hipdnn_left_to_window; - hand-written wiring bands move up by one to keep their key counts; - the window_right feature is now noncausal_window; - paged requests carry real page tables (new paged() helper), which the contract now requires. Co-Authored-By: Claude <noreply@anthropic.com>
ThruptiRajLakshmanaGowda
added a commit
that referenced
this pull request
Nov 4, 2026
Delete the stale sdpa_reference_from_request oracle (platform's copy, which predates #14483 and gives windowed rows one extra key; develop's tests/sdpa/request_reference.py is the one every test uses). Export attention_band/attention_problem/dense_scale_range as public names from dispatch.attention so the SDPA gates stop reaching into another package's private _attention_band/_problem and a kernel's _MIN_SCALE/ _MAX_SCALE lookup; add a test pinning dense_scale_range("gfx950"). Switch bind_unified.py's launch plan to torch_dtype()/empty_workspace()/ resolve_stream(), matching bind_dense.py, instead of getattr(torch, ...)/ torch.empty/torch.zeros/torch.cuda.current_stream(device); fix bind_shared's docstring claim that was no longer accurate. Consolidate the unified and dense gates' near-identical decline-reason tables into one family-templated table in bind_shared.py. This also fixes a real inconsistency it was hiding: bias/bias_broadcast/alibi are now tagged NOT_YET_IMPLEMENTED on both families (nothing about the dense hardware rules it out either, so it was never a permanent limit like the comment claimed). Name every unified route candidate's algorithm as a module constant instead of a literal, export their union as dispatch.attention.UNIFIED_ALGORITHMS, and build bind_unified's _UNIFIED_ALGORITHMS from it so a renamed algorithm breaks an import instead of silently orphaning a string nobody matches. Add a coverage test partitioning every ATTENTION_ROUTE_REGISTRY candidate into unified/dense/tuning/not-served-by-SDPA. Restore the dropped KERNEL_AUTHORING.md pointer in dispatch/sdpa/__init__.py and add the doc's missing binding-layer checklist for a family routed over dispatch.attention (new Appendix H), the known SDPA routing gaps, and a library/dispatch/sdpa/ row in the library-layout table. Not changed: the unified binding still copies K/V into the paged cache every launch rather than viewing them in place, since the in-place path would drop the zero-padding guarantee for rows past a sequence's real length and there is no GPU available here to validate it numerically. Verified: tests/sdpa (5617 passed / 4 skipped with torch, 5537 passed / 11 skipped without) and tests/dispatch/attention (796 passed / 253 skipped with torch, same without) both green before and after.
ThruptiRajLakshmanaGowda
added a commit
that referenced
this pull request
Nov 4, 2026
- N8/N12: the unified LSE gate was narrowed to split-KV only (_LSE_ALGORITHMS, the entry points' spec.path == "3d", and bind_unified's forced backend="3d"), but tiled 2D also writes LSE on gfx942/gfx950 (generic.py's _reject_lse). Each unified route/tuning candidate's own Capability + support() already gate LSE correctly, so all three re-decisions are removed; the launch now always uses backend="auto" and defers to the kernel's own path selection. Admission grows from 202/261 to 283/405 cases on gfx942/gfx950. - N9: the seq_len bound check in bind_unified._lengths duplicated kernels.common._sdpa_seqlen's (now public) validate_lengths instead of reusing it, and its new tests only ran under torch. _lengths now calls validate_lengths directly, and tests/test_sdpa_seqlen.py pins the same six cases torch-free. - N9: test_sdpa_family_entry_points.py's _expected() mirrored _serves() exactly rather than pinning an independent outcome; added two tests on named candidates (gfx942 full attention reaches dense but not the unified tuning variants; gfx950 padded reaches unified but not dense) and a CPU test pinning kept_keys against the reference oracle's hipdnn_left_to_window at left_bound = 1, 4, 63, 64 (the #14483 off-by-one). - N10: attention_band / attention_problem were public wrappers around private _attention_band / _problem. Collapsed to one name each (internal callers now use the public name); attention_band returns an AttentionBand instead of a raw tuple. Exported DENSE_ALGORITHMS alongside the existing UNIFIED_ALGORITHMS so bind_dense.py's gate and the attention route-coverage test build from one source instead of re-listing the three dense algorithm constants. - Updated KERNEL_AUTHORING.md Appendix H and the PR description for the above (new public API, corrected admission counts, removed the "LSE only on split-KV" known gap). Co-Authored-By: Claude Sonnet 5 <noreply@anthropic.com>
brockhargreaves-amd
added a commit
that referenced
this pull request
Nov 5, 2026
…rated bundles (#14622) ## Summary This PR adds the SDPA backward test plan required by RFC 0022 (Per-Op Test Plans, merged in #13937). It also adds a generator that writes the SdpaBackward bundles from the plan, and the bundles it writes. The 16 inherited SdpaBackward cases (all B = H = 1, packed BHSD, fixed scale) are replaced by 503 generated cases in four levels. Each case carries a purpose and traces to the plan items it covers. > Written with AI assistance (Claude Code). The plan follows the merged RFC 0022; it records one remaining deviation (DV-4) for reviewers to approve. JIRA ID : ALMIOPEN-2894 ## Risk Assessment Risk 2 of 5. The PR changes test data, test tooling, test configuration and documentation only; no library or provider code changes. The changes that reach beyond the SdpaBackward bundles: - `support_matrix_units.collect_sweep_units()` leaves out cases whose `metadata.purpose` is `probe`, as RFC 0022 §9 requires. Today only SdpaBackward cases carry a `purpose`, so other ops' matrices are unchanged. The committed `SUPPORT_MATRIX_OVERVIEW.md` is re-rendered. - `ASM_SDPA_ENGINE.toml`: two dtype-only SdpaBackward tolerance rows are replaced by 12 generated rows (seven tolerance classes), plus temporary skips for the open engine defects, each naming its ticket. - `ASM_SDPA_ENGINE_test_categories_integration.yaml`: six rejection probes that ASM_SDPA_ENGINE wrongly accepts are quarantined with their tickets (RFC 0022 §6). Both files affect the ASM lane only. - #14358's eight PyTorch golden cases move, unchanged, from `quick/SdpaBackward/{Default,AdditiveMask}/` to `quick/SdpaBackward/Golden/{Default,AdditiveMask}/`, so the generator owns the `Default` sweeps. They keep their fp16 tolerance row. `bhsd/` and the `*_fwd_pair` bundles are untouched. The main risk is CI noise from new cases. Every level has been run on gfx942 and gfx950 with no failure (see Testing Checklist). ## ASIC Coverage The bundles run in every provider lane whose tier YAML matches `*Sdpa*`. Only `ASM_SDPA_ENGINE` accepts SdpaBackward graphs today, and only on gfx942: its backward plan builder declines every other device ("Device string does not match gfx942"). Claims are harvested for gfx942 only. PR CI tests gfx942. On gfx950 every generated backward case is declined, so a gfx950 run only shows that nothing fails; one is recorded below. PR CI on gfx942 is sufficient. ## Testing Summary - **Generator tests** (`hipdnn_sdpa_bwd_bundle_generator_python_tests`, label `quick`, 19 tests). They fail when the checked-in sweeps or the TOML block drift from the generator. They also check legality, all-pairs coverage up to nightly, the per-case ceilings and cumulative budgets of each level, two-way traceability between plan items and cases, that every TOML skip still matches a written case, and that the tier-YAML quarantine holds only rejection probes. - **`generate_sdpa_bwd_bundles.py --dry-run --report`.** Reports counts, budgets, the all-pairs proof, legality, and each declared-but-gated case with its gate. - **Claims run** (2026-10-26, MI300X, gfx942). All generated bundles ran on `ASM_SDPA_ENGINE`, configured like CI. `harvest_support_claims.py` turned the passing cases into the 272 claims. Every claimed case passes its tolerance class with a margin of at least 2.4 (median 4.2). - **Measured run, every level on GPU** (2026-11-04, plan §4.5). `hipdnn_integration_tests` ran on `ASM_SDPA_ENGINE` configured like the CI lane: `--test-engine` (claims enforced), the PR's `ASM_SDPA_ENGINE.toml`, the tier YAML's quarantine as a GTest filter, device input fill and the GPU reference as oracle. It ran all four levels, on a fresh MI300X (gfx942) and an MI355X (gfx950) node, built from this branch. The bundles, TOML and YAML are identical to `6e014ad50e0`. - **PR CI** runs the quick and standard levels on gfx942. ## Testing Checklist - [x] Generator tests - CTest `hipdnn_sdpa_bwd_bundle_generator_python_tests` - 19 of 19 - Status: Passed - [x] Plan report - `python3 reference-data-scripts/generate_sdpa_bwd_bundles.py --dry-run --report` - 503 written, 77 gated, 0 legality violations, all-pairs 669 of 669 - Status: Passed - [x] Bundle and support-matrix checks - `verify_support_claims.py`, `render_support_matrix.py --output -`, `feature_matrix.py --check`; the bundle, claim, render and support-feature pytest suites - Status: Passed (`verify_golden_bundles.py` reports no SdpaBackward error; its 322 errors in other ops' bundles are on develop too) - [x] pre-commit on the changed files - Status: Passed - [x] Claims run - `hipdnn_integration_tests`, `ASM_SDPA_ENGINE`, MI300X (gfx942), 2026-10-26 - Status: Passed - [x] Measured run, gfx942 - `hipdnn_integration_tests --test-engine ASM_SDPA_ENGINE --test-config ASM_SDPA_ENGINE.toml '--gtest_filter=*SdpaBackward*-<the tier YAML quarantine>'` - MI300X, Slurm job 68275259 - Status: Passed. 553 tests in 43.3 s, 0 failed, no claim broken: - 289 executed and passed against the GPU reference, and 15 expected rejections rejected; - 226 declined by ASM, and 23 hit the DEF skips; - per level: quick 2.4 s, PR 2.8 s, nightly 16.5 s, full 21.4 s. - [x] Quarantine check, gfx942 - the six quarantined cases run alone - Status: all six fail as expected ("Engine ASM_SDPA_ENGINE accepted expected-rejection bundle"), which shows the quarantine is needed. - [x] Every level, gfx950 - same command - MI355X, Slurm job 68275260 - Status: Passed. 0 failed: ASM declines every backward graph on gfx950, and the rejection probes, quarantined ones included, are correctly rejected. - [x] The other SDPA bundles, same lane - `--gtest_filter=*Sdpa*:-*SdpaBackward*` - Status: Passed. gfx942: 47 passed, 84 skipped. gfx950: 16 passed, 115 skipped. 0 failed on either. - [ ] PR CI on gfx942 - TheRock multi-arch CI - Status: Pending on `6e014ad50e0`. The 2026-11-02 run failed one case, `prbr01`, which is now gated. Later runs failed only on the hipCCL configure break from #14976, which develop reverted in #15089 (merged into this branch). ## Technical Changes ### What this adds - **The plan:** `dnn-providers/integration-tests/docs/test-plans/sdpa-bwd.md`. It follows the RFC 0022 template: the operation, the feature inventory, the shape catalog, building the sweep (levels, budgets, probes, regressions, trade-off register), the golden reference, pass criteria, and exhaustive mode. - **The generator:** `reference-data-scripts/sdpa_bwd_plan.py` holds the declarations: axes, legality, the shape catalog, level rules and the cost model, one record per case. `generate_sdpa_bwd_bundles.py` writes the sweeps, the support-claim files and the engine-TOML tolerance block from them. Nobody edits the bundles by hand. RFC 0022 open question 1 cites this module as the per-op Python option. - **The bundles:** 503 written cases, spread across the levels: | Level | Directory | Cases | |---|---|---| | quick | `quick/` | 56 | | PR | `standard/` | 152 | | nightly | `comprehensive/` | 220 | | full | `full/` | 75 | - Each case carries `purpose` (`claim`, `probe` or `regression`) and its plan items. - 77 more cases are declared but not written yet, each waiting on a named gate: an oracle extension, a harness feature or golden data. - One of the 77 is PRB-R01 (`dropout_probability = 0`). Since #14358 the references decline any graph that sets `dropout_probability`, so it waits on ALMIOPEN-3149 or ALMIOPEN-3150. - **Levels:** the cost model assigns each case to the lowest level whose per-case limits it fits: | Level | Per-case time | Per-case bytes | Cumulative budget | |---|---|---|---| | quick | 1.5 s | 4 MiB | 20 s | | PR | 5 s | 16 MiB | 120 s | | nightly | 75 s | 64 MiB | 480 s | | full | 300 s | no limit | 1,800 s | Each cumulative budget covers that level and every level below it, and keeps 5 % in reserve. The budgets come from a CPU-reference cost model; with the GPU reference, the measured run above is far below them. The ASM lane YAML picks the cases up by GTest prefix (`quick_*Sdpa*`, `standard_*Sdpa*` and so on), and each level also runs every level below it. - **Rejection probes** (21) carry `metadata.expected_rejection` (#14346): the harness never executes them, and an engine that accepts one fails the lane. PRB-M02 and PRB-M02f (`right_bound = -2`) are not written, because the frontend rejects them since #14358; `TestSdpaBwdNode.PreValidateChecksDiagonalBandBounds` covers that rule. - **dBias:** the five dBias probes and the live GL-14 case are written, now that #14358's references model dBias. To keep PR and nightly inside their reserve, the plan's trim rule moved M07.pr to nightly and M26.n to full. - **Tolerance classes:** seven classes, written as 12 `[[tolerance_overrides]]` rows between generated BEGIN and END markers in `ASM_SDPA_ENGINE.toml`. - **Support claims:** 272 claims for `ASM_SDPA_ENGINE` on gfx942. - **Probes left out of the support matrices** (RFC 0022 §9). This is the `support_matrix_units` change above. - **Generator file I/O is UTF-8**, so its tests pass on Windows. ### Temporary ASM_SDPA_ENGINE exclusions These stay until each fix lands; the PR with the fix deletes its entries. **Open engine defects** (`[[test_skips]]` in `ASM_SDPA_ENGINE.toml`). These are valid graphs that ASM accepts and computes wrong. Other engines still run them. | Cases | What goes wrong | Ticket | Fix | |---|---|---|---| | DEF-01 | head-dim stride other than 1 | ALMIOPEN-3036 | #14684 | | DEF-02 | non-packed Stats | ALMIOPEN-3037 | #14684 | | DEF-04 | Hk = Hq > Hv | ALMIOPEN-3039 | #14684 | | DEF-05 and H61 | odd element strides | ALMIOPEN-3038 | #14684 | | DEF-06 | scale of 0 or below | ALMIOPEN-3043 | #14688 | | DEF-03 | fp16 dV precision in the AITER kernel | ALMIOPEN-3046 | none in hipDNN (kernel) | **Quarantined rejection probes** (`_common_excludes` in `ASM_SDPA_ENGINE_test_categories_integration.yaml`, as RFC 0022 §6 asks). These are undefined graphs ASM should decline but accepts. | Cases | What ASM accepts | Ticket | Fix | |---|---|---|---| | prbu01, prbu03 | O or dQ dims that differ from Q's | ALMIOPEN-3039 | #14684 | | prbu13 | a dQ sequence stride below D (overlapping rows) | ALMIOPEN-3038 | #14684 | | prbt04, prbt04n | an O dtype that differs from Q's | ALMIOPEN-3050 | none yet | | prbr05 | `dropout_scale_inv` without a dropout mask | ALMIOPEN-3051 | none yet | ### Dependencies (all merged) - #13937, RFC 0022 (2026-10-30). Its first-phase documents, the methodology guide and the `docs/test-plans/` index, are not on develop yet, so the plan links the RFC's sections directly. - #14312: O and Stats derived from the forward of the same inputs. - #14483: `left_bound` uses cuDNN's meaning. The bundles write `left_bound = W`. - #14358 (2026-11-02): SDPA references cover the full graph API, and the GPU reference is the first oracle in `auto` mode. It also adds frontend rule R14 (`right_bound >= -1`). - #14346 (2026-11-03): the `expected_rejection` and `no_*_reference` bundle flags. - #14709 (2026-11-02): zero strides in the data SDK `Tensor`. ### Interaction with other open PRs - **#14684, #14686, #14687, #14688** (ASM declines, ALMIOPEN-3035). Each deletes its own entries: TOML skips for the DEF cases, and tier-YAML lines for the quarantined probes it fixes (#14684: prbu01, prbu03, prbu13). Whichever of these and this PR lands second removes the entries. - **#14999** (SDPA forward plan bundles, ALMIOPEN-2893). Both PRs edit two files the same way: - `ASM_SDPA_ENGINE.toml`: keep this PR's generated block and put #14999's `SdpaFwdPlan` skips after the `# END SdpaBackward tolerance classes` line. The generator test only checks the BEGIN to END span. The filters do not overlap (`SdpaBackward` vs `SdpaFwdPlan`). - `SUPPORT_MATRIX_OVERVIEW.md`: re-render it; do not hand-merge. The two PRs also add lines near each other in `integration-tests/CMakeLists.txt` and `test_categories_external.yaml`; these auto-merge. ### Before review - [x] `prbr01` (`dropout_probability = 0`) gated; ALMIOPEN-3149 and ALMIOPEN-3150 track the decision. - [x] Rejection probes carry `expected_rejection`; the six ASM accepts are quarantined in the ASM tier YAML with their tickets. - [x] `prbm02` and `prbm02f` removed; the frontend unit test of R14 covers them. - [x] The five dBias probes and GL-14 written; REG-08 (rank-3 Stats) still waits for the derivation extension. - [x] The plan refreshed against the merged RFC: DV-1 and DV-2 now agree with it, DV-3 is an RFC §11 gap, and only DV-4 remains a deviation. - [x] The measured run (plan §4.5) recorded above. 🤖 Generated with [Claude Code](https://claude.com/claude-code) --------- Co-authored-by: Claude Opus 5.5 (1M context) <noreply@anthropic.com>
This file contains hidden or bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
Sign up for free
to join this conversation on GitHub.
Already have an account?
Sign in to comment
Add this suggestion to a batch that can be applied as a single commit.This suggestion is invalid because no changes were made to the code.Suggestions cannot be applied while the pull request is closed.Suggestions cannot be applied while viewing a subset of changes.Only one suggestion per line can be applied in a batch.Add this suggestion to a batch that can be applied as a single commit.Applying suggestions on deleted lines is not supported.You must change the existing code in this line in order to create a valid suggestion.Outdated suggestions cannot be applied.This suggestion has been applied or marked resolved.Suggestions cannot be applied from pending reviews.Suggestions cannot be applied on multi-line comments.Suggestions cannot be applied while the pull request is queued to merge.Suggestion cannot be applied right now. Please check back later.
Summary
hipDNN's SDPA
left_bound(andset_sliding_window_length) now means what it means in cuDNN, whose API hipDNN mirrors:left_bound = Lkeeps the L keys ending at the diagonal, diagonal included. Until now every hipDNN component kept L + 1 keys (FlashAttention'swindow_size_leftconvention), so a graph written against the cuDNN contract attended to one extra key per query, forward and backward.JIRA ID : ALMIOPEN-3024
This supersedes #14287 by Daming Feng: his commit is included unchanged (same patch, authorship kept), rebased on current
develop, with follow-ups for code that landed since and a generator flag that names each convention explicitly.Description
cuDNN's left bound is a
CMP_GTdiagonal-band mask that keepscol + L > row(cudnn-frontendattn_score_modifiers.hsliding_window_mask; referencetest/python/sdpa/fp16_ref.pymasksrel <= diag_offset - left_bound). Example,L = 2, query 5, TOP_LEFT: cuDNN keeps keys {4, 5}; hipDNN kept {3, 4, 5}. The right bound already matched. Nothing caught it because every check shared hipDNN's convention: the golden generator built its own mask and cross-checked against AITER with the same value, the engine tests compared against our own CPU reference, and no golden bundle uses a left bound.L = 3, query on the diagonalleft_bound,set_sliding_window_lengthsliding_window = Wleft_bound = W)window_size_left/window_leftleft_bound = W + 1)Changes:
CpuFpReferenceSdpa(forward and backward),CpuFpReferenceSdpaRagged,GpuRefSdpaFwdandGpuRefSdpaRaggedFwdmaskskv <= sq + off - L.detail::validateSdpaLeftBoundinSdpaFwdNode/SdpaBwdNoderejectsleft_bound = 0and values below -1. Doxygen onset_diagonal_band_left_boundandset_sliding_window_lengthstates the cuDNN meaning and the FlashAttention mapping.extractDiagonalBandParamsand the GPUGpuSdpaRaggedFwdPlanreject 0.plan_utils::resolveDiagonalBandis the single conversion point;DiagonalBand::leftBoundholds the kernels' FlashAttention window (L - 1) for asm/AITER forward and backward, hip_flash2 and the gfx950 dense pack (a window that exactly covers the sequence is now correctly classified as causal).Gfx1151WmmaAttentionNative::maskFor(prefill and decode) converts the same way and declines 0. Kernels and rocKE keep their own conventions.generate_sdpa_fwd_golden.pygains--left-bound L(graph / cuDNN convention, written unchanged).--window-left Wkeeps FlashAttention's convention (written asleft_bound = W + 1) for--validateagainst AITER. The two are mutually exclusive; the README records the conversion. No checked-in bundle uses a left bound, so no golden data changes.left_boundand rocKE'ssliding_windowcount keys the same way, so the value passes through unchanged (wasleft + 1), and 0 is refused.left_bound = 1024for their 1024-key window; a heuristics test no longer builds aleft_bound = 0graph; the CPU and GPU plans' rejection of 0 is tested directly.An audit of every other file on
developthat mentionsleft_boundfound no other consumer of the raw value: the remaining plans and engines take the band already converted byresolveDiagonalBand/extractDiagonalBandParams, and the rest carry the field through (descriptors, packers, schema, Python bindings, corpus tools).Behavior change and migration
left_bound = Lnow attends to one fewer key than before. Callers that passed FlashAttention-style values straight intoleft_boundmust passW + 1. Out-of-tree providers that read the graph'sleft_boundmust convert to their kernel's convention (FlashAttention-style kernels takeL - 1).left_bound = 0and values below -1 are rejected; -1 still means unbounded. (cuDNN's frontend rejects a forwardleft_bound <= 0only on backends before 9.10 and a backward one always; hipDNN rejects 0 in both directions, because a provider'sL - 1would turn 0 into -1, which kernels read as unbounded.)flatbuffers_sdk/frontendversions is being discussed separately and is not part of this PR.Risk Assessment
Low. It is an intentional behavior change to a public SDPA attribute, but SDPA was off by default in every hipDNN build in TheRock until less than a week ago, so no released build has users relying on the old meaning. The in-tree consumers all change together in this PR.
resolveDiagonalBand(and gfx1151'smaskFor), each with tests that pinL -> L - 1,1 ->diagonal only,0 ->declined.LeftBoundKeepsLKeysIncludingDiagonalLikeCudnnpins the cuDNN rule by hand; the generator's mask is compared against an independent mask built from cudnn-frontend'sfp16_ref.pyrule (top-left, bottom-right,S_q != S_kv).--left-boundvs--window-left); the shape miner and rocKE agree on counting the diagonal.developafter #14287 still uses the old valuesleft_boundconsumer on currentdevelop; the gfx1151 decode tests (#13897) were the only ones affected and are updated.ASIC Coverage
The semantic change is arch-independent. Affected engines: asm/AITER and hip_flash2 (gfx942/gfx950) and the gfx1151 WMMA pack (gfx11/gfx12). The gfx950 dense pack declines windows and is unaffected. Validated on gfx942 (below); gfx950 and gfx1151 have no device run in this PR: their changes are the shared band resolution and
maskFor, which the host-side tests cover.Testing
New and updated tests:
TestCpuFpReferenceSdpaFp64.LeftBoundKeepsLKeysIncludingDiagonalLikeCudnn(L = 2, query 5 keeps {4, 5}; L = 1 keeps only the diagonal); existing sliding-window tests keep their hand-checked masks withleftBoundraised by one.TestSdpaDiagonalBandParams.RejectsZeroLeftBound(CPU plans) and aleft_bound = 0case inTestGpuSdpaRaggedFwdPlanBuilder.BuildRejectsInvalidMaskAttributes(GPU plan).TestSdpaFwdResolveDiagonalBand,LeftBoundZero/LeftBoundOneIsDiagonalOnlymask cases, covering-window boundaries (backward 64, hip_flash2 256 vs 255), gfx1151 binding (7 binds as 6, 1 as 0, 0 declined) and the decode-engine window cases.test_generate_sdpa_fwd_golden.py: the generator's mask against cudnn-frontend's rule;--left-bound 3and--window-left 2give the same graph; invalid flag combinations are rejected.Results, gfx942 (MI300X), this branch built with
-Werror,HIPDNN_ENABLE_SDPA=ON,HIPDNN_ENABLE_KERNEL_INGESTOR=ON:*Sdpa*:*Flash2*:*Gfx950Attention*:*Gfx1151WmmaAttention*)The CI CTest for the generator tests has no PyTorch in its environment and skips them; the PyTorch run above is what exercises the generator.
Related
🤖 Generated with Claude Code