fix(hip-kernel-provider): ASM SDPA backward declines layouts and dims its kernels address wrongly by brockhargreaves-amd · Pull Request #14684 · ROCm/rocm-libraries

brockhargreaves-amd · 2026-10-29T19:07:12Z

Motivation

ASM_SDPA_ENGINE backward accepts graphs that its gfx942 kernels address wrongly, and returns wrong gradients instead of declining. Some results change from run to run, and one case writes past the end of dV. This PR adds isApplicable declines for them. Tickets ALMIOPEN-3036 to ALMIOPEN-3040, epic ALMIOPEN-3035. The kernels are unchanged.

Technical Details

One commit per class of defect, plus one test fix:

Unaddressable layouts (ALMIOPEN-3036, 3037, 3038). Declines:
- a head-dim stride other than 1;
- a sequence stride below D;
- an odd stride on an axis that DQDKDV indexes in 4-byte records;
- stats (LSE) that are not packed [B, H_q, S_q(, 1)].
buildPlan now derives the LSE/D strides from Q's dims.
Mismatched dims (ALMIOPEN-3039). O, dO and dQ must have Q's dims, V, dK and dV must have K's dims, and K's batch must equal Q's. H_k = H_q > H_v used to write dV past its buffer.
Zero-stride outputs (ALMIOPEN-3040). Declines dQ, dK or dV with a zero batch or head stride on an axis longer than 1. Inputs with a zero batch or head stride (broadcasts), and a zero stride on a length-1 batch or head axis, stay accepted. A zero or short sequence stride is declined under item 1, on inputs too and on a length-1 sequence axis. Outputs whose nonzero strides overlap are still accepted, which is a known gap: a general footprint rule belongs in hipDNN graph validation.
Test fix. fix(hipdnn): range-check every SDPA kernarg narrowed to 32 bits #14075's IsApplicableBoundsTheTsProductByTheResolvedTileSize probed with an odd stride, which the new parity check also declines, so the test no longer covered the Ts guard. It now uses even probes, so deleting the guard makes it fail again.
Review follow-up. IntegrationGpuSdpaBackward gains B=2, H=2 cases, with no mask and top-left causal, both packed and with the non-packed layouts isApplicable accepts (Q, K, V and dO stored [B, S, H, D], O and dQ rows padded to D+1). Every earlier GPU case had B=1, H=1, where the LSE/D strides buildPlan derives from Q's dims are not discriminated. The unit tests probe each decline at more boundary values (head-dim stride 0, sequence stride 0, a short row at D=192, mismatched S and D, wrong rank or stride count), and stale comments are corrected.

The new checks run after #14075's dims gate. Each commit message has the GPU reproducers and the mutation results, which were run on develop 61c26758398, before #14075 and #14483 changed the backward builder and its test; the results for this branch alone are under Test Result.

Interaction with #14622 (draft SdpaBackward test plan): this PR lets #14622 delete its DEF-01, DEF-02, DEF-04, DEF-05/H61, prbu01/03 and prbu13 [[test_skips]] (20 cases). Whichever PR lands second deletes them. A leftover skip is not an error.

ASIC Coverage

Backward isApplicable accepts only gfx942 and declines every other device before the new checks, so the change affects gfx942 only. TheRock PR CI covers it on gfx94X-dcgpu (unit tests, provider integration tests and the ASM SDPA bundle lane). The gfx950 column below is supplementary: the backward tests skip there.

Test Plan

Unit tests and provider integration tests on gfx942 and gfx950.
develop's SDPA bundles and test(integration-tests): add the SDPA backward test plan and its generated bundles #14622's SdpaBackward bundles (temporary skips removed) on ASM_SDPA_ENGINE, configured like the CI lane.

Test Result

This branch alone at 7e19386ebce (the later 90d97a9691a changes test comments only), built with the hip-kernel-provider preset (-DGPU_TARGETS=gfx942 -DHIPDNN_ENABLE_SDPA=ON) and run on one MI300X (gfx942):

Command	Result
`hip_kernel_provider_tests --gtest_filter='TestSdpaBwd*'`	110 pass, 1 skip (`IsApplicableRejectsGfx950`), 0 fail
`hip_kernel_provider_integration_tests --gtest_filter='Smoke/IntegrationGpuSdpaBwd*'`	24 pass, 0 skip, 0 fail, the four new B=2, H=2 cases in bf16 and fp16 included
`ctest -R '^hip-kernel-provider-asm-sdpa-external-integration_standard_suite$'`	42 pass, 39 skip, 0 fail

With statsStrideBatch and statsStrideHead swapped in buildPlan, or with numHeadsQ dropped from the batch stride, exactly the eight new integration tests fail and the other sixteen pass.
With the three fixes' product changes reverted (host build, gfx942 device identity faked, no kernel launched), the five Rejects tests fail on all 98 decline cases; they pass at this head.
Host-side mutants of the dims-agreement check (heads only; B, H and S only), of the rank and stride-count guard, and of the head-dim stride check (!= 1 to > 1) each make a new unit test fail.

Combined build on develop 98b504d92c5 (after #14075; supplementary). The four ALMIOPEN-3035 ASM fix PRs (this PR, #14688, #14687, #14686) were built together for gfx942 and gfx950 and run on one MI325X (gfx942) and one MI355X (gfx950). Each branch was also checked on its own with a host-side runner of the real isApplicable and unit tests.

Check	gfx942 (MI325X)	gfx950 (MI355X)
`hip_kernel_provider_tests`	912 pass, 7 skip, 0 fail	886 pass, 33 skip, 0 fail
`hip_kernel_provider_integration_tests`	50 pass, 21 skip, 0 fail	23 pass, 48 skip, 0 fail
develop SDPA bundles, `ASM_SDPA_ENGINE` lane config, claims enforced	62 pass, 56 skip, 0 fail, no claim failure	15 pass, 103 skip, 0 fail, no claim failure
#14622 SdpaBackward bundles (`138218c0755`), its temporary skips removed	275 pass, 225 skip, 0 fail, no claim failure	not run (ASM backward is gfx942 only)

clang-tidy 20 (--warnings-as-errors='*') and clang-format 18 are clean on the changed files.

With test(integration-tests): add the SDPA backward test plan and its generated bundles #14622's skips removed, its 20 DEF-01/02/04/05/H61 and prbu01/03/13 cases are skipped as declined by ASM, and no other case changes.
The new isApplicable tests pass on gfx942: IsApplicableAcceptsAddressableLayouts, RejectsNonUnitHeadDimStride, RejectsMisalignedOrShortRows, RejectsNonPackedStats, RejectsMismatchedTensorDims and RejectsZeroStrideOutputs. Like the other backward isApplicable tests, they skip on gfx950.

Submission Checklist

Look over the contributing guidelines at https://github.com/ROCm/TheRock/blob/main/GOVERNANCE.md#pull-requests.

🤖 Generated with Claude Code

…youts SdpaBwdPlanBuilder::isApplicable accepted tensor layouts that the gfx942 backward kernels address wrongly. On gfx942 (MI300X) each one gave silent wrong gradients instead of a decline. The evidence comes from two sources. One is GPU runs of the plugin from a CI build of #14312, which accepts all of these layouts. The other is disassembly of the dispatched .co files. Q/K/V/O/dO/dQ/dK/dV strides. The ODO, DQDKDV and DQ_CONVERT argument blocks carry only batch, head and sequence byte strides. isApplicable now declines: - A head-dim (innermost) stride other than 1. The kernels read it as 1: a stride-2 Q gave Inf dK/dV, and a stride-2 dQ left elements unwritten. - A sequence stride below D, also on a length-1 sequence axis. Each buffer ends S * seq stride bytes into its (batch, head) slice, so the end of the last row reads as 0, and writes there are dropped. A seq stride of 1 on a length-1 axis gave wrong gradients for each of the eight tensors. Seq strides of D/2 and 2 did the same for Q, K, V and dO. - An odd stride on an axis whose offset DQDKDV puts in its 4-byte record index (byte offset >> 2). That is the sequence axis of Q, K, V, dO, dK and dV, and the head axis of dK and dV. Rows that start 2 bytes into a dword are read or written one element early. A seq stride of D+1 failed on each of the six. A dO row pitch of D+1 gave 47-97 % wrong dQ, dK and dV. Odd strides on the other axes are addressed correctly and stay accepted: - batch strides, which are added to the 64-bit buffer base; - head strides of Q, K, V, O, dO and dQ; - seq strides of O and dQ, which ODO and DQ_CONVERT address by byte offset. The causal kernels also add the dK/dV head offset to the base, but the check does not depend on the mask. Stats (LSE). All 28 dispatched batch-mode DQDKDV kernels compute the LSE and D offset as ((b * H_q + h) * S_q + s) * 4 and never load Hs_lsed. A stats head stride padded to S_q + 64 gave wrong gradients that changed between runs with identical inputs. A padded batch stride, a seq stride of 2 and [B, S_q, H] storage also gave wrong gradients. isApplicable now requires stats dims [B, H_q, S_q] or [B, H_q, S_q, 1] matching Q, with packed strides on every axis longer than 1. buildPlan sets the LSE/D strides from Q's dims, so the stride of a size-1 stats axis no longer reaches a kernel argument or the 32-bit stride check. These layouts stay accepted, all GPU-verified on gfx942: BSHD on all tensors, dO strides that differ from O, seq strides D+2 and D+8, B and H above 1, hd192, fp16, causal and sliding-window masks, BR causal with Sq above Skv, Sq in {1, 16, 17}, and tails. New tests in TestSdpaBwdPlanBuilder.cpp (gfx942 only, like the other isApplicable tests). Each decline arm has a case that only it declines, and each exemption has an accept case: - IsApplicableAcceptsAddressableLayouts - IsApplicableRejectsNonUnitHeadDimStride - IsApplicableRejectsMisalignedOrShortRows - IsApplicableRejectsNonPackedStats Checked on gfx942 (MI300X). This commit's plugin replaced the one in a CI build of #14312, and results were compared with the CPU reference: - Each declined layout above failed with the old plugin, which accepts it, and now declines. Each layout named above as addressed correctly passed with the old plugin and still passes. The probes cover hd128 and hd192, bf16 and fp16, and the no-mask, causal and sliding-window kernels. - The shipped bf16 tolerance hides some short-row errors, so those were also compared at atol=1e-9 with packed controls. Their error was 7-34x the controls'; for Q, the dQ error sits in the cut-off last row. - The develop quick/standard SdpaBackward bundles and the CI Sdpa bundles give the same results as before. - The unit tests pass. Each of 29 mutants that removes or widens one arm of these checks makes one of the new tests fail. Rebased onto develop 98b504d, after #14075, whose range gate now runs first in isApplicable. The GPU results above are from develop 61c2675. On the new base, the four ALMIOPEN-3035 ASM SDPA fix branches (backward layouts, attention scale, a16 accumulator, forward layouts) were built and run together: - hip_kernel_provider_tests: 912 pass and 7 skip on gfx942 (MI325X), 886 pass and 33 skip on gfx950 (MI355X); none fail. - hip_kernel_provider_integration_tests: none fail on either. - develop's SDPA bundles on ASM_SDPA_ENGINE, run like the CI lane (claims enforced): 62 pass and 56 skip on gfx942, 15 pass and 103 skip on gfx950, and no support claim fails. A host-side run of the real isApplicable over these bundles finds no outcome change from develop. - #14622's SdpaBackward bundles (138218c) with its temporary [[test_skips]] removed: 275 pass and 225 skip on gfx942, none fail, and no support claim fails. This branch's checks decline the 20 cases behind the DEF-01, DEF-02, DEF-04, DEF-05/H61, prbu01, prbu03 and prbu13 skips, and no other case (host-checked for this branch alone). - clang-tidy 20 and clang-format 18 are clean on the changed files. Co-Authored-By: Claude Opus 5.5 (1M context) <noreply@anthropic.com>

…r dims Every backward kernel takes B, H, S_q, S_kv and D once, from Q, K and V, and addresses all eight tensors with them. isApplicable compared only K's head count with Q's. The frontend lets V have fewer heads than K, because it checks only that both divide H_q. DQDKDV then reads V and writes dV for every Q head, past the last head of both. On gfx942 (MI300X), with H_q = H_k = 4 and H_v = 2 or 1, 51 % of dQ and dK and 87-92 % of dV were wrong. The other dims were not compared with Q's and K's either: the frontend checks only dO against O. isApplicable now requires O, dO and dQ to have Q's dims, V, dK and dV to have K's dims, and K's batch to equal Q's. The check runs after the GQA, asymmetric head-dim and head-dim checks, so those keep their own decline reasons. New test IsApplicableRejectsMismatchedTensorDims covers both of these graphs, plus one case per check that only that check declines. Checked on gfx942 (MI300X), with this commit's plugin in a CI build of #14312. Both of these graphs failed with the old plugin and now decline, and no other probe or bundle case changes from the previous commit. Of the 182 cases of an earlier gfx942 sweep, only the 4 head-dim stride and stats layout failures change: they now decline. Each of 7 mutants that removes one of these checks makes the new test fail. Co-Authored-By: Claude Opus 5.5 (1M context) <noreply@anthropic.com>

SdpaBwdPlanBuilder::isApplicable accepted dQ, dK and dV with a zero batch or head stride on an axis longer than 1. Such a graph has no well-defined result in general, because the slices that share memory hold different gradients. No layer rejects it: the frontend and backend check only that dims are positive and that strides are set, and isApplicable's stride checks pass a zero stride on these axes, which fits in 32 bits and is even. Each kernel launches its own workgroups for each (batch, head) slice (grid y and z are head and batch). DQDKDV writes dK and dV, and DQ_CONVERT (A32 path) writes dQ, with plain stores at batch * batch stride + head * head stride. With a zero stride several slices share memory, so each element ends up holding whichever slice was stored last. On gfx942 (MI300X), B=2, H=2 (H=3 where noted), S=256, bf16, three runs in each of two GPU jobs: - dK and dV with batch stride 0: each memory location held one batch's gradient, so half of the dK and dV elements differed from the packed run, and 17k-54k of them changed between runs with identical inputs. - dQ with batch stride 0: the same, with 33k-65k changing between runs. - dK and dV with head stride 0, H=3: two thirds differed from the packed run, and 23k-147k changed between runs. - dQ with head stride 0, H=3: two thirds differed from the packed run, and 64k-148k changed between runs. There was no NaN or Inf and no write into the guard regions around each buffer, and dQ was unaffected when only dK and dV aliased. isApplicable now declines dQ, dK and dV with a zero batch or head stride on an axis longer than 1. The sequence and head-dim axes are already covered: their strides must be at least D and exactly 1. These stay accepted: - Zero strides on the inputs Q, K, V, O and dO (broadcasts), which the kernels just re-read. On gfx942 these gave gradients that matched the CPU reference and were bitwise equal to the packed runs, in each of three runs: K and V together, or dO alone, with a zero batch stride, head stride or both; Q, O or V alone with a zero batch or head stride; Q and O together with a zero head stride; and all five inputs with a zero batch stride. The K and V batch broadcast also passed with fp16, with a top-left causal mask, and at S=64. - A zero stride on a length-1 axis, on outputs too, since it only ever multiplies index 0. On gfx942, dQ, dK and dV with batch stride 0 at B=1, or head stride 0 at H=1, passed the same way. Nonzero strides that make an output overlap itself are not checked. On gfx942, dK and dV with equal batch and head strides (B=H=2) are still accepted. They differed from the packed run in a quarter of their elements, 6% of dK and 17% of dV were outside the CPU reference's bf16 tolerance, and the result was not always the same from run to run. The zero stride is checked on its own because it is how a broadcast is written, so an output gets one when it is given the strides of a broadcast input; other overlap has to be built by hand. PyTorch's check for tensors that an operation writes to also rejects only a zero stride on an axis longer than 1. A general rule (sort the axes by stride and require each stride to cover the span of the axes inside it) is left for a follow-up, in this plugin or in hipDNN graph validation, where every engine would get it. It would also need the range tests that #14075 added to switch to non-overlapping strides: they give dQ, dK and dV overlapping strides, such as equal batch and head strides with H=8, and expect them to be accepted. New test IsApplicableRejectsZeroStrideOutputs has one case per output tensor and axis, and covers the four layouts that failed on gfx942. IsApplicableAcceptsAddressableLayouts gains the broadcast inputs and the zero strides on length-1 output axes. Checked on gfx942 (MI300X) with a standalone hipDNN probe that drives ASM_SDPA_ENGINE on its own buffers, because the bundle harness's test tensors reject zero strides. It compares dQ, dK and dV with a float64 CPU reference and with the same data stored packed, three runs per case. - The develop plugin accepts every layout above. With this commit's plugin the four failing layouts decline, every broadcast and length-1 case above still passes bitwise, and the forward cases give the same results as with the develop plugin. - The unit and integration tests pass. Each of 14 mutants that removes or widens one arm of the check (dropping a length-1 exemption, or treating an input as an output) makes one of the two tests fail. Co-Authored-By: Claude Opus 5.5 (1M context) <noreply@anthropic.com>

IsApplicableBoundsTheTsProductByTheResolvedTileSize (added by #14075) probes K sequence strides at and one past the Ts limit. With the gfx942 bf16 hd128 tile (ts = 192) the past-limit stride is odd, and isApplicable now also declines an odd stride on K's sequence axis and dK's head axis, because DQDKDV indexes them in 4-byte records. The test still passed with the Ts guard deleted, so it no longer tested the guard. The at-limit stride would also fail the parity check for a tile size that makes the limit odd. Both probes are now the nearest even strides on each side of the limit, so the Ts guard is the only check that separates them. Deleting the guard makes the test fail again. Co-Authored-By: Claude Opus 5.5 (1M context) <noreply@anthropic.com>

therock-pr-bot · 2026-10-29T19:13:34Z

✅ All Checks Passed — Ready for Review

Check	Status	Details
📝 PR Description	✅ Pass	—
⛔ Forbidden Files	✅ Pass	—
🧪 Unit Test	✅ Pass	—
🔎 pre-commit	✅ Pass	—
🚫 Draft PR	🔜 To Be Enabled	—
🚩 Feature Flag	🔜 To Be Enabled	—
📊 Code Coverage	🔜 To Be Enabled	—
🤖 therock-pr-bot	✅ Pass	—

🎉 All checks passed! This PR is ready for review.

📖 Need help? See the Policy FAQ for details on every check and how to fix failures.

🙋 Wish to Override Policy?

therock-pr-bot · 2026-10-29T19:13:38Z

🎉 All checks passed! This PR is ready for review.

…n-packed layouts Review follow-up for the backward decline series. - IntegrationGpuSdpaBackward: add B=2, H=2 cases (no mask and top-left causal), so the LSE/D strides buildPlan derives from Q's dims are exercised with more than one batch and head. Every earlier case was B=1, H=1, where swapping the two derived values left every kernel argument unchanged. Add the same two cases with the non-packed layouts isApplicable accepts: Q, K, V and dO stored [B, S, H, D], O and dQ rows padded to D+1. - TestSdpaBwdPlanBuilder: probe each decline at more than one value (a head-dim stride of 0, a sequence stride of 0 with S=256, a 128-element row stride at D=192 on each output-side tensor), add sequence-length and head-dim mismatches to the dims test, and decline a dK with 5 strides or rank-3 dims. Make the accept test's header match its cases and name the device cases above, and drop an undefined test-plan ID from a comment. - Comments: buffer bounds are S * seq stride elements, not bytes; the sequence-stride rule is conservative for Q, K and V at D=192; the LSE/D strides are derived from Q's dims, not taken from the stats tensor. Co-Authored-By: Claude Opus 5.5 (1M context) <noreply@anthropic.com>

… wording The accept test's header said "any stride on a length-1 axis", but with S=1 a sequence stride below D is still declined, and the byte-stride gate declines a negative or oversized stride on any axis. Say what is accepted: odd or zero batch and head strides on a length-1 axis, and an odd row stride of at least D with S=1. Comments only. Co-Authored-By: Claude Opus 5.5 (1M context) <noreply@anthropic.com>

…rated bundles (#14622) ## Summary This PR adds the SDPA backward test plan required by RFC 0022 (Per-Op Test Plans, merged in #13937). It also adds a generator that writes the SdpaBackward bundles from the plan, and the bundles it writes. The 16 inherited SdpaBackward cases (all B = H = 1, packed BHSD, fixed scale) are replaced by 503 generated cases in four levels. Each case carries a purpose and traces to the plan items it covers. > Written with AI assistance (Claude Code). The plan follows the merged RFC 0022; it records one remaining deviation (DV-4) for reviewers to approve. JIRA ID : ALMIOPEN-2894 ## Risk Assessment Risk 2 of 5. The PR changes test data, test tooling, test configuration and documentation only; no library or provider code changes. The changes that reach beyond the SdpaBackward bundles: - `support_matrix_units.collect_sweep_units()` leaves out cases whose `metadata.purpose` is `probe`, as RFC 0022 §9 requires. Today only SdpaBackward cases carry a `purpose`, so other ops' matrices are unchanged. The committed `SUPPORT_MATRIX_OVERVIEW.md` is re-rendered. - `ASM_SDPA_ENGINE.toml`: two dtype-only SdpaBackward tolerance rows are replaced by 12 generated rows (seven tolerance classes), plus temporary skips for the open engine defects, each naming its ticket. - `ASM_SDPA_ENGINE_test_categories_integration.yaml`: six rejection probes that ASM_SDPA_ENGINE wrongly accepts are quarantined with their tickets (RFC 0022 §6). Both files affect the ASM lane only. - #14358's eight PyTorch golden cases move, unchanged, from `quick/SdpaBackward/{Default,AdditiveMask}/` to `quick/SdpaBackward/Golden/{Default,AdditiveMask}/`, so the generator owns the `Default` sweeps. They keep their fp16 tolerance row. `bhsd/` and the `*_fwd_pair` bundles are untouched. The main risk is CI noise from new cases. Every level has been run on gfx942 and gfx950 with no failure (see Testing Checklist). ## ASIC Coverage The bundles run in every provider lane whose tier YAML matches `*Sdpa*`. Only `ASM_SDPA_ENGINE` accepts SdpaBackward graphs today, and only on gfx942: its backward plan builder declines every other device ("Device string does not match gfx942"). Claims are harvested for gfx942 only. PR CI tests gfx942. On gfx950 every generated backward case is declined, so a gfx950 run only shows that nothing fails; one is recorded below. PR CI on gfx942 is sufficient. ## Testing Summary - **Generator tests** (`hipdnn_sdpa_bwd_bundle_generator_python_tests`, label `quick`, 19 tests). They fail when the checked-in sweeps or the TOML block drift from the generator. They also check legality, all-pairs coverage up to nightly, the per-case ceilings and cumulative budgets of each level, two-way traceability between plan items and cases, that every TOML skip still matches a written case, and that the tier-YAML quarantine holds only rejection probes. - **`generate_sdpa_bwd_bundles.py --dry-run --report`.** Reports counts, budgets, the all-pairs proof, legality, and each declared-but-gated case with its gate. - **Claims run** (2026-10-26, MI300X, gfx942). All generated bundles ran on `ASM_SDPA_ENGINE`, configured like CI. `harvest_support_claims.py` turned the passing cases into the 272 claims. Every claimed case passes its tolerance class with a margin of at least 2.4 (median 4.2). - **Measured run, every level on GPU** (2026-11-04, plan §4.5). `hipdnn_integration_tests` ran on `ASM_SDPA_ENGINE` configured like the CI lane: `--test-engine` (claims enforced), the PR's `ASM_SDPA_ENGINE.toml`, the tier YAML's quarantine as a GTest filter, device input fill and the GPU reference as oracle. It ran all four levels, on a fresh MI300X (gfx942) and an MI355X (gfx950) node, built from this branch. The bundles, TOML and YAML are identical to `6e014ad50e0`. - **PR CI** runs the quick and standard levels on gfx942. ## Testing Checklist - [x] Generator tests - CTest `hipdnn_sdpa_bwd_bundle_generator_python_tests` - 19 of 19 - Status: Passed - [x] Plan report - `python3 reference-data-scripts/generate_sdpa_bwd_bundles.py --dry-run --report` - 503 written, 77 gated, 0 legality violations, all-pairs 669 of 669 - Status: Passed - [x] Bundle and support-matrix checks - `verify_support_claims.py`, `render_support_matrix.py --output -`, `feature_matrix.py --check`; the bundle, claim, render and support-feature pytest suites - Status: Passed (`verify_golden_bundles.py` reports no SdpaBackward error; its 322 errors in other ops' bundles are on develop too) - [x] pre-commit on the changed files - Status: Passed - [x] Claims run - `hipdnn_integration_tests`, `ASM_SDPA_ENGINE`, MI300X (gfx942), 2026-10-26 - Status: Passed - [x] Measured run, gfx942 - `hipdnn_integration_tests --test-engine ASM_SDPA_ENGINE --test-config ASM_SDPA_ENGINE.toml '--gtest_filter=*SdpaBackward*-<the tier YAML quarantine>'` - MI300X, Slurm job 68275259 - Status: Passed. 553 tests in 43.3 s, 0 failed, no claim broken: - 289 executed and passed against the GPU reference, and 15 expected rejections rejected; - 226 declined by ASM, and 23 hit the DEF skips; - per level: quick 2.4 s, PR 2.8 s, nightly 16.5 s, full 21.4 s. - [x] Quarantine check, gfx942 - the six quarantined cases run alone - Status: all six fail as expected ("Engine ASM_SDPA_ENGINE accepted expected-rejection bundle"), which shows the quarantine is needed. - [x] Every level, gfx950 - same command - MI355X, Slurm job 68275260 - Status: Passed. 0 failed: ASM declines every backward graph on gfx950, and the rejection probes, quarantined ones included, are correctly rejected. - [x] The other SDPA bundles, same lane - `--gtest_filter=*Sdpa*:-*SdpaBackward*` - Status: Passed. gfx942: 47 passed, 84 skipped. gfx950: 16 passed, 115 skipped. 0 failed on either. - [ ] PR CI on gfx942 - TheRock multi-arch CI - Status: Pending on `6e014ad50e0`. The 2026-11-02 run failed one case, `prbr01`, which is now gated. Later runs failed only on the hipCCL configure break from #14976, which develop reverted in #15089 (merged into this branch). ## Technical Changes ### What this adds - **The plan:** `dnn-providers/integration-tests/docs/test-plans/sdpa-bwd.md`. It follows the RFC 0022 template: the operation, the feature inventory, the shape catalog, building the sweep (levels, budgets, probes, regressions, trade-off register), the golden reference, pass criteria, and exhaustive mode. - **The generator:** `reference-data-scripts/sdpa_bwd_plan.py` holds the declarations: axes, legality, the shape catalog, level rules and the cost model, one record per case. `generate_sdpa_bwd_bundles.py` writes the sweeps, the support-claim files and the engine-TOML tolerance block from them. Nobody edits the bundles by hand. RFC 0022 open question 1 cites this module as the per-op Python option. - **The bundles:** 503 written cases, spread across the levels: | Level | Directory | Cases | |---|---|---| | quick | `quick/` | 56 | | PR | `standard/` | 152 | | nightly | `comprehensive/` | 220 | | full | `full/` | 75 | - Each case carries `purpose` (`claim`, `probe` or `regression`) and its plan items. - 77 more cases are declared but not written yet, each waiting on a named gate: an oracle extension, a harness feature or golden data. - One of the 77 is PRB-R01 (`dropout_probability = 0`). Since #14358 the references decline any graph that sets `dropout_probability`, so it waits on ALMIOPEN-3149 or ALMIOPEN-3150. - **Levels:** the cost model assigns each case to the lowest level whose per-case limits it fits: | Level | Per-case time | Per-case bytes | Cumulative budget | |---|---|---|---| | quick | 1.5 s | 4 MiB | 20 s | | PR | 5 s | 16 MiB | 120 s | | nightly | 75 s | 64 MiB | 480 s | | full | 300 s | no limit | 1,800 s | Each cumulative budget covers that level and every level below it, and keeps 5 % in reserve. The budgets come from a CPU-reference cost model; with the GPU reference, the measured run above is far below them. The ASM lane YAML picks the cases up by GTest prefix (`quick_*Sdpa*`, `standard_*Sdpa*` and so on), and each level also runs every level below it. - **Rejection probes** (21) carry `metadata.expected_rejection` (#14346): the harness never executes them, and an engine that accepts one fails the lane. PRB-M02 and PRB-M02f (`right_bound = -2`) are not written, because the frontend rejects them since #14358; `TestSdpaBwdNode.PreValidateChecksDiagonalBandBounds` covers that rule. - **dBias:** the five dBias probes and the live GL-14 case are written, now that #14358's references model dBias. To keep PR and nightly inside their reserve, the plan's trim rule moved M07.pr to nightly and M26.n to full. - **Tolerance classes:** seven classes, written as 12 `[[tolerance_overrides]]` rows between generated BEGIN and END markers in `ASM_SDPA_ENGINE.toml`. - **Support claims:** 272 claims for `ASM_SDPA_ENGINE` on gfx942. - **Probes left out of the support matrices** (RFC 0022 §9). This is the `support_matrix_units` change above. - **Generator file I/O is UTF-8**, so its tests pass on Windows. ### Temporary ASM_SDPA_ENGINE exclusions These stay until each fix lands; the PR with the fix deletes its entries. **Open engine defects** (`[[test_skips]]` in `ASM_SDPA_ENGINE.toml`). These are valid graphs that ASM accepts and computes wrong. Other engines still run them. | Cases | What goes wrong | Ticket | Fix | |---|---|---|---| | DEF-01 | head-dim stride other than 1 | ALMIOPEN-3036 | #14684 | | DEF-02 | non-packed Stats | ALMIOPEN-3037 | #14684 | | DEF-04 | Hk = Hq > Hv | ALMIOPEN-3039 | #14684 | | DEF-05 and H61 | odd element strides | ALMIOPEN-3038 | #14684 | | DEF-06 | scale of 0 or below | ALMIOPEN-3043 | #14688 | | DEF-03 | fp16 dV precision in the AITER kernel | ALMIOPEN-3046 | none in hipDNN (kernel) | **Quarantined rejection probes** (`_common_excludes` in `ASM_SDPA_ENGINE_test_categories_integration.yaml`, as RFC 0022 §6 asks). These are undefined graphs ASM should decline but accepts. | Cases | What ASM accepts | Ticket | Fix | |---|---|---|---| | prbu01, prbu03 | O or dQ dims that differ from Q's | ALMIOPEN-3039 | #14684 | | prbu13 | a dQ sequence stride below D (overlapping rows) | ALMIOPEN-3038 | #14684 | | prbt04, prbt04n | an O dtype that differs from Q's | ALMIOPEN-3050 | none yet | | prbr05 | `dropout_scale_inv` without a dropout mask | ALMIOPEN-3051 | none yet | ### Dependencies (all merged) - #13937, RFC 0022 (2026-10-30). Its first-phase documents, the methodology guide and the `docs/test-plans/` index, are not on develop yet, so the plan links the RFC's sections directly. - #14312: O and Stats derived from the forward of the same inputs. - #14483: `left_bound` uses cuDNN's meaning. The bundles write `left_bound = W`. - #14358 (2026-11-02): SDPA references cover the full graph API, and the GPU reference is the first oracle in `auto` mode. It also adds frontend rule R14 (`right_bound >= -1`). - #14346 (2026-11-03): the `expected_rejection` and `no_*_reference` bundle flags. - #14709 (2026-11-02): zero strides in the data SDK `Tensor`. ### Interaction with other open PRs - **#14684, #14686, #14687, #14688** (ASM declines, ALMIOPEN-3035). Each deletes its own entries: TOML skips for the DEF cases, and tier-YAML lines for the quarantined probes it fixes (#14684: prbu01, prbu03, prbu13). Whichever of these and this PR lands second removes the entries. - **#14999** (SDPA forward plan bundles, ALMIOPEN-2893). Both PRs edit two files the same way: - `ASM_SDPA_ENGINE.toml`: keep this PR's generated block and put #14999's `SdpaFwdPlan` skips after the `# END SdpaBackward tolerance classes` line. The generator test only checks the BEGIN to END span. The filters do not overlap (`SdpaBackward` vs `SdpaFwdPlan`). - `SUPPORT_MATRIX_OVERVIEW.md`: re-render it; do not hand-merge. The two PRs also add lines near each other in `integration-tests/CMakeLists.txt` and `test_categories_external.yaml`; these auto-merge. ### Before review - [x] `prbr01` (`dropout_probability = 0`) gated; ALMIOPEN-3149 and ALMIOPEN-3150 track the decision. - [x] Rejection probes carry `expected_rejection`; the six ASM accepts are quarantined in the ASM tier YAML with their tickets. - [x] `prbm02` and `prbm02f` removed; the frontend unit test of R14 covers them. - [x] The five dBias probes and GL-14 written; REG-08 (rank-3 Stats) still waits for the derivation extension. - [x] The plan refreshed against the merged RFC: DV-1 and DV-2 now agree with it, DV-3 is an RFC §11 gap, and only DV-4 remains a deviation. - [x] The measured run (plan §4.5) recorded above. 🤖 Generated with [Claude Code](https://claude.com/claude-code) --------- Co-authored-by: Claude Opus 5.5 (1M context) <noreply@anthropic.com>

brockhargreaves-amd and others added 4 commits October 29, 2026 14:44

github-actions Bot added the project: hip-kernel-provider label Oct 29, 2026

assistant-librarian Bot added the organization: ROCm label Oct 29, 2026

This was referenced Oct 29, 2026

fix(hip-kernel-provider): ASM SDPA backward rejects the a16 accumulator #14687

Open

fix(hip-kernel-provider): ASM SDPA declines non-positive attention scales #14688

Open

brockhargreaves-amd mentioned this pull request Oct 29, 2026

fix(hip-kernel-provider): ASM SDPA forward declines unaddressable layouts #14686

Open

1 task

brockhargreaves-amd requested a review from DarylHawkinsAMD October 29, 2026 20:59

brockhargreaves-amd and others added 2 commits October 30, 2026 13:07

brockhargreaves-amd marked this pull request as ready for review October 30, 2026 19:26

brockhargreaves-amd requested a review from a team as a code owner October 30, 2026 19:26

brockhargreaves-amd mentioned this pull request Nov 4, 2026

test(integration-tests): add the SDPA backward test plan and its generated bundles #14622

Merged

15 of 16 tasks

Navigation Menu

Repository navigation

fix(hip-kernel-provider): ASM SDPA backward declines layouts and dims its kernels address wrongly - #14684

fix(hip-kernel-provider): ASM SDPA backward declines layouts and dims its kernels address wrongly#14684
brockhargreaves-amd wants to merge 6 commits into
developfrom
users/brockhargreaves-amd/asm-sdpa-bwd-decline-strides

brockhargreaves-amd commented Oct 29, 2026 •

edited

Loading

Uh oh!

therock-pr-bot Bot commented Oct 29, 2026 •

edited

Loading

Uh oh!

therock-pr-bot Bot commented Oct 29, 2026

Uh oh!

This branch has not been deployed

Reviewers

Uh oh!

Assignees

Labels

Projects

Milestone

Development

Uh oh!

1 participant

Conversation

brockhargreaves-amd commented Oct 29, 2026 • edited Loading Uh oh! There was an error while loading. Please reload this page.

Uh oh!

Motivation

Technical Details

ASIC Coverage

Test Plan

Test Result

Submission Checklist

Uh oh!

therock-pr-bot Bot commented Oct 29, 2026 • edited Loading Uh oh! There was an error while loading. Please reload this page.

Uh oh!

✅ All Checks Passed — Ready for Review

Uh oh!

therock-pr-bot Bot commented Oct 29, 2026

Uh oh!

This branch has not been deployed

Reviewers

Uh oh!

Assignees

Labels

Projects

Milestone

Development

Uh oh!

1 participant

brockhargreaves-amd commented Oct 29, 2026 •

edited

Loading

therock-pr-bot Bot commented Oct 29, 2026 •

edited

Loading