Repository navigation
test(integration-tests): add the SDPA backward test plan and its generated bundles - #14622
Merged
brockhargreaves-amd merged 21 commits intoNov 5, 2026
Merged
Conversation
✅ All Checks Passed — Ready for Review
📖 Need help? See the Policy FAQ for details on every check and how to fix failures. |
|
🎉 All checks passed! This PR is ready for review. |
brockhargreaves-amd
force-pushed
the
users/brockhargreaves-amd/sdpa-bwd-test-plan
branch
from
October 29, 2026 02:55
a0de08f to
2e14bfd
Compare
Pre-commit check failed⛔ pre-commit failed Please run locally:
This repo uses |
brockhargreaves-amd
added a commit
that referenced
this pull request
Oct 29, 2026
…youts SdpaBwdPlanBuilder::isApplicable accepted tensor layouts that the gfx942 backward kernels address wrongly. On gfx942 (MI300X) each one gave silent wrong gradients instead of a decline. The evidence comes from two sources. One is GPU runs of the plugin from a CI build of #14312, which accepts all of these layouts. The other is disassembly of the dispatched .co files. Q/K/V/O/dO/dQ/dK/dV strides. The ODO, DQDKDV and DQ_CONVERT argument blocks carry only batch, head and sequence byte strides. isApplicable now declines: - A head-dim (innermost) stride other than 1. The kernels read it as 1: a stride-2 Q gave Inf dK/dV, and a stride-2 dQ left elements unwritten. - A sequence stride below D, also on a length-1 sequence axis. Each buffer ends S * seq stride bytes into its (batch, head) slice, so the end of the last row reads as 0, and writes there are dropped. A seq stride of 1 on a length-1 axis gave wrong gradients for each of the eight tensors. Seq strides of D/2 and 2 did the same for Q, K, V and dO. - An odd stride on an axis whose offset DQDKDV puts in its 4-byte record index (byte offset >> 2). That is the sequence axis of Q, K, V, dO, dK and dV, and the head axis of dK and dV. Rows that start 2 bytes into a dword are read or written one element early. A seq stride of D+1 failed on each of the six. A dO row pitch of D+1 gave 47-97 % wrong dQ, dK and dV. Odd strides on the other axes are addressed correctly and stay accepted: - batch strides, which are added to the 64-bit buffer base; - head strides of Q, K, V, O, dO and dQ; - seq strides of O and dQ, which ODO and DQ_CONVERT address by byte offset. The causal kernels also add the dK/dV head offset to the base, but the check does not depend on the mask. Stats (LSE). All 28 dispatched batch-mode DQDKDV kernels compute the LSE and D offset as ((b * H_q + h) * S_q + s) * 4 and never load Hs_lsed. A stats head stride padded to S_q + 64 gave wrong gradients that changed between runs with identical inputs. A padded batch stride, a seq stride of 2 and [B, S_q, H] storage also gave wrong gradients. isApplicable now requires stats dims [B, H_q, S_q] or [B, H_q, S_q, 1] matching Q, with packed strides on every axis longer than 1. buildPlan sets the LSE/D strides from Q's dims, so the stride of a size-1 stats axis no longer reaches a kernel argument or the 32-bit stride check. These layouts stay accepted, all GPU-verified on gfx942: BSHD on all tensors, dO strides that differ from O, seq strides D+2 and D+8, B and H above 1, hd192, fp16, causal and sliding-window masks, BR causal with Sq above Skv, Sq in {1, 16, 17}, and tails. New tests in TestSdpaBwdPlanBuilder.cpp (gfx942 only, like the other isApplicable tests). Each decline arm has a case that only it declines, and each exemption has an accept case: - IsApplicableAcceptsAddressableLayouts - IsApplicableRejectsNonUnitHeadDimStride - IsApplicableRejectsMisalignedOrShortRows - IsApplicableRejectsNonPackedStats Checked on gfx942 (MI300X). This commit's plugin replaced the one in a CI build of #14312, and results were compared with the CPU reference: - Each declined layout above failed with the old plugin, which accepts it, and now declines. Each layout named above as addressed correctly passed with the old plugin and still passes. The probes cover hd128 and hd192, bf16 and fp16, and the no-mask, causal and sliding-window kernels. - The shipped bf16 tolerance hides some short-row errors, so those were also compared at atol=1e-9 with packed controls. Their error was 7-34x the controls'; for Q, the dQ error sits in the cut-off last row. - The develop quick/standard SdpaBackward bundles and the CI Sdpa bundles give the same results as before. - The unit tests pass. Each of 29 mutants that removes or widens one arm of these checks makes one of the new tests fail. Rebased onto develop 98b504d, after #14075, whose range gate now runs first in isApplicable. The GPU results above are from develop 61c2675. On the new base, the four ALMIOPEN-3035 ASM SDPA fix branches (backward layouts, attention scale, a16 accumulator, forward layouts) were built and run together: - hip_kernel_provider_tests: 912 pass and 7 skip on gfx942 (MI325X), 886 pass and 33 skip on gfx950 (MI355X); none fail. - hip_kernel_provider_integration_tests: none fail on either. - develop's SDPA bundles on ASM_SDPA_ENGINE, run like the CI lane (claims enforced): 62 pass and 56 skip on gfx942, 15 pass and 103 skip on gfx950, and no support claim fails. A host-side run of the real isApplicable over these bundles finds no outcome change from develop. - #14622's SdpaBackward bundles (138218c) with its temporary [[test_skips]] removed: 275 pass and 225 skip on gfx942, none fail, and no support claim fails. This branch's checks decline the 20 cases behind the DEF-01, DEF-02, DEF-04, DEF-05/H61, prbu01, prbu03 and prbu13 skips, and no other case (host-checked for this branch alone). - clang-tidy 20 and clang-format 18 are clean on the changed files. Co-Authored-By: Claude Opus 5.5 (1M context) <noreply@anthropic.com>
brockhargreaves-amd
added a commit
that referenced
this pull request
Oct 29, 2026
…ales SdpaFwdPlanBuilder::isApplicable and SdpaBwdPlanBuilder::isApplicable accepted any attention scale, and execute passed it to the kernels unchecked. cuDNN and hipDNN accept any scale value. On gfx942 (MI300X) a scale of 0, -0.0, a negative value or a subnormal gave NaN or Inf outputs and gradients instead of a decline. The mechanism comes from the .co disassembly. The forward and backward kernels write -inf into every masked score (causal and sliding-window masks, and the padding past S_kv in the last K tile), and only then multiply the scores by scale * log2(e). A scale of 0 turns those scores into NaN, and a negative scale turns them into +inf. The bf16 forward kernels also start the running row max at -FLT_MAX / (scale * log2(e)), which is positive for a negative scale or -0.0, so every exp2 overflows even with no mask. 32-bit denormals are flushed, so a subnormal scale acts as 0. An infinite scale gives NaN everywhere. The gfx942 runs used the plugin from a CI build of #14312, which accepts all of these scales, and compared results with the CPU reference: - Backward: 0, -0.0, 1e-40 and -1.2 gave NaN or Inf dQ, dK and dV with top-left or bottom-right causal masks and sliding windows, in bf16 and fp16, hd128 and hd192. With no mask and S_kv = 128 (the hd128 K tile is 192), all of dQ was NaN. - Forward: -1.2, -0.088 and -0.0 gave NaN O and Inf LSE even with no mask. 0 gave NaN O with a bottom-right causal mask or S_kv = 200. - A runtime pass-by-value scale tensor gave the same results as attn_scale_value. Both isApplicable functions now decline a scale that is not a finite, positive, normal float (FLT_MIN <= scale <= FLT_MAX; NaN fails). The check covers attn_scale_value, and the value of a compile-time constant or runtime-with-default pass-by-value scale tensor, narrowed to the f32 kernarg as execute narrows it. They also decline a scale tensor whose value the plugin SDK cannot read, on which buildPlan would throw. The value of a pure runtime pass-by-value scale tensor arrives only at execute, so both execute functions now throw BAD_PARAM for such a scale before the first launch, as the gfx1151 WMMA attention pack does for its runtime scale. The check is shared in SdpaPlanUtils.hpp. An unset scale (1.0) and every finite, positive, normal scale stay accepted; 1e-37 and 1/sqrt(128) were correct on gfx942. The rule also declines some cases that happened to compute correctly: no mask with S_kv a multiple of the K tile, for any of these scales in the backward and for +0.0 or a subnormal scale in the forward. A rule keyed on the mask and the tile would break once group mode makes the K tail per-sequence, so the decline does not depend on them. Scales near FLT_MAX (1e37, 1e38) still give NaN in the forward; that is a separate issue. New tests: - TestSdpaPlanUtils (host only): IsSupportedAttnScaleRequiresFinitePositiveNormalFloat and AttnScaleBeforeExecute. - TestSdpaFwdPlanBuilder and TestSdpaBwdPlanBuilder (gfx942, and gfx950 for the forward, like the other isApplicable tests): IsApplicableRejectsUnsupportedAttnScale declines 0, -0.0, -1.2, -1/sqrt(128), -FLT_MIN, three subnormals, NaN and +-inf, each from attn_scale_value, a constant scale tensor and a runtime scale tensor default, plus an FP8_E4M3 scale tensor. IsApplicableAcceptsSupportedAttnScale accepts 1e-37, 1/sqrt(128) and 1 from each source, the unset default and a pure runtime scale tensor. - TestSdpaFwdPlanExecute and TestSdpaBwdPlanExecute: execute returns BAD_PARAM for each unsupported runtime scale (host only, on a plan with no loaded kernels), and passes supported ones on to the launch. Checked on gfx942 (MI300X). This commit's plugin replaced the one in a CI build of #14312, and results were compared with the CPU reference on the 82 scale cases behind the results above: - The 61 cases whose unsupported scale comes from attn_scale_value or a constant scale tensor are now declined. With the old plugin 48 of them failed, 12 passed (the no-mask cases named above) and one was already declined (fp16 forward). - The 4 cases with an unsupported runtime scale tensor (0 and -1.2, forward and backward) now fail at execute with BAD_PARAM, and the message gives the scale. The old plugin gave NaN or Inf in 3 of them. - The 15 cases with a supported scale pass with both plugins: 1/sqrt(128) with each mask, dtype and scale source, and 1e-37. The forward with 1e37 or 1e38 fails with both, as noted above. - On the develop CI build, develop's quick and standard SdpaFwd, SdpaFwdRuntimeScale and SdpaBackward bundles give the same result for each case as with develop's plugin: 42 pass and 39 are declined, for the same reasons, none of them the scale. - The unit tests pass. Each of 14 mutants that removes, widens or narrows one arm of these checks makes one of the new tests fail. Rebased onto develop 98b504d, after #14075, whose range gate now runs first in isApplicable. The GPU results above are from develop 61c2675. On the new base, the four ALMIOPEN-3035 ASM SDPA fix branches (backward layouts, attention scale, a16 accumulator, forward layouts) were built and run together: - hip_kernel_provider_tests: 912 pass and 7 skip on gfx942 (MI325X), 886 pass and 33 skip on gfx950 (MI355X); none fail. - hip_kernel_provider_integration_tests: none fail on either. - develop's SDPA bundles on ASM_SDPA_ENGINE, run like the CI lane (claims enforced): 62 pass and 56 skip on gfx942, 15 pass and 103 skip on gfx950, and no support claim fails. A host-side run of the real isApplicable over these bundles finds no outcome change from develop. - #14622's SdpaBackward bundles (138218c) with its temporary [[test_skips]] removed: 275 pass and 225 skip on gfx942, none fail, and no support claim fails. This change declines def06a, def06b and def06c (scale 0 and -1.2), the cases behind the DEF-06 skip, and no other case (host-checked for this branch alone). The execute-path scale tests (PassesSupportedRuntimeAttnScaleToLaunch) pass on both GPUs. - clang-tidy 20 and clang-format 18 are clean on the changed files. Co-Authored-By: Claude Opus 5.5 (1M context) <noreply@anthropic.com>
brockhargreaves-amd
added a commit
that referenced
this pull request
Oct 29, 2026
SdpaBwdPlanBuilder advertised the sdpa.bwd.accumulator_type knob with valid values a32 and a16, and initializeExecutionSettings accepted a16. isApplicable checks only the A32 kernels. With an A16 setting, buildPlan asks for a DQDKDV row with atomic32=0, pssk=1, pddv=1, and the gfx942 registry has none: its 24 a16 rows all have pssk=0, hd64 or hd128, and mask none or top-left causal. So buildPlan threw INTERNAL_ERROR "failed to resolve dqdkdv kernel ... (isApplicable should have rejected)" for every graph isApplicable accepts. On gfx942 (MI300X), develop's builder built all 42 accepted graphs of a sweep with a32 and failed all 42 with a16: bf16 and fp16; hd128 with no mask, top-left and bottom-right causal and sliding window; hd192 with no mask and both causal masks. hipDNN users cannot reach this today. AsmSdpaEngine publishes no builder knobs and never calls initializeExecutionSettings, so it always builds A32 plans, and the frontend refuses or drops knobs that an engine does not advertise. This change leaves the engine alone: with only a32 valid there is nothing to forward, and forwarding the knobs before this fix would have exposed the failure. Selecting the a16 rows that do exist would not fix it either. The gfx942 a16 kernels add dQ atomically, in the 16-bit output type, into the output dQ, address dQ and dO with Q's strides, and never load seqlen_k, the window bounds, or any V, dO, dK or dV stride other than dK's seq stride. In a bf16 diagnostic run with those rows patched in, cases inside that envelope matched a32 to 1 bf16 ulp. A dQ holding stale values, a dQ or dO layout that differs from Q's, Sq != Skv, and S not a multiple of 64 each gave wrong gradients, and a padded Q/dO row pitch and S=300 also wrote past the end of dQ. Now: - getCustomKnobs lists only a32. - initializeExecutionSettings rejects a16 with INVALID_VALUE and a message of its own. Other unknown strings still get INVALID_VALUE and non-string values BAD_PARAM. - buildPlan refuses an A16 setting with an INTERNAL_ERROR that says so, before it queries the device or resolves a kernel. It is the only gate: getMaxWorkspaceSize still sizes an A16 setting without dq_acc, but no A16 plan is ever built to use that size, and the A16 code in buildPlan and SdpaBwdPlan is kept for a future enablement. - Comments in the builder, the plan, and the params, kernel-args and workspace-size headers that said A16 can be selected, or that its kernels write dQ directly in BF16 and need no dQ zeroing, now say that the a16 kernels add dQ atomically into the output dQ, which must be zeroed first, and that A16 is refused. The comment on the rejection lists the graphs that getCustomKnobs and initializeExecutionSettings would have to limit A16 to before it could be enabled. isApplicable cannot do that: it sees no knob, and it also validates A32, which runs graphs A16 cannot. An unset knob and a32 behave as before, and isApplicable and the A32 plans are unchanged. Tests in TestSdpaBwdPlanBuilder.cpp: - GetCustomKnobsReturnsAccumulatorKnob now expects only a32 (gfx942 only, like the other isApplicable tests). - InitializeExecutionSettingsAcceptsA32 covers a32, and an unset knob from an invalid config, a valid config with no knobs, and a valid config with only another knob. - InitializeExecutionSettingsRejectsA16 - InitializeExecutionSettingsRejectsUnknownAccumulator - BuildPlanRejectsA16Setting, which needs no GPU. - Gfx942HasNoA16DqdkdvForDispatchedLayout pins the registry fact that the rejection comment cites, with the pssk=0 a16 rows as positive controls. Checked on gfx942 (MI300X). All 228 asm_sdpa unit tests pass except IsApplicableRejectsGfx950, which skips on gfx942 as it does on develop (223 tests there). The host-only tests also pass on a machine without a gfx942 device. Each of 9 mutants makes one of the tests above fail: mapping a16 back to A16, dropping the a16 arm, letting a32 fall through, listing a16 again, dropping the buildPlan check (buildPlan then fails to resolve a DQDKDV kernel, as on develop), accepting unknown strings, dropping the has-knob check (a valid config without the knob then throws std::out_of_range), pointing the registry check at gfx950, which has such rows, and making every lookup in it miss. develop's quick and standard SdpaBackward bundles (8 cases) pass against the CPU reference with this plugin, as with develop's, and still build A32 plans. clang-format 18 and clang-tidy 20 are clean on the changed files. Rebased onto develop 98b504d, after #14075, whose range gate now runs first in isApplicable. The GPU results above are from develop 61c2675. On the new base, the four ALMIOPEN-3035 ASM SDPA fix branches (backward layouts, attention scale, a16 accumulator, forward layouts) were built and run together: - hip_kernel_provider_tests: 912 pass and 7 skip on gfx942 (MI325X), 886 pass and 33 skip on gfx950 (MI355X); none fail. - hip_kernel_provider_integration_tests: none fail on either. - develop's SDPA bundles on ASM_SDPA_ENGINE, run like the CI lane (claims enforced): 62 pass and 56 skip on gfx942, 15 pass and 103 skip on gfx950, and no support claim fails. A host-side run of the real isApplicable over these bundles finds no outcome change from develop. - No #14622 SdpaBackward case changes outcome: no bundle sets a knob (host-checked for this branch alone). - clang-tidy 20 and clang-format 18 are clean on the changed files. Co-Authored-By: Claude Opus 5.5 (1M context) <noreply@anthropic.com>
brockhargreaves-amd
added a commit
that referenced
this pull request
Oct 29, 2026
…outs SdpaFwdPlanBuilder::isApplicable accepted Q/K/V/O and stats (LSE) layouts that the forward kernels address wrongly. Its only stride check was that byte strides fit in 32 bits. On gfx942 (MI300X) each kind of layout below gave a silently wrong O or wrong stats instead of a decline, in the cases that were run (the exceptions are noted). The evidence comes from two sources. One is GPU runs of the develop CI plugin, which accepts all of these layouts, compared with the CPU reference. The other is disassembly of the 14 forward .co files. Q/K/V/O strides. The forward argument block carries only batch, head and sequence byte strides, and the kernels read or write each row as D contiguous elements. isApplicable now declines: - A head-dim (innermost) stride other than 1. The kernels read it as 1. Stride-2 Q, K or V gave a wrong O that changed between runs, because the kernels read the gaps between elements. A stride-2 O left half of O unwritten. Q stored as [B, H, D, S] and hd192 Q with stride 2 also failed. - A sequence stride below the tensor's own head dim, also 0 and also on a length-1 sequence axis. Each buffer ends S * seq stride bytes into its (batch, head) slice, so the end of the last row reads as 0 and writes there are dropped. A seq stride of D/2 gave a wrong O for Q and V, and for K at S_kv = 2. At S_kv = 256 the K case stayed within the bf16 tolerance, but its error was about 6x the packed run's. With S_q = 1 and an O seq stride of 1, O was left NaN. With S_kv = 1 and a V seq stride of 1, O was 0. K and V with seq stride 0 gave O and stats off by up to 1.0 and 1.3, Q with seq stride 0 gave a wrong O and stats, and O with seq stride 0 was left unwritten. With S_kv = 1 a K seq stride of 1 still gave the right O, since the one key takes all the softmax weight; by the address formula its score, and so the stats, are wrong. Each tensor is compared with its own head dim. That relies on K having Q's head dim and O being [B, H_q, S_q, D_v], because the kernels size K rows by Q's head dim and O's buffer by S_q. The forward isApplicable does not check those dims yet; a forward counterpart of the backward's mismatched-dims decline is a separate change. - A zero O batch or head stride on an axis longer than 1. Each (batch, head) slice of O is written by its own workgroups, so such slices overwrite each other: with an O batch or head stride of 0, half or two thirds of O differed from the packed run, and O changed between some runs with identical inputs. Nonzero strides that make O overlap itself are not checked. On gfx942, O with equal batch and head strides (B = H = 2), or with a head stride of half a slice, differed from the packed run in a quarter of its elements, with both plugins. As in the backward, only a zero stride is checked: a zero stride is how a broadcast is written, so O gets one when it is given a broadcast input's strides, while overlapping nonzero strides have to be built by hand. PyTorch's check for tensors that an operation writes to also rejects only a zero stride on an axis longer than 1. Stats do get an overlap check (below), because their address formula leaves a single comparison to make; for O it would take a footprint check over three axes in any order, which would fit better in hipDNN's graph validation, where every engine gets it. The zero O batch stride check assumes batch mode, the only mode with forward kernels. In group (ragged) mode the kernels locate each batch by its sequence offsets and get no O batch stride, so when group-mode forward kernels are added, a ragged O must be exempt from the batch-axis part of this check. Stats. Every forward kernel writes the LSE of (b, h, s) at ((b * H_q + h) * lse_Hs + s) * 4 bytes, with lse_Hs the stats head stride, and takes no stats batch or sequence stride. isApplicable now declines stats whose seq stride is not 1 when S_q > 1, or whose batch stride is not H_q times the head stride when B > 1. It also declines a head stride below S_q when B * H_q > 1, with which (batch, head) slices overlap. On gfx942 each of these gave wrong or NaN stats while O was correct: - a batch stride padded to H_q * S_q + 128, with hd128, hd192 and GQA; - a seq stride of 2; - [B, S_q, H] storage, and H_q = 1 with strides [S_q, 1, 1, 1]; - [H, B, S_q] storage, which by the kernels' address formula also writes 2 KiB past the end of the stats buffer. This is looser than the backward's packed-only stats rule on purpose. The backward kernels ignore the stats strides, but the forward kernels use lse_Hs: a head stride padded to S_q + 64, with a batch stride of H_q times that, gave correct stats on gfx942 and stays accepted. The stride of a size-1 stats batch or sequence axis is not checked. A size-1 head axis is, on purpose: buildPlan passes the head stride as lse_Hs, so with H_q = 1 the kernel steps between batches by the head stride. Passing the batch stride instead when H_q = 1 would also accept packed [B, 1, S_q] stats with any head stride, such as the H_q = 1 case above; they are rare and decline. The stats decline reasons name the stats dims and strides and Q's dims. These stay accepted, all GPU-verified with bf16 on gfx942: - BSHD on all tensors, and O BSHD with BHSD inputs; - seq strides D+2 and D+8; - odd strides: a seq stride of D+1 and odd head and batch strides on each of Q, K, V and O, with hd128 and hd192, gave O bitwise equal to the packed runs (the backward declines some of these); - hd192, GQA, bottom-right causal, tails, and S_q = 1 or S_kv = 1 packed; - a zero batch or head stride on Q, K or V (a broadcast): K and V with a zero batch or head stride, and Q with a zero batch stride, gave O bitwise equal to the packed runs and stats that matched the CPU reference. A zero stride on a length-1 O axis is accepted too, since it only ever multiplies index 0. The gfx950 kernels have the same argument block and the same LSE address formula (from their disassembly), but load rows with 16-byte LDS-DMA where the gfx942 kernels use 4-byte. On gfx950 (MI355X), this commit's plugin gave the BSHD, D+2, D+8 and odd-stride layouts above, with hd128 and hd192, the same mismatch counts and max and mean error against the CPU reference at atol=1e-9 as the packed layouts. The fp8 kernels also have the same argument block and LSE formula, so the same rules apply to them. They were not run; in particular odd strides, which put fp8 rows 1 or 3 bytes past a 4-byte boundary, are unverified with fp8. New tests in TestSdpaFwdPlanBuilder.cpp (gfx942 and gfx950, like the other forward isApplicable tests). Each decline arm has a case that only it declines, and each exemption has an accept case: - IsApplicableAcceptsAddressableLayouts - IsApplicableRejectsNonUnitHeadDimStride - IsApplicableRejectsShortRows, including hd192 Q and K rows long enough for V's and O's head dim but not their own - IsApplicableRejectsZeroStrideOutput - IsApplicableRejectsUnaddressableStats The zero-stride and overlapping layouts were run with a standalone hipDNN probe that drives ASM_SDPA_ENGINE on its own buffers, because the bundle harness's test tensors reject zero strides. Checked on gfx942 (MI300X). This commit's plugin replaced the one in a develop CI build whose ASM SDPA engine source matches this commit's parent, and results were compared with the CPU reference: - 84 forward layout probes. The 25 that this commit declines now skip, each with a reason that names the tensor, its dims and its strides. With the develop plugin, 21 of them gave a wrong O or wrong stats. Two could not be checked there: the harness rejects a zero stride (the standalone probe covers it), and the CPU reference rejects rank-3 stats. The last two passed but are wrong in ways the check missed: the K seq stride D/2 case stays within the bf16 tolerance, and with S_kv = 1 a K seq stride of 1 only gives the right O because one key takes all the softmax weight. - The other 59 give the same results as with the develop plugin, down to the mismatch counts and the max and mean error at atol=1e-9. 47 pass; they cover every layout named above as staying accepted except the broadcasts, which the next item covers. 3 were already declined. 9 still fail: non-positive or denormal scales, which are left to a separate change, and rank-3 stats, which the CPU reference cannot check. - On the standalone probe, K/V and Q broadcasts still match the CPU reference and are bitwise equal to their packed twins in each of 3 runs. An O batch or head stride of 0 and a Q, K/V or O seq stride of 0 now decline; with the develop plugin each gave a wrong O. The two overlapping O layouts are accepted by both plugins, with the same mismatch counts. - 19 probes with tensors of up to 4 GiB, which exercise the kernels' 32-bit address math, give the same results as before (10 of them still fail with both plugins; the 32-bit address math is a separate change). - The develop SdpaFwd and SdpaFwdRuntimeScale bundles (102 tests, quick to full) give the same results as before: 46 pass and 56 skip, with the same decline reasons. - The provider's own forward GPU integration tests (hip_kernel_provider_integration_tests, *IntegrationGpuSdpaFwd*: Smoke, SmokeFp8, SmokeStats, ShapeSweep, Gqa and AsymmetricSeq) pass against this commit's plugin: 29 pass, and 21 skip as gfx950-only or causal with stats. - The unit tests pass. Each of 33 mutants that removes, narrows or widens one arm of these checks, or compares a row stride with another tensor's head dim or a fixed 128, makes one of the new tests fail. On gfx950 the 84 forward layout probes ran with this commit's plugin only, since the develop CI build has no gfx950 kernels. The same 28 skip, and every probe that passes on gfx942 and has a gfx950 kernel passes. The forward unit tests pass there too. Rebased onto develop 98b504d, after #14075, whose range gate now runs first in isApplicable. The GPU results above are from develop 61c2675. On the new base, the four ALMIOPEN-3035 ASM SDPA fix branches (backward layouts, attention scale, a16 accumulator, forward layouts) were built and run together: - hip_kernel_provider_tests: 912 pass and 7 skip on gfx942 (MI325X), 886 pass and 33 skip on gfx950 (MI355X); none fail. - hip_kernel_provider_integration_tests: none fail on either. - develop's SDPA bundles on ASM_SDPA_ENGINE, run like the CI lane (claims enforced): 62 pass and 56 skip on gfx942, 15 pass and 103 skip on gfx950, and no support claim fails. A host-side run of the real isApplicable over these bundles finds no outcome change from develop. - No develop or #14622 bundle case changes outcome (host-checked for this branch alone). The new forward tests pass on gfx942 and gfx950. - clang-tidy 20 and clang-format 18 are clean on the changed files. Co-Authored-By: Claude Opus 5.5 (1M context) <noreply@anthropic.com>
This was referenced Oct 29, 2026
brockhargreaves-amd
added a commit
that referenced
this pull request
Oct 30, 2026
…ales SdpaFwdPlanBuilder::isApplicable and SdpaBwdPlanBuilder::isApplicable accepted any attention scale, and execute passed it to the kernels unchecked. cuDNN and hipDNN accept any scale value. On gfx942 (MI300X) a scale of 0, -0.0, a negative value or a subnormal gave NaN or Inf outputs and gradients instead of a decline. The mechanism comes from the .co disassembly. The forward and backward kernels write -inf into every masked score (causal and sliding-window masks, and the padding past S_kv in the last K tile), and only then multiply the scores by scale * log2(e). A scale of 0 turns those scores into NaN, and a negative scale turns them into +inf. The bf16 forward kernels also start the running row max at -FLT_MAX / (scale * log2(e)), which is positive for a negative scale or -0.0, so every exp2 overflows even with no mask. 32-bit denormals are flushed, so a subnormal scale acts as 0. An infinite scale gives NaN everywhere. The gfx942 runs used the plugin from a CI build of #14312, which accepts all of these scales, and compared results with the CPU reference: - Backward: 0, -0.0, 1e-40 and -1.2 gave NaN or Inf dQ, dK and dV with top-left or bottom-right causal masks and sliding windows, in bf16 and fp16, hd128 and hd192. With no mask and S_kv = 128 (the hd128 K tile is 192), all of dQ was NaN. - Forward: -1.2, -0.088 and -0.0 gave NaN O and Inf LSE even with no mask. 0 gave NaN O with a bottom-right causal mask or S_kv = 200. - A runtime pass-by-value scale tensor gave the same results as attn_scale_value. Both isApplicable functions now decline a scale that is not a finite, positive, normal float (FLT_MIN <= scale <= FLT_MAX; NaN fails). The check covers attn_scale_value, and the value of a compile-time constant or runtime-with-default pass-by-value scale tensor, narrowed to the f32 kernarg as execute narrows it. They also decline a scale tensor whose value the plugin SDK cannot read, on which buildPlan would throw. The value of a pure runtime pass-by-value scale tensor arrives only at execute, so both execute functions now throw BAD_PARAM for such a scale before the first launch, as the gfx1151 WMMA attention pack does for its runtime scale. The check is shared in SdpaPlanUtils.hpp. An unset scale (1.0) and every finite, positive, normal scale stay accepted; 1e-37 and 1/sqrt(128) were correct on gfx942. The rule also declines some cases that happened to compute correctly: no mask with S_kv a multiple of the K tile, for any of these scales in the backward and for +0.0 or a subnormal scale in the forward. A rule keyed on the mask and the tile would break once group mode makes the K tail per-sequence, so the decline does not depend on them. Scales near FLT_MAX (1e37, 1e38) still give NaN in the forward; that is a separate issue. New tests: - TestSdpaPlanUtils (host only): IsSupportedAttnScaleRequiresFinitePositiveNormalFloat and AttnScaleBeforeExecute. - TestSdpaFwdPlanBuilder and TestSdpaBwdPlanBuilder (gfx942, and gfx950 for the forward, like the other isApplicable tests): IsApplicableRejectsUnsupportedAttnScale declines 0, -0.0, -1.2, -1/sqrt(128), -FLT_MIN, three subnormals, NaN and +-inf, each from attn_scale_value, a constant scale tensor and a runtime scale tensor default, plus an FP8_E4M3 scale tensor. IsApplicableAcceptsSupportedAttnScale accepts 1e-37, 1/sqrt(128) and 1 from each source, the unset default and a pure runtime scale tensor. - TestSdpaFwdPlanExecute and TestSdpaBwdPlanExecute: execute returns BAD_PARAM for each unsupported runtime scale (host only, on a plan with no loaded kernels), and passes supported ones on to the launch. Checked on gfx942 (MI300X). This commit's plugin replaced the one in a CI build of #14312, and results were compared with the CPU reference on the 82 scale cases behind the results above: - The 61 cases whose unsupported scale comes from attn_scale_value or a constant scale tensor are now declined. With the old plugin 48 of them failed, 12 passed (the no-mask cases named above) and one was already declined (fp16 forward). - The 4 cases with an unsupported runtime scale tensor (0 and -1.2, forward and backward) now fail at execute with BAD_PARAM, and the message gives the scale. The old plugin gave NaN or Inf in 3 of them. - The 15 cases with a supported scale pass with both plugins: 1/sqrt(128) with each mask, dtype and scale source, and 1e-37. The forward with 1e37 or 1e38 fails with both, as noted above. - On the develop CI build, develop's quick and standard SdpaFwd, SdpaFwdRuntimeScale and SdpaBackward bundles give the same result for each case as with develop's plugin: 42 pass and 39 are declined, for the same reasons, none of them the scale. - The unit tests pass. Each of 14 mutants that removes, widens or narrows one arm of these checks makes one of the new tests fail. Rebased onto develop 98b504d, after #14075, whose range gate now runs first in isApplicable. The GPU results above are from develop 61c2675. On the new base, the four ALMIOPEN-3035 ASM SDPA fix branches (backward layouts, attention scale, a16 accumulator, forward layouts) were built and run together: - hip_kernel_provider_tests: 912 pass and 7 skip on gfx942 (MI325X), 886 pass and 33 skip on gfx950 (MI355X); none fail. - hip_kernel_provider_integration_tests: none fail on either. - develop's SDPA bundles on ASM_SDPA_ENGINE, run like the CI lane (claims enforced): 62 pass and 56 skip on gfx942, 15 pass and 103 skip on gfx950, and no support claim fails. A host-side run of the real isApplicable over these bundles finds no outcome change from develop. - #14622's SdpaBackward bundles (138218c) with its temporary [[test_skips]] removed: 275 pass and 225 skip on gfx942, none fail, and no support claim fails. This change declines def06a, def06b and def06c (scale 0 and -1.2), the cases behind the DEF-06 skip, and no other case (host-checked for this branch alone). The execute-path scale tests (PassesSupportedRuntimeAttnScaleToLaunch) pass on both GPUs. - clang-tidy 20 and clang-format 18 are clean on the changed files. Co-Authored-By: Claude Opus 5.5 (1M context) <noreply@anthropic.com>
The SdpaBackward bundles come from the SDPA backward test plan (docs/test-plans/sdpa-bwd.md, RFC 0022), in the shape of the convolution-forward generator: - sdpa_bwd_plan.py declares the plan: levels and the baseline switches, the window semantics (left_bound = W - 1 on develop, W after SEM-001), the stride kinds of every layout, the cost model, ceilings and budgets, the legality layers (F, WF, EXT, U), the oracle gates with their tracking items, the purpose rule, the value domains, the tolerance classes and their rule, the magnitude estimator, the all-pairs axes and check, and the S, H, X, M and L catalog with the model-layer reduction rule. - generate_sdpa_bwd_bundles.py holds the level rules, the graph templates of all twelve topologies, the sweep writer (metadata with purpose, plan_items, size class, seed, per-uid inputs, notes and tolerance class), the TOML tolerance-row writer and the report. It writes only cases whose oracle is ready; the others stay declared with level notrun and their gate. --dry-run --report prints counts, budgets, the all-pairs proof, legality and gates. - sdpa_bwd_calibration.json records the NumPy magnitude estimates behind the tolerance classes, keyed by plan item and dtype with a fingerprint of the estimate's inputs. A run fails on a missing or stale estimate; --calibrate recomputes it. The generator reproduces the reviewed case list (sdpa_bwd_cases.json, 580 cases) case for case: ids, levels, topologies, dims, strides, attributes, inputs, purposes, plan items and tolerance classes. The differences are deliberate: S46 keeps its odd row pitch, H62 its zero K/V batch stride and H87 its Q = K = V = 133120 fill (all three not run), and the DEF entry gates name the companion engine pull requests. Co-Authored-By: Claude Opus 5.5 (1M context) <noreply@anthropic.com>
…est plan Replaces the 16 captured SdpaBackward cases with the 500 cases of the test plan, written by generate_sdpa_bwd_bundles.py: | Level | Cases | Sweeps | |---|---|---| | quick | 57 | Default, AdditiveMask, ScaleTensor | | standard | 151 | Default, AdditiveMask, DBiasOnly, ScaleTensor, DropoutScaleInv, SeqLenQOnly | | comprehensive | 222 | Default, AdditiveMask, DBiasOnly, ScaleTensor, FwdBwdChained | | full | 70 | Default | 80 more cases wait for a gate (#14358, the #14312 derivation extension, harness items) and are not written. Each sweep gets an empty support.json; claims are harvested. The templates make the band, attn_scale_value (null = unset), the deprecated causal flags, alibi_mask, dropout_probability, the node compute type and every tensor's alignment placeholders. A runtime pass-by-value scale patches the template's constant scale tensor. ASM_SDPA_ENGINE.toml: the two dtype rows become one row per tolerance class and dtype (12 rows), written between the generator's markers. Each case id ends with its class token, so every id matches one row. A [[test_skips]] entry skips the fp16 DEF-03 probes on ASM (fp16 dV loses precision at Skv <= 2, Sq >= 512 in the kernel; tracking issue to be filed). The probes stay in their levels so the skip is reported on every run and other engines run them. The DEF-01, DEF-02, DEF-04 and DEF-05 probes expect a typed decline from ASM_SDPA_ENGINE (companion engine pull request); the DEF-06 probes expect the decline being diagnosed. Land those first. Co-Authored-By: Claude Opus 5.5 (1M context) <noreply@anthropic.com>
test_generate_sdpa_bwd_bundles.py (pytest) checks that: - the checked-in SdpaBackward sweeps and the generated block of ASM_SDPA_ENGINE.toml equal the generator output, and the generator is deterministic; - every written case is legal (WF or EXT; rejection probes in U) and its sweep values and metadata match the case; - not-run cases are not written and each has a known gate; - the cases up to nightly cover every required pair of axis values, and every pair of claim values with a claim-only case; - every case id matches exactly one [[tolerance_overrides]] row, the row of its class, and no other graph name matches a row; every SdpaBackward [[test_skips]] filter still selects a written case; - every metadata.plan_items entry is a known feature, shape or plan row, and every feature has a case or a disposition; - the per-case ceilings and the cumulative budgets hold; - kept_pairs matches a brute-force count, the model reduction meets its target, and (with numpy) the estimator reproduces the calibration. Registered as hipdnn_sdpa_bwd_bundle_generator_python_tests, a pytest suite like the other pytest-style suites, and labelled quick and ffm-full through test_categories_external.yaml. Co-Authored-By: Claude Opus 5.5 (1M context) <noreply@anthropic.com>
… passes on gfx942 Adds 272 claims (ASM_SDPA_ENGINE, gfx942, linux) to seven SdpaBackward sidecars. They come from one gfx942 run of the generated bundles: - Node: MI300X (gfx942), 2026-10-26 23:48 UTC. - Binaries: hipdnn_integration_tests and the backend from the #14312 CI build (run 40350338181, head 186a38f). The ASM plugin comes from asm-sdpa-bwd-decline-strides (7c8f12a871d; its engine source equals 9c1d25e). That branch only adds declines, so the develop plugin accepts every claimed case too. - Settings as in CI: device fill, auto verification (the CPU reference), enforcing claims, a fresh process per level. Results: | Level | Cases | Pass | Skip | Fail | |---|---|---|---|---| | quick | 57 | 50 | 7 | 0 | | standard | 151 | 71 | 77 | 3 | | comprehensive | 222 | 109 | 113 | 0 | | full | 70 | 45 | 25 | 0 | The three failures are the DEF-06 probes (scale <= 0). The engine returns NaN or Inf for them. harvest_support_claims.py read the four level logs. No block was rejected and no claim failed. It kept 275 cells. Three of them are probes that passed (prbg01, prbr01, h12). RFC 0022 section 9 never claims a probe, so they are left out. That leaves 260 claim cases and 12 regression cases. A second run with atol 1e-9 and device fill printed the error of every passing case. Each claimed case passes its tolerance class with margin 2.4 or more (median 4.2). A rerun of quick, standard and comprehensive with these sidecars confirmed all 227 of their claims, with no claim failure. SUPPORT_MATRIX_OVERVIEW.md is not re-rendered. support_features/feature_matrix.py does not catalogue 43 SdpaBackward cases (head dims such as 80 and 256, FLOAT and FP8 IO, both deprecated causal flags, a transposed Q), so it renders nothing. This has been so since the bundles were generated. Co-Authored-By: Claude Opus 5.5 (1M context) <noreply@anthropic.com>
…eview) Adds dnn-providers/integration-tests/docs/test-plans/sdpa-bwd.md, the human-readable half of the SdpaBackward plan (ALMIOPEN-2894). sdpa_bwd_plan.py, generate_sdpa_bwd_bundles.py and the README already point at this path. The plan covers the feature inventory, the shape catalog (S, L, M, H, X), how the sweep is built (core cells, all-pairs, probes, regressions, budgets), the golden reference choice and the pass criteria. It is marked as a draft for review. The case counts in section 4.1 match the generator's --dry-run --report at this commit: 580 cases, 500 written (quick 57, PR 151, nightly 222, full 70) and 80 gated. The banner maps the plan's engine defects DEF-01 to DEF-08 to their ALMIOPEN tickets. The methodology and index links resolve once the first phase of RFC 0022 (#13937) lands. Co-Authored-By: Claude Opus 5.5 (1M context) <noreply@anthropic.com>
The DEF-03 [[test_skips]] entry in ASM_SDPA_ENGINE.toml said its tracking issue was still to be filed. It is ALMIOPEN-3046. Co-Authored-By: Claude Opus 5.5 (1M context) <noreply@anthropic.com>
…n's current sections The generator and plan-module comments, the calibration note and the generated TOML block still cited section numbers from an earlier draft of the plan (for example §4.8, §4.11 and §6.2 for the tolerance classes, which are now §6.1). They now cite the sections of docs/test-plans/sdpa-bwd.md as it stands. No case changes. Co-Authored-By: Claude Opus 5.5 (1M context) <noreply@anthropic.com>
… model probes The plan's trim rule (§4.4, register R-M6) moves probes up a level, largest first, until a level is inside its 5 % budget reserve. The plan already describes the result, but the generator never applied it, so PR (115.6 s of 120) and nightly (476.0 s of 480) ran over the reserve. M09.pr, M28.pr and M31.pr (6.6 s) move from PR to nightly; M03.n, M12.n, M20.n, M36.n and M43.n (25.4 s) move from nightly to full. All eight are probes, so no support claim moves. Cumulative budgets are now quick 18.4, PR 109.0, nightly 450.6 and full 1698.4 s, each inside its reserve; the all-pairs cover is unchanged (669 of 669 pairs). Case ids do not change. Co-Authored-By: Claude Opus 5.5 (1M context) <noreply@anthropic.com>
… until the declines land The PR level holds the open-defect probes of the SDPA backward test plan (§4.2): valid graphs ASM_SDPA_ENGINE accepts today and computes wrongly. Without a skip, every gfx942 PR run fails 20 of them. One [[test_skips]] entry per defect, each naming its ticket and the branch whose decline replaces it: - DEF-01, head-dim stride other than 1: ALMIOPEN-3036 - DEF-02, non-packed Stats: ALMIOPEN-3037 - DEF-04, Hk = Hq > Hv: ALMIOPEN-3039 - DEF-05 and H61, odd element strides: ALMIOPEN-3038 - DEF-06, attention scale of 0 or below: ALMIOPEN-3043 The pull request that adds each decline deletes its entry. Other engines still run these cases. None of them carries a support claim. Co-Authored-By: Claude Opus 5.5 (1M context) <noreply@anthropic.com>
…ne with the generator The level counts, size classes, topology counts, legality counts, cost table and shard split are recomputed from the generator after the trim (generate_sdpa_bwd_bundles.py --report and its case records): 500 written cases, quick 57, PR 148, nightly 220, full 75; 444 WF, 113 EXT and 23 U; PR 109.0 s and nightly 450.6 s cumulative. The open-defect section now describes the interim disposition the engine TOML uses: a DEF case sits in its level, and an engine that has the defect skips it, naming the ticket, until its decline lands. DEF-08 is marked as listed but not generated yet. Co-Authored-By: Claude Opus 5.5 (1M context) <noreply@anthropic.com>
RFC 0022 §9 tags each generated sweep case with a purpose. A probe exercises a value no engine claims, or an undefined graph a correct engine rejects, and the published support matrices exclude probes. The support-matrix units still took every sweep case, so the SdpaBackward test-plan probes (head dims outside the SDPA feature catalogue, fp32 IO, a transposed q, both deprecated causal flags) reached support_features/ and failed hipdnn_support_features_python_tests with "is not catalogued" errors. collect_sweep_units() now skips a case whose metadata.purpose is "probe", after the template and patch checks, so a broken probe is still reported. Cases without a purpose, claims and regressions are unchanged. Co-Authored-By: Claude Opus 5.5 (1M context) <noreply@anthropic.com>
…undles #14483 gave the SDPA left_bound the cuDNN meaning on develop (SEM-001): left_bound = L keeps the L keys ending at the diagonal, and 0 is illegal. The frontend, both references and ASM_SDPA_ENGINE moved with it. The generator still wrote develop's old form, left_bound = W - 1, so every window case described a window one key narrower than its id says, and the W = 1 cases wrote the now-illegal 0. WINDOW_SEMANTICS is now "after_sem001": the 117 window cases are regenerated with left_bound = W. Case ids, levels, tolerance classes and the TOML block are unchanged, and the claims still describe the same windows. The plan and the generator now cite #14483, which replaced the closed #14287, including in regression_of; the window-semantics test sets both meanings explicitly instead of assuming the default. Co-Authored-By: Claude Opus 5.5 (1M context) <noreply@anthropic.com>
…he SdpaBackward claims The SdpaBackward test plan added 272 ASM_SDPA_ENGINE claims on gfx942 and 264 SdpaBackward graphs that are claims or regressions; probes are left out of the matrices. Regenerated with support_features/feature_matrix.py, which test_committed_overview_is_current checks: Attention on gfx942 goes from 16 % to 42 % of its options, and SdpaBackward appears under ASM_SDPA_ENGINE at 64 %. Co-Authored-By: Claude Opus 5.5 (1M context) <noreply@anthropic.com>
…es ASM accepts The first gfx942 PR run of the SdpaBackward bundles on develop's ASM_SDPA_ENGINE failed three rejection probes of the PR level. The engine accepts each undefined graph instead of declining it: - prbu01 (O and dO with half the head dim) and prbu03 (dQ with half the sequence length): the CPU reference then throws on the shape. These are the dims mismatches of ALMIOPEN-3039. - prbu13 (dQ with a sequence stride below D, so its rows overlap): the aliased dQ fails the comparison on 74 % of its elements. This is the short-row class of ALMIOPEN-3038. The 2026-10-26 run used the asm-sdpa-bwd-decline-strides plugin, which declines all three, so it did not see them. Two temporary [[test_skips]] entries name the tickets and the fix branch, as for the DEF probes. The plan's list of rejection probes that reach a failing check, and the generator's recorded verdicts, now include these three. Co-Authored-By: Claude Opus 5.5 (1M context) <noreply@anthropic.com>
…es as UTF-8
hipdnn_sdpa_bwd_bundle_generator_python_tests failed on the Windows
superbuild: test_engine_toml_block_is_generated read ASM_SDPA_ENGINE.toml
with the platform default encoding, so the section sign in the generated
comment ("sdpa-bwd.md §6.1") did not round-trip. Every read_text and
write_text in the generator, the plan module and the tests now names
UTF-8, and writes use "\n" line endings so a Windows run writes the same
bytes. The tests also pass under an ASCII locale.
Co-Authored-By: Claude Opus 5.5 (1M context) <noreply@anthropic.com>
The UTF-8 change left three lines over black's limit, which failed the pre-commit job. Co-Authored-By: Claude Opus 5.5 (1M context) <noreply@anthropic.com>
brockhargreaves-amd
force-pushed
the
users/brockhargreaves-amd/sdpa-bwd-test-plan
branch
from
November 2, 2026 18:46
138218c to
5744d47
Compare
10 of 13 tasks
brockhargreaves-amd
added a commit
that referenced
this pull request
Nov 3, 2026
…ales SdpaFwdPlanBuilder::isApplicable and SdpaBwdPlanBuilder::isApplicable accepted any attention scale, and execute passed it to the kernels unchecked. cuDNN and hipDNN accept any scale value. On gfx942 (MI300X) a scale of 0, -0.0, a negative value or a subnormal gave NaN or Inf outputs and gradients instead of a decline. The mechanism comes from the .co disassembly. The forward and backward kernels write -inf into every masked score (causal and sliding-window masks, and the padding past S_kv in the last K tile), and only then multiply the scores by scale * log2(e). A scale of 0 turns those scores into NaN, and a negative scale turns them into +inf. The bf16 forward kernels also start the running row max at -FLT_MAX / (scale * log2(e)), which is positive for a negative scale or -0.0, so every exp2 overflows even with no mask. 32-bit denormals are flushed, so a subnormal scale acts as 0. An infinite scale gives NaN everywhere. The gfx942 runs used the plugin from a CI build of #14312, which accepts all of these scales, and compared results with the CPU reference: - Backward: 0, -0.0, 1e-40 and -1.2 gave NaN or Inf dQ, dK and dV with top-left or bottom-right causal masks and sliding windows, in bf16 and fp16, hd128 and hd192. With no mask and S_kv = 128 (the hd128 K tile is 192), all of dQ was NaN. - Forward: -1.2, -0.088 and -0.0 gave NaN O and Inf LSE even with no mask. 0 gave NaN O with a bottom-right causal mask or S_kv = 200. - A runtime pass-by-value scale tensor gave the same results as attn_scale_value. Both isApplicable functions now decline a scale that is not a finite, positive, normal float (FLT_MIN <= scale <= FLT_MAX; NaN fails). The check covers attn_scale_value, and the value of a compile-time constant or runtime-with-default pass-by-value scale tensor, narrowed to the f32 kernarg as execute narrows it. They also decline a scale tensor whose value the plugin SDK cannot read, on which buildPlan would throw. The value of a pure runtime pass-by-value scale tensor arrives only at execute, so both execute functions now throw BAD_PARAM for such a scale before the first launch, as the gfx1151 WMMA attention pack does for its runtime scale. The check is shared in SdpaPlanUtils.hpp. An unset scale (1.0) and every finite, positive, normal scale stay accepted; 1e-37 and 1/sqrt(128) were correct on gfx942. The rule also declines some cases that happened to compute correctly: no mask with S_kv a multiple of the K tile, for any of these scales in the backward and for +0.0 or a subnormal scale in the forward. A rule keyed on the mask and the tile would break once group mode makes the K tail per-sequence, so the decline does not depend on them. Scales near FLT_MAX (1e37, 1e38) still give NaN in the forward; that is a separate issue. New tests: - TestSdpaPlanUtils (host only): IsSupportedAttnScaleRequiresFinitePositiveNormalFloat and AttnScaleBeforeExecute. - TestSdpaFwdPlanBuilder and TestSdpaBwdPlanBuilder (gfx942, and gfx950 for the forward, like the other isApplicable tests): IsApplicableRejectsUnsupportedAttnScale declines 0, -0.0, -1.2, -1/sqrt(128), -FLT_MIN, three subnormals, NaN and +-inf, each from attn_scale_value, a constant scale tensor and a runtime scale tensor default, plus an FP8_E4M3 scale tensor. IsApplicableAcceptsSupportedAttnScale accepts 1e-37, 1/sqrt(128) and 1 from each source, the unset default and a pure runtime scale tensor. - TestSdpaFwdPlanExecute and TestSdpaBwdPlanExecute: execute returns BAD_PARAM for each unsupported runtime scale (host only, on a plan with no loaded kernels), and passes supported ones on to the launch. Checked on gfx942 (MI300X). This commit's plugin replaced the one in a CI build of #14312, and results were compared with the CPU reference on the 82 scale cases behind the results above: - The 61 cases whose unsupported scale comes from attn_scale_value or a constant scale tensor are now declined. With the old plugin 48 of them failed, 12 passed (the no-mask cases named above) and one was already declined (fp16 forward). - The 4 cases with an unsupported runtime scale tensor (0 and -1.2, forward and backward) now fail at execute with BAD_PARAM, and the message gives the scale. The old plugin gave NaN or Inf in 3 of them. - The 15 cases with a supported scale pass with both plugins: 1/sqrt(128) with each mask, dtype and scale source, and 1e-37. The forward with 1e37 or 1e38 fails with both, as noted above. - On the develop CI build, develop's quick and standard SdpaFwd, SdpaFwdRuntimeScale and SdpaBackward bundles give the same result for each case as with develop's plugin: 42 pass and 39 are declined, for the same reasons, none of them the scale. - The unit tests pass. Each of 14 mutants that removes, widens or narrows one arm of these checks makes one of the new tests fail. Rebased onto develop 98b504d, after #14075, whose range gate now runs first in isApplicable. The GPU results above are from develop 61c2675. On the new base, the four ALMIOPEN-3035 ASM SDPA fix branches (backward layouts, attention scale, a16 accumulator, forward layouts) were built and run together: - hip_kernel_provider_tests: 912 pass and 7 skip on gfx942 (MI325X), 886 pass and 33 skip on gfx950 (MI355X); none fail. - hip_kernel_provider_integration_tests: none fail on either. - develop's SDPA bundles on ASM_SDPA_ENGINE, run like the CI lane (claims enforced): 62 pass and 56 skip on gfx942, 15 pass and 103 skip on gfx950, and no support claim fails. A host-side run of the real isApplicable over these bundles finds no outcome change from develop. - #14622's SdpaBackward bundles (138218c) with its temporary [[test_skips]] removed: 275 pass and 225 skip on gfx942, none fail, and no support claim fails. This change declines def06a, def06b and def06c (scale 0 and -1.2), the cases behind the DEF-06 skip, and no other case (host-checked for this branch alone). The execute-path scale tests (PassesSupportedRuntimeAttnScaleToLaunch) pass on both GPUs. - clang-tidy 20 and clang-format 18 are clean on the changed files. Co-Authored-By: Claude Opus 5.5 (1M context) <noreply@anthropic.com>
Preserve both probe and expected-rejection matrix coverage, and regenerate the support overview from the merged bundle data.
Since #14358 the SDPA references decline every graph that sets dropout_probability (isSupportedSdpaDropout), including 0 with no seed, offset or mask. The input fill and ASM_SDPA_ENGINE read 0 as no dropout, so ASM accepts PRB-R01 (prbr01) and nothing can check its output: PR CI on 2026-11-02 failed it as Unverifiable in the ASM standard lane. PRB-R01 keeps quick as its target level but now waits on a new gate, p0: a SEM entry for p = 0 and a reference that models it. ALMIOPEN-3149 (compute it as no dropout) and ALMIOPEN-3150 (make it an invalid graph) are the two ways to open it. The quick level drops from 57 to 56 cases (499 written, 81 gated); the plan's counts, budgets and oracle-gap row follow, and the calibration entry is refreshed for the gated case. Co-Authored-By: Claude Opus 5.5 (1M context) <noreply@anthropic.com>
…ged references and flags With #14358 and #14346 on develop: - Every rejection probe now carries metadata.expected_rejection, so an engine that accepts one fails the lane instead of the case reporting SKIP. On gfx942 ASM_SDPA_ENGINE declines 17 of the 23 and accepts six, which are quarantined in ASM_SDPA_ENGINE_test_categories_integration.yaml with their tickets, as RFC 0022 §6 asks: prbu01 and prbu03 (ALMIOPEN-3039), prbu13 (ALMIOPEN-3038), prbt04 and prbt04n (ALMIOPEN-3050), prbr05 (ALMIOPEN-3051). The two rejection-probe [[test_skips]] rows leave ASM_SDPA_ENGINE.toml; the DEF rows stay. A generator test keeps the quarantine to written rejection probes. - PRB-M02 and PRB-M02f (right_bound = -2) leave the bundles: the frontend rejects them since #14358 (R14), and TestSdpaBwdNode's PreValidateChecksDiagonalBandBounds already asserts it. Legality is now checked against that frontend (F_R13R14). - The five dBias probes (PRB-B11 to PRB-B14, PRB-B16) and the live GL-14 case enter PR: both references model dBias since #14358. The trim rule moves M07.pr to nightly and M26.n to full to keep PR and nightly inside their reserve. REG-08 (rank-3 Stats) waits on the derivation extension instead. Written cases: 503 (quick 56, PR 152, nightly 220, full 75); 77 declared and gated. Co-Authored-By: Claude Opus 5.5 (1M context) <noreply@anthropic.com>
…rged RFC 0022 and develop - The status note, the level and golden-criteria links point at the merged RFC 0022 (#13937); its methodology guide and test-plans index do not exist yet. The baseline is develop 961dd76, with #14358, #14346 and #14709. - Deviations: the merged RFC adopted DV-1 (§7 exception) and DV-2 (§12.3 uplift default) and lists DV-3 as an oracle gap (§11); only DV-4 still needs approval. - Gates that named #14358 now name what is left (I6, I4, HG4, HG20, GPU forward, timing run); the reference-status table, the oracle columns and the §6.4 gap table show #14358's references as in place. - §2.8 to §4: F_active is F_R13R14; PRB-M02 and PRB-M02f are frontend unit tests; rejection probes carry expected_rejection, and the six ASM accepts are quarantined in the tier YAML; the dBias probes and GL-14 are written; the second trim (M07.pr, M26.n); counts, budgets and topology rows from the generator. - §4.5 records the 2026-11-04 measured run on gfx942 and the gfx950 run. - §5: #14358's golden cases stay under quick/SdpaBackward/Golden; GT1 and the SEM-010 −inf exception are in place, so GL-09 and GL-16b can join the first golden data. Co-Authored-By: Claude Opus 5.5 (1M context) <noreply@anthropic.com>
brockhargreaves-amd
marked this pull request as ready for review
November 4, 2026 21:22
adickin-amd
approved these changes
Nov 5, 2026
adickin-amd
left a comment
adickin-amd
left a comment
Contributor
There was a problem hiding this comment.
LGTM, This is a great start on the test bundles for SDPA BWD and it has already exposed issues were actively closing. Lets get this in an proceed with filling out more test cases as we see fit.
adickin-amd
reviewed
Nov 5, 2026
brockhargreaves-amd
deleted the
users/brockhargreaves-amd/sdpa-bwd-test-plan
branch
November 5, 2026 19:17
AnaghaRaoAMD
added a commit
that referenced
this pull request
Nov 5, 2026
…2-dense-aot The only conflict is the generated SUPPORT_MATRIX_OVERVIEW.md, which #14622 (SDPA backward bundles) and this branch's stats bundles both changed; it is regenerated from the merged tree with feature_matrix.py. The 224 shipped gfx942 dense kernels lower to the same LLVM IR before and after the merge. Co-Authored-By: Claude <noreply@anthropic.com>
3 of 4 tasks
adickin-amd
added a commit
that referenced
this pull request
Nov 6, 2026
… claim PRs (#15293) ## Summary Stop regenerating `SUPPORT_MATRIX_OVERVIEW.md` in every PR that changes bundles or support claims. Every such PR edited the same generated file, so each merge made the other open claim PRs conflict on it and forced another develop merge and regenerate (for example #14984 after #14259 and #14622 merged). The overview is now refreshed by hand in a PR of its own, and no check fails when it lags. JIRA ID : ALMIOPEN-3185 ## Risk Assessment Risk 1. Python tooling, a pre-commit hook and docs only; no product code, no build change. The only behavior removed is the staleness check on a generated file. Errors in the overview renderer and in the sidecars it reads are still caught by the render hook and the `hipdnn_support_features_python_tests` ctest. ## ASIC Coverage None. Host-only Python tooling and docs; no GPU code path changes. ## Testing Summary - Unit: `test_harvest_support_claims.py` and `test_support_features.py`, including the real-tree render and size checks. - Smoke: with the committed overview deliberately edited out of date, the `TestRealBundleTree` ctest tests pass and `feature_matrix.py --check` reports it as stale (exit 1). - Pre-commit on the changed files, including the new `render-support-matrix-overview` hook. ## Testing Checklist - [x] Support-claim tooling tests - `python -m pytest dnn-providers/integration-tests/scripts/test_harvest_support_claims.py dnn-providers/integration-tests/scripts/test_support_features.py` - Status: Passed (1914 tests) - [x] Stale overview no longer fails - `pytest test_support_features.py -k RealBundleTree` against an edited overview - Status: Passed - [x] Pre-commit - `pre-commit run --files <changed files>` - Status: Passed - [ ] PR CI - GitHub PR checks - Status: Pending ## Technical Changes - `test_support_features.py`: remove `test_committed_overview_is_current`. The real-tree render and the size limit stay. - `.pre-commit-config.yaml`: replace `check-support-matrix-overview` (`--check`) with `render-support-matrix-overview`, which renders to stdout and discards, like `render-support-matrix`. - `harvest_support_claims.py`: `--write` no longer re-renders the overview; `refresh_overview` and its tests are removed, and a test checks that `--write` leaves the overview alone. - `feature_matrix.py`: the rendered overview states that it can lag develop and points to the current copy in the `hipdnn-support-matrix` CI artifact. `--check` stays for telling whether a refresh is due. - `SUPPORT_MATRIX_OVERVIEW.md`: regenerated for that header note. - `docs/support-matrix.md`, `docs/adding-tests.md`, RFC 0015: the overview is not committed with sidecar or bundle changes; after a harvest or a batch of claim PRs merges, whoever landed them refreshes it in a PR with only that file. Open claim PRs that already carry an overview change should drop it once this merges: `git checkout origin/develop -- dnn-providers/integration-tests/SUPPORT_MATRIX_OVERVIEW.md`.
This file contains hidden or bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
Sign up for free
to join this conversation on GitHub.
Already have an account?
Sign in to comment
Add this suggestion to a batch that can be applied as a single commit.This suggestion is invalid because no changes were made to the code.Suggestions cannot be applied while the pull request is closed.Suggestions cannot be applied while viewing a subset of changes.Only one suggestion per line can be applied in a batch.Add this suggestion to a batch that can be applied as a single commit.Applying suggestions on deleted lines is not supported.You must change the existing code in this line in order to create a valid suggestion.Outdated suggestions cannot be applied.This suggestion has been applied or marked resolved.Suggestions cannot be applied from pending reviews.Suggestions cannot be applied on multi-line comments.Suggestions cannot be applied while the pull request is queued to merge.Suggestion cannot be applied right now. Please check back later.
Summary
This PR adds the SDPA backward test plan required by RFC 0022 (Per-Op Test Plans, merged in #13937). It also adds a generator that writes the SdpaBackward bundles from the plan, and the bundles it writes. The 16 inherited SdpaBackward cases (all B = H = 1, packed BHSD, fixed scale) are replaced by 503 generated cases in four levels. Each case carries a purpose and traces to the plan items it covers.
JIRA ID : ALMIOPEN-2894
Risk Assessment
Risk 2 of 5. The PR changes test data, test tooling, test configuration and documentation only; no library or provider code changes. The changes that reach beyond the SdpaBackward bundles:
support_matrix_units.collect_sweep_units()leaves out cases whosemetadata.purposeisprobe, as RFC 0022 §9 requires. Today only SdpaBackward cases carry apurpose, so other ops' matrices are unchanged. The committedSUPPORT_MATRIX_OVERVIEW.mdis re-rendered.ASM_SDPA_ENGINE.toml: two dtype-only SdpaBackward tolerance rows are replaced by 12 generated rows (seven tolerance classes), plus temporary skips for the open engine defects, each naming its ticket.ASM_SDPA_ENGINE_test_categories_integration.yaml: six rejection probes that ASM_SDPA_ENGINE wrongly accepts are quarantined with their tickets (RFC 0022 §6). Both files affect the ASM lane only.quick/SdpaBackward/{Default,AdditiveMask}/toquick/SdpaBackward/Golden/{Default,AdditiveMask}/, so the generator owns theDefaultsweeps. They keep their fp16 tolerance row.bhsd/and the*_fwd_pairbundles are untouched.The main risk is CI noise from new cases. Every level has been run on gfx942 and gfx950 with no failure (see Testing Checklist).
ASIC Coverage
The bundles run in every provider lane whose tier YAML matches
*Sdpa*. OnlyASM_SDPA_ENGINEaccepts SdpaBackward graphs today, and only on gfx942: its backward plan builder declines every other device ("Device string does not match gfx942"). Claims are harvested for gfx942 only. PR CI tests gfx942. On gfx950 every generated backward case is declined, so a gfx950 run only shows that nothing fails; one is recorded below. PR CI on gfx942 is sufficient.Testing Summary
hipdnn_sdpa_bwd_bundle_generator_python_tests, labelquick, 19 tests). They fail when the checked-in sweeps or the TOML block drift from the generator. They also check legality, all-pairs coverage up to nightly, the per-case ceilings and cumulative budgets of each level, two-way traceability between plan items and cases, that every TOML skip still matches a written case, and that the tier-YAML quarantine holds only rejection probes.generate_sdpa_bwd_bundles.py --dry-run --report. Reports counts, budgets, the all-pairs proof, legality, and each declared-but-gated case with its gate.ASM_SDPA_ENGINE, configured like CI.harvest_support_claims.pyturned the passing cases into the 272 claims. Every claimed case passes its tolerance class with a margin of at least 2.4 (median 4.2).hipdnn_integration_testsran onASM_SDPA_ENGINEconfigured like the CI lane:--test-engine(claims enforced), the PR'sASM_SDPA_ENGINE.toml, the tier YAML's quarantine as a GTest filter, device input fill and the GPU reference as oracle. It ran all four levels, on a fresh MI300X (gfx942) and an MI355X (gfx950) node, built from this branch. The bundles, TOML and YAML are identical to6e014ad50e0.Testing Checklist
hipdnn_sdpa_bwd_bundle_generator_python_tests- 19 of 19 - Status: Passedpython3 reference-data-scripts/generate_sdpa_bwd_bundles.py --dry-run --report- 503 written, 77 gated, 0 legality violations, all-pairs 669 of 669 - Status: Passedverify_support_claims.py,render_support_matrix.py --output -,feature_matrix.py --check; the bundle, claim, render and support-feature pytest suites - Status: Passed (verify_golden_bundles.pyreports no SdpaBackward error; its 322 errors in other ops' bundles are on develop too)hipdnn_integration_tests,ASM_SDPA_ENGINE, MI300X (gfx942), 2026-10-26 - Status: Passedhipdnn_integration_tests --test-engine ASM_SDPA_ENGINE --test-config ASM_SDPA_ENGINE.toml '--gtest_filter=*SdpaBackward*-<the tier YAML quarantine>'- MI300X, Slurm job 68275259 - Status: Passed. 553 tests in 43.3 s, 0 failed, no claim broken:--gtest_filter=*Sdpa*:-*SdpaBackward*- Status: Passed. gfx942: 47 passed, 84 skipped. gfx950: 16 passed, 115 skipped. 0 failed on either.6e014ad50e0. The 2026-11-02 run failed one case,prbr01, which is now gated. Later runs failed only on the hipCCL configure break from feat(rocthrust,hipcub): add compatibility with libhipcxx 3.4 via legacy headers #14976, which develop reverted in Revert "feat(rocthrust,hipcub): add compatibility with libhipcxx 3.4 via legacy headers" #15089 (merged into this branch).Technical Changes
What this adds
The plan:
dnn-providers/integration-tests/docs/test-plans/sdpa-bwd.md. It follows the RFC 0022 template: the operation, the feature inventory, the shape catalog, building the sweep (levels, budgets, probes, regressions, trade-off register), the golden reference, pass criteria, and exhaustive mode.The generator:
reference-data-scripts/sdpa_bwd_plan.pyholds the declarations: axes, legality, the shape catalog, level rules and the cost model, one record per case.generate_sdpa_bwd_bundles.pywrites the sweeps, the support-claim files and the engine-TOML tolerance block from them. Nobody edits the bundles by hand. RFC 0022 open question 1 cites this module as the per-op Python option.The bundles: 503 written cases, spread across the levels:
quick/standard/comprehensive/full/purpose(claim,probeorregression) and its plan items.dropout_probability = 0). Since feat(hipdnn): SDPA CPU and GPU references cover the full SDPA graph API #14358 the references decline any graph that setsdropout_probability, so it waits on ALMIOPEN-3149 or ALMIOPEN-3150.Levels: the cost model assigns each case to the lowest level whose per-case limits it fits:
Each cumulative budget covers that level and every level below it, and keeps 5 % in reserve. The budgets come from a CPU-reference cost model; with the GPU reference, the measured run above is far below them. The ASM lane YAML picks the cases up by GTest prefix (
quick_*Sdpa*,standard_*Sdpa*and so on), and each level also runs every level below it.Rejection probes (21) carry
metadata.expected_rejection(feat(hipdnn-integration-tests): no-reference and expected-rejection bundle metadata flags #14346): the harness never executes them, and an engine that accepts one fails the lane. PRB-M02 and PRB-M02f (right_bound = -2) are not written, because the frontend rejects them since feat(hipdnn): SDPA CPU and GPU references cover the full SDPA graph API #14358;TestSdpaBwdNode.PreValidateChecksDiagonalBandBoundscovers that rule.dBias: the five dBias probes and the live GL-14 case are written, now that feat(hipdnn): SDPA CPU and GPU references cover the full SDPA graph API #14358's references model dBias. To keep PR and nightly inside their reserve, the plan's trim rule moved M07.pr to nightly and M26.n to full.
Tolerance classes: seven classes, written as 12
[[tolerance_overrides]]rows between generated BEGIN and END markers inASM_SDPA_ENGINE.toml.Support claims: 272 claims for
ASM_SDPA_ENGINEon gfx942.Probes left out of the support matrices (RFC 0022 §9). This is the
support_matrix_unitschange above.Generator file I/O is UTF-8, so its tests pass on Windows.
Temporary ASM_SDPA_ENGINE exclusions
These stay until each fix lands; the PR with the fix deletes its entries.
Open engine defects (
[[test_skips]]inASM_SDPA_ENGINE.toml). These are valid graphs that ASM accepts and computes wrong. Other engines still run them.Quarantined rejection probes (
_common_excludesinASM_SDPA_ENGINE_test_categories_integration.yaml, as RFC 0022 §6 asks). These are undefined graphs ASM should decline but accepts.dropout_scale_invwithout a dropout maskDependencies (all merged)
docs/test-plans/index, are not on develop yet, so the plan links the RFC's sections directly.left_bounduses cuDNN's meaning. The bundles writeleft_bound = W.automode. It also adds frontend rule R14 (right_bound >= -1).expected_rejectionandno_*_referencebundle flags.Tensor.Interaction with other open PRs
fix(hip-kernel-provider): ASM SDPA backward declines layouts and dims its kernels address wrongly #14684, fix(hip-kernel-provider): ASM SDPA forward declines unaddressable layouts #14686, fix(hip-kernel-provider): ASM SDPA backward rejects the a16 accumulator #14687, fix(hip-kernel-provider): ASM SDPA declines non-positive attention scales #14688 (ASM declines, ALMIOPEN-3035). Each deletes its own entries: TOML skips for the DEF cases, and tier-YAML lines for the quarantined probes it fixes (fix(hip-kernel-provider): ASM SDPA backward declines layouts and dims its kernels address wrongly #14684: prbu01, prbu03, prbu13). Whichever of these and this PR lands second removes the entries.
test(integration-tests): SDPA forward plan bundles (draft set) #14999 (SDPA forward plan bundles, ALMIOPEN-2893). Both PRs edit two files the same way:
ASM_SDPA_ENGINE.toml: keep this PR's generated block and put test(integration-tests): SDPA forward plan bundles (draft set) #14999'sSdpaFwdPlanskips after the# END SdpaBackward tolerance classesline. The generator test only checks the BEGIN to END span. The filters do not overlap (SdpaBackwardvsSdpaFwdPlan).SUPPORT_MATRIX_OVERVIEW.md: re-render it; do not hand-merge.The two PRs also add lines near each other in
integration-tests/CMakeLists.txtandtest_categories_external.yaml; these auto-merge.Before review
prbr01(dropout_probability = 0) gated; ALMIOPEN-3149 and ALMIOPEN-3150 track the decision.expected_rejection; the six ASM accepts are quarantined in the ASM tier YAML with their tickets.prbm02andprbm02fremoved; the frontend unit test of R14 covers them.🤖 Generated with Claude Code