Skip to content

docs(hipdnn): RFC 0022 per-op test plans (ALMIOPEN-2947) - #13937

Merged
adickin-amd merged 7 commits into
developfrom
users/addickin/rfc_per_op_test_plans
Oct 30, 2026
Merged

adickin-amd merged 7 commits into
developfrom
users/addickin/rfc_per_op_test_plans

Conversation

@adickin-amd

@adickin-amd adickin-amd commented Oct 19, 2026 •

Copy link
Copy Markdown
Contributor

Summary

Adds RFC 0022, the high-level approach for per-op test bundles that run in CI (quick / PR / nightly / full). Each op family gets one reviewed, engine-independent plan that fixes which cases its bundles hold, why each case exists, and what was left out. Exhaustive testing is a separate tool with its own RFC. This RFC defines only the hook that plans declare for it.

JIRA ID : ALMIOPEN-2947

Risk Assessment

Risk 1/5. One new RFC document and one index row under projects/hipdnn/docs/rfcs/. No product, test or build changes.

ASIC Coverage

No ASIC impact. Docs only. Standard PR CI is sufficient.

Testing Summary

  • Docs only. No code, so no build or test run applies.
  • Pre-commit hooks ran on commit. The applicable hooks passed.

Testing Checklist

  • pre-commit hooks - run automatically at git commit - Status: Passed
  • PR CI - GitHub PR checks - Status: Pending

Technical Changes

  • Adds projects/hipdnn/docs/rfcs/0022_PerOpTestPlans.md and its row in the RFC index.
  • Per-op plan method: features derived from the graph API, shape catalog and size ladder, sweep rules, case purpose (claim, probe, regression), trade-off register, oracle prerequisites, plan template and review checklist.
  • Four levels, each with the question it answers, and a mapping from plan to sweep.json, support.json claims, per-engine TOML, tier YAML and CTest lanes.
  • Golden policy: baseline third-party check per op (G-A), with other criteria only where they apply. Not exhaustive.
  • Exhaustive mode: each plan declares a hook. The tool and its RFC are separate and out of scope.
  • New ops: an op cannot land without a test plan and the bundles built from it (§16.2).
  • Rollout order: attention, convolution, then normalization. Lists the docs and stories to update on merge.
  • An audit of the convolution-forward bundles is the problem evidence. No per-op plan or bundle is in this PR.
  • Related: RFC 0011 (bundle test framework), RFC 0015 (engine support claims), RFC 0017 and 0018 (descriptor fuzzing, out of scope).

Adds RFC 0021: a per-op test plan methodology for the integration suite. It covers feature inventory, shape catalog and size ladder, tier rules, case purpose, trade-off register, oracle prerequisites, golden selection, and the test generator and fuzzing tool.

JIRA ID : ALMIOPEN-2895
@therock-pr-bot

therock-pr-bot Bot commented Oct 19, 2026 •

Copy link
Copy Markdown

✅ All Checks Passed — Ready for Review

Check Status Details
📝 PR Description ✅ Pass —
⛔ Forbidden Files ✅ Pass —
🧪 Unit Test ✅ Pass PR does not contain code files — Unit Test auto-passed
🔎 pre-commit ✅ Pass —
🚫 Draft PR 🔜 To Be Enabled —
🚩 Feature Flag 🔜 To Be Enabled —
📊 Code Coverage 🔜 To Be Enabled —
🤖 therock-pr-bot ✅ Pass —

🎉 All checks passed! This PR is ready for review.

📖 Need help? See the Policy FAQ for details on every check and how to fix failures.

🙋 Wish to Override Policy?

@therock-pr-bot

Copy link
Copy Markdown

🎉 All checks passed! This PR is ready for review.

The convolution-forward plan lands in a separate PR. Remove its link, worked-example register and rollout phases. Keep the convolution-forward audit as problem evidence.

JIRA ID : ALMIOPEN-2895
@adickin-amd adickin-amd changed the title docs(hipdnn): RFC 0021 per-op test plans (ALMIOPEN-2895) docs(hipdnn): RFC 0021 per-op test plans (ALMIOPEN-2947) Oct 19, 2026
Make exhaustive testing a separate tool with a hook only. Keep plans engine-independent. Add the level questions, the plan-to-file mapping, the rollout order (attention, convolution, normalization), the baseline golden policy, and the documents to update.

JIRA ID : ALMIOPEN-2947
…ire plans for new ops

- Rename to 0022 (0021 is taken by kernel slots, #13839).
- Add the status header and the index row.
- Fix the RFC 0011 link (0011_BundleTestFramework.md).
- Link every section and RFC reference.
- Add 16.2: a new op cannot land without a test plan and its bundles.
@adickin-amd adickin-amd changed the title docs(hipdnn): RFC 0021 per-op test plans (ALMIOPEN-2947) docs(hipdnn): RFC 0022 per-op test plans (ALMIOPEN-2947) Oct 26, 2026
@adickin-amd
adickin-amd marked this pull request as ready for review October 27, 2026 20:37
@adickin-amd
adickin-amd requested a review from a team as a code owner October 27, 2026 20:37
brockhargreaves-amd added a commit that referenced this pull request Oct 27, 2026
…eview)

Adds dnn-providers/integration-tests/docs/test-plans/sdpa-bwd.md, the
human-readable half of the SdpaBackward plan (ALMIOPEN-2894).
sdpa_bwd_plan.py, generate_sdpa_bwd_bundles.py and the README already
point at this path.

The plan covers the feature inventory, the shape catalog (S, L, M, H,
X), how the sweep is built (core cells, all-pairs, probes,
regressions, budgets), the golden reference choice and the pass
criteria.

It is marked as a draft for review. The case counts in section 4.1
match the generator's --dry-run --report at this commit: 580 cases,
500 written (quick 57, PR 151, nightly 222, full 70) and 80 gated.
The banner maps the plan's engine defects DEF-01 to DEF-08 to their
ALMIOPEN tickets.

The methodology and index links resolve once the first phase of RFC 0022
(#13937) lands.

Co-Authored-By: Claude Opus 5.5 (1M context) <noreply@anthropic.com>

@brockhargreaves-amd brockhargreaves-amd left a comment

Copy link
Copy Markdown
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

🤖 AI-generated review (Claude Code), posted by @brockhargreaves-amd. Feedback from applying this RFC to SdpaBackward (ALMIOPEN-2894). The plan is dnn-providers/integration-tests/docs/test-plans/sdpa-bwd.md on users/brockhargreaves-amd/sdpa-bwd-test-plan, with its generator and 500 generated cases.

The method held up. Nothing here blocks the RFC. These are the places where the SdpaBackward plan had to deviate, or that may be worth folding into the text:

  • Source of truth (line 137). The text says the plan is the source of truth and the generator reads it. The inline comment on that line says the plan describes the approach and is not the source of truth for the generated tests. The SdpaBackward branch follows the text: the cases are declared in a per-op Python module that the generator reads, and the Markdown plan explains them. Settling the wording before merge would tell plan authors which file reviewers should treat as authoritative. The per-op module also worked well as an answer to open question 1: the generator's tests prove all-pairs coverage, legality and plan-item traceability.
  • §7 ceilings. The quick starting value (≤ 20 MFLOP, ≤ 1 MB) is below every existing SdpaBackward quick case (37–126 MFLOP) and below a useful default shape (66 MFLOP). The CPU reference is about 90 % of each SdpaBackward case's wall time, so the plan budgets in measured wall time instead (quick ≤ 1.5 s per case, 20 s in total), from a cost model fitted on gfx942. The RFC could say that ceilings may be stated as wall time when the reference dominates.
  • §7 non-degenerate defaults. A head-group ratio above 1, or Dqk ≠ Dv, in the default shape would put the head-grouping and Dv paths into every S and L case. One defect would then fail them all, and S and L would no longer isolate one feature. The plan covers both as their own S, H and L cases and all-pairs axes instead. A carve-out for ratios whose non-unit values select a separate code path may be worth adding.
  • §12.3 precision, open question 2. For bf16 and fp16 backward, golden data computed in FP64 and rounded once was reproducible across BLAS builds: 0 differing elements between NumPy/OpenBLAS and torch/MKL over 57 bundles × 5 tensors. Matched FP32 moved 1–5 ulps between torch builds. The kernel's own P and dS rounding sets the tolerance either way. This argues against matched precision as the default for SdpaBackward.
  • §12.2 non-finite rule. Fully masked rows legitimately have LSE = −inf, so no SDPA case with such a row can be golden under the current rule. Proposed addition: −inf in an SDPA stats tensor, on exactly the rows that the case's bounds and lengths mask completely, does not count as a non-finite value for this rule.
  • §8.1 core cover for fused topologies. A forward-plus-backward graph cannot meet core cover or get a G-A anchor until a reference executes it and the golden generator writes fused graphs. The plan registers this as a known gap.

@CMiservaAMD CMiservaAMD left a comment

Copy link
Copy Markdown
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

A couple thoughts for consideration.

Declarations are the source of truth for bundles. Rejection coverage goes beyond frontend validation via expected_rejection bundles; a rejection gap is an engine applicability bug (fix or quarantine). Probe is tolerant, expected_rejection strict. Optional motivation note for engine-specific cases. Wall-time ceilings and default-ratio carve-out in section 7. FP64 uplift is the default golden precision. SDPA -inf stats exception. Fused-topology reference gap.
@adickin-amd

Copy link
Copy Markdown
Contributor Author

@brockhargreaves-amd thanks for the SdpaBackward feedback. All six points are addressed in abf9887:

  • Source of truth (L137): §4 now names the declarations as the source of truth. The plan records the approach and traces to them. Your per-op Python module is cited as evidence under open question 1.
  • §7 ceilings: ceilings may be stated as measured wall time when the CPU reference dominates. The numeric starting values are unchanged.
  • §7 non-degenerate defaults: carve-out added. A ratio whose non-unit values select a separate code path stays at 1 in the default shape. It is covered as its own S, H and L cases and as all-pairs axes.
  • §12.3 precision: FP64 uplift, rounded once, is now the default, citing your 0-differing-elements result. Open question 2 now asks whether that holds for convolution and normalization.
  • §12.2 non-finite rule: exception added for -inf on exactly the rows that the case masks completely. The RFC also says the verifier must apply it. That verifier change is not made yet.
  • §8.1 fused topologies: not a rule change. Added to the §11 oracle-gap table as a fused topology with no executing reference.

brockhargreaves-amd added a commit that referenced this pull request Oct 29, 2026
…eview)

Adds dnn-providers/integration-tests/docs/test-plans/sdpa-bwd.md, the
human-readable half of the SdpaBackward plan (ALMIOPEN-2894).
sdpa_bwd_plan.py, generate_sdpa_bwd_bundles.py and the README already
point at this path.

The plan covers the feature inventory, the shape catalog (S, L, M, H,
X), how the sweep is built (core cells, all-pairs, probes,
regressions, budgets), the golden reference choice and the pass
criteria.

It is marked as a draft for review. The case counts in section 4.1
match the generator's --dry-run --report at this commit: 580 cases,
500 written (quick 57, PR 151, nightly 222, full 70) and 80 gated.
The banner maps the plan's engine defects DEF-01 to DEF-08 to their
ALMIOPEN tickets.

The methodology and index links resolve once the first phase of RFC 0022
(#13937) lands.

Co-Authored-By: Claude Opus 5.5 (1M context) <noreply@anthropic.com>
@adickin-amd
adickin-amd merged commit 1ac351c into develop Oct 30, 2026
35 checks passed
@adickin-amd
adickin-amd deleted the users/addickin/rfc_per_op_test_plans branch October 30, 2026 21:13
brockhargreaves-amd added a commit that referenced this pull request Nov 2, 2026
…eview)

Adds dnn-providers/integration-tests/docs/test-plans/sdpa-bwd.md, the
human-readable half of the SdpaBackward plan (ALMIOPEN-2894).
sdpa_bwd_plan.py, generate_sdpa_bwd_bundles.py and the README already
point at this path.

The plan covers the feature inventory, the shape catalog (S, L, M, H,
X), how the sweep is built (core cells, all-pairs, probes,
regressions, budgets), the golden reference choice and the pass
criteria.

It is marked as a draft for review. The case counts in section 4.1
match the generator's --dry-run --report at this commit: 580 cases,
500 written (quick 57, PR 151, nightly 222, full 70) and 80 gated.
The banner maps the plan's engine defects DEF-01 to DEF-08 to their
ALMIOPEN tickets.

The methodology and index links resolve once the first phase of RFC 0022
(#13937) lands.

Co-Authored-By: Claude Opus 5.5 (1M context) <noreply@anthropic.com>
@assistant-librarian

Copy link
Copy Markdown
Contributor

TheRock Submodule Bump Activity

Newest first

brockhargreaves-amd added a commit that referenced this pull request Nov 4, 2026
…rged RFC 0022 and develop

- The status note, the level and golden-criteria links point at the merged
  RFC 0022 (#13937); its methodology guide and test-plans index do not exist
  yet. The baseline is develop 961dd76, with #14358, #14346 and #14709.
- Deviations: the merged RFC adopted DV-1 (§7 exception) and DV-2 (§12.3
  uplift default) and lists DV-3 as an oracle gap (§11); only DV-4 still
  needs approval.
- Gates that named #14358 now name what is left (I6, I4, HG4, HG20, GPU
  forward, timing run); the reference-status table, the oracle columns and
  the §6.4 gap table show #14358's references as in place.
- §2.8 to §4: F_active is F_R13R14; PRB-M02 and PRB-M02f are frontend unit
  tests; rejection probes carry expected_rejection, and the six ASM accepts
  are quarantined in the tier YAML; the dBias probes and GL-14 are written;
  the second trim (M07.pr, M26.n); counts, budgets and topology rows from
  the generator.
- §4.5 records the 2026-11-04 measured run on gfx942 and the gfx950 run.
- §5: #14358's golden cases stay under quick/SdpaBackward/Golden; GT1 and
  the SEM-010 −inf exception are in place, so GL-09 and GL-16b can join the
  first golden data.

Co-Authored-By: Claude Opus 5.5 (1M context) <noreply@anthropic.com>
brockhargreaves-amd added a commit that referenced this pull request Nov 5, 2026
…rated bundles (#14622)

## Summary

This PR adds the SDPA backward test plan required by RFC 0022 (Per-Op
Test Plans, merged in #13937). It also adds a generator that writes the
SdpaBackward bundles from the plan, and the bundles it writes. The 16
inherited SdpaBackward cases (all B = H = 1, packed BHSD, fixed scale)
are replaced by 503 generated cases in four levels. Each case carries a
purpose and traces to the plan items it covers.

> Written with AI assistance (Claude Code). The plan follows the merged
RFC 0022; it records one remaining deviation (DV-4) for reviewers to
approve.

JIRA ID : ALMIOPEN-2894

## Risk Assessment

Risk 2 of 5. The PR changes test data, test tooling, test configuration
and documentation only; no library or provider code changes. The changes
that reach beyond the SdpaBackward bundles:

- `support_matrix_units.collect_sweep_units()` leaves out cases whose
`metadata.purpose` is `probe`, as RFC 0022 §9 requires. Today only
SdpaBackward cases carry a `purpose`, so other ops' matrices are
unchanged. The committed `SUPPORT_MATRIX_OVERVIEW.md` is re-rendered.
- `ASM_SDPA_ENGINE.toml`: two dtype-only SdpaBackward tolerance rows are
replaced by 12 generated rows (seven tolerance classes), plus temporary
skips for the open engine defects, each naming its ticket.
- `ASM_SDPA_ENGINE_test_categories_integration.yaml`: six rejection
probes that ASM_SDPA_ENGINE wrongly accepts are quarantined with their
tickets (RFC 0022 §6). Both files affect the ASM lane only.
- #14358's eight PyTorch golden cases move, unchanged, from
`quick/SdpaBackward/{Default,AdditiveMask}/` to
`quick/SdpaBackward/Golden/{Default,AdditiveMask}/`, so the generator
owns the `Default` sweeps. They keep their fp16 tolerance row. `bhsd/`
and the `*_fwd_pair` bundles are untouched.

The main risk is CI noise from new cases. Every level has been run on
gfx942 and gfx950 with no failure (see Testing Checklist).

## ASIC Coverage

The bundles run in every provider lane whose tier YAML matches `*Sdpa*`.
Only `ASM_SDPA_ENGINE` accepts SdpaBackward graphs today, and only on
gfx942: its backward plan builder declines every other device ("Device
string does not match gfx942"). Claims are harvested for gfx942 only. PR
CI tests gfx942. On gfx950 every generated backward case is declined, so
a gfx950 run only shows that nothing fails; one is recorded below. PR CI
on gfx942 is sufficient.

## Testing Summary

- **Generator tests** (`hipdnn_sdpa_bwd_bundle_generator_python_tests`,
label `quick`, 19 tests). They fail when the checked-in sweeps or the
TOML block drift from the generator. They also check legality, all-pairs
coverage up to nightly, the per-case ceilings and cumulative budgets of
each level, two-way traceability between plan items and cases, that
every TOML skip still matches a written case, and that the tier-YAML
quarantine holds only rejection probes.
- **`generate_sdpa_bwd_bundles.py --dry-run --report`.** Reports counts,
budgets, the all-pairs proof, legality, and each declared-but-gated case
with its gate.
- **Claims run** (2026-10-26, MI300X, gfx942). All generated bundles ran
on `ASM_SDPA_ENGINE`, configured like CI. `harvest_support_claims.py`
turned the passing cases into the 272 claims. Every claimed case passes
its tolerance class with a margin of at least 2.4 (median 4.2).
- **Measured run, every level on GPU** (2026-11-04, plan §4.5).
`hipdnn_integration_tests` ran on `ASM_SDPA_ENGINE` configured like the
CI lane: `--test-engine` (claims enforced), the PR's
`ASM_SDPA_ENGINE.toml`, the tier YAML's quarantine as a GTest filter,
device input fill and the GPU reference as oracle. It ran all four
levels, on a fresh MI300X (gfx942) and an MI355X (gfx950) node, built
from this branch. The bundles, TOML and YAML are identical to
`6e014ad50e0`.
- **PR CI** runs the quick and standard levels on gfx942.

## Testing Checklist

- [x] Generator tests - CTest
`hipdnn_sdpa_bwd_bundle_generator_python_tests` - 19 of 19 - Status:
Passed
- [x] Plan report - `python3
reference-data-scripts/generate_sdpa_bwd_bundles.py --dry-run --report`
- 503 written, 77 gated, 0 legality violations, all-pairs 669 of 669 -
Status: Passed
- [x] Bundle and support-matrix checks - `verify_support_claims.py`,
`render_support_matrix.py --output -`, `feature_matrix.py --check`; the
bundle, claim, render and support-feature pytest suites - Status: Passed
(`verify_golden_bundles.py` reports no SdpaBackward error; its 322
errors in other ops' bundles are on develop too)
- [x] pre-commit on the changed files - Status: Passed
- [x] Claims run - `hipdnn_integration_tests`, `ASM_SDPA_ENGINE`, MI300X
(gfx942), 2026-10-26 - Status: Passed
- [x] Measured run, gfx942 - `hipdnn_integration_tests --test-engine
ASM_SDPA_ENGINE --test-config ASM_SDPA_ENGINE.toml
'--gtest_filter=*SdpaBackward*-<the tier YAML quarantine>'` - MI300X,
Slurm job 68275259 - Status: Passed. 553 tests in 43.3 s, 0 failed, no
claim broken:
- 289 executed and passed against the GPU reference, and 15 expected
rejections rejected;
  - 226 declined by ASM, and 23 hit the DEF skips;
  - per level: quick 2.4 s, PR 2.8 s, nightly 16.5 s, full 21.4 s.
- [x] Quarantine check, gfx942 - the six quarantined cases run alone -
Status: all six fail as expected ("Engine ASM_SDPA_ENGINE accepted
expected-rejection bundle"), which shows the quarantine is needed.
- [x] Every level, gfx950 - same command - MI355X, Slurm job 68275260 -
Status: Passed. 0 failed: ASM declines every backward graph on gfx950,
and the rejection probes, quarantined ones included, are correctly
rejected.
- [x] The other SDPA bundles, same lane -
`--gtest_filter=*Sdpa*:-*SdpaBackward*` - Status: Passed. gfx942: 47
passed, 84 skipped. gfx950: 16 passed, 115 skipped. 0 failed on either.
- [ ] PR CI on gfx942 - TheRock multi-arch CI - Status: Pending on
`6e014ad50e0`. The 2026-11-02 run failed one case, `prbr01`, which is
now gated. Later runs failed only on the hipCCL configure break from
#14976, which develop reverted in #15089 (merged into this branch).

## Technical Changes

### What this adds

- **The plan:**
`dnn-providers/integration-tests/docs/test-plans/sdpa-bwd.md`. It
follows the RFC 0022 template: the operation, the feature inventory, the
shape catalog, building the sweep (levels, budgets, probes, regressions,
trade-off register), the golden reference, pass criteria, and exhaustive
mode.
- **The generator:** `reference-data-scripts/sdpa_bwd_plan.py` holds the
declarations: axes, legality, the shape catalog, level rules and the
cost model, one record per case. `generate_sdpa_bwd_bundles.py` writes
the sweeps, the support-claim files and the engine-TOML tolerance block
from them. Nobody edits the bundles by hand. RFC 0022 open question 1
cites this module as the per-op Python option.
- **The bundles:** 503 written cases, spread across the levels:

  | Level | Directory | Cases |
  |---|---|---|
  | quick | `quick/` | 56 |
  | PR | `standard/` | 152 |
  | nightly | `comprehensive/` | 220 |
  | full | `full/` | 75 |

- Each case carries `purpose` (`claim`, `probe` or `regression`) and its
plan items.
- 77 more cases are declared but not written yet, each waiting on a
named gate: an oracle extension, a harness feature or golden data.
- One of the 77 is PRB-R01 (`dropout_probability = 0`). Since #14358 the
references decline any graph that sets `dropout_probability`, so it
waits on ALMIOPEN-3149 or ALMIOPEN-3150.
- **Levels:** the cost model assigns each case to the lowest level whose
per-case limits it fits:

  | Level | Per-case time | Per-case bytes | Cumulative budget |
  |---|---|---|---|
  | quick | 1.5 s | 4 MiB | 20 s |
  | PR | 5 s | 16 MiB | 120 s |
  | nightly | 75 s | 64 MiB | 480 s |
  | full | 300 s | no limit | 1,800 s |

Each cumulative budget covers that level and every level below it, and
keeps 5 % in reserve. The budgets come from a CPU-reference cost model;
with the GPU reference, the measured run above is far below them. The
ASM lane YAML picks the cases up by GTest prefix (`quick_*Sdpa*`,
`standard_*Sdpa*` and so on), and each level also runs every level below
it.
- **Rejection probes** (21) carry `metadata.expected_rejection`
(#14346): the harness never executes them, and an engine that accepts
one fails the lane. PRB-M02 and PRB-M02f (`right_bound = -2`) are not
written, because the frontend rejects them since #14358;
`TestSdpaBwdNode.PreValidateChecksDiagonalBandBounds` covers that rule.
- **dBias:** the five dBias probes and the live GL-14 case are written,
now that #14358's references model dBias. To keep PR and nightly inside
their reserve, the plan's trim rule moved M07.pr to nightly and M26.n to
full.
- **Tolerance classes:** seven classes, written as 12
`[[tolerance_overrides]]` rows between generated BEGIN and END markers
in `ASM_SDPA_ENGINE.toml`.
- **Support claims:** 272 claims for `ASM_SDPA_ENGINE` on gfx942.
- **Probes left out of the support matrices** (RFC 0022 §9). This is the
`support_matrix_units` change above.
- **Generator file I/O is UTF-8**, so its tests pass on Windows.

### Temporary ASM_SDPA_ENGINE exclusions

These stay until each fix lands; the PR with the fix deletes its
entries.

**Open engine defects** (`[[test_skips]]` in `ASM_SDPA_ENGINE.toml`).
These are valid graphs that ASM accepts and computes wrong. Other
engines still run them.

| Cases | What goes wrong | Ticket | Fix |
|---|---|---|---|
| DEF-01 | head-dim stride other than 1 | ALMIOPEN-3036 | #14684 |
| DEF-02 | non-packed Stats | ALMIOPEN-3037 | #14684 |
| DEF-04 | Hk = Hq > Hv | ALMIOPEN-3039 | #14684 |
| DEF-05 and H61 | odd element strides | ALMIOPEN-3038 | #14684 |
| DEF-06 | scale of 0 or below | ALMIOPEN-3043 | #14688 |
| DEF-03 | fp16 dV precision in the AITER kernel | ALMIOPEN-3046 | none
in hipDNN (kernel) |

**Quarantined rejection probes** (`_common_excludes` in
`ASM_SDPA_ENGINE_test_categories_integration.yaml`, as RFC 0022 §6
asks). These are undefined graphs ASM should decline but accepts.

| Cases | What ASM accepts | Ticket | Fix |
|---|---|---|---|
| prbu01, prbu03 | O or dQ dims that differ from Q's | ALMIOPEN-3039 |
#14684 |
| prbu13 | a dQ sequence stride below D (overlapping rows) |
ALMIOPEN-3038 | #14684 |
| prbt04, prbt04n | an O dtype that differs from Q's | ALMIOPEN-3050 |
none yet |
| prbr05 | `dropout_scale_inv` without a dropout mask | ALMIOPEN-3051 |
none yet |

### Dependencies (all merged)

- #13937, RFC 0022 (2026-10-30). Its first-phase documents, the
methodology guide and the `docs/test-plans/` index, are not on develop
yet, so the plan links the RFC's sections directly.
- #14312: O and Stats derived from the forward of the same inputs.
- #14483: `left_bound` uses cuDNN's meaning. The bundles write
`left_bound = W`.
- #14358 (2026-11-02): SDPA references cover the full graph API, and the
GPU reference is the first oracle in `auto` mode. It also adds frontend
rule R14 (`right_bound >= -1`).
- #14346 (2026-11-03): the `expected_rejection` and `no_*_reference`
bundle flags.
- #14709 (2026-11-02): zero strides in the data SDK `Tensor`.

### Interaction with other open PRs

- **#14684, #14686, #14687, #14688** (ASM declines, ALMIOPEN-3035). Each
deletes its own entries: TOML skips for the DEF cases, and tier-YAML
lines for the quarantined probes it fixes (#14684: prbu01, prbu03,
prbu13). Whichever of these and this PR lands second removes the
entries.
- **#14999** (SDPA forward plan bundles, ALMIOPEN-2893). Both PRs edit
two files the same way:
- `ASM_SDPA_ENGINE.toml`: keep this PR's generated block and put
#14999's `SdpaFwdPlan` skips after the `# END SdpaBackward tolerance
classes` line. The generator test only checks the BEGIN to END span. The
filters do not overlap (`SdpaBackward` vs `SdpaFwdPlan`).
  - `SUPPORT_MATRIX_OVERVIEW.md`: re-render it; do not hand-merge.

The two PRs also add lines near each other in
`integration-tests/CMakeLists.txt` and `test_categories_external.yaml`;
these auto-merge.

### Before review

- [x] `prbr01` (`dropout_probability = 0`) gated; ALMIOPEN-3149 and
ALMIOPEN-3150 track the decision.
- [x] Rejection probes carry `expected_rejection`; the six ASM accepts
are quarantined in the ASM tier YAML with their tickets.
- [x] `prbm02` and `prbm02f` removed; the frontend unit test of R14
covers them.
- [x] The five dBias probes and GL-14 written; REG-08 (rank-3 Stats)
still waits for the derivation extension.
- [x] The plan refreshed against the merged RFC: DV-1 and DV-2 now agree
with it, DV-3 is an RFC §11 gap, and only DV-4 remains a deviation.
- [x] The measured run (plan §4.5) recorded above.

🤖 Generated with [Claude Code](https://claude.com/claude-code)

---------

Co-authored-by: Claude Opus 5.5 (1M context) <noreply@anthropic.com>
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Projects

None yet

Development

Successfully merging this pull request may close these issues.

3 participants