Repository navigation
feat(hipdnn-integration-tests): no-reference and expected-rejection bundle metadata flags - #14346
Merged
Merged
Conversation
…d expected-rejection bundles ALMIOPEN-3022: no_cpu_reference / no_gpu_reference metadata flags drop a reference from the oracle chain; disabling both requires declared golden data (loader and verify_golden_bundles.py); explicit-mode and not-pulled skips; golden-data lanes register a visible skip, or a failure beside a known-gap entry. ALMIOPEN-3023: expected_rejection bundles must be refused by the pinned engine at ranked-list or plan-build time; kept out of support claims, the matrix and coverage; stage shown on the VERIFIER line and in the Verified-by summary. Adds quick/Matmul/nc/fp16 invalid Matmul examples (twins of a hipBLASLt-claimed case). Docs updated. Both tickets land in one commit because the metadata, harness and docs changes interleave.
…n registration counter test ALMIOPEN-3023 follow-up.
…ch-mismatch Matmul rejection bundle ALMIOPEN-3023: hipBLASLt accepted the K-mismatch twin on gfx90a (alola run), so it was not a rejection; the batch-dimension mismatch is invalid per the Matmul contract and refused by hipBLASLt MatmulParams.
…ejection example ALMIOPEN-3023: the compute-type variant is merely unsupported by hipBLASLt; the batch mismatch is invalid per the Matmul contract and is refused by hipBLASLt at the ranked list (alola gfx90a). Docs and verifier test follow the bundle.
…lags; authoring exclusion testable ALMIOPEN-3023 review round 1: the both-references-off rule no longer applies to an expected_rejection bundle (C++ loader and verify_golden_bundles.py); single-graph golden-declared now means the .tensors.dvc pointer or a blob for every output; write-support-claims is read through HarnessPolicy so the authoring exclusion is unit tested; two tautological tests removed.
…s the engine table ALMIOPEN-3023 review round 1 follow-up.
…olden-declared; doc wording ALMIOPEN-3023 review round 2: a graph with a virtual intermediate output and a blob only for the real output declares golden data (verify_golden_bundles.py), matching the loader; file-formats.md says every non-virtual output.
✅ All Checks Passed — Ready for Review
📖 Need help? See the Policy FAQ for details on every check and how to fix failures. |
|
🎉 All checks passed! This PR is ready for review. |
adickin-amd
marked this pull request as ready for review
October 26, 2026 14:06
…dle_noref_invalid_flags # Conflicts: # dnn-providers/integration-tests/docs/file-formats.md # dnn-providers/integration-tests/src/harness/bundle/IntegrationBundleVerificationHarness.cpp # dnn-providers/integration-tests/src/harness/bundle/IntegrationBundleVerificationHarness.hpp
…dle_noref_invalid_flags # Conflicts: # dnn-providers/integration-tests/reference-data-scripts/README.md # dnn-providers/integration-tests/reference-data-scripts/tests/test_verify_golden_bundles.py
…dle_noref_invalid_flags
…ap list judgeLane looked up a gap entry for a metadata-disabled lane in the global knownReferenceGaps() instead of the gaps it was given, so an injected list was ignored on that path. Use the argument. The two tests covering this now use the fixture gap list rather than real SDPA bundle ids, which no longer have gap entries.
…lid_flags' into users/addickin/bundle_noref_invalid_flags
Contributor
|
Suggestion (docs): Could
A matching line under "When no reference executor supports the op" (the "add the op to a reference executor" bullet) would put the hint where someone, or an agent, adding reference support will read it. Not blocking. |
mousdahl-amd
approved these changes
Nov 3, 2026
mousdahl-amd
left a comment
mousdahl-amd
left a comment
Contributor
There was a problem hiding this comment.
LGTM, non-blocking comment.
15 of 16 tasks
brockhargreaves-amd
added a commit
that referenced
this pull request
Nov 4, 2026
…ged references and flags With #14358 and #14346 on develop: - Every rejection probe now carries metadata.expected_rejection, so an engine that accepts one fails the lane instead of the case reporting SKIP. On gfx942 ASM_SDPA_ENGINE declines 17 of the 23 and accepts six, which are quarantined in ASM_SDPA_ENGINE_test_categories_integration.yaml with their tickets, as RFC 0022 §6 asks: prbu01 and prbu03 (ALMIOPEN-3039), prbu13 (ALMIOPEN-3038), prbt04 and prbt04n (ALMIOPEN-3050), prbr05 (ALMIOPEN-3051). The two rejection-probe [[test_skips]] rows leave ASM_SDPA_ENGINE.toml; the DEF rows stay. A generator test keeps the quarantine to written rejection probes. - PRB-M02 and PRB-M02f (right_bound = -2) leave the bundles: the frontend rejects them since #14358 (R14), and TestSdpaBwdNode's PreValidateChecksDiagonalBandBounds already asserts it. Legality is now checked against that frontend (F_R13R14). - The five dBias probes (PRB-B11 to PRB-B14, PRB-B16) and the live GL-14 case enter PR: both references model dBias since #14358. The trim rule moves M07.pr to nightly and M26.n to full to keep PR and nightly inside their reserve. REG-08 (rank-3 Stats) waits on the derivation extension instead. Written cases: 503 (quick 56, PR 152, nightly 220, full 75); 77 declared and gated. Co-Authored-By: Claude Opus 5.5 (1M context) <noreply@anthropic.com>
brockhargreaves-amd
added a commit
that referenced
this pull request
Nov 4, 2026
…rged RFC 0022 and develop - The status note, the level and golden-criteria links point at the merged RFC 0022 (#13937); its methodology guide and test-plans index do not exist yet. The baseline is develop 961dd76, with #14358, #14346 and #14709. - Deviations: the merged RFC adopted DV-1 (§7 exception) and DV-2 (§12.3 uplift default) and lists DV-3 as an oracle gap (§11); only DV-4 still needs approval. - Gates that named #14358 now name what is left (I6, I4, HG4, HG20, GPU forward, timing run); the reference-status table, the oracle columns and the §6.4 gap table show #14358's references as in place. - §2.8 to §4: F_active is F_R13R14; PRB-M02 and PRB-M02f are frontend unit tests; rejection probes carry expected_rejection, and the six ASM accepts are quarantined in the tier YAML; the dBias probes and GL-14 are written; the second trim (M07.pr, M26.n); counts, budgets and topology rows from the generator. - §4.5 records the 2026-11-04 measured run on gfx942 and the gfx950 run. - §5: #14358's golden cases stay under quick/SdpaBackward/Golden; GT1 and the SEM-010 −inf exception are in place, so GL-09 and GL-16b can join the first golden data. Co-Authored-By: Claude Opus 5.5 (1M context) <noreply@anthropic.com>
Contributor
TheRock Submodule Bump ActivityNewest first
|
brockhargreaves-amd
added a commit
that referenced
this pull request
Nov 5, 2026
…rated bundles (#14622) ## Summary This PR adds the SDPA backward test plan required by RFC 0022 (Per-Op Test Plans, merged in #13937). It also adds a generator that writes the SdpaBackward bundles from the plan, and the bundles it writes. The 16 inherited SdpaBackward cases (all B = H = 1, packed BHSD, fixed scale) are replaced by 503 generated cases in four levels. Each case carries a purpose and traces to the plan items it covers. > Written with AI assistance (Claude Code). The plan follows the merged RFC 0022; it records one remaining deviation (DV-4) for reviewers to approve. JIRA ID : ALMIOPEN-2894 ## Risk Assessment Risk 2 of 5. The PR changes test data, test tooling, test configuration and documentation only; no library or provider code changes. The changes that reach beyond the SdpaBackward bundles: - `support_matrix_units.collect_sweep_units()` leaves out cases whose `metadata.purpose` is `probe`, as RFC 0022 §9 requires. Today only SdpaBackward cases carry a `purpose`, so other ops' matrices are unchanged. The committed `SUPPORT_MATRIX_OVERVIEW.md` is re-rendered. - `ASM_SDPA_ENGINE.toml`: two dtype-only SdpaBackward tolerance rows are replaced by 12 generated rows (seven tolerance classes), plus temporary skips for the open engine defects, each naming its ticket. - `ASM_SDPA_ENGINE_test_categories_integration.yaml`: six rejection probes that ASM_SDPA_ENGINE wrongly accepts are quarantined with their tickets (RFC 0022 §6). Both files affect the ASM lane only. - #14358's eight PyTorch golden cases move, unchanged, from `quick/SdpaBackward/{Default,AdditiveMask}/` to `quick/SdpaBackward/Golden/{Default,AdditiveMask}/`, so the generator owns the `Default` sweeps. They keep their fp16 tolerance row. `bhsd/` and the `*_fwd_pair` bundles are untouched. The main risk is CI noise from new cases. Every level has been run on gfx942 and gfx950 with no failure (see Testing Checklist). ## ASIC Coverage The bundles run in every provider lane whose tier YAML matches `*Sdpa*`. Only `ASM_SDPA_ENGINE` accepts SdpaBackward graphs today, and only on gfx942: its backward plan builder declines every other device ("Device string does not match gfx942"). Claims are harvested for gfx942 only. PR CI tests gfx942. On gfx950 every generated backward case is declined, so a gfx950 run only shows that nothing fails; one is recorded below. PR CI on gfx942 is sufficient. ## Testing Summary - **Generator tests** (`hipdnn_sdpa_bwd_bundle_generator_python_tests`, label `quick`, 19 tests). They fail when the checked-in sweeps or the TOML block drift from the generator. They also check legality, all-pairs coverage up to nightly, the per-case ceilings and cumulative budgets of each level, two-way traceability between plan items and cases, that every TOML skip still matches a written case, and that the tier-YAML quarantine holds only rejection probes. - **`generate_sdpa_bwd_bundles.py --dry-run --report`.** Reports counts, budgets, the all-pairs proof, legality, and each declared-but-gated case with its gate. - **Claims run** (2026-10-26, MI300X, gfx942). All generated bundles ran on `ASM_SDPA_ENGINE`, configured like CI. `harvest_support_claims.py` turned the passing cases into the 272 claims. Every claimed case passes its tolerance class with a margin of at least 2.4 (median 4.2). - **Measured run, every level on GPU** (2026-11-04, plan §4.5). `hipdnn_integration_tests` ran on `ASM_SDPA_ENGINE` configured like the CI lane: `--test-engine` (claims enforced), the PR's `ASM_SDPA_ENGINE.toml`, the tier YAML's quarantine as a GTest filter, device input fill and the GPU reference as oracle. It ran all four levels, on a fresh MI300X (gfx942) and an MI355X (gfx950) node, built from this branch. The bundles, TOML and YAML are identical to `6e014ad50e0`. - **PR CI** runs the quick and standard levels on gfx942. ## Testing Checklist - [x] Generator tests - CTest `hipdnn_sdpa_bwd_bundle_generator_python_tests` - 19 of 19 - Status: Passed - [x] Plan report - `python3 reference-data-scripts/generate_sdpa_bwd_bundles.py --dry-run --report` - 503 written, 77 gated, 0 legality violations, all-pairs 669 of 669 - Status: Passed - [x] Bundle and support-matrix checks - `verify_support_claims.py`, `render_support_matrix.py --output -`, `feature_matrix.py --check`; the bundle, claim, render and support-feature pytest suites - Status: Passed (`verify_golden_bundles.py` reports no SdpaBackward error; its 322 errors in other ops' bundles are on develop too) - [x] pre-commit on the changed files - Status: Passed - [x] Claims run - `hipdnn_integration_tests`, `ASM_SDPA_ENGINE`, MI300X (gfx942), 2026-10-26 - Status: Passed - [x] Measured run, gfx942 - `hipdnn_integration_tests --test-engine ASM_SDPA_ENGINE --test-config ASM_SDPA_ENGINE.toml '--gtest_filter=*SdpaBackward*-<the tier YAML quarantine>'` - MI300X, Slurm job 68275259 - Status: Passed. 553 tests in 43.3 s, 0 failed, no claim broken: - 289 executed and passed against the GPU reference, and 15 expected rejections rejected; - 226 declined by ASM, and 23 hit the DEF skips; - per level: quick 2.4 s, PR 2.8 s, nightly 16.5 s, full 21.4 s. - [x] Quarantine check, gfx942 - the six quarantined cases run alone - Status: all six fail as expected ("Engine ASM_SDPA_ENGINE accepted expected-rejection bundle"), which shows the quarantine is needed. - [x] Every level, gfx950 - same command - MI355X, Slurm job 68275260 - Status: Passed. 0 failed: ASM declines every backward graph on gfx950, and the rejection probes, quarantined ones included, are correctly rejected. - [x] The other SDPA bundles, same lane - `--gtest_filter=*Sdpa*:-*SdpaBackward*` - Status: Passed. gfx942: 47 passed, 84 skipped. gfx950: 16 passed, 115 skipped. 0 failed on either. - [ ] PR CI on gfx942 - TheRock multi-arch CI - Status: Pending on `6e014ad50e0`. The 2026-11-02 run failed one case, `prbr01`, which is now gated. Later runs failed only on the hipCCL configure break from #14976, which develop reverted in #15089 (merged into this branch). ## Technical Changes ### What this adds - **The plan:** `dnn-providers/integration-tests/docs/test-plans/sdpa-bwd.md`. It follows the RFC 0022 template: the operation, the feature inventory, the shape catalog, building the sweep (levels, budgets, probes, regressions, trade-off register), the golden reference, pass criteria, and exhaustive mode. - **The generator:** `reference-data-scripts/sdpa_bwd_plan.py` holds the declarations: axes, legality, the shape catalog, level rules and the cost model, one record per case. `generate_sdpa_bwd_bundles.py` writes the sweeps, the support-claim files and the engine-TOML tolerance block from them. Nobody edits the bundles by hand. RFC 0022 open question 1 cites this module as the per-op Python option. - **The bundles:** 503 written cases, spread across the levels: | Level | Directory | Cases | |---|---|---| | quick | `quick/` | 56 | | PR | `standard/` | 152 | | nightly | `comprehensive/` | 220 | | full | `full/` | 75 | - Each case carries `purpose` (`claim`, `probe` or `regression`) and its plan items. - 77 more cases are declared but not written yet, each waiting on a named gate: an oracle extension, a harness feature or golden data. - One of the 77 is PRB-R01 (`dropout_probability = 0`). Since #14358 the references decline any graph that sets `dropout_probability`, so it waits on ALMIOPEN-3149 or ALMIOPEN-3150. - **Levels:** the cost model assigns each case to the lowest level whose per-case limits it fits: | Level | Per-case time | Per-case bytes | Cumulative budget | |---|---|---|---| | quick | 1.5 s | 4 MiB | 20 s | | PR | 5 s | 16 MiB | 120 s | | nightly | 75 s | 64 MiB | 480 s | | full | 300 s | no limit | 1,800 s | Each cumulative budget covers that level and every level below it, and keeps 5 % in reserve. The budgets come from a CPU-reference cost model; with the GPU reference, the measured run above is far below them. The ASM lane YAML picks the cases up by GTest prefix (`quick_*Sdpa*`, `standard_*Sdpa*` and so on), and each level also runs every level below it. - **Rejection probes** (21) carry `metadata.expected_rejection` (#14346): the harness never executes them, and an engine that accepts one fails the lane. PRB-M02 and PRB-M02f (`right_bound = -2`) are not written, because the frontend rejects them since #14358; `TestSdpaBwdNode.PreValidateChecksDiagonalBandBounds` covers that rule. - **dBias:** the five dBias probes and the live GL-14 case are written, now that #14358's references model dBias. To keep PR and nightly inside their reserve, the plan's trim rule moved M07.pr to nightly and M26.n to full. - **Tolerance classes:** seven classes, written as 12 `[[tolerance_overrides]]` rows between generated BEGIN and END markers in `ASM_SDPA_ENGINE.toml`. - **Support claims:** 272 claims for `ASM_SDPA_ENGINE` on gfx942. - **Probes left out of the support matrices** (RFC 0022 §9). This is the `support_matrix_units` change above. - **Generator file I/O is UTF-8**, so its tests pass on Windows. ### Temporary ASM_SDPA_ENGINE exclusions These stay until each fix lands; the PR with the fix deletes its entries. **Open engine defects** (`[[test_skips]]` in `ASM_SDPA_ENGINE.toml`). These are valid graphs that ASM accepts and computes wrong. Other engines still run them. | Cases | What goes wrong | Ticket | Fix | |---|---|---|---| | DEF-01 | head-dim stride other than 1 | ALMIOPEN-3036 | #14684 | | DEF-02 | non-packed Stats | ALMIOPEN-3037 | #14684 | | DEF-04 | Hk = Hq > Hv | ALMIOPEN-3039 | #14684 | | DEF-05 and H61 | odd element strides | ALMIOPEN-3038 | #14684 | | DEF-06 | scale of 0 or below | ALMIOPEN-3043 | #14688 | | DEF-03 | fp16 dV precision in the AITER kernel | ALMIOPEN-3046 | none in hipDNN (kernel) | **Quarantined rejection probes** (`_common_excludes` in `ASM_SDPA_ENGINE_test_categories_integration.yaml`, as RFC 0022 §6 asks). These are undefined graphs ASM should decline but accepts. | Cases | What ASM accepts | Ticket | Fix | |---|---|---|---| | prbu01, prbu03 | O or dQ dims that differ from Q's | ALMIOPEN-3039 | #14684 | | prbu13 | a dQ sequence stride below D (overlapping rows) | ALMIOPEN-3038 | #14684 | | prbt04, prbt04n | an O dtype that differs from Q's | ALMIOPEN-3050 | none yet | | prbr05 | `dropout_scale_inv` without a dropout mask | ALMIOPEN-3051 | none yet | ### Dependencies (all merged) - #13937, RFC 0022 (2026-10-30). Its first-phase documents, the methodology guide and the `docs/test-plans/` index, are not on develop yet, so the plan links the RFC's sections directly. - #14312: O and Stats derived from the forward of the same inputs. - #14483: `left_bound` uses cuDNN's meaning. The bundles write `left_bound = W`. - #14358 (2026-11-02): SDPA references cover the full graph API, and the GPU reference is the first oracle in `auto` mode. It also adds frontend rule R14 (`right_bound >= -1`). - #14346 (2026-11-03): the `expected_rejection` and `no_*_reference` bundle flags. - #14709 (2026-11-02): zero strides in the data SDK `Tensor`. ### Interaction with other open PRs - **#14684, #14686, #14687, #14688** (ASM declines, ALMIOPEN-3035). Each deletes its own entries: TOML skips for the DEF cases, and tier-YAML lines for the quarantined probes it fixes (#14684: prbu01, prbu03, prbu13). Whichever of these and this PR lands second removes the entries. - **#14999** (SDPA forward plan bundles, ALMIOPEN-2893). Both PRs edit two files the same way: - `ASM_SDPA_ENGINE.toml`: keep this PR's generated block and put #14999's `SdpaFwdPlan` skips after the `# END SdpaBackward tolerance classes` line. The generator test only checks the BEGIN to END span. The filters do not overlap (`SdpaBackward` vs `SdpaFwdPlan`). - `SUPPORT_MATRIX_OVERVIEW.md`: re-render it; do not hand-merge. The two PRs also add lines near each other in `integration-tests/CMakeLists.txt` and `test_categories_external.yaml`; these auto-merge. ### Before review - [x] `prbr01` (`dropout_probability = 0`) gated; ALMIOPEN-3149 and ALMIOPEN-3150 track the decision. - [x] Rejection probes carry `expected_rejection`; the six ASM accepts are quarantined in the ASM tier YAML with their tickets. - [x] `prbm02` and `prbm02f` removed; the frontend unit test of R14 covers them. - [x] The five dBias probes and GL-14 written; REG-08 (rank-3 Stats) still waits for the derivation extension. - [x] The plan refreshed against the merged RFC: DV-1 and DV-2 now agree with it, DV-3 is an RFC §11 gap, and only DV-4 remains a deviation. - [x] The measured run (plan §4.5) recorded above. 🤖 Generated with [Claude Code](https://claude.com/claude-code) --------- Co-authored-by: Claude Opus 5.5 (1M context) <noreply@anthropic.com>
Open
1 of 4 tasks
This file contains hidden or bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
Sign up for free
to join this conversation on GitHub.
Already have an account?
Sign in to comment
Add this suggestion to a batch that can be applied as a single commit.This suggestion is invalid because no changes were made to the code.Suggestions cannot be applied while the pull request is closed.Suggestions cannot be applied while viewing a subset of changes.Only one suggestion per line can be applied in a batch.Add this suggestion to a batch that can be applied as a single commit.Applying suggestions on deleted lines is not supported.You must change the existing code in this line in order to create a valid suggestion.Outdated suggestions cannot be applied.This suggestion has been applied or marked resolved.Suggestions cannot be applied from pending reviews.Suggestions cannot be applied on multi-line comments.Suggestions cannot be applied while the pull request is queued to merge.Suggestion cannot be applied right now. Please check back later.
Summary
Adds three bundle metadata flags to the cross-provider integration tests:
no_cpu_reference/no_gpu_referencelet a bundle opt out of a reference oracle.expected_rejectionmarks a bundle as intentionally invalid: the engine under test must reject it.Also adds one invalid Matmul bundle that proves the expected-rejection flow end to end.
JIRA ID : ALMIOPEN-3022
JIRA ID : ALMIOPEN-3023
Risk Assessment
Risk 2/5. Changes are confined to the integration-test harness, its Python tooling, docs, and one new quick-tier bundle. There is no change to hipDNN core, provider plugins, public API/ABI or FlatBuffers schemas. All three flags default to
false, so existing bundles behave as before. The new quick bundle runs in every provider lane, and an engine that accepts it fails that lane by design.ASIC Coverage
The harness logic is host-side and arch-independent. The new bundle is rejected by a host-side hipBLASLt parameter check (
Unsupported input matrix batch dimensions), so arch-independence is expected but has not been run beyond gfx90a. Local validation ran only on gfx90a (MI210, Linux); MI300 was unavailable. PR CI provides the first run on gfx942/gfx950 and on the Windows hipBLASLt lane. Passing PR CI is sufficient; no multi-arch sweep is required.Testing Summary
hipdnn_integration_tests_unit_tests, mocked engine runner and reference executors) cover:from_binaryfailure, unpinned skip);verify_golden_bundles.py,verify_support_claims.py, the support-matrix units and feature coverage.quick/Matmul/ncl/fp32/ExpectedRejectionMatmulBatchMismatchis a twin of the hipBLASLt-claimed casequick/Matmul/Default3_16_128_fp32_ncl_6d8970. Only B's batch dimension changes, from 1 to 2, which is not broadcast-compatible with 3. hipBLASLt rejects it at the ranked list, and the valid twin passes in the same lane. Other lanes reject it vacuously because they have no Matmul support. No checked-in bundle uses the no-reference flags yet, so ALMIOPEN-3022 is proven by unit and Python tests.Testing Checklist
hipdnn-providers) - Linux gfx90a - Status: Passedctest -R test_name_validation -V- hipdnn 11062 / integration-tests 15009 / miopen-provider 1360 tests, 0 invalid - Status: Passedctest -R hipdnn_integration_tests_unit_tests -V- 1076 tests: 1074 passed, 2 skipped (DVC not pulled) - Status: Passedctest -L quick- 50/50 suites - Status: Passedctest -V -R hipblaslt-provider-external-integration_quick_suite- 17 passed / 618 skipped / 0 failed;[ VERIFIER ] rejected(ranked_list)for the new bundle; valid twin passes on gpu_ref - Status: Passedctest -V -R miopen-provider-external-integration_quick_suite- 247 passed / 388 skipped / 0 failed - Status: Passedctest -R python_tests -V(Linux) and pytest (Windows) - Status: Passedverify_support_claims.py,feature_matrix.py --check(overview unchanged),verify_golden_bundles.pyon the new bundle - Status: Passedpre-commit run --files <changed files>- Status: PassedTechnical Changes
BundleMetadata.hpp: strict-booleanno_cpu_reference,no_gpu_reference,expected_rejection. A non-boolean value is aBundleMetadataError, which registers a failing test.IntegrationTestBundle.hpp, mirrored inverify_golden_bundles.py:{Name}.tensors.dvc, a blob for every non-virtual output, or a sweep case'sgolden.path.expected_rejectionplus golden data is an error.expected_rejectionignores the reference flags.automode.--verification-modenaming it, or both disabled with golden data not pulled, skips before inputs are filled or the engine runs.TestBodytry): pinned engine absent from a resolved ranked list, or plan build declines/fails → PASS; plans build → FAIL naming the engine; unresolved query orfrom_binaryfailure → FAIL; unpinned → SKIP. Never executes or compares. The per-test line is[ VERIFIER ] rejected(<stage>), and the summary countsrejected.--write-support-claims(the flag is now read throughHarnessPolicy), and keep registration counters in parity. A claim that names one fails the C++ test andverify_support_claims.py.classifyBundle: an expected-rejection bundle that cannot load is a redFailedLoad, not a silent skip.knownReferenceGapsentry FAILs, so the stale gap entry is caught.support_matrix_units.py: expected-rejection bundles and cases add nothing to the matrix, coverage or feature counts.SUPPORT_MATRIX_OVERVIEW.mdis unchanged.file-formats.md,running-tests.md,adding-tests.md,support-claim-enforcement.md, thedocs/README.mdsignals table, andreference-data-scripts/README.md.quick/Matmul/nc/fp16examples. Those were replaced by the singlequick/Matmul/ncl/fp32batch-mismatch bundle in later commits. For the squash merge, use this PR description.