fix(rocke): compare arch-domain baselines across compiler builds

dnn-providers/hip-kernel-provider/rocke/TESTING.md

@@ -434,19 +434,31 @@ This gate and §4.4 overlap on purpose and neither subsumes the other:
   often an unnamed fatal error. This reports `(key, arch, flavor)` plus the
   diagnostic the probe recorded.
 
-### 4.6 Does the measured arch-domain artifact reproduce?
+### 4.6 Does the toolchain retain the baseline's support?
 
 The warning gate consumes committed measurements; it does not prove that this
-host's toolchain reproduces them. When the host compiler identity matches the
-provenance recorded in the committed column, the
+host's toolchain retains the recorded support. The existing
 `rocke_arch_domain_reproducibility` CTest runs
 `tools/gen_arch_domain.py --check` over the full intrinsic/architecture matrix
-and compares the result with that column. LLVM development builds with the same
-major can disagree about intrinsic availability and ABI, so a different compiler
-identity is explicitly unvalidated rather than compared as though it asked the
-same question. This is an artifact-reproduction gate backed by the host
-toolchain, not a Python unit test: it owns external compiler processes, a
-dedicated timeout and resource policy, and an explicit CTest skip result.
+for the host's LLVM flavor and checks a **monotonic support floor**, not exact
+snapshot equality. A formerly `ok` cell must still be `ok`; becoming unavailable,
+crashing, timing out, or losing a measurement fails. Previously accepted processor
+names must remain accepted too.
+
+Support gains and unknown-to-unavailable transitions pass and are reported.
+Unavailable-to-unknown transitions are reported as degraded measurement, not
+support loss; remaining no-data cells are explicitly not qualified. Diagnostic
+wording, canonical intrinsic names, and compiler identity do not define support.
+Compiler identity is retained and printed as provenance. Invalid probes,
+malformed data, missing coverage, and incompatible scope or probe methodology
+still fail. This is a real-toolchain gate, not a Python unit test: it owns
+external compiler processes, a dedicated timeout and resource policy, and an
+explicit environmental CTest skip result.
+
+`--check` never re-blesses the baseline. Reviewed gains must be recorded separately
+to protect them from future loss and update the runtime availability table. That
+table also controls lowering warnings/refusals, so a baseline update is a product
+policy change, not merely a way to make this test pass.
 
 The gate is registered in both execution environments. A developer build runs
 the source tool against the source package data. Provider CI runs a packaged,
@@ -456,13 +468,12 @@ stages every input, CTest supplies artifact-relative paths, and the test never
 configures or reaches back into a source checkout.
 
 The result mapping distinguishes applicability from failure: no clang, no
-committed host-flavor column, a clang/resolved-flavor mismatch, or a compiler
-identity different from the column's provenance skips with a named reason. A
-partial or internally inconsistent LLVM installation fails. Flavor
-auto-detection errors, including unavailable COMGR evidence, also remain
+committed host-flavor column, or a clang/resolved-flavor mismatch skips with a
+named reason. A partial or internally inconsistent LLVM installation fails.
+Flavor auto-detection errors, including unavailable COMGR evidence, also remain
 failures during this migration.
 
-The reproduction scheduler preserves exhaustive coverage while avoiding
+The probe scheduler preserves exhaustive coverage while avoiding
 redundant compiler startup. It batches declaration discovery, preflights each
 target once, and combines expected-positive calls per architecture. The
 committed artifact schedules those batches but cannot make them pass: every call
@@ -474,17 +485,20 @@ Expected crashes and timeouts whose normalized evidence still matches are not
 retried; changed evidence, unexpected verdicts, and unclassified outcomes still
 receive a contention-free retry.
 
-The fast structure and generator units remain source-only in
-`platform/tests/tools/test_arch_domain_artifact.py`. Normal pytest and the
-`ROCKE_BACKEND=both` pytest pass do not re-run the compiler sweep. The dedicated
+Structure and compiler-probe units remain source-only in
+`platform/tests/tools/test_arch_domain_artifact.py`. Host-only support-policy
+controls in `platform/tests/tools/test_arch_domain_support.py` run in both source
+pytest and the installed `rocke_pytest` CI entry, using the staged generator in
+installed artifacts. They check loss-fail, gain-pass, invalid records, and method/
+metadata guards for both gfx942 and gfx950 without a real compiler. Normal pytest
+and the `ROCKE_BACKEND=both` pytest pass do not re-run the compiler sweep. The dedicated
 CTest is categorized as standard in both build-tree and installed manifests, so
 installed-artifact CI can schedule the reproduction result without adding the
 external process sweep to the quick or FFM pytest pools. A named skip is not
-reproduction coverage: check the CTest result and reason, especially when the
-compiler identity differs from the committed provenance, its major differs from
-rocKE's resolved flavor, or no committed column exists. Keeping one generator
-implementation behind two thin registrations preserves local coverage while
-avoiding duplicated test logic.
+support-validation coverage: check the CTest result and reason, especially when the
+compiler's major differs from rocKE's resolved flavor or no committed column
+exists. Keeping one generator implementation behind two thin registrations
+preserves local coverage while avoiding duplicated test logic.
 
 **Kernel-author routing.** Ordinary kernel work does not need a separate focused
 run beyond the full pre-merge `tests/run_all.py`; once that runner has configured

dnn-providers/hip-kernel-provider/rocke/platform/CMakeLists.txt

@@ -461,9 +461,9 @@ if(ROCKE_INSTALL_TESTS)
     # the differential harness (needs compiler + source; its Python-only golden
     # check is installed separately above), the WaveScope sidecar,
     # LDS analyzer and dsl_probes tests (load tools from the adjacent dsl_doc
-    # source tree), and the arch-domain artifact unit tests. The latter remain
-    # source-only units, while the full regeneration sweep is installed below as
-    # its own CTest with an independent timeout and resource policy.
+    # source tree), and the arch-domain artifact/probe unit tests. The host-only
+    # support-policy units are installed with the rest of tests/tools; the full
+    # regeneration sweep has its own CTest timeout and resource policy.
     # Include run_all.py: installed runner unit tests import it as a dependency.
     # Linux detector fixtures link a stdin wrapper against the production archive.
     # Keep it with the pytest assets so TheRock's test artifact captures it even
@@ -503,9 +503,9 @@ if(ROCKE_INSTALL_TESTS)
               "rocKE: top-level run_checks.py not found; skipping its test-only install")
     endif()
 
-    # Test-only implementation for the installed arch-domain reproduction gate.
-    # Keep it outside the production package: CTest owns the external compiler
-    # sweep, while package resources provide the committed columns being checked.
+    # Test-only implementation shared by the installed support-policy units and
+    # arch-domain reproduction gate. Keep it outside the production package:
+    # CTest owns the compiler sweep; pytest checks the policy without one.
     # tests/conftest.py also points rocke.tests.ir_validity at this copy, so the
     # installed pytest runs compile lowered IR with the same tool.
     install(FILES

dnn-providers/hip-kernel-provider/rocke/platform/dsl_docs/development/arch_axis_proposal.md

@@ -37,7 +37,7 @@ the C++ lowerer does not yet own the same in-engine lookup.
 The staged design is:
 
 - **S1** *(done)* — generate an `arch_domain` table by probing the installed
-  toolchain; commit it, gate regeneration as a no-op, exactly like golden. One
+  toolchain; commit it, gate retained support with a monotonic floor. One
   column per flavor, `(key, arch) → status + evidence`; a plain `key → set of
   arches` proved too lossy (§3.2, §4/S1).
 - **S2** *(warning/corpus gate done; hard rejection proposed)* — make `_need()`
@@ -249,13 +249,16 @@ Two further findings from the sweep, neither visible to a 15-probe sample:
 `tools/gen_arch_domain.py`, for each decl key × each wired arch, emits a minimal
 module, compiles **and links** it in a **subprocess**, and classifies the
 outcome. Commit the result; `tests/tools/test_arch_domain_artifact.py` checks
-its structure and the `rocke_arch_domain_reproducibility` CTest asserts
-regeneration is a no-op when the host uses the exact compiler build recorded in
-the artifact. A different build is explicitly unvalidated because development
-builds within one LLVM major can disagree. The CTest is registered for both
-developer build trees and relocated provider test artifacts; the installed form
-checks the package data CI actually consumes without requiring a source
-checkout. This is exactly how golden is managed.
+its structure and the `rocke_arch_domain_reproducibility` CTest checks a
+monotonic support floor with compatible flavor, scope, and probe methodology.
+Previously working cells and accepted processor names must remain working.
+Compiler identity is provenance, not an applicability gate. Gains and improved
+classification pass and are reported; remaining no-data and degraded negative
+measurements are not qualified. Invalid or missing comparison records fail.
+The CTest is registered for both developer build trees and relocated provider
+test artifacts; the installed form checks the package data CI actually consumes
+without requiring a source checkout. Unlike an IR golden, this gate compares
+capability transitions rather than requiring identical generated text.
 
 Shipped schema — per-cell provenance is the point, per
 [`multi_toolchain_strategy.md`](multi_toolchain_strategy.md) §3.2, since one host

dnn-providers/hip-kernel-provider/rocke/platform/dsl_docs/development/multi_toolchain_strategy.md

@@ -365,12 +365,14 @@ with the same install's `llvm-mc`: the backend can select an intrinsic for a
 target that lacks the instruction and write it into the object, and only the
 text assembler checks each instruction against the target (`arch_absent`
 otherwise). Every decl key × every wired arch — the committed columns are the count of
-record, not this sentence — ~15 s wall across cores on `llvm22`. Managed like
-golden — a generator tool, a committed artifact, and a test asserting
-regeneration is a no-op when run with the exact compiler build recorded in the
-artifact. A different build is explicitly unvalidated because development builds
-within one LLVM major can disagree. Depends only on `clang`, never on install
-layout (§2.4).
+record, not this sentence — ~15 s wall across cores on `llvm22`. The committed artifact supplies a monotonic support floor: the test requires
+previously working cells and accepted processor names to remain working, with
+compatible flavor, scope, and probe methodology. Compiler identity is recorded
+as provenance rather than an applicability gate. Gains and improved
+classifications pass and are reported; remaining no-data and degraded negative
+measurements are not qualified. Invalid or missing records still fail. The
+probe uses matching `clang`, `opt`, and `llvm-mc` tools from a complete LLVM
+installation (§2.4).
 
 Per-cell provenance landed as **one file per flavor, named after it**, rather
 than a `verified`/`unvalidated` marker inside a shared file as §3.2 suggested. A

dnn-providers/hip-kernel-provider/rocke/platform/tests/README.md

@@ -73,7 +73,7 @@ for what is actually exercised vs. manual/diagnostic/demo.
 | `analysis/` | (covered via `test_rocke.py`) | - |
 | `benchmark/` | `test_occupancy_targets.py` (the occupancy model on every target rocKE lowers for; gfx908 checked against llc) | - |
 | `support/` | tests of the `rocke.tests` helpers: `test_gpu.py`, `test_ir_validity.py`, `test_arch_marks.py`, `test_universal_gemm.py` | - |
-| `tools/` | `test_arch_domain_artifact.py` (source-only structure/generator units), `test_run_all.py`, tool-sidecar units | `rocke_arch_domain_reproducibility` (**CTest**, artifact reproduction through the host toolchain) |
+| `tools/` | `test_arch_domain_artifact.py` (source-only structure/probe units), `test_arch_domain_support.py` (source + installed host-only policy controls), `test_run_all.py`, tool-sidecar units | `rocke_arch_domain_reproducibility` (**CTest**, artifact reproduction through the host toolchain) |
 | (root) | `test_rocke.py` (multi-layer monolith), `test_rocke_ci_static.py` | - |
 
 > Notes: `instances/rocke_ir_parity_harness.py` supplies the representative corpus
@@ -167,7 +167,10 @@ Four distinct things run here; don't conflate them:
 `rocke_arch_domain_reproducibility` owns the full
 `tools/gen_arch_domain.py --check` sweep. It is deliberately not collected by
 pytest: the sweep starts external compiler/linker processes, has its own timeout
-and resource contract, and is independent of `ROCKE_BACKEND`.
+and resource contract, and is independent of `ROCKE_BACKEND`. The separate
+`tools/test_arch_domain_support.py` units exercise the comparison policy without
+real compiler processes. They run in source pytest and installed `rocke_pytest`,
+using the source or staged generator respectively.
 
 The same logical gate has two registrations: the build-tree entry invokes the
 source tool and committed source column, while the installed entry invokes a
@@ -190,23 +193,29 @@ expected crash or timeout whose normalized evidence still matches is not rerun
 serially; changed evidence, changed verdicts, unclassified results, and otherwise
 transient-looking results are.
 
-The artifact records the exact compiler build that measured it. LLVM development
-builds with the same major can disagree about intrinsic availability and ABI, so
-exact reproduction is applicable only when the host identity matches that
-provenance. A mismatch is reported with both identities and is an explicit
-unvalidated result, not a pass.
+The artifact records the compiler build that measured it as provenance. The
+check runs with a compatible LLVM flavor and probe method even when that compiler
+identity differs. It requires monotonic support: every formerly working cell and
+accepted processor name must still work. Gains and improved classifications are
+reported, not failures. Remaining no-data cells and degraded negative measurements
+are reported without claiming their qualification. Both documents must carry
+complete processor answers and explicit matching per-cell `verified_on` values.
 
 | Condition | CTest result |
 |---|---|
-| Recorded compiler identity matches and measurements reproduce cell-for-cell | Pass |
-| Host compiler identity differs from the recorded identity | Skip |
+| Compatible flavor/method and baseline working support retained | Pass |
+| Different compiler identity within the same flavor | Run the comparison; identity is not a pass/fail criterion |
+| Support gain or unknown-to-unavailable classification improvement | Pass, report changes |
+| Negative measurement degrades to unknown, or no-data remains | Report without qualifying those cells; not support loss |
+| Previously working support or accepted processor name is lost | Fail |
+| Missing/invalid comparison records or incompatible scope/probe method | Fail |
 | No clang | Skip |
 | Clang flavor differs from rocKE's resolved flavor | Skip |
 | No committed column for the host flavor | Skip |
 | Clang exists but `opt` is absent | Fail |
 | Clang and `opt` report different flavors | Fail |
 | Flavor auto-detection fails, including unavailable COMGR evidence | Fail |
-| Measurements drift or a probe is unclassified | Fail |
+| Probe is unclassified (`probe_error`) | Fail |
 
 Run it from a configured platform build with: