AIRUNTIME-2744: Require reason on populated test skip sections by sruscica · Pull Request #12703 · ROCm/rocm-systems

sruscica · 2026-10-02T18:50:38Z

Motivation

Follow-up to AIRUNTIME-2743 (commit 0bf70ef), which added the unsupported
skip field alongside disabled in the Catch2 test-config system. The skip
sections accept a reason, but it was optional and unenforced, so skipped
tests could accumulate with no recorded justification. This PR makes a reason
mandatory whenever a skip section is populated, requires that reason to cite a
real Jira reference so the skip is trackable, and migrates the existing
disabled entries to carry their justification as structured metadata so
disabled tests can be monitored and re-triaged.

Technical Details

check_config.py: adds a validation pass that errors (exit 1) when any
disabled/unsupported section has non-empty targets but (a) an empty or
whitespace-only reason, or (b) a reason that does not contain a real Jira
reference (AIRUNTIME-<number>; a digitless placeholder does not count).
Empty sections (targets: []) need no reason; the flat-list form remains
valid only when empty. Ordered after the existing missing-level and
invalid-skip-shape checks.
Migrated 39 config YAMLs: existing flat-list disabled: entries converted to
the mapping form {targets: [...], reason: "..."}. Prior inline comments were
folded into reason, each reason leads with its tracking ticket
(AIRUNTIME-<number> - ...), and untriaged entries use the placeholder
AIRUNTIME-2744 - needs to be re-triaged so each disabled test has a
trackable follow-up. No empty disabled/unsupported sections were
scaffolded; definitions.yaml (anchor definitions) is untouched.
reason is metadata only — parse_config.py ignores it — so the generated
Catch2 test headers are unchanged by the migration.
Updated README.md and TESTING.md to document the enforced behavior (both
the mandatory reason and the required Jira reference) and the disabled
(temporary, monitored) vs unsupported (permanent) intent distinction.

Issue Tracking

JIRA ID: AIRUNTIME-2744 (follow-up to AIRUNTIME-2743)

Test Plan

Regenerated the Catch2 config headers from the migrated YAMLs and compared
against the pre-migration baseline.
Exercised check_config.py against the real configs plus synthetic cases
(populated section with empty reason; populated section with a reason that has
no Jira ref; empty-targets section).
Ran black/pre-commit on the modified Python.

Test Result

Golden gate: generated headers byte-identical (sha256) before vs after the
migration across all three config trees — a pure no-op on generated test output.
Enforcement: exit 0 on the real configs; exit 1 naming the offending entry on a
populated section with an empty reason; exit 1 naming the offending entry on a
populated section whose reason lacks an AIRUNTIME-<number> ref; exit 0 on an
empty-targets section.
black/pre-commit: pass.

Submission Checklist

Look over the contributing guidelines at https://github.com/ROCm/rocm-systems/blob/develop/CONTRIBUTING.md.

therock-pr-bot · 2026-10-02T18:51:41Z

✅ All Policy Checks Passed

Check	Status	Details
📝 PR Description	✅ Pass	—
⛔ Forbidden Files	✅ Pass	—
🧪 Unit Test	⚠️ Warning	Error: Source/code files changed without an accompanying unit test. Expected: add at least one test file named like `test_<name>.py` / `test_<name>.cpp` (or `<name>_test.`). Current:* code file(s) changed: `projects/hip-tests/catch/config/check_config.py`; no test file found
🚫 Draft PR	🔜 To Be Enabled	—
🚩 Feature Flag	🔜 To Be Enabled	—
📊 Code Coverage	🔜 To Be Enabled	—

🎉 All policy checks passed!

📖 Need help? See the Policy FAQ for details on every check and how to fix failures.

🙋 Wish to Override Policy?

mangupta

🤖 AI-assisted pre-review (for @mangupta)

Phase-2 follow-up to AIRUNTIME-2743 (0bf70ef). Migrate every flat-list `disabled:` entry in the HIP Catch2 test configs to the mapping form carrying a `reason`, and enforce in check_config.py that any populated `disabled`/`unsupported` section has a non-empty reason. Existing inline skip comments are transferred into the reason and tagged for re-triage. Generated headers are unchanged (reason is parser-ignored; targets untouched). Changes: - Migrate 937 flat-list `disabled:` entries across 39 config YAMLs to mapping form; transfer 830 inline comments into `reason` and append "Needs to be re-triaged AIRUNTIME-2744" (or set it alone when no comment exists). - check_config.py: enforce non-empty `reason` for any populated `disabled`/`unsupported` section; empty/absent sections unaffected; remove the AIRUNTIME-2744 deferral comment. - TESTING.md, catch/README.md: document the reason requirement and the temporary-monitored (disabled) vs permanent (unsupported) distinction. 🤖 Claude Code 🤖

…l triage Fold triage evidence, Jira issue refs (AIRUNTIME-2862, ROCM-29915) or AIRUNTIME-XXXX placeholders, and platform-qualified re-triage sentinels into the reason fields of disabled test entries. Metadata only; generated headers unchanged.

Second-round triage adapting disabled-test dispositions to a fresh gfx1030 rebuild (TheRock main 2026-10-02, amd-llvm 2026-09-28) after the first round was found to have run against a 5-month-old stale build. Validator check_config.py configs exits 0. Changes: - Re-enable 6 tests now passing 3/3 on gfx1030 (drop amd_linux from skip targets, keep untriaged platforms with a re-triage sentinel): hipExecutionCtxStreamDestroy_Negative, hipGraphAddMemAllocNode_Negative_Free_Alloc_Memory_Again, Device_tgammaf_Accuracy_Limited_Positive, hipMemRangeGetAttribute_Positive_AccessedBy_Basic, hipMemRangeGetAttribute_Positive_AccessedBy_Partial_Range, hipMemAdvise_No_Flag_Interference. - Correct two reasons to fresh truth: cluster_split_barrier (self-skips since cluster launch is unsupported on gfx1030) and KernelStackSize (arch-gated to gfx1250/1251 but present in the fat multi-arch build; prior "not built on other asics" reason was wrong). - Move hipStreamKilledWorkerThread from disabled to unsupported (Windows-only via if(WIN32); never builds/runs on Linux).

…an TheRock build) Re-triages every disabled test whose targets include amd_windows against a fresh, fully coherent TheRock build (single amd-llvm toolchain + HIP 7.17 runtime + hip-tests, all built together from this branch for gfx1201 only, per the CLRT "TheRock build" guide). This supersedes the earlier mixed-component attempt: no stale runtime, so the device/deviceLib/cooperativeGrps/module/ callback tests that previously failed to build now run. Each test run on Windows / gfx1201 (RX 9070 XT) by exact Catch2 name to bypass the skip tag. 556 entries updated; validator check_config.py configs exits 0. - Re-enable 229 tests passing on gfx1201 (10x; the templated cooperative-group tail confirmed 3x given their cost): drop amd_windows from targets and re-qualify the re-triage sentinel to the remaining untriaged platform(s); stale prefixes kept verbatim. - Move 228 entries to unsupported for amd_windows (source-gated off native Windows): 192 math + syncthreads (if(UNIX)); 19 deviceLib ext_ocp fp4/fp6/fp8 (if(NOT WIN32) - OCP host-reference bitfield ABI); 8 virtualMemoryManagement HostNuma (if(UNIX) - libnuma); 4 stream Aql (#if HT_AMD && HT_LINUX); 5 oob (add_subdirectory(oob) under UNIX). amd_wsl etc. stay disabled where they still build. - Keep disabled with a gfx1201 (amd_windows) finding + Jira placeholder: 34 failures, 4 hangs (>300s), 4 flaky; 57 self-skips (capability/build gates: cooperative launch unsupported, managed memory unsupported, large-BAR unsupported, assertions-disabled build) recorded without a defect ref. Not triaged (left disabled with existing sentinel, ~13): executionControl x2, performance x3, graph x2, kernel KernelStackSize, streamperthread, memory, and 2 cases that report no matching test - registered in source but not built or not registered on the Windows build; need per-case follow-up. Co-Authored-By: Claude <noreply@anthropic.com>

mangupta · 2026-10-04T14:16:41Z

🤖 AI-assisted pre-review (for @mangupta)

Re-reviewed the updated head (f68f2b0906cb) — no new findings at or above the posting threshold.

…-enable; move more not-built suites to unsupported Follow-up to the gfx1201 clean-build triage, addressing PR feedback: - Re-enabled tests: strip clauses scoped to (amd_windows) from the reason. Once a test passes on amd_windows, an "(amd_windows) Note: ..." clause is stale/false, so it is removed; clauses for platforms still disabled (amd_wsl etc.) and generic prefixes are kept verbatim, and the sentinel stays re-qualified to the remaining untriaged platform(s). - Move 4 more suites to unsupported for amd_windows (confirmed never built on the native Windows build): kernel Unit_KernelStackSize and memory Unit_hipPointerSetAttribute_Positive_SyncMemops (both added only if(UNIX) in their CMakeLists; the latter notes a Windows undefined-symbol build error), and graph Unit_hipStreamBeginCaptureToGraph_CaptureDepGraph / _IndepGraphsThreads (wrapped in #ifdef __linux__). - Two cases the first pass reported as "no matching test" are in fact registered and were re-run: deviceLib Unit_MemcpyToSymbolInParallelWithStreamLaunch hangs (>timeout) and vmm Unit_hipMemSetAccessHost_devicealloc crashes (access violation); both kept disabled with the finding. Still disabled, documented (not a clean never-on-Windows gate): performance x3 (gated by build-option BUILD_PERF_TESTS, intentional local perf tests), executionControl x2, streamperthread StrmWaitEvt, hrr CaptureSmoke_Direct (hidden [.] child test). Validator check_config.py configs exits 0. Co-Authored-By: Claude <noreply@anthropic.com>

Re-triages every disabled test whose targets include amd_wsl by running it on WSL2 Ubuntu / gfx1201 (RX 9070 XT via the DXG passthrough, /dev/dxg). Runtime is the validated ROCm-for-WSL rc3 (therock-dist-linux-multiarch-tests-10.1.0rc3, DXG-backed) with its prebuilt multiarch hip-tests; the if(UNIX)-gated suites (math, syncthreads, atomics, ...) that never build on native Windows DO build and run here. Each test run by exact Catch2 name, 10x (templated tail 3x). 822 amd_wsl entries; 633 edited; check_config.py configs exits 0. - Re-enable 495 tests passing on gfx1201/WSL: drop amd_wsl from targets, re-qualify the sentinel to the remaining untriaged platform(s), and strip now-stale (amd_wsl)-scoped clauses. Tests that passed on both amd_windows and amd_wsl (sole remaining target) have their disabled block removed entirely. - Keep disabled with a gfx1201 (amd_wsl) finding + Jira placeholder: 35 fails (e.g. "no valid pci bdf in wsl"), 1 hang, 1 flaky; 101 self-skips (managed memory unsupported, cooperative launch unsupported, etc.). Notably all texture tests fail on WSL (DXG image path), unlike native Windows where they passed. Not triaged here (189, left disabled): these branch tests are absent from the rc3 prebuilt WSL hip-tests binaries (the release predates them) - memory 106, virtualMemoryManagement 52, gl_interop 17, and a few others. The fully from-source WSL build that would carry the branch's exact tests was blocked by missing sudo for WSL dev packages; these remain on the amd_wsl re-triage sentinel for a future from-source pass. Co-Authored-By: Claude <noreply@anthropic.com>

…ymptoms A crashing test corrupts the shared Catch2 XML for its chunk, so chunk-mates were mis-scored. Re-ran the 8 affected amd_wsl entries in isolation (10x): - hipIpcCloseMemHandle_Positive_Reference_Counting and hipIpcOpenMemHandle_Negative_Open_In_Two_Contexts_Same_Device pass 10/10 -> re-enabled on amd_wsl (were spurious "xml parse failed" fails). - hipStreamAddCallback_StrmSyncTiming is 6/10 -> FLAKY (kept disabled). - Device___exp10f/__expf accuracy tests genuinely crash (core dump) and fdim/fma/remquo genuinely miss 0-ULP accuracy: replaced the placeholder findings with the real symptom so the next triage starts from a real base. check_config.py configs exits 0. Co-Authored-By: Claude <noreply@anthropic.com>

mangupta · 2026-10-06T02:42:16Z

🤖 AI-assisted pre-review (for @mangupta)

Re-reviewed the updated head (2ca8e80b6dff) — no new findings at or above the posting threshold.

…hip-tests Re-ran the 189 amd_wsl cases that were unmapped on the rc3 prebuilt against a branch-coherent, from-source TheRock WSL build (HIP 7.17, gfx1201, DXG runtime on a local RX 9070 XT). 166 of them now map and run: - 131 pass -> re-enabled on amd_wsl (dropped from disabled targets; stale (amd_wsl) prefix clauses removed; re-triage sentinel re-qualified to the remaining untriaged platforms) - 8 fail -> kept disabled with the amd_wsl failure symptom + AIRUNTIME-XXXX (NUMA host-memory VMM + one HRR capture variant) - 27 self-skip -> kept disabled with the gfx1201 (amd_wsl) self-skip note Source-gate findings for cases absent from the branch build: - executionControl hipFuncSetAttribute/hipFuncSetSharedMemConfig -> amd_wsl unsupported (compiled only on non-AMD per the HIP_PLATFORM gate) - cluster split_barrier -> kept disabled: cluster launch unsupported on gfx1201 (arch-gated to gfx1101/gfx1250) gl_interop (OpenGL not present at build), performance (BUILD_PERF_TESTS flag), and the commented-out streamperthread StrmWaitEvt / hidden HRR CaptureSmoke_Direct helper are left untouched (not target source-gates). check_config.py configs: PASS. Co-Authored-By: Claude <noreply@anthropic.com>

check_config.py accepted any non-empty reason value, including YAML lists and mappings, because it validated via str(reason).strip(). A non-scalar reason stringified to a truthy value and silently passed, even though the documented mapping form requires a reason scalar. Require reason to be a non-empty string: a populated skip section whose reason is a list, mapping, or other non-string value is now routed to the missing-reason error and fails validation. Empty-targets no-op sections and the flat-list form are unchanged, and the existing config tree still passes. Changes: - check_config.py: guard reason with isinstance(reason, str) and reason.strip() 🤖 Claude Code 🤖

memory.yaml carried a mojibake-corrupted em-dash (bytes including 0x9d) in two skip-reason strings. check_config.py reads config files via common.py's open() without an explicit encoding, so on Windows (cp1252) the 0x9d byte is undefined and the check_hip_tests_config build stage failed with UnicodeDecodeError. Replace the garbled dash with ASCII -- to keep the config tree readable on all platforms. Changes: - memory.yaml: replace corrupted em-dash with ASCII -- in two reason strings 🤖 Claude Code 🤖

AIRUNTIME-2744 Phase 3. Every disabled/unsupported skip reason in the Catch2 config YAMLs now leads with a real AIRUNTIME-<number> tracking reference. Prior placeholder tags (AIRUNTIME-XXXX) are replaced with the AIRUNTIME-2892 follow-up ticket, and all reasons are rephrased for clarity. Non-ASCII characters are converted to ASCII to satisfy the Windows cp1252 check_hip_tests_config build constraint. check_config.py is extended to enforce that every skip reason contains a real AIRUNTIME-<digits> reference. Changes: - check_config.py: add missing_jira enforcement so a skip reason without a real AIRUNTIME-<digits> reference fails the config check - Replace 75 AIRUNTIME-XXXX placeholders with the AIRUNTIME-2892 follow-up ticket, leading the reason string - Rephrase all 630 skip reasons to lead with AIRUNTIME-<num> for clarity and consistent triage attribution - Convert 12 non-ASCII reason strings to ASCII (Windows cp1252 build constraint) 🤖 Claude Code 🤖

The reason check also requires a real AIRUNTIME-<number> ref, but the docs only mentioned a non-empty reason. Update README.md and TESTING.md to state the Jira-ref requirement and use the leading-ref placeholder form that matches the migrated configs. Changes: - README.md / TESTING.md: document the AIRUNTIME-<number> requirement; replace the stale "Needs to be re-triaged AIRUNTIME-2744" placeholder with the leading-ref form "AIRUNTIME-2744 - needs to be re-triaged". - check_config.py: drop the phantom AIRUNTIME-XXXX placeholder from the comment; describe the missing-ref case generically. - memory.yaml: behaviour -> behavior. - hrr.yaml: replace a machine-specific user path with %LOCALAPPDATA%.

sruscica · 2026-10-07T00:39:57Z

Manual CI: full hip-tests catch run

Dispatched a therock-multi-arch-ci.yml run to actually execute the re-enabled hip-tests cases on hardware (the PR's pull_request CI only runs the sanity lane, so the re-enabled catch cases aren't exercised there). This is a workflow_dispatch run against this PR's head, so it won't surface in the checks section above:

Run: https://github.com/ROCm/rocm-systems/actions/runs/37553101265
Scope: test:hip-tests on Linux gfx94X (amd_linux) and Windows gfx110X (amd_windows).

Caveats:

amd_wsl re-enables (the majority) have no catch lane in this workflow and are not covered here.
gfx125X/MI455 stays skipped (runner offline).

mangupta · 2026-10-07T04:10:48Z

🤖 AI-assisted pre-review (for @mangupta)

Re-reviewed the updated head (1b75cfab8083) — no new findings at or above the posting threshold.

…dows The re-enabled Catch2 test Unit_Uuid_FntlTstsFor_SetEnv_HIP_VISIBLE_DEVICES passes locally but fails on the Windows gfx110X ROCR hip-tests lane. Its child-process section "Set Env in parent and verify UUID in child" hits REQUIRE(proc.run(t_uuid) == 1) == 0 at hipDeviceGetUuid.cc:322, meaning the spawned child that re-reads the UUID under HIP_VISIBLE_DEVICES reports failure on Windows ROCR CI. Re-disable only amd_windows (the asic that regressed); amd_wsl has no CI lane and showed no failure, so it stays re-enabled per the original PR intent. Changes: - device.yaml: add disabled{targets:[amd_windows]} to Unit_Uuid_FntlTstsFor_SetEnv_HIP_VISIBLE_DEVICES with an AIRUNTIME-2892 reason documenting the Windows ROCR child-process failure

sruscica · 2026-10-07T07:24:31Z

CI watch update — one re-enabled test regressed on Windows ROCR

Dispatched full hip-tests run 37553101265 results:

Linux gfx94X — sanity + all 4 hip-tests shards: pass
Windows gfx110X (PAL) — all 4 shards: pass
Windows gfx110X (ROCR) — shards 1, 3, 4: pass; shard 2/4: 1 test failed out of 980

Failing test: Unit_Uuid_FntlTstsFor_SetEnv_HIP_VISIBLE_DEVICES
Section "Set Env in parent and verify UUID in child" — REQUIRE(proc.run(t_uuid) == 1) evaluated to 0 == 1 at hipDeviceGetUuid.cc:322. The spawned child that re-reads the device UUID under HIP_VISIBLE_DEVICES returns failure on Windows ROCR. This test passed on our local system but regressed in CI — exactly the scenario we were watching for.

This test was re-enabled by this PR on amd_windows + amd_wsl. I've re-disabled it for amd_windows only (the asic that regressed; amd_wsl has no CI lane and showed no failure) in projects/hip-tests/catch/config/configs/unit/device.yaml, tracked under AIRUNTIME-2892.

Pushed as a94c060910. check_config.py passes locally. Continuing the hourly watch.

mangupta · 2026-10-07T07:44:22Z

🤖 AI-assisted pre-review (for @mangupta)

Re-reviewed the updated head (a94c0609104c) — no new findings at or above the posting threshold.

sruscica · 2026-10-07T12:21:24Z

Full hip-tests verification complete (fix commit `a94c060`)

Dispatched a full hip-tests run on the latest commit: 37587149950 (Linux gfx94X + Windows gfx110X, PAL & ROCR backends, 4 shards each).

The AIRUNTIME-2892 fix is confirmed working. Windows ROCR shard 2/4 — which previously failed on Unit_Uuid_FntlTstsFor_SetEnv_HIP_VISIBLE_DEVICES — now passes with that test skipped on amd_windows. All Windows PAL (1-4) and ROCR (1,2,4) shards and Linux shards 2/3/4 are green.

Two remaining failures, neither caused by this PR:

Windows ROCR shard 3/4 — pre-existing 120-min timeout cascade. The first test to hang is Unit_hipStreamWaitValue_Default - uint32_t (SWDEV-558839 - Implement hipMemPrefetchBatchAsync API #1626), which this PR does not re-enable. Once it wedges the device for 600s, subsequent stream/event tests also time out. This cascade reproduces identically on the pre-fix commit (37553101265), confirming it is unrelated to the config-reason changes here. Only one re-enabled test (Unit_hipStreamSynchronize_spt_FinishWork) is caught in the cascade, and it times out ~70 min after the device is already wedged — collateral, not a cause. Re-disabling it would not clear the shard.
Linux gfx94X shard 1/4 — flaky managed-memory prefetch. Unit_hipMemPrefetchAsync_v2_Device_Host (data mismatch at hipMemPrefetchAsync_v2.cc:106) and Unit_hipMemPrefetchBatchAsync_RoundTripDataIntegrity (hipMemPrefetchBatchAsync.cc:73). Neither is re-enabled by this PR, and both passed on the earlier run where all 4 Linux shards were green. Classic timing-sensitive prefetch flakiness.

Net: every test this PR re-enables that executed on a CI lane passes, except the one Uuid case already re-disabled for amd_windows under AIRUNTIME-2892. The two outstanding shard failures are pre-existing / flaky and independent of this change.

The re-enabled Catch2 test Unit_hipStreamSynchronize_spt_FinishWork hangs past the 600s ctest timeout on the Windows gfx110X ROCR hip-tests shard 3 lane, as part of a stream/event timeout cascade on that shard. It passes locally. Re-disable it on amd_windows only (the asic where it is flaky in CI); amd_wsl has no CI lane and stays re-enabled per the original PR intent. Changes: - stream.yaml: add disabled{targets:[amd_windows]} to Unit_hipStreamSynchronize_spt_FinishWork with an AIRUNTIME-2892 reason documenting the Windows ROCR shard 3 timeout

mangupta · 2026-10-07T15:03:18Z

🤖 AI-assisted pre-review (for @mangupta)

Re-reviewed the updated head (3d945f85af83) — no new findings at or above the posting threshold.

Resolve conflicts in the hip-tests Catch2 config tree (36 files) between develop's level-system redesign and this branch's reason-enforcement migration. develop renamed the default level anchor (*level_2 -> *level_default = [0,1,2]), redesigned definitions.yaml, added new tests, and added a few disables in the old bare-list style. This branch rewrote every populated disabled/unsupported section into the mapping form with an AIRUNTIME-referenced reason, re-enabled a large set of tests, and re-disabled two on amd_windows. Resolution policy: develop is authoritative for levels/anchors, params, and new tests; this branch is authoritative for skip-state (disabled/unsupported/reason). Verified post-merge that every common case's resolved level matches develop (0 mismatches, 0 develop tests lost) and every common case's skip-state matches this branch (0 mismatches, 0 branch cases lost). check_config.py configs exits 0 and all reason strings are ASCII-only. Changes: - configs/**: take develop's anchors/levels/new-tests; transplant this branch's reason-form disabled/unsupported blocks per test case - definitions.yaml: take develop's redesigned level system (level_default, level_stress, level_all; level: [N] lists; math_* cmd_options) - memory.yaml: convert develop's 3 new bare-list disables to reason form (Unit_hipFreeInteriorPtrDev/Host asan double-free; batch-async SdmaDisabled_Child) with AIRUNTIME-2744 reasons so they satisfy the merged check_config enforcement

fabiomestre

By re-enabling all these tests in one go, we are at risk of having significant amount of new Asan failures (i.e. some of these tests have never been run under Asan). It might be good to triage this PR with an Asan build before merging and keep any test that fails with Asan disabled so that it can be triaged later.

PR #12703 re-enabled 225 Catch2 unit cases on amd_windows that were previously disabled. The individual tests pass, but the full Windows ROCR shards now exceed the 120-minute per-shard CI timeout (timeout_minutes in TheRock's fetch_test_configurations.py), causing shard failures. Re-disable these on amd_windows as a revertable experiment: if CI goes green, fix the runtime/timeout in TheRock and re-enable. Changes: - 170 previously skip-free cases: add a disabled block {targets: [amd_windows], reason: "AIRUNTIME-2892 ..."}. - 55 cases already carrying a disabled block for other targets: add amd_windows to targets and append the AIRUNTIME-2892 timeout note to the existing reason (original rationale preserved). - All reasons cite AIRUNTIME-2892; check_config.py passes. Intended to be reverted once Windows test runtimes are improved or the TheRock total timeout is raised. 🤖 Claude Code 🤖

Four VMM/IPC tests fail under device-side ASAN (heap-use-after-free in libamdhip64 plus LeakSanitizer leaks) on gfx942. These failures are pre-existing in the HIP runtime and are not introduced by the reason-note enforcement work, but they currently break the PR's ASAN lane. Skip them on the asan target so the lane goes green; the underlying runtime bugs are tracked under AIRUNTIME-2892 for re-triage. Changes: - Add disabled: {targets: [asan]} to Unit_hipMemImportFromShareableHandle_HostBackedAlloc - Add disabled: {targets: [asan]} to Unit_hipMemImportFromShareableHandle_Capture - Add disabled: {targets: [asan]} to Unit_hipMemGetHandleForAddressRange_VM - Merge asan into existing disabled list of Unit_hipMemCreate_HostNuma_ExportImport 🤖 Claude Code 🤖

check_config.py enforced that every populated disabled:/unsupported: skip section cites a tracking ticket, but the matcher was hard-coded to AIRUNTIME-<number>. Skips are also tracked under other projects (ROCM, SWDEV, EXSWHTEC already appear in the real configs, and AIRADSW is in use), so generalize enforcement to an explicit allowlist of known Jira project keys expressed as a named constant. The change is strictly more permissive for the AIRUNTIME anchor, so all existing reasons still validate; it is an allowlist rather than a generic [A-Z]+-<number> pattern so typos like BLAH-1 are still rejected. Changes: - check_config.py: replace the AIRUNTIME-only regex with a JIRA_PROJECTS allowlist tuple (AIRUNTIME, ROCM, SWDEV, EXSWHTEC, AIRADSW); build the regex and the error message from that constant so they cannot drift - catch/README.md: document the allowlist, list the keys, and note that a new key is added by editing JIRA_PROJECTS - TESTING.md: list the accepted project keys in the skip policy 🤖 Claude Code 🤖

…untime-2744-hip-tests-config-reason-note-enforcement # Conflicts: # projects/hip-tests/catch/config/configs/unit/graph.yaml

mangupta · 2026-10-16T03:20:27Z

🤖 AI-assisted pre-review (for @mangupta)

Re-reviewed the updated head (4ffdca7df8d9) — no new findings at or above the posting threshold.

…-2892) Both cases fail intermittently on the Windows ROCR hip-tests lane. On the identical head SHA, consecutive CI runs failed disjoint test sets (one run hit 4 timeout-range failures, the next hit these 2 fast-assertion failures), confirming flakiness rather than a deterministic regression. Changes: - stream.yaml: add amd_windows to Unit_hipMultiStream_SharedHwQueuePacketOrdering disabled targets; REQUIRE(host_ordering_errors == 0) intermittently fails 3==0. - graph.yaml: add amd_windows to Unit_hipGraphAddMemAllocNode_Negative_Free_Alloc_Memory_Again disabled targets; same ret==hipErrorInvalidValue (0==1) failure seen on amd_wsl, with intermittent SEH on Windows. 🤖 Claude Code 🤖

Comparing gfx90a results from an ASAN device build against a non-ASAN build on the same GPU surfaced 28 cases that fail only under device ASAN and pass on the non-ASAN build. Disable these on the asan target pending re-triage so ASAN lanes stop flagging known-environmental failures. Changes: - memory.yaml: 14 new asan disables + 4 merged into existing blocks - graph.yaml: 2 new asan disables - texture.yaml: 4 new asan disables + 2 merged into existing blocks - virtualMemoryManagement.yaml: 2 inline cases expanded with asan disable - All tagged AIRUNTIME-2892 for phase-3 re-triage 🤖 Claude Code 🤖

mangupta · 2026-10-19T04:41:29Z

🤖 AI-assisted pre-review (for @mangupta)

Re-reviewed the updated head (ebeaaa6951d6) — no new findings at or above the posting threshold.

…untime-2744-hip-tests-config-reason-note-enforcement # Conflicts: # projects/hip-tests/catch/config/configs/unit/graph.yaml

…untime-2744-hip-tests-config-reason-note-enforcement # Conflicts: # projects/hip-tests/catch/config/configs/unit/virtualMemoryManagement.yaml

mangupta · 2026-10-20T01:05:44Z

🤖 AI-assisted pre-review (for @mangupta)

Re-reviewed the updated head (4a3dd2f3b7c2) — no new findings at or above the posting threshold.

Unit_hipStreamWaitValue_Default fails on amd_windows (gfx110X) with an illegal memory access in hipStreamValue.cc. The fault corrupts the GPU queue state, cascading every subsequent stream/event test in the shard to exit 0xc0000409 and hanging until the 120-minute shard timeout. Disable on amd_windows (the config has no ROCR-only granularity) pending re-triage under AIRUNTIME-2892. Changes: - stream.yaml: add disabled {targets: [amd_windows], reason} to Unit_hipStreamWaitValue_Default 🤖 Claude Code 🤖

…on (#13892) * test(hip-tests): give the disabled 3D batch DuringApiCall case a reason check_config.py requires a reason on every populated skip field since #12703, so the hip-tests config check fails on develop. Move the comment into the reason field, linked to ROCM-31101.

## Motivation Follow-up to AIRUNTIME-2743 (commit 0bf70ef), which added the `unsupported` skip field alongside `disabled` in the Catch2 test-config system. The skip sections accept a `reason`, but it was optional and unenforced, so skipped tests could accumulate with no recorded justification. This PR makes a reason mandatory whenever a skip section is populated, requires that reason to cite a real Jira reference so the skip is trackable, and migrates the existing `disabled` entries to carry their justification as structured metadata so disabled tests can be monitored and re-triaged. ## Technical Details - `check_config.py`: adds a validation pass that errors (exit 1) when any `disabled`/`unsupported` section has non-empty targets but (a) an empty or whitespace-only `reason`, or (b) a `reason` that does not contain a real Jira reference (`AIRUNTIME-<number>`; a digitless placeholder does not count). Empty sections (`targets: []`) need no reason; the flat-list form remains valid only when empty. Ordered after the existing missing-`level` and invalid-skip-shape checks. - Migrated 39 config YAMLs: existing flat-list `disabled:` entries converted to the mapping form `{targets: [...], reason: "..."}`. Prior inline comments were folded into `reason`, each reason leads with its tracking ticket (`AIRUNTIME-<number> - ...`), and untriaged entries use the placeholder `AIRUNTIME-2744 - needs to be re-triaged` so each disabled test has a trackable follow-up. No empty `disabled`/`unsupported` sections were scaffolded; `definitions.yaml` (anchor definitions) is untouched. - `reason` is metadata only — `parse_config.py` ignores it — so the generated Catch2 test headers are unchanged by the migration. - Updated `README.md` and `TESTING.md` to document the enforced behavior (both the mandatory reason and the required Jira reference) and the disabled (temporary, monitored) vs unsupported (permanent) intent distinction. ## Issue Tracking JIRA ID: AIRUNTIME-2744 (follow-up to AIRUNTIME-2743) ## Test Plan - Regenerated the Catch2 config headers from the migrated YAMLs and compared against the pre-migration baseline. - Exercised `check_config.py` against the real configs plus synthetic cases (populated section with empty reason; populated section with a reason that has no Jira ref; empty-targets section). - Ran `black`/pre-commit on the modified Python. ## Test Result - Golden gate: generated headers byte-identical (sha256) before vs after the migration across all three config trees — a pure no-op on generated test output. - Enforcement: exit 0 on the real configs; exit 1 naming the offending entry on a populated section with an empty reason; exit 1 naming the offending entry on a populated section whose reason lacks an `AIRUNTIME-<number>` ref; exit 0 on an empty-targets section. - `black`/pre-commit: pass. ## Submission Checklist - [x] Look over the contributing guidelines at https://github.com/ROCm/rocm-systems/blob/develop/CONTRIBUTING.md. --------- Co-authored-by: sruscica <�Sam.Ruscica@amd.com> Co-authored-by: Claude <noreply@anthropic.com>

…on (#13892) * test(hip-tests): give the disabled 3D batch DuringApiCall case a reason check_config.py requires a reason on every populated skip field since #12703, so the hip-tests config check fails on develop. Move the comment into the reason field, linked to ROCM-31101.

systems-assistant · 2026-10-22T19:12:55Z

TheRock Submodule Bump Activity

Newest first

2026-10-22 — Included in TheRock via ROCm/TheRock#9425.

sruscica requested a review from a team as a code owner October 2, 2026 18:50

github-actions Bot added the project: hip-tests label Oct 2, 2026

systems-assistant Bot added the organization: ROCm label Oct 2, 2026

mangupta reviewed Oct 3, 2026

View reviewed changes

Comment thread projects/hip-tests/catch/config/check_config.py Resolved

mangupta added the lrt-bot: ready label Oct 3, 2026

sruscica added 3 commits October 3, 2026 05:35

sruscica force-pushed the users/sruscica/airuntime-2744-hip-tests-config-reason-note-enforcement branch from cad6f6b to f68f2b0 Compare October 3, 2026 14:48

sruscica and others added 3 commits October 5, 2026 16:10

sruscica and others added 5 commits October 5, 2026 23:38

chrispaquot approved these changes Oct 14, 2026

View reviewed changes

fabiomestre reviewed Oct 14, 2026

View reviewed changes

sruscica added 4 commits October 15, 2026 14:51

Merge remote-tracking branch 'origin/develop' into users/sruscica/air…

4ffdca7

…untime-2744-hip-tests-config-reason-note-enforcement # Conflicts: # projects/hip-tests/catch/config/configs/unit/graph.yaml

sruscica added the ci:asan Opt-in to building ASAN label Oct 15, 2026

mangupta added lrt-bot: review lrt-bot: ready and removed lrt-bot: ready lrt-bot: review labels Oct 16, 2026

sruscica and others added 2 commits October 16, 2026 15:13

sruscica added 2 commits October 19, 2026 14:58

Merge remote-tracking branch 'origin/develop' into users/sruscica/air…

656a9a3

…untime-2744-hip-tests-config-reason-note-enforcement # Conflicts: # projects/hip-tests/catch/config/configs/unit/graph.yaml

Merge remote-tracking branch 'origin/develop' into users/sruscica/air…

4a3dd2f

…untime-2744-hip-tests-config-reason-note-enforcement # Conflicts: # projects/hip-tests/catch/config/configs/unit/virtualMemoryManagement.yaml

sruscica merged commit f5fc5b5 into develop Oct 20, 2026
157 of 226 checks passed

sruscica deleted the users/sruscica/airuntime-2744-hip-tests-config-reason-note-enforcement branch October 20, 2026 23:26

mangupta removed the lrt-bot: ready label Oct 21, 2026

andraghetti mentioned this pull request Oct 21, 2026

test(hip-tests): give the disabled 3D batch DuringApiCall case a reason #13892

Merged

1 task done

This was referenced Oct 21, 2026

[rocjitsu] Correct legacy floating min/max and median selection #13898

Merged

[rocjitsu] Expose backed CDNA3 and CDNA4 hardware identity fields #13899

Merged

[rocjitsu] Implement scratch LDS transfers and scalar cache discard #13900

Merged

Conversation

sruscica commented Oct 2, 2026 • edited Loading Uh oh! There was an error while loading. Please reload this page.

Uh oh!

Motivation

Technical Details

Issue Tracking

Test Plan

Test Result

Submission Checklist

Uh oh!

therock-pr-bot Bot commented Oct 2, 2026

✅ All Policy Checks Passed

Uh oh!

mangupta left a comment

Choose a reason for hiding this comment

Uh oh!

Uh oh!

mangupta commented Oct 4, 2026

Uh oh!

mangupta commented Oct 6, 2026

Uh oh!

sruscica commented Oct 7, 2026

Manual CI: full hip-tests catch run

Uh oh!

mangupta commented Oct 7, 2026

Uh oh!

sruscica commented Oct 7, 2026

CI watch update — one re-enabled test regressed on Windows ROCR

Uh oh!

mangupta commented Oct 7, 2026

Uh oh!

sruscica commented Oct 7, 2026

Full hip-tests verification complete (fix commit a94c060)

Uh oh!

mangupta commented Oct 7, 2026

Uh oh!

fabiomestre left a comment

Choose a reason for hiding this comment

Uh oh!

mangupta commented Oct 16, 2026

Uh oh!

mangupta commented Oct 19, 2026

Uh oh!

mangupta commented Oct 20, 2026

Uh oh!

Uh oh!

systems-assistant Bot commented Oct 22, 2026

TheRock Submodule Bump Activity

Uh oh!

Reviewers

Uh oh!

Assignees

Labels

Projects

Milestone

Development

Uh oh!

4 participants

sruscica commented Oct 2, 2026 •

edited

Loading

Full hip-tests verification complete (fix commit `a94c060`)