[anugodavar]

Summary

Adds a manually-triggered workflow and a HIP_TESTS_REGEX env var to reproduce the intermittent hipMemPrefetchAsync failures reported in ROCm/TheRock#8271.

Failing tests:

• Unit_hipMemPrefetchAsync_Basic_AllDevices — 42 == 0 (value mismatch at index 0)
• Unit_hipMemPrefetchBatchAsync_SingleOperationSingleLocation — false == true

Root cause hypothesis: hipMemPrefetchAsync dispatches hsaKmtSVMSetAttr(HSA_SVM_ATTR_PREFETCH_LOC) on a CPU signal-handler thread. The kernel's migrate_pages() can silently fail to migrate some pages (due to concurrent access or memory pressure during parallel CTest execution). With XNACK disabled, a CPU read of an unmapped GPU-resident page returns stale/zero data instead of faulting and retrying. The failure is intermittent because migration only sometimes succeeds.

Changes:

• .github/workflows/debug-prefetch-flake.yml — new workflow_dispatch workflow that runs only the two failing tests on gfx942 CSP runners, with a configurable repeat count (5–30 parallel matrix jobs) to stress the flake
• projects/hip-tests/catch/packaging/run_hiptests.py — adds HIP_TESTS_REGEX env var support so ctest --tests-regex can target a specific subset without modifying test source

How to use:

1. Get a TheRock artifact run ID from a recent gfx94X-dcgpu CI run
2. Go to Actions → "Debug hipMemPrefetchAsync flake (issue 8271)" → Run workflow
3. Set the artifact run ID and repeat count (10–30 recommended)
4. Check per-matrix-job logs for 42 == 0 or false == true failures

🤖 Generated with Claude Code

[therock-pr-bot]

✅ All Policy Checks Passed

Check	Status	Details
📝 PR Description	✅ Pass	—
⛔ Forbidden Files	✅ Pass	—
🧪 Unit Test	⚠️ Warning	Error: Source/code files changed without an accompanying unit test. Expected: add at least one test file named like test_<name>.py / test_<name>.cpp (or <name>_test.*). Current: code file(s) changed: projects/hip-tests/catch/packaging/run_hiptests.py; no test file found
🚫 Draft PR	🔜 To Be Enabled	—
🚩 Feature Flag	🔜 To Be Enabled	—
📊 Code Coverage	🔜 To Be Enabled	—

🎉 All policy checks passed!

📖 Need help? See the Policy FAQ for details on every check and how to fix failures.

🙋 Wish to Override Policy?

[Copilot]

🟡 Changes recommended

The workflow setup is broken and lacks required GPU-isolation configuration.

Get a fresh assessment by requesting another Copilot review.

Pull request overview

Adds a manual, repeatable workflow to reproduce the hipMemPrefetchAsync flake and supports targeted HIP test selection.

Changes:

• Adds configurable parallel gfx942 debugging workflow.
• Adds HIP_TESTS_REGEX support to the HIP test runner.

File summaries

File	Summary
projects/hip-tests/catch/packaging/run_hiptests.py	Adds regex-based CTest filtering.
.github/workflows/debug-prefetch-flake.yml	Adds repeated prefetch-flake test jobs. Critical (3 votes): checkout/setup paths prevent execution. Moderate (1 vote): missing GPU-isolation environment configuration.

Review details

Suppressed comments (5)

.github/workflows/debug-prefetch-flake.yml:113

• Only prejob/build_tools is fetched above, and the subsequent checkout is rocm-systems, so ./build_tools/github_actions/cleanup_processes.sh does not exist at the workspace root. Because this step runs with always(), every job will fail during cleanup even after a successful test run. Use the fetched tools path (or explicitly check out build_tools at the root).

        run: |

.github/workflows/debug-prefetch-flake.yml:77

• The setup action's FETCH_ARTIFACT_ARGS input defaults to empty, while the hip-tests configuration requires --tests to download the catch test package. Without that argument, build/share/hip/catch_tests is not provisioned and the following test command cannot run the requested tests.
  .github/workflows/debug-prefetch-flake.yml:108
• This invokes run_hiptests.py from the source tree, but derive_rocm_path() only accepts an installed .../share/hip/catch_tests layout. Since ROCM_PATH is not set here, the script raises before CTest starts; point it at the unpacked ROCm root (or invoke the installed copy).

          print("(query skipped - check rocminfo output above)")

.github/workflows/debug-prefetch-flake.yml:41

• This runner is an OSSCI MIXXX GPU runner, for which the repository's other containerized GPU workflows pass /etc/podinfo/gha-gpu-isolation-settings (for example, rocprofiler-systems-continuous-integration.yml:160-168). Omitting that env file can run the matrix jobs without the required GPU-isolation configuration. Add the env-file option to this container.

      options: "--ipc host --user 0:0 --ulimit memlock=-1:-1 --ulimit nofile=1048576:1048576 --security-opt seccomp=unconfined --group-add video --device /dev/kfd --device /dev/dri -e ROCR_VISIBLE_DEVICES -e KUBE_CPU_REQUEST"

.github/workflows/debug-prefetch-flake.yml:102

• The diagnostic step claims the SVM query is skipped, but it still loads libhsa-runtime64.so.1. The no-ROCm image only receives the ROCm libraries under the unpacked build tree, and no library path is configured for this standalone Python process, so this can raise OSError and abort the workflow before the tests. Remove the unused library load.

          echo ""
          echo "=== Large BAR (BAR0 size per GPU) ==="
          for f in /sys/class/drm/card*/device/resource; do
            awk 'NR==1{s=strtonum($1);e=strtonum($2);if(e>s)printf "  %s BAR0: %d MB\n",FILENAME,(e-s+1)/1024/1024}' "$f"
          done

• Files reviewed: 2/2 changed files
• Comments generated: 1
• Review effort level: Lite

---

💡 Add a code-review agent skill or configure MCP servers for context-aware, tailored reviews. Learn more in the docs.