Fix H2D staging race with host function callback on all GPUs by anugodavar · Pull Request #13275 · ROCm/rocm-systems

anugodavar · 2026-10-12T15:36:06Z

Summary

Fix race condition where H2D staging copies read from hostSrc (via CPU memcpy) before a pending hipLaunchHostFunc callback has finished writing to it
Affects all GPUs using the ROCR backend — both discrete (MI300X, RX 9070 XT) and unified-memory APUs (Strix Halo)
Two staging paths are fixed:
- hsaCopyStagedOrPinned (DMA blit) — used for large copies (> sdmaCopyThreshold_)
- KernelBlitManager::writeBuffer (shader blit) — used for small copies, was completely unprotected

JIRA ID: SWDEV-542012

Root Cause

The graph captures: MallocAsync → HostFunc(fill h_array) → H2D → Kernel → D2H → FreeAsync

During hipGraphLaunch, the host function runs asynchronously via a signal handler thread. Both H2D staging paths do a CPU memcpy(stagingBuffer, hostSrc, ...) without waiting for the host callback to complete, racing with it and copying stale data.

The pinned/SDMA path does not have this race because the DMA engine's read from hostSrc is ordered by GPU signal dependencies.

Fix

hsaCopyStagedOrPinned: Add releaseGpuMemoryFence(false) in the H2D staging path (when pinnedMem_ == nullptr) before the CPU memcpy. The pinned path is not affected — SDMA reads are ordered by the signal chain.
KernelBlitManager::writeBuffer: Add releaseGpuMemoryFence(false) before the shader-blit staging loop. This path was completely missing the wait.

Test Results

GPU	OS	Backend	Path	Before	After
MI300X	Linux	ROCR	Kernel blit (4KB)	0/100	100/100
MI300X	Linux	ROCR	DMA blit (1MB)	0/100	100/100
RX 9070 XT	Windows	ROCR	Kernel blit (4KB)	12/100	100/100
RX 9070 XT	Windows	PAL	Kernel blit (4KB)	100/100	N/A (PAL unaffected)

Test Plan

hipLaunchHostFunc -> hipMemcpyAsync H2D repro passes on MI300X (Linux, ROCR)
hipLaunchHostFunc -> hipMemcpyAsync H2D repro passes on RX 9070 XT (Windows, ROCR)
PAL backend unaffected (Windows, RX 9070 XT) — 100/100 without fix
No regression on discrete GPUs — skip-wait optimization preserved for pinned path
Unit_hipLaunchHostFunc_H2D_Kernel_D2H_Capture passes on gfx1151 (Strix Halo APU)

🤖 Generated with Claude Code

On unified-memory APUs (e.g. gfx1151 Strix Halo), hsaCopyStagedOrPinned() uses a staging buffer for H2D copies: CPU memcpy from hostSrc into a staging buffer, then SDMA from staging to device memory. The existing code unconditionally skipped the CPU wait for all H2D copies (releaseGpuMemoryFence(true)), which is safe on discrete GPUs where the pinning path is used and GPU barriers ensure ordering. However, on APUs with hostUnifiedMemory_=true, the staging path's CPU memcpy reads from hostSrc before any pending host-function callback (hipLaunchHostFunc) has finished writing to it. This causes a race condition where the memcpy copies stale data (zeros instead of the values the host function was supposed to fill). Fix: Only skip the CPU wait for H2D when the device does NOT have unified memory (i.e. discrete GPUs using the pinning path). On unified-memory APUs, the CPU wait ensures all prior operations — including host function callbacks — complete before the staging memcpy reads the source buffer. Fixes: SWDEV-542012 Co-Authored-By: Claude <noreply@anthropic.com>

therock-pr-bot · 2026-10-12T15:37:04Z

✅ All Policy Checks Passed

Check	Status	Details
📝 PR Description	✅ Pass	—
⛔ Forbidden Files	✅ Pass	—
🧪 Unit Test	⚠️ Warning	Error: Source/code files changed without an accompanying unit test. Expected: add at least one test file named like `test_<name>.py` / `test_<name>.cpp` (or `<name>_test.`). Current:* code file(s) changed: `projects/clr/rocclr/device/rocm/rocblit.cpp`; no test file found
🚫 Draft PR	🔜 To Be Enabled	—
🚩 Feature Flag	🔜 To Be Enabled	—
📊 Code Coverage	🔜 To Be Enabled	—

🎉 All policy checks passed!

📖 Need help? See the Policy FAQ for details on every check and how to fix failures.

🙋 Wish to Override Policy?

therock-pr-bot · 2026-10-12T15:37:06Z

🚫 Please fix the failed policies before requesting reviews.

The following policy checks failed:

❌ PR Description

The Not ready to Review label has been added to this PR.
Once all policies pass, the label will be removed automatically.

Copilot

🟡 Changes recommended

Discrete-GPU staging paths still retain the same race for small transfers or pinning failures.

1 open finding

Correct misleading D2H skip-wait comment · New

What changed in this PR

Updates H2D synchronization to prevent host-callback staging races on unified-memory APUs.

Changes:

Waits for preceding stream work before staged APU H2D copies.
Preserves skipped waits for discrete GPUs.

File	Description
`projects/clr/rocclr/device/rocm/rocblit.cpp`	Adjusts H2D fence-wait selection.

🧠 Review effort: Balanced

💡 Add a code-review agent skill or configure MCP servers for context-aware, tailored reviews. Learn more in the docs.

mangupta

🤖 AI-assisted pre-review (for @mangupta)

The original fix only waited for host callback completion on unified-memory APUs (hostUnifiedMemory_=true). Testing on discrete GPUs (MI300X on Linux, RX 9070 XT on Windows with ROCR backend) showed the same race: hipLaunchHostFunc callback writes to hostSrc while the staging path's CPU memcpy reads stale data (0/100 pass rate on MI300X, 12/100 on RX 9070 XT). Changes: - hsaCopyStagedOrPinned: wait for H2D only in the staging path (pinnedMem_==nullptr), not unconditionally. The pinned/SDMA path is ordered by GPU signal dependencies and does not need a CPU wait. - KernelBlitManager::writeBuffer: add releaseGpuMemoryFence(false) before the shader-blit staging loop — this path was completely unprotected and is hit for small copies (< sdmaCopyThreshold_). Verified 100/100 pass on both paths (4KB kernel-blit, 1MB DMA-blit) on RX 9070 XT (Windows, ROCR) and MI300X (Linux). Fixes: SWDEV-542012 Co-Authored-By: Claude <noreply@anthropic.com>

mangupta · 2026-10-22T15:30:58Z

🤖 AI-assisted pre-review (for @mangupta)

Re-reviewed the updated head (f133e0bdff93) — no new findings at or above the posting threshold.

mangupta · 2026-10-26T15:16:55Z

🤖 AI-assisted pre-review (for @mangupta)

Re-reviewed the updated head (2eaec5348766) — no new findings at or above the posting threshold.

(#13275) MIME-Version: 1.0 Content-Type: text/plain; charset=UTF-8 Content-Transfer-Encoding: 8bit ## Summary - Fix race condition where H2D staging copies read from `hostSrc` (via CPU `memcpy`) before a pending `hipLaunchHostFunc` callback has finished writing to it - Affects **all GPUs** using the ROCR backend — both discrete (MI300X, RX 9070 XT) and unified-memory APUs (Strix Halo) - Two staging paths are fixed: - `hsaCopyStagedOrPinned` (DMA blit) — used for large copies (> `sdmaCopyThreshold_`) - `KernelBlitManager::writeBuffer` (shader blit) — used for small copies, was completely unprotected JIRA ID: SWDEV-542012 ## Root Cause The graph captures: `MallocAsync → HostFunc(fill h_array) → H2D → Kernel → D2H → FreeAsync` During `hipGraphLaunch`, the host function runs asynchronously via a signal handler thread. Both H2D staging paths do a CPU `memcpy(stagingBuffer, hostSrc, ...)` without waiting for the host callback to complete, racing with it and copying stale data. The pinned/SDMA path does not have this race because the DMA engine's read from `hostSrc` is ordered by GPU signal dependencies. ## Fix 1. **hsaCopyStagedOrPinned**: Add `releaseGpuMemoryFence(false)` in the H2D staging path (when `pinnedMem_ == nullptr`) before the CPU `memcpy`. The pinned path is not affected — SDMA reads are ordered by the signal chain. 2. **KernelBlitManager::writeBuffer**: Add `releaseGpuMemoryFence(false)` before the shader-blit staging loop. This path was completely missing the wait. ## Test Results | GPU | OS | Backend | Path | Before | After | |-----|----|---------|----- |--------|-------| | MI300X | Linux | ROCR | Kernel blit (4KB) | 0/100 | 100/100 | | MI300X | Linux | ROCR | DMA blit (1MB) | 0/100 | 100/100 | | RX 9070 XT | Windows | ROCR | Kernel blit (4KB) | 12/100 | 100/100 | | RX 9070 XT | Windows | PAL | Kernel blit (4KB) | 100/100 | N/A (PAL unaffected) | ## Test Plan - [x] hipLaunchHostFunc -> hipMemcpyAsync H2D repro passes on MI300X (Linux, ROCR) - [x] hipLaunchHostFunc -> hipMemcpyAsync H2D repro passes on RX 9070 XT (Windows, ROCR) - [x] PAL backend unaffected (Windows, RX 9070 XT) — 100/100 without fix - [ ] No regression on discrete GPUs — skip-wait optimization preserved for pinned path - [ ] Unit_hipLaunchHostFunc_H2D_Kernel_D2H_Capture passes on gfx1151 (Strix Halo APU) 🤖 Generated with [Claude Code](https://claude.com/claude-code) [rocm-systems] ROCm/rocm-systems#13275 (commit 8a45f2c)

## Summary - Fix race condition where H2D staging copies read from `hostSrc` (via CPU `memcpy`) before a pending `hipLaunchHostFunc` callback has finished writing to it - Affects **all GPUs** using the ROCR backend — both discrete (MI300X, RX 9070 XT) and unified-memory APUs (Strix Halo) - Two staging paths are fixed: - `hsaCopyStagedOrPinned` (DMA blit) — used for large copies (> `sdmaCopyThreshold_`) - `KernelBlitManager::writeBuffer` (shader blit) — used for small copies, was completely unprotected JIRA ID: SWDEV-542012 ## Root Cause The graph captures: `MallocAsync → HostFunc(fill h_array) → H2D → Kernel → D2H → FreeAsync` During `hipGraphLaunch`, the host function runs asynchronously via a signal handler thread. Both H2D staging paths do a CPU `memcpy(stagingBuffer, hostSrc, ...)` without waiting for the host callback to complete, racing with it and copying stale data. The pinned/SDMA path does not have this race because the DMA engine's read from `hostSrc` is ordered by GPU signal dependencies. ## Fix 1. **hsaCopyStagedOrPinned**: Add `releaseGpuMemoryFence(false)` in the H2D staging path (when `pinnedMem_ == nullptr`) before the CPU `memcpy`. The pinned path is not affected — SDMA reads are ordered by the signal chain. 2. **KernelBlitManager::writeBuffer**: Add `releaseGpuMemoryFence(false)` before the shader-blit staging loop. This path was completely missing the wait. ## Test Results | GPU | OS | Backend | Path | Before | After | |-----|----|---------|----- |--------|-------| | MI300X | Linux | ROCR | Kernel blit (4KB) | 0/100 | 100/100 | | MI300X | Linux | ROCR | DMA blit (1MB) | 0/100 | 100/100 | | RX 9070 XT | Windows | ROCR | Kernel blit (4KB) | 12/100 | 100/100 | | RX 9070 XT | Windows | PAL | Kernel blit (4KB) | 100/100 | N/A (PAL unaffected) | ## Test Plan - [x] hipLaunchHostFunc -> hipMemcpyAsync H2D repro passes on MI300X (Linux, ROCR) - [x] hipLaunchHostFunc -> hipMemcpyAsync H2D repro passes on RX 9070 XT (Windows, ROCR) - [x] PAL backend unaffected (Windows, RX 9070 XT) — 100/100 without fix - [ ] No regression on discrete GPUs — skip-wait optimization preserved for pinned path - [ ] Unit_hipLaunchHostFunc_H2D_Kernel_D2H_Capture passes on gfx1151 (Strix Halo APU) 🤖 Generated with [Claude Code](https://claude.com/claude-code) --------- Co-authored-by: Claude <noreply@anthropic.com>

systems-assistant · 2026-10-28T16:29:05Z

TheRock Submodule Bump Activity

Newest first

2026-10-28 — Included in TheRock via ROCm/TheRock#9555.

Copilot AI balanced review requested due to automatic review settings October 12, 2026 15:36

anugodavar requested a review from a team as a code owner October 12, 2026 15:36

github-actions Bot added the project: clr label Oct 12, 2026

therock-pr-bot Bot added the Not ready to Review label Oct 12, 2026

Copilot started reviewing on behalf of anugodavar October 12, 2026 15:37

View session

Copilot AI reviewed Oct 12, 2026

View reviewed changes

Comment thread projects/clr/rocclr/device/rocm/rocblit.cpp Resolved

systems-assistant Bot added the organization: ROCm label Oct 12, 2026

mangupta reviewed Oct 13, 2026

View reviewed changes

Comment thread projects/clr/rocclr/device/rocm/rocblit.cpp Resolved

Comment thread projects/clr/rocclr/device/rocm/rocblit.cpp Resolved

Comment thread projects/clr/rocclr/device/rocm/rocblit.cpp Resolved

mangupta added the lrt-bot: ready label Oct 13, 2026

AidanBeltonS reviewed Oct 14, 2026

View reviewed changes

Comment thread projects/clr/rocclr/device/rocm/rocblit.cpp Resolved

mangupta added lrt-bot: review and removed lrt-bot: ready labels Oct 16, 2026

anugodavar mentioned this pull request Oct 18, 2026

Issue] Windows nightly: hip-tests (ROCR) Unit_hipLaunchHostFunc_H2D_Kernel_D2H_Capture fails #13096

Closed

anugodavar changed the title ~~Fix H2D staging race with host function callback on unified-memory APUs~~ Fix H2D staging race with host function callback on all GPUs Oct 22, 2026

saleelk approved these changes Oct 23, 2026

View reviewed changes

mangupta added lrt-bot: ready and removed lrt-bot: review labels Oct 23, 2026

anugodavar removed the Not ready to Review label Oct 26, 2026

Merge branch 'develop' into users/agodavar/fix-host-func-h2d-race

2eaec53

anugodavar merged commit 8a45f2c into develop Oct 28, 2026
600 of 607 checks passed

anugodavar deleted the users/agodavar/fix-host-func-h2d-race branch October 28, 2026 06:08

mangupta removed the lrt-bot: ready label Oct 28, 2026

hartmark mentioned this pull request Nov 2, 2026

[Issue]: GPU queues stay evicted ~80% of the time when clr pins pageable memory under host memory pressure; hipMemcpy stalls for minutes while a core spins #12528

Open

This was referenced Nov 3, 2026

[Issue]: small async hipMemcpyAsync from pageable memory reads stale data from the staging ring (agent-scope acquire); breaks RCCL all-gather on gfx1100 #14886

Open

fix(clr): acquire at system scope for every staged H2D shader copy #14897

Closed

Navigation Menu

Repository navigation

Fix H2D staging race with host function callback on all GPUs - #13275

Fix H2D staging race with host function callback on all GPUs#13275
anugodavar merged 3 commits into
developfrom
users/agodavar/fix-host-func-h2d-race

anugodavar commented Oct 12, 2026 •

edited

Loading

Uh oh!

therock-pr-bot Bot commented Oct 12, 2026 •

edited

Loading

Uh oh!

therock-pr-bot Bot commented Oct 12, 2026

Uh oh!

Copilot AI left a comment

Uh oh!

Uh oh!

mangupta left a comment

Uh oh!

Uh oh!

Uh oh!

Uh oh!

Uh oh!

mangupta commented Oct 22, 2026

Uh oh!

mangupta commented Oct 26, 2026

Uh oh!

Uh oh!

systems-assistant Bot commented Oct 28, 2026

Uh oh!

Reviewers

Uh oh!

Assignees

Labels

Projects

Milestone

Development

Uh oh!

5 participants

Conversation

anugodavar commented Oct 12, 2026 • edited Loading Uh oh! There was an error while loading. Please reload this page.

Uh oh!

Summary

Root Cause

Fix

Test Results

Test Plan

Uh oh!

therock-pr-bot Bot commented Oct 12, 2026 • edited Loading Uh oh! There was an error while loading. Please reload this page.

Uh oh!

✅ All Policy Checks Passed

Uh oh!

therock-pr-bot Bot commented Oct 12, 2026

Uh oh!

Copilot AI left a comment

Choose a reason for hiding this comment

🟡 Changes recommended

Uh oh!

Uh oh!

mangupta left a comment

Choose a reason for hiding this comment

Uh oh!

Uh oh!

Uh oh!

Uh oh!

Uh oh!

mangupta commented Oct 22, 2026

Uh oh!

mangupta commented Oct 26, 2026

Uh oh!

Uh oh!

systems-assistant Bot commented Oct 28, 2026

TheRock Submodule Bump Activity

Uh oh!

Reviewers

Uh oh!

Assignees

Labels

Projects

Milestone

Development

Uh oh!

5 participants

anugodavar commented Oct 12, 2026 •

edited

Loading

therock-pr-bot Bot commented Oct 12, 2026 •

edited

Loading