[Sundance636]

Motivation

Bind asyncHandler thread to same NUMA domain as main thread, so when HSA_OVERRIDE_CPU_AFFINITY_DEBUG is unset (default), ROCr now checks the main thread's CPU affinity:

• If restricted: background threads are bound to the same NUMA domain
• If unrestricted: background threads remain unrestricted.

This prevents the asyncHandler thread from being scheduled on CPU cores
in a different NUMA domain than the GPU, which was causing performance
degradation on MI-300A systems.

When HSA_OVERRIDE_CPU_AFFINITY_DEBUG=0, ROCr abides by OS-set affinity.

JIRA ID

JIRA ID : AILIROCR-373

Test plan

• Build hsa-runtime on Linux with libnuma found through pkg-config and through the ROCm NUMA CMake package
• Confirm helper-thread affinity stays on the creator thread's NUMA domain when that thread is restricted, and remains unrestricted otherwise

[Copilot]

Pull request overview

This PR updates ROCr’s Linux thread creation logic to derive a NUMA-aware CPU affinity mask from the creating thread’s current CPU affinity and apply it to background threads, aiming to keep async/background work scheduled within the same NUMA domain (notably improving MI-300A performance).

Changes:

• Added libnuma-based helpers to compute allowed NUMA nodes from the current thread’s CPU affinity and to build a CPU set from those nodes.
• Updated os_thread CPU affinity setup to use the derived NUMA-based CPU set (with fallbacks).

Comments suppressed due to low confidence (1)

projects/rocr-runtime/runtime/hsa-runtime/core/util/lnx/os_linux.cpp:269

• When HAVE_PTHREAD_ATTR_SETAFFINITY_NP is not defined, the cpuset allocated earlier in the override_cpu_affinity() block is never freed (since CPU_FREE(cpuset) is inside this #ifdef). Later, the non-attr path allocates another cpuset, so the original allocation leaks. Ensure cpuset is freed on all build configurations (or reuse the same allocation in the non-attr path).

#ifdef HAVE_PTHREAD_ATTR_SETAFFINITY_NP
      err = pthread_attr_setaffinity_np(&attrib, CPU_ALLOC_SIZE(cores), cpuset);
      CPU_FREE(cpuset);
      if (err != 0) {

---

💡 Add Copilot custom instructions for smarter, more guided reviews. Learn how to get started.

[cfreeamd]

🤖 AI-assisted pre-review (for @cfreeamd) — re-review after 2026-08-27 rework

Two new commits substantially rework both NUMA detection (CMakeLists.txt) and the NUMA affinity implementation (os_linux.cpp).

shwetagkhatri's CHANGES_REQUESTED concerns — all addressed:

• bool/bitmask mix: fixed (separate all_nodes_allowed bool and allowed_node_bitmask)

• 64 NUMA nodes UB: fixed (explicit bounds check with stderr fallback warning)

• TheRock build: fixed (3-tier detection: pkg-config → CMake package → manual find_library, rocm_sysdeps_numa name included, INTERFACE_INCLUDE_DIRECTORIES on IMPORTED target)
• numa_available() guard: added

Original Important concern (CMakeLists/HAS_NUMA coupling) — mitigated. The new IMPORTED target sets INTERFACE_INCLUDE_DIRECTORIES, which propagates include dirs when target_link_libraries(... numa) is used. __has_include(<numa.h>) should now resolve correctly in practice. The cleaner fix (CMake-defined HAVE_NUMA=1) is still preferable but no longer blocking.

Remaining (non-blocking): FATAL_ERROR for all Linux builds (original Optional unchanged); indentation inconsistency in FillCpuSetFromNumaNodes and double GetAllowedNumaNodeMask() call in the #ifndef HAVE_PTHREAD_ATTR_SETAFFINITY_NP path (original Nits, unchanged).

Note: shwetagkhatri still has an outstanding CHANGES_REQUESTED. The rework addresses their specific concerns — worth pinging for a re-review.

[github-actions]

This pull request has been inactive for 25 days and will be marked as stale.

If you would like to keep this PR open, please:

• Add new commits
• Add a comment explaining why it should remain open

This PR will be automatically closed in 5 days if no further activity occurs.

[github-actions]

This pull request has been automatically closed due to inactivity (30 days with no updates).

If you'd like to continue working on this, feel free to reopen the PR or create a new one.

[dayatsin-amd]

@shwetagkhatri @cfreeamd I made some changes to this PR:

• When CMake finds the ROCm NUMA package, its numa::numa target is forwarded through the numa target used by the link lines.
• If a restricted NUMA mask yields no CPUs, the helper thread falls back to the full CPU set instead of receiving an empty affinity mask.

Please review

[cfreeamd]

🤖 AI-assisted pre-review (for @cfreeamd)

Re-review after the 2026-10-19 reopen + 94f951c3 ("Fix NUMA package linking and empty affinity fallback") and the 2026-10-21 develop merge.

The two fixes in 94f951c3 look correct:

• numa::numa is now forwarded through an INTERFACE numa target with ROCM_PATH hints — resolves the TheRock linking concern cleanly.
• The added_cpu guard falls back to the full CPU set when a restricted NUMA mask yields zero CPUs, preventing pthread_setaffinity_np failure → helper thread never created. Good defensive fix.

Two things still open, both one-line fixes (inline). The FATAL_ERROR now contradicts three graceful-degradation paths that the same PR added, and there's a cpuset leak in the non-pthread_attr build path.

(Note: this PR still carries the "Not ready to Review" label.)

[dayatsin-amd]

This is what we say in our docs about HSA_OVERRIDE_CPU_AFFINITY_DEBUG -

$ grep -r "HSA_OVERRIDE_CPU_AFFINITY_DEBUG" runtime/hsa-runtime/core/util/flag.h: var = os::GetEnvVar("HSA_OVERRIDE_CPU_AFFINITY_DEBUG"); runtime/docs/data/env_variables.rst: * - | HSA_OVERRIDE_CPU_AFFINITY_DEBUG

• • | HSA_OVERRIDE_CPU_AFFINITY_DEBUG
    | Controls whether ROCm helper threads inherit the parent process's CPU affinity mask.

    • 1
    • | 0: Enable inheritance. Helper threads use the parent process's core affinity mask, which should be set with enough cores for all threads.
      | 1: Disable inheritance. Helper threads spawn on all available cores, ignoring the parent's affinity settings, which may affect performance in certain environments.

The patch is preserving the override_cpu_affinity guard, but it changes the documented behavior of HSA_OVERRIDE_CPU_AFFINITY_DEBUG=1 /unset above... The docs says that this mode disables affinity inheritance and lets helper threads run on all cores, while this patch restricts helper threads to the creator thread’s NUMA domain when the creator is restricted.

I think we need to also update the docs and commit message to explicitly state that default behavior is changing from full-core override to NUMA-domain inheritance.

The new behavior was discussed and agree'd here:
https://amd-hub.atlassian.net/browse/AILIROCR-373?focusedCommentId=261376

@bimaniamd Can you please check whether the current docs align with the description in that comment. Please update the docs if needed.