[PGO][HIP] Decouple device profile drain via HSA introspection

[lfmeadow]

Status

Supersedes draft PR #2714
(same logical patch content, rebased onto current amd-staging; the
intermediate multi-device fix in commit b1b20686afe becomes redundant
under this drain and its tests under compiler-rt/test/profile/GPU/
continue to work via the retained ABI forwarder).

This revision also addresses the static-review feedback gathered against
#2714 — see "Review-driven changes" below.

Build, install & usage

Instructions for building the amdgcn device profile runtime, installing it
into the clang resource directory, and compiling/running/reporting HIP
device coverage live in the new "HIP / AMDGPU device code coverage"
section of clang/docs/SourceBasedCodeCoverage.rst
(added by this PR).

Summary

Replaces the host-shadow / per-CUID / hipModuleLoad-interceptor device
profile drain with an HSA-introspection drain. At process exit the
drain walks every loaded HSA code object on every GPU agent, finds the
canonical __llvm_profile_sections bounds table emitted by
compiler-rt/lib/profile/InstrProfilingPlatformGPU.c, D2H-copies its
counters/data/names back to the host, and writes an arch-prefixed
.profraw via __llvm_write_custom_profile.

What this changes vs. the existing approach

• Host and device drains are fully decoupled. The device drain runs
  from an atexit handler installed by a library constructor in
  libclang_rt.profile. Device counters are collected whether or not
  the host TUs were instrumented and without any host-side per-TU
  shadow, CUID matching, or module-load interception.

• Cases the old 1-1 host↔device model could not handle now work:
  separate device-only modules loaded at runtime
  (hipModuleLoad / hsa_executable_*), an uninstrumented host, and
  multi-GPU. Multi-GPU no longer relies on hipGetSymbolAddress,
  removing the comgr-at-atexit null deref that the predecessor
  band-aid worked around by restricting collection to a single device.

• Clang stops emitting any PGO-specific machinery for HIP.
  CGCUDANV.cpp loses the offload-profiling shadow/registration code
  entirely. For HIP+PGO host links the driver force-links the host drain
  object via -u__llvm_profile_hip_collect_device_data (the drain's
  atexit handler is otherwise unreferenced now that the host emits no
  shadow).

• Device profile runtime is linked on the offload device link.
  LinkerWrapper::ConstructJob now forwards the static device profile
  runtime (libclang_rt.profile-<arch>.a, defining
  __llvm_profile_instrument_gpu and the __llvm_profile_sections bounds
  table) to each GPU device linker when instrumentation is enabled. Without
  this a plain clang -x hip -fprofile-instr-generate -fcoverage-mapping
  full link fails with undefined symbol: __llvm_profile_instrument_gpu,
  since the offload device link otherwise only pulls in libc/builtins (and
  only for OpenMP). The device runtime must be installed at the per-target
  resource path lib/<device-triple>/libclang_rt.profile.a for the driver
  to find it.

• ABI compatibility: legacy
  __llvm_profile_offload_register_{shadow,section_shadow,dynamic_module}
  and __llvm_profile_hip_collect_device_data are retained as
  no-ops / forwarders so binaries compiled against the previous runtime
  still link and produce output.

Review-driven changes (vs the #2714 snapshot)

• drainDevices() idempotency — split into DrainInProgress /
  DrainCompleted; only latches "done" after a successful walk that
  actually drained data. Transient no-ops ("HSA/HIP not yet
  resolvable", "no GPU agents", "no loaded segments", "no instrumented
  sections") stay retryable so a late atexit call still picks up code
  objects that loaded after an early host-write trigger. Covered by
  the new compiler-rt/test/profile/AMDGPU/device-early-collect.hip.

• HIP runtime resolution — tries RTLD_DEFAULT first (catches the
  common case of HIP already being in the process namespace, including
  runtime-only ROCm installs without an unversioned dev symlink), then
  falls back to dlopen of libamdhip64.so.{7,6,5,4} and finally the
  unversioned .so.

• Section bounds validation — processDeviceSections now does
  uintptr_t-based size math, rejects End < Begin and per-section
  spans above 256 MiB, requires
  DataSize % sizeof(__llvm_profile_data) == 0, and per-record
  validates that each CounterPtr resolves inside the copied counters
  region (out-of-range entries are zeroed and warned about instead of
  producing a .profraw that points at unrelated memory).

• HSA symbol-iter error path — non-SUCCESS/non-INFO_BREAK
  return from hsa_executable_iterate_agent_symbols is now warned and
  reflected in the drain's exit status.

• lit gate tightened — compiler-rt/test/profile/lit.cfg.py now
  requires /dev/kfd plus a usable HIP install (probed via
  $ROCM_PATH//opt/rocm) plus the amdgcn device profile runtime
  in the resource directory before enabling the hip / amdgpu
  features. Exports %hip_lib_path and %amdgpu_arch so the new
  AMDGPU/*.hip tests stay portable (and consistent with the existing
  GPU/instrprof-hip-* tests).

Validation

Built clean against current amd-staging on Linux x86_64 + AMDGPU:
host clang/lld/compiler-rt, and libclang_rt.profile-amdgcn.a
(device profile runtime) for the amdgcn target.

End-to-end exercised with RCCL under its --enable-device-coverage
fast build path (debug + -gline-tables-only -O1, default device
linker, gfx90a, ~10 min incremental relink against the rebased runtime):

• librccl.so carries the expected
  __llvm_prf_{names,cnts,data,vnds} + __llvm_covfun +
  __llvm_covmap sections and the
  __llvm_profile_hip_collect_device_data forwarder.
• all_reduce_perf -g 2 produces 1 host .profraw + 26 device
  gfx90a.*.profraw per rank (one per loaded HSA executable), all
  LLVM raw profile version 10.
• Verbose drain log:
  HIP resolved via existing process namespace (RTLD_DEFAULT) and
  walk complete: agents=4 pairs=28 found=26 drained=26 iter-failures=0
  — zero out-of-range counter pointer warnings.
• llvm-profdata merge + llvm-cov report produce realistic per-file
  coverage for both the host librccl.so and the per-arch device ELF.

[ronlieb]

does this need to up upstream at some point ?

[yxsamliu]

I like this approach. The main concern for me is whether it works on Windows. My PR used hipRegister* mainly due to needing to support offload pgo on Windows.

[lfmeadow]

does this need to up upstream at some point ?

I don't really understand our upstream strategy. This is largely dependent on HIP. I don't know if it would ever be ported to NVIDIA, they have their own tools. I guess someone else needs to answer that question.

[ronlieb]

if this PR is needed to get device code coverage working with a trunk build, then yes, upstream

[yxsamliu]

On Windows, HIP runs on top of PAL (Platform Abstraction Library) rather than ROCm/HSA, so hsa-runtime64.dll is not available — the HSA runtime API is not exposed as a separate DLL on Windows at all. The only HSA-named files in the Windows system directory are hsa-thunk64.dll / hsa-thunk.dll, which are the kernel-mode KMT thunk and only export hsaKmt* symbols, not the HSA runtime API. So hsa_init, hsa_iterate_agents, hsa_executable_iterate_agent_symbols, hsa_system_get_major_extension_table, and query_segment_descriptors are all absent on Windows. Only hipMemcpy is available.

So the HSA introspection approach can't work on Windows. My original hipRegister*-based approach worked there because it only needed HIP APIs, which are available on both platforms.

Would it be possible to keep the old shadow-registration path for Windows (#ifdef _WIN32) and use the new HSA introspection drain on Linux? That way Windows users don't lose device PGO support, and Linux gets the cleaner decoupled approach.

[lfmeadow]

Folks, I just realized that this project isn't properly set up for therock build and CI. I'm working on revamping that. Good news it will make private builds easier, but you'll end up having to use more therock infra. Stay tuned, I'm new to this myself.

[lfmeadow]

@yxsamliu Done — implemented exactly the dual path you suggested (commit ad69004).

• Windows keeps the legacy hipRegister* host-shadow drain; Linux uses the HSA introspection drain.
• To avoid #ifdef clutter, the Windows code lives in its own file (InstrProfilingPlatformROCmWindows.cpp) and the CMake compiles exactly one of the two by host platform — no interleaved #if _WIN32 … #else … #endif.
• The CodeGen shadow emission (__llvm_profile_sections_<CUID> device bounds table + host shadow + __hipRegisterVar/registration) is restored behind a single "host is Windows" guard, checked via the aux triple on device compiles; it's inert on other hosts.

One intentional semantic difference: the decoupled, host-uninstrumented collection is Linux-only. Windows still requires the host TUs to be instrumented (same as your original PR), since the shadow registration lives in host code.

Caveat: I haven't been able to validate the Windows path end-to-end (no Windows+GPU CI leg here). It's effectively your previously-working mechanism moved into a separate file, but a Windows CI run would be good before we rely on it. offload-pgo-sections.hip now covers both paths at the IR level.

[lfmeadow]

Closing in favor of a fresh restart on the current amd-staging baseline.

Rationale: amd-staging now carries the relanded HIP offload PGO runtime as the opt-in clang_rt.profile_rocm library (llvm#201606), which diverges from this branch's design. The 16 commits here were implementation/testing artifacts; rather than rebase them, the work is being recreated cleanly on top of amd-staging:

• Linux keeps the HSA-introspection drain (needed for device modules without a host shadow, e.g. RCCL-style synthesized kernels).
• Windows keeps the reland's host-shadow drain unchanged.
• Driver host-link wiring follows Reland HIP offload PGO compiler support and link the device-profile runtime llvm/llvm-project#201607; device-link and the standalone (non-TheRock) cmake build recipe + GPU test runner are brought along.

A new PR will follow from the restarted branch.