Skip to content

refactor(rocprofiler-sdk): migrate PC sampling off the per-queue callback registry - #11966

Closed
amd-vkale wants to merge 8 commits into
developfrom
users/vkale/remove-callbacks-pc-sampling
Closed

amd-vkale wants to merge 8 commits into
developfrom
users/vkale/remove-callbacks-pc-sampling

Conversation

@amd-vkale

Copy link
Copy Markdown
Contributor

Rehosted copy of #8895. Identical content, but the head branch users/vkale/remove-callbacks-pc-sampling lives in
ROCm/rocm-systems instead of a fork, so the CodeQL workflow can read the container
registry secrets it needs. Review history stays on #8895, which is now stacked on this
branch.

Motivation

The purpose of this PR is to isolate the PC-sampling slice of the callback-removal effort (reference PRs #8730 / #8586) into a small, single-service change, per review guidance to land callback removal one service at a time. Helps toward but independent of the kernel replay feature in #7960.

Underlying Problem

PC sampling currently registers a per-queue completion callback with the HSA queue controller so kernel completions reach the CID manager. That ties completion delivery to the shared per-queue callback registry even though marker-packet injection was never registry-based.

This PR:

  1. Removes the registry registration from pc_sampling_service_finish_configuration and delivers completion explicitly via pc_sampling::signal_completion_hook from the async signal handler.
  2. Replaces the get_notifiers() gate with is_configured_on_agent() so PC-only runs still enter the write interceptor and receive marker packets on configured agents.

PC sampling has no enqueue hook — marker injection stays inline in WriteInterceptor. This is the smallest migration: completion-only, no batching change. The per-queue callback registry remains for counters, thread trace, and SPM until those services land in their own PRs.

Notes on Bigger Picture

Sibling migrations: thread trace (#8790), SPM (#8887), counter collection (#8891).

Kernel replay (#7960) benefits from callback removal but this PR is independent and behavior-preserving on its own.

Technical Details

flowchart LR
  WriteInterceptor --> Marker["generate_marker_packet_for_kernel if configured on agent"]
  AsyncSignalHandler --> CompletionHook["pc_sampling::signal_completion_hook"]
  CompletionHook --> CID["kernel_completion_cb → cid_manager"]
Loading

Permalinks pin to ec87a2f.

Hooks and call sites

PC sampling does not use client_ids.hpp producer tags on this branch (marker injection is unchanged). No is_any_active() — the gate is per-agent configuration state.

Reviewer Guide

  1. Confirm no add_callback / remove_callback in hsa_adapter.cpp configure path.
  2. AsyncSignalHandler calls pc_sampling::signal_completion_hook after registry walk (queue.cpp#L172-L179).
  3. no_real_consumers uses is_configured_on_agent(agent) per dispatch queue's agent (queue.cpp#L318-L321) — PC-sampling-only on one GPU must still intercept that GPU's dispatches.
  4. Marker injection path unchanged except gate rename to is_configured_on_agent (queue.cpp).
  5. kernel_completion_cb still guards on is_pc_sample_service_configured and correlation id (hsa_adapter.cpp#L138-L149).
  6. #if ROCPROFILER_SDK_HSA_PC_SAMPLING — hook stubs must link when support is off (queue_hooks.cpp).
  7. Unit test: queue_hooks_test.cpp#L34-L39.

Questions to answer:

  • With PC sampling configured on GPU-0 only, does GPU-0 still get marker injection and completion CID retirement?

  • With no PC sampling configured, does is_configured_on_agent return false for all agents?

Out of scope: enqueue hook extraction; client_ids.hpp tagging; per-agent scoping API; deleting Queue::_callbacks.

Issue Tracking

JIRA ticket:
AIPROFSDK-1019

Test Plan

Test Result

Unit gate test as above. Treat the Checks tab as source of truth for CI.

Submission Checklist

amd-vkale and others added 8 commits July 20, 2026 18:55
…back registry

Isolates the PC-sampling slice of the callback-removal effort (reference PRs
#8730 / #8586) into a small, single-service change, per review guidance to land
callback removal one service at a time. Independent of kernel replay.

PC sampling no longer registers a queue_callbacks_t with the HSA queue
controller in pc_sampling_service_finish_configuration. Instead:
  - pc_sampling::signal_completion_hook (new pc_sampling/queue_hooks.{hpp,cpp})
    delivers kernel completion to the CID manager; it is called explicitly from
    the HSA async signal handler in hsa/queue.cpp. kernel_completion_cb is moved
    out of the anonymous namespace and declared in hsa_adapter.hpp.
  - pc_sampling::is_configured_on_agent replaces the queue.get_notifiers()
    signal the old registration used to provide, so the write interceptor still
    injects the marker packet for PC-only runs. It is always linkable (guarded
    by ROCPROFILER_SDK_HSA_PC_SAMPLING) and compiled before the pc_sampling
    CMake early-return so it links even without HSA PC sampling support.

The marker packet injection in the write interceptor is unchanged (it was never
registry-based). Removes the now-unused intercept_cb_id from PCSAgentSession and
adds a queue_hooks unit test. The per-queue callback registry is retained for
the remaining service (thread-trace/spm/counters migrate in their own PRs).

Co-authored-by: Cursor <cursoragent@cursor.com>
Co-authored-by: Vivek Kale <vivek.kale@amd.com>
… at the shim

Review asked whether the ROCPROFILER_SDK_HSA_PC_SAMPLING guard could move into
is_pc_sample_service_configured instead of living in a separate wrapper. It
cannot: queue_hooks.cpp is added to the object library before the early return
in CMakeLists.txt so it is always built, while service.cpp defines that function
after the early return and is not built when PC sampling is compiled out.
Moving the guard would leave the write interceptor with an undefined symbol in
that configuration.

Record the constraint at the shim so the next reader does not retry the change,
and bump the copyright year on the new files to 2026.

(cherry picked from commit fa38a01)
…completion hook

Three review findings on the completion path:

The hook took the queue by reference, so the unit test covering the null-session
early return had to synthesize one by dereferencing a null pointer. That is
undefined behaviour at argument construction even though the hook returns before
touching the queue. Take the queue by pointer instead, which lets the test pass
nullptr honestly and documents that the callee must not assume a queue exists.

kernel_completion_cb took a mutable reference to a packet it never modifies,
forcing the hook to copy the kernel packet on every completion. Make the
parameter const and drop the copy.

The configured-agent check lived inside kernel_completion_cb, after that copy, so
a process with any interceptor active paid the cost on every dispatch even when
PC sampling was never configured. The registry-based path cost nothing until a
callback was registered. Check is_pc_sample_service_configured in the hook before
doing any work, restoring that property; kernel_completion_cb has one caller, so
the check is not lost.

Also drops a stale kernel_completion_cb declaration in pc_sampling_internals.hpp
that still described the removed registry signature, and corrects two comments
that referred to the old callback mechanism.

(cherry picked from commit c93f8dd)
…ity jobs

Both workflows failed before compiling any rocprofiler-sdk source. Code
coverage died installing ROCR-Runtime with 'CMake 3.25 or higher is
required. You are running version 3.22.1', and release compatibility died
resolving find_dependency(NUMA) from hsakmt-config.cmake on the 'latest'
ROCm container.

This branch predates the fixes for both on develop, so take develop's
versions of the two workflows and of the rocprofiler-sdk-util action they
depend on. The action change is additive -- python-venv-path defaults to
empty and is guarded -- so the continuous integration workflow, which does
not pass it, is unaffected.

The pinned CMake is held at 3.25 rather than develop's 4.x. The external
submodules on this branch (json, pybind11, sqlite, yaml-cpp) still declare
cmake_minimum_required below 3.5, which CMake 4 rejects outright, so 4.x
would trade one configure failure for another.

Co-authored-by: Vivek Kale <vivek.kale@amd.com>
…ility

Develop dropped -DROCPROFILER_DISABLE_ATT_QUICK_SCAN=ON because it builds
rocprof-trace-decoder, but this branch does not, so rocprofiler-sdk-tool's
find_package(rocprof-trace-decoder REQUIRED) fails to configure against
every ROCm release container. Restore the flag.

Co-authored-by: Vivek Kale <vivek.kale@amd.com>
…emove-callbacks-pc-sampling

Co-authored-by: Cursor <cursoragent@cursor.com>

# Conflicts:
#	projects/rocprofiler-sdk/source/lib/rocprofiler-sdk/hsa/queue.cpp
#	projects/rocprofiler-sdk/source/lib/rocprofiler-sdk/pc_sampling/tests/CMakeLists.txt
…sampling

Co-authored-by: Cursor <cursoragent@cursor.com>
Copilot AI lite review requested due to automatic review settings September 21, 2026 20:01
@amd-vkale
amd-vkale requested review from a team as code owners September 21, 2026 20:01
@github-actions github-actions Bot added project: rocprofiler-sdk github actions Pull requests that update GitHub Actions code labels Sep 21, 2026
@therock-pr-bot

Copy link
Copy Markdown

✅ All Policy Checks Passed

Check Status Details
📝 PR Description ✅ Pass —
⛔ Forbidden Files ✅ Pass —
🧪 Unit Test ✅ Pass —
🚫 Draft PR 🔜 To Be Enabled —
🚩 Feature Flag 🔜 To Be Enabled —
📊 Code Coverage 🔜 To Be Enabled —

🎉 All policy checks passed!

📖 Need help? See the Policy FAQ for details on every check and how to fix failures.

🙋 Wish to Override Policy?

@amd-vkale

Copy link
Copy Markdown
Contributor Author

Superseded: recreating from the --with-cursor-fixes branch name so the base branches are consistently named.

@amd-vkale amd-vkale closed this Sep 21, 2026

Copilot AI left a comment

Copy link
Copy Markdown
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Copilot review overview

🔵 Needs a closer look

One or more issues must be addressed before approval.

Review effort: Lite
Findings: None

What changed in this PR

Refactors PC sampling completion delivery away from the per-queue callback registry while preserving marker injection and adding hook coverage.

Changes:

  • Adds explicit PC sampling configuration and completion hooks.
  • Removes callback registration state and updates HSA completion handling.
  • Adds unit tests and pins compatibility workflow CMake.
File Description
projects/​rocprofiler-sdk/​source/​lib/​rocprofiler-sdk/​pc_sampling/​types.hpp Updated as part of this pull request.
projects/​rocprofiler-sdk/​source/​lib/​rocprofiler-sdk/​pc_sampling/​tests/​queue_hooks_test.cpp Updated as part of this pull request.
projects/​rocprofiler-sdk/​source/​lib/​rocprofiler-sdk/​pc_sampling/​tests/​pc_sampling_internals.hpp Updated as part of this pull request.
projects/​rocprofiler-sdk/​source/​lib/​rocprofiler-sdk/​pc_sampling/​tests/​CMakeLists.txt Updated as part of this pull request.
projects/​rocprofiler-sdk/​source/​lib/​rocprofiler-sdk/​pc_sampling/​queue_hooks.hpp Updated as part of this pull request.
projects/​rocprofiler-sdk/​source/​lib/​rocprofiler-sdk/​pc_sampling/​queue_hooks.cpp Updated as part of this pull request.
projects/​rocprofiler-sdk/​source/​lib/​rocprofiler-sdk/​pc_sampling/​hsa_adapter.hpp Updated as part of this pull request.
projects/​rocprofiler-sdk/​source/​lib/​rocprofiler-sdk/​pc_sampling/​hsa_adapter.cpp Updated as part of this pull request.
projects/​rocprofiler-sdk/​source/​lib/​rocprofiler-sdk/​pc_sampling/​CMakeLists.txt Updated as part of this pull request.
projects/​rocprofiler-sdk/​source/​lib/​rocprofiler-sdk/​hsa/​queue.cpp Updated as part of this pull request.
.github/​workflows/​rocprofiler-sdk-rocm_release_compatibility.yml Updated as part of this pull request.

💡 Add a code-review agent skill or configure MCP servers for context-aware, tailored reviews. Learn more in the docs.

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

github actions Pull requests that update GitHub Actions code project: rocprofiler-sdk

Projects

None yet

Development

Successfully merging this pull request may close these issues.

3 participants