Skip to content

feat(rocprofiler-sdk): Add delayed start options - #11736

Merged
ApoKalipse-V merged 4 commits into
developfrom
users/gbaraldi/delaystart
Sep 26, 2026
Merged

ApoKalipse-V merged 4 commits into
developfrom
users/gbaraldi/delaystart

Conversation

@ApoKalipse-V

@ApoKalipse-V ApoKalipse-V commented Sep 16, 2026 •

Copy link
Copy Markdown
Contributor

Motivation

Thread trace creates hsa or kfd queues for every ROCR_VISIBLE_DEVICES.
For applications spanning dozens of processes, it results in too many queues being created.

This PR adds an API option, and accompanying rocprofv3 option, to define how to initialize thread trace resources. For example:

--att-resource-mode code-object

Delays queue creation until the first code object is uploaded, therefore not creating queues in GPUs never used, while

--att-resource-mode hsa

Keeps the old hsa-based initialization behavior.
A few more initializer options are added to the API for future proofing, but are not implemented yet. Particularly, 'ALL' initializer allocates all resources without waiting for any runtime specific callbacks, depending only on KFD.

JIRA ID : AIPROFSDK-1147
JIRA ID : ROCM-30915

Submission Checklist

@ApoKalipse-V
ApoKalipse-V requested review from a team as code owners September 16, 2026 23:40
@github-actions github-actions Bot added documentation Improvements or additions to documentation project: rocprofiler-sdk labels Sep 16, 2026
@therock-pr-bot

Copy link
Copy Markdown

✅ All Policy Checks Passed

Check Status Details
📝 PR Description ✅ Pass —
⛔ Forbidden Files ✅ Pass —
🧪 Unit Test ✅ Pass —
🚫 Draft PR 🔜 To Be Enabled —
🚩 Feature Flag 🔜 To Be Enabled —
📊 Code Coverage 🔜 To Be Enabled —

🎉 All policy checks passed!

📖 Need help? See the Policy FAQ for details on every check and how to fix failures.

🙋 Wish to Override Policy?

Copilot AI left a comment

Copy link
Copy Markdown
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Copilot review overview

🟡 Changes recommended

Enum-string metadata is incomplete, and the published ALL API name conflicts with the described NODELAY contract.

Get a fresh assessment by requesting another Copilot review.

Review effort: Balanced
Findings: 2 Medium severity

Open (2)
What changed in this PR

Adds configurable ATT resource initialization, defaulting to deferred allocation on the first code-object load while preserving eager HSA initialization.

Changes:

  • Adds public API and rocprofv3 resource-mode options.
  • Implements deferred per-GPU resource creation and pending device-trace starts.
  • Adds unit, launcher, integration, and documentation coverage.
File Description
tests/​rocprofv3/​advanced-thread-trace/​CMakeLists.txt Adds code-object-mode integration coverage.
thread_trace/​tests/​att_packet_test.cpp Tests configuration and deferred lifecycle behavior.
thread_trace/​service.cpp Validates and maps resource modes.
thread_trace/​core.hpp Adds mode state and deferred-start helpers.
thread_trace/​core.cpp Implements lazy resource allocation.
rocprofiler-sdk-tool/​tool.cpp Passes the configured mode to the SDK.
rocprofiler-sdk-tool/​config.hpp Stores and serializes mode configuration.
rocprofiler-sdk-tool/​config.cpp Parses resource-mode environment values.
experimental/​thread-trace/​core.h Defines the public resource-mode API.
cxx/​enum_string.hpp Adds resource-mode labels.
docs/​rocprofv3_input_schema.json Adds input-schema support.
docs/​how-to/​using-thread-trace.rst Documents the CLI option.
docs/​api-reference/​thread_trace.rst Documents API modes and semantics.
bin/​tests/​test_att_resource_mode.py Tests CLI and input-file forwarding.
bin/​tests/​CMakeLists.txt Registers launcher tests.
bin/​rocprofv3.py Adds CLI parsing and environment forwarding.

💡 Add a code-review agent skill or configure MCP servers for context-aware, tailored reviews. Learn more in the docs.

@ApoKalipse-V
ApoKalipse-V merged commit 35b8f2b into develop Sep 26, 2026
388 of 400 checks passed
@ApoKalipse-V
ApoKalipse-V deleted the users/gbaraldi/delaystart branch September 26, 2026 03:41
cursor Bot pushed a commit that referenced this pull request Sep 26, 2026
Resolves the conflict with #11736 (thread trace delayed start options) in
thread_trace/core.cpp: keep develop's null check on the start packet, which
is now absent until the agent's first code object loads, ahead of this
branch's owner tagging.
cursor Bot pushed a commit that referenced this pull request Sep 26, 2026
…eue hook tests

Since #11736 a dispatch thread-trace context defers its queue and packet
factory until the agent's first code object loads. The queue hook tests
dispatch on fake queues that never load one, so no start packet was
injected and three of them failed. Request RESOURCE_MODE_HSA for the tests
that expect a packet.

Co-authored-by: Vivek Kale <vivek.kale@amd.com>
@systems-assistant

Copy link
Copy Markdown
Contributor

TheRock Submodule Bump Activity

Newest first

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

documentation Improvements or additions to documentation organization: ROCm project: rocprofiler-sdk

Projects

None yet

Development

Successfully merging this pull request may close these issues.

3 participants