feat(rocdbgapi): introduce clusters of workgroups (alternative) by aktemur · Pull Request #10241 · ROCm/rocm-systems

aktemur · 2026-08-17T17:08:40Z

This is the same as PR #7992, except that it implements the "there is always a cluster" model.

therock-pr-bot · 2026-08-17T17:10:08Z

✅ All Policy Checks Passed

Check	Status	Details
📝 PR Description	✅ Pass	—
⛔ Forbidden Files	✅ Pass	—
🧪 Unit Test	⚠️ Warning	Error: Source/code files changed without an accompanying unit test. Expected: add at least one test file named like `test_<name>.py` / `test_<name>.cpp` (or `<name>_test.`). Current:* code file(s) changed: `projects/rocdbgapi/src/architecture.cpp`, `projects/rocdbgapi/src/architecture.h`, `projects/rocdbgapi/src/cluster.cpp`, `projects/rocdbgapi/src/cluster.h`, `projects/rocdbgapi/src/logging.cpp` (+10 more); no test file found
🚫 Draft PR	🔜 To Be Enabled	—
🚩 Feature Flag	🔜 To Be Enabled	—
📊 Code Coverage	🔜 To Be Enabled	—

🎉 All policy checks passed!

📖 Need help? See the Policy FAQ for details on every check and how to fix failures.

🙋 Wish to Override Policy?

aktemur · 2026-09-21T12:54:29Z

The force-push above is a rebase.

aktemur · 2026-09-22T10:04:45Z

The force-push above updated the branch for the next review round.

aktemur · 2026-09-23T16:50:47Z

The force-push above is a rebase. As part of that, the PR now bumps the version to 0.82.

lancesix

With latest changes, things look good to me, thanks.

Be sure to retest on the final target before merging.

In two places, we get a workgroup's ids by calling `group_ids ()`, which returns an optional tuple, and if the optional is non-empty, we call `group_ids ()` again. Avoid the duplicated call.

A workgroup is alive as long as it has a wave. Set the epoch mark unconditionally, whenever the mark of any wave of the workgroup is being set. In practice, this should not change the epoch value of a workgroup. This change is a refactoring to obtain clearer and simpler code.

Starting with gfx12-5, workgroups may belong to clusters. Add methods to read cluster ids from a CWSR record. The cluster and group ids are read from TTMP registers. The meaning of the registers depend on whether the wave is in cluster mode. If not in a cluster, group ids are obtained from TTMP7 and TTMP9 as in previous gfx12 architecture. If in a cluster, what we read from TTMP7 and TTMP9 are cluster ids. The group ids in this case are read from TTMP6. To figure out if the wave is in cluster mode, we keep track of the "first_of_cluster" and "last_of_cluster" bits in the compute relaunch wave register. All CWSR records created for waves between the first and the last of cluster (inclusive) are marked to be in a cluster. If a wave/workgroup is not in cluster mode, we nevertheless conceptually consider the workgroup to belong to a single-workgroup cluster and return cluster ids and sizes accordingly. Bug: AIROCGDB-77

Add a first definition for clusters, which are groups of workgroups. Bug: AIROCGDB-77

Starting with gfx12-5, workgroups may be grouped into clusters. When creating workgroups, take this into account. Note that a dispatch is not necessarily set in cluster mode. We nevertheless create a single-workgroup cluster even in that case, so that conceptually a workgroup is always in a cluster. This patch adds clusters into the wave -> workgroup -> cluster -> dispatch hierarchy. API queries are added in the next patch. Bug: AIROCGDB-77

Define new enums in the API to query the cluster coordinate of a wave and a workgroup, and other cluster-related information. Bug: AIROCGDB-77

Add API queries to obtain cluster support, mode, and sizes. Clusters are supported at the architecture level. Clusters are enabled/disabled and their size information is available at the dispatch level. Regarding cluster sizes, to be in consistence with the existing queries that return workgroup sizes and grid sizes (i.e. AMD_DBGAPI_DISPATCH_INFO_WORKGROUP_SIZES and AMD_DBGAPI_DISPATCH_INFO_GRID_SIZES), the new query (i.e. AMD_DBGAPI_DISPATCH_INFO_CLUSTER_SIZES) returns the sizes in units of work-items. Bug: AIROCGDB-77

…uster Add a workgroup query and a wave query to get the workgroup coordinates within the cluster the workgroup belongs to.

aktemur · 2026-09-29T12:39:17Z

In nwg_in_cluster, the values obtained from ttmp6 are "minus 1". Hence, I made the following change in the last update:

-  nwg[0] = utils::bit_extract (ttmp6, 12, 15);
-  nwg[1] = utils::bit_extract (ttmp6, 16, 19);
-  nwg[2] = utils::bit_extract (ttmp6, 20, 23);
+  nwg[0] = utils::bit_extract (ttmp6, 12, 15) + 1;
+  nwg[1] = utils::bit_extract (ttmp6, 16, 19) + 1;
+  nwg[2] = utils::bit_extract (ttmp6, 20, 23) + 1;

aktemur · 2026-09-29T12:41:37Z

I noticed I forgot the update the version in CMakeLists.txt. The last update makes this change:

+ project(amd-dbgapi VERSION 0.81.0 LANGUAGES CXX DESCRIPTION "ROCm Debugger API")
- project(amd-dbgapi VERSION 0.82.0 LANGUAGES CXX DESCRIPTION "ROCm Debugger API")

aktemur · 2026-09-29T12:42:18Z

Tested on gfx1250. I'll merge when the CI run finishes.

systems-assistant · 2026-10-05T04:19:40Z

TheRock Submodule Bump Activity

Newest first

2026-10-05 — Included in TheRock via ROCm/TheRock#8732.

aktemur requested a review from a team as a code owner August 17, 2026 17:08

github-actions Bot added the project: rocdbgapi label Aug 17, 2026

aktemur changed the title ~~feat(rocdbgapi): introduce clusters of workgroups~~ feat(rocdbgapi): introduce clusters of workgroups (alternative) Aug 17, 2026

systems-assistant Bot added the organization: ROCm label Aug 17, 2026

aktemur mentioned this pull request Aug 18, 2026

handle clusters (alternative) ROCm/ROCgdb#287

Merged

lumachad force-pushed the users/aktemur/cluster-111 branch from 978e21a to cb81692 Compare August 20, 2026 11:47

lumachad mentioned this pull request Aug 20, 2026

[Issue]: Multi-Arch CI rebuilds unnecessary stages for rocdbgapi/rocr-debug-agent ROCm/TheRock#7520

Open

aktemur mentioned this pull request Sep 14, 2026

feat(rocdbgapi): introduce clusters of workgroups #7992

Closed

lancesix reviewed Sep 16, 2026

View reviewed changes

Comment thread projects/rocdbgapi/src/architecture.cpp Resolved

Comment thread projects/rocdbgapi/src/architecture.h Resolved

Comment thread projects/rocdbgapi/include/amd-dbgapi.h.in Resolved

Comment thread projects/rocdbgapi/src/workgroup.cpp Resolved

lancesix assigned aktemur Sep 21, 2026

aktemur force-pushed the users/aktemur/cluster-111 branch from ff2a049 to 7c8fbf5 Compare September 21, 2026 12:54

aktemur force-pushed the users/aktemur/cluster-111 branch from 7c8fbf5 to 0bd0824 Compare September 22, 2026 10:02

aktemur assigned lancesix and unassigned aktemur Sep 22, 2026

lancesix reviewed Sep 23, 2026

View reviewed changes

aktemur force-pushed the users/aktemur/cluster-111 branch from 0bd0824 to d4b4c31 Compare September 23, 2026 16:49

aktemur force-pushed the users/aktemur/cluster-111 branch from d4b4c31 to 705a012 Compare September 23, 2026 17:08

lancesix approved these changes Sep 23, 2026

View reviewed changes

aktemur added 3 commits September 24, 2026 17:10

doc(rocdbgapi): fix a documentation typo

a247ee0

refactor(rocdbgapi): avoid calling workgroup::group_ids () twice

f8e20a7

In two places, we get a workgroup's ids by calling `group_ids ()`, which returns an optional tuple, and if the optional is non-empty, we call `group_ids ()` again. Avoid the duplicated call.

aktemur force-pushed the users/aktemur/cluster-111 branch from 705a012 to 939673f Compare September 24, 2026 15:11

aktemur added 3 commits September 24, 2026 18:27

feature(rocdbgapi): add cluster.h and cluster.cpp

436efd8

Add a first definition for clusters, which are groups of workgroups. Bug: AIROCGDB-77

aktemur force-pushed the users/aktemur/cluster-111 branch from 939673f to 2e780e1 Compare September 24, 2026 16:28

aktemur added 3 commits September 29, 2026 14:35

feature(rocdbgapi): add API definitions to query cluster information

ff5c03c

Define new enums in the API to query the cluster coordinate of a wave and a workgroup, and other cluster-related information. Bug: AIROCGDB-77

feature(rocdbgapi): add API definition to query workgroup coord in cl…

b8b2cd8

…uster Add a workgroup query and a wave query to get the workgroup coordinates within the cluster the workgroup belongs to.

aktemur force-pushed the users/aktemur/cluster-111 branch from 2e780e1 to b8b2cd8 Compare September 29, 2026 12:40

aktemur merged commit 6b97c66 into develop Sep 29, 2026
82 of 83 checks passed

aktemur deleted the users/aktemur/cluster-111 branch September 29, 2026 15:20

Conversation

aktemur commented Aug 17, 2026

Uh oh!

therock-pr-bot Bot commented Aug 17, 2026 • edited Loading Uh oh! There was an error while loading. Please reload this page.

Uh oh!

✅ All Policy Checks Passed

Uh oh!

Uh oh!

Uh oh!

Uh oh!

Uh oh!

aktemur commented Sep 21, 2026

Uh oh!

aktemur commented Sep 22, 2026

Uh oh!

Uh oh!

Uh oh!

Uh oh!

Uh oh!

Uh oh!

aktemur commented Sep 23, 2026

Uh oh!

lancesix left a comment

Choose a reason for hiding this comment

Uh oh!

aktemur commented Sep 29, 2026

Uh oh!

aktemur commented Sep 29, 2026

Uh oh!

aktemur commented Sep 29, 2026

Uh oh!

Uh oh!

systems-assistant Bot commented Oct 5, 2026

TheRock Submodule Bump Activity

Uh oh!

Reviewers

Uh oh!

Assignees

Labels

Projects

Milestone

Development

Uh oh!

2 participants

therock-pr-bot Bot commented Aug 17, 2026 •

edited

Loading