Skip to content

fix(rocprofiler-sdk): report LDS_Block_Size from the dispatch, not the code-object symbol - #12311

Closed
tianxiaojiang4 wants to merge 1 commit into
ROCm:developfrom
tianxiaojiang4:fix/lds-block-size-dispatch
Closed

tianxiaojiang4 wants to merge 1 commit into
ROCm:developfrom
tianxiaojiang4:fix/lds-block-size-dispatch

Conversation

@tianxiaojiang4

@tianxiaojiang4 tianxiaojiang4 commented Sep 25, 2026 •

Copy link
Copy Markdown

Closes #12312

Summary

LDS_Block_Size in the CSV outputs is sourced from the code-object symbol, which carries only
the static group segment. Kernels that request group memory dynamically at dispatch declare 0
there, so the column reads 0 for them no matter how much LDS they actually use — that is every
Triton and Gluon kernel, and any HIP kernel using extern __shared__.

projects/rocprofiler-sdk/source/lib/output/generateCSV.cpp, lines 300 and 635:

const auto* kernel_info = tool_metadata.get_kernel_symbol(record.dispatch_info.kernel_id);
auto lds_block_size_v =
    (kernel_info->group_segment_size + (lds_block_size - 1)) & ~(lds_block_size - 1);

rocprofiler_kernel_dispatch_info_t::group_segment_size is documented in fwd.h as

"Size of group segment memory (static + runtime) required by the kernel (per work-group), in
bytes. AKA: LDS size"

and it is already on the record both call sites hold. This switches to it.

Evidence

gfx950, ROCm 7.2.4, one Triton kernel from aiter's sparse-MLA decode, plus a control in the same
trace:

source _sparse_mla (Triton, dynamic) at::native::reduce_kernel (static __shared__)
Triton metadata shared 68608 n/a
.amdhsa_group_segment_fixed_size 0 512
rocpd dispatch table group_segment_size 68608 —
rocpd symbol table group_segment_size 0 —
CSV LDS_Block_Size 0 ❌ 512 ✅

The control matters: the column is not always zero, it is specifically blind to dynamic allocation.
Scratch_Size agreed between both tables (44), which is consistent with it already reading the
dispatch side.

Why this looks like an oversight rather than intent

  • The adjacent Scratch_Size column already reads record.dispatch_info.private_segment_size.
    The register columns come from the symbol because registers cannot be dynamic — so of the two
    fields that can have a runtime component, LDS is the only one not taking it.
  • generateRocpd.cpp already writes the dispatch table's group_segment_size from the dispatch
    info, so rocpd output is correct today while the CSV is not — which is how the table above was
    produced.

Impact

Occupancy analysis from a kernel trace is silently wrong for dynamically-allocating kernels. In our
case a kernel using 67 KiB/workgroup reported 0, which reads as unbounded occupancy; the true figure
allows 2 workgroups per CU on a 160 KiB/CU part. That sent a kernel investigation down the wrong
path for several days.

Testing

I have not been able to run ctest — building the SDK needs a full ROCm toolchain I do not have
set up, and I did not want to gate a two-token change on it. The correctness argument is instead that
generateRocpd.cpp already reads exactly the field this switches to, and its output carries the
right value (68608) for the same dispatch, as shown above. No test references LDS_Block_Size.

The policy bot also flags this as a source change without an accompanying unit test. I have not
added one because a meaningful test needs a kernel that allocates LDS dynamically plus CSV
validation, which I cannot run end to end here — I would rather say so than add a test I have not
executed. Happy to write one to your preferred shape, adjust the formatting, or split the two call
sites.

🤖 Generated with Claude Code

@therock-pr-bot

therock-pr-bot Bot commented Sep 25, 2026 •

Copy link
Copy Markdown

✅ All Policy Checks Passed

Check Status Details
📝 PR Description ✅ Pass —
⛔ Forbidden Files ✅ Pass —
🧪 Unit Test ⚠️ Warning Error: Source/code files changed without an accompanying unit test.
Expected: add at least one test file named like test_<name>.py / test_<name>.cpp (or <name>_test.*).
Current: code file(s) changed: projects/rocprofiler-sdk/source/lib/output/generateCSV.cpp; no test file found
🚫 Draft PR 🔜 To Be Enabled —
🚩 Feature Flag 🔜 To Be Enabled —
📊 Code Coverage 🔜 To Be Enabled —

🎉 All policy checks passed!

📖 Need help? See the Policy FAQ for details on every check and how to fix failures.

🙋 Wish to Override Policy?

@therock-pr-bot

Copy link
Copy Markdown

🚫 Please fix the failed policies before requesting reviews.

The following policy checks failed:

  • ❌ PR Description

The Not ready to Review label has been added to this PR.
Once all policies pass, the label will be removed automatically.

generate_csv fills LDS_Block_Size from get_kernel_symbol(...)->group_segment_size, which is the code object's group_segment_fixed_size -- the STATIC allocation. Kernels that request group memory dynamically at dispatch declare 0 there, so the column reads 0 for them however much LDS they use. That is every Triton and Gluon kernel, and any HIP kernel using extern __shared__.

rocprofiler_kernel_dispatch_info_t::group_segment_size is documented in fwd.h as "Size of group segment memory (static + runtime) required by the kernel (per work-group), in bytes. AKA: LDS size", and it is already on the record both call sites hold.

Measured on gfx950 / ROCm 7.2.4 with one Triton kernel (aiter sparse MLA decode): Triton metadata shared=68608, .amdhsa_group_segment_fixed_size=0, rocpd dispatch table group_segment_size=68608, rocpd symbol table=0, CSV LDS_Block_Size=0. A control in the same trace, at::native::reduce_kernel with static __shared__, reports 512 correctly, so the column is not always zero -- it is specifically blind to dynamic allocation.

Closes ROCm#12312

Co-Authored-By: Claude Opus 5 <noreply@anthropic.com>
@tianxiaojiang4
tianxiaojiang4 force-pushed the fix/lds-block-size-dispatch branch from 8997981 to aa02030 Compare September 28, 2026 18:20
@tianxiaojiang4

Copy link
Copy Markdown
Author

@ROCm/rocm-devtools-team — mentioning you directly because CODEOWNERS routing does not fire for
external contributions: this PR has zero requested reviewers, where recent internal PRs (#12279,
#12245, #12233) each got 1–2 assigned automatically. CONTRIBUTING §3 lists
/projects/rocprofiler-sdk/ as yours for first review, so this is me doing by hand what the
automation would have done.

Status: all policy checks pass, rebased onto current develop (behind_by=0), +4/-2 in one
file, issue linked as #12312. The only open item is the unit-test warning, which I explain in the
description — a meaningful test needs a dynamically-allocating kernel plus CSV validation run end to
end, which I cannot do here, and I would rather say so than add a test I have not executed.

The change in one line: LDS_Block_Size is read from the code-object symbol, which carries only the
static group segment, so it reports 0 for every Triton and Gluon kernel and any HIP kernel using
extern __shared__. The dispatch record has the correct value and is already on the same object —
generateRocpd.cpp uses it today, which is how the evidence table in the description was produced.

No rush from my side; flagging it mainly so it does not sit unnoticed.

@github-actions

Copy link
Copy Markdown
Contributor

This pull request has been inactive for 25 days and will be marked as stale.

If you would like to keep this PR open, please:

  • Add new commits
  • Add a comment explaining why it should remain open

This PR will be automatically closed in 5 days if no further activity occurs.

@github-actions github-actions Bot added the Stale PR has no activity for 25+ days label Oct 24, 2026
@github-actions

Copy link
Copy Markdown
Contributor

This pull request has been automatically closed due to inactivity (30 days with no updates).

If you'd like to continue working on this, feel free to reopen the PR or create a new one.

@github-actions github-actions Bot closed this Oct 30, 2026
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Projects

None yet

Development

Successfully merging this pull request may close these issues.

rocprofiler-sdk: LDS_Block_Size reports static group segment only, so it is 0 for all Triton/Gluon kernels

1 participant