[bimaniamd]

## Motivation

New feature implementation for the GL2 residency control.
## Technical Details

rocr API's
HSA_AMD_AGENT_INFO_LIMIT_PERSISTING_L2_CACHE_SIZE
HSA_AMD_AGENT_ATTRIBUTE_REQUEST_PERSISTING_L2_CACHE_SIZE.
And their related structures added

## Issue Tracking

JIRA ID: AILIROCR-405

## Test Plan

Added with this change

## Test Result

Awaiting results.

## Submission Checklist

- [x] Look over the contributing guidelines at https://github.com/ROCm/rocm-systems/blob/develop/CONTRIBUTING.md.

[therock-pr-bot]

✅ All Policy Checks Passed

Check	Status	Details
📝 PR Description	✅ Pass	—
⛔ Forbidden Files	✅ Pass	—
🧪 Unit Test	✅ Pass	—
🚫 Draft PR	🔜 To Be Enabled	—
🚩 Feature Flag	🔜 To Be Enabled	—
📊 Code Coverage	🔜 To Be Enabled	—

🎉 All policy checks passed!

📖 Need help? See the Policy FAQ for details on every check and how to fix failures.

🙋 Wish to Override Policy?

[cfreeamd]

🤖 AI-assisted pre-review (for @cfreeamd)

Reviewed the projects/rocr-runtime/ changes for the GL2 residency-control path (libhsakmt, KFD driver, GpuAgent, public hsa_ext_amd surface, rocrtst). The UAPI amdgpu_drm.h edits are consistently synced across the three trees. Left inline comments on several blocking items — see below.

[cfreeamd]

🤖 AI-assisted pre-review (for @cfreeamd) — follow-up after author update

The update adds the previously-missing gl2_cache_test.cc/.h, so the rocrtst target now compiles, and the new GL2CacheTest::SetUp() correctly calls TestBase::SetUp() (platform filtering preserved). Good.

However, all four prior [Important] correctness findings remain unaddressed at head:

1. libhsakmt/src/memory.c — hsaKmtSetPersistingCacheSizeCtx still fills a drm_amdgpu_vm struct but submits it via AMDKFD_IOC_AVAILABLE_MEMORY; still has stray printf, pr_debug, //TODO: BINCY marker, and return 0;.
2. amd_gpu_agent.cpp (HSA_AMD_AGENT_INFO_MAX_PERSISTING_L2_CACHE_SIZE) — braces were added to the if, but return HSA_STATUS_SUCCESS; is still inside the for loop, so only cache_props_[0] is examined and *value is left uninitialized when the first entry isn't the L2 persisting cache.
3. ABI width — hsa_ext_amd.h still documents size_t while GetInfo/SetAgentAttribute read/write via *((uint32_t*)value) (backing field still size_t).
4. rocrtst YAML registration — the six new GL2_PersistingCache_* tests still aren't in test_categories.yaml or platform_config.yaml, so CI won't schedule/filter them.

Debug/WIP markers also remain across the diff (//TODO: BINCY, Bincy TODO remove). Still request-changes.

[cfreeamd]

🤖 AI-assisted pre-review (for @cfreeamd) — follow-up after force-push (0d9df8d)

Thanks for the updates. Two of the prior [Important] issues are now resolved:

• ioctl fixed: hsaKmtSetPersistingCacheSizeCtx now resolves the amdgpu render-node fd and issues drmIoctl(DRM_IOCTL_AMDGPU_VM, ...) (with pr_debug + proper HSAKMT_STATUS returns) instead of the mismatched AMDKFD_IOC_AVAILABLE_MEMORY on a drm_amdgpu_vm struct.
• loop early-return fixed: the max-size query is now GetMaxPersistingL2CacheSize(), which iterates correctly and returns 0 when absent, and persisting_l2_cache_size_ is initialized to 0 in InitCacheList().

Still outstanding before merge:

• [Important] size_t vs uint32_t ABI mismatch — the public header still documents these attributes as size_t, but GetInfo/SetAgentAttribute read/write them as uint32_t. See inline comments.
• [Important] rocrtst config not updated — neither rocrtst/config/test_categories.yaml nor platform_config.yaml registers the six new GL2_PersistingCache_* tests, so CI won't bucket/filter them. (GL2CacheTest::SetUp does call TestBase::SetUp, so platform filtering itself is preserved — good.)
• [Nit] WIP markers — 4 //TODO:BINCY remove / Bincy TODO remove lines remain, two of them in the public hsa_ext_amd.h header on the newly exported ABI. Public exports are effectively permanent; please strip these before merge.

[cfreeamd]

🤖 AI-assisted pre-review (for @cfreeamd) — follow-up after author replies (2026-09-01)

Author response accepted: The argument for hsa_amd_agent_set_attribute using an extensible enum (mirroring the existing hsa_amd_agent_info_t pattern) is valid — new attributes can be added over time without new exported symbols. Dropping the prior [Optional] API-shape concern.

Two [Important] issues remain unaddressed:

1. ABI width mismatch (hsa_ext_amd.h + amd_gpu_agent.cpp): the public header documents these attributes as size_t and the enum comment repeats that, but GetInfo/SetAgentAttribute cast through (uint32_t*) and GetMaxPersistingL2CacheSize() returns uint32_t. A caller that allocates a size_t (8-byte) buffer per the documented contract gets only 4 bytes written on a query. Choose one width and align header docs, casts, and the backing member.

2. rocrtst YAML registration (test_categories.yaml + platform_config.yaml): the six new GL2_PersistingCache_* tests still have no entries in either file — CI won't bucket or filter them.

WIP markers (//TODO:BINCY remove, /** Bincy TODO remove */) still present in hsa_ext_amd.h on newly exported ABI.

Note: @marifamd has CHANGES_REQUESTED on the amdsmi/rocm-smi-lib portions — the rocm-smi-lib update was flagged as unnecessary (slated for deprecation), and the amdsmi copy needs confirmation that the kernel UAPI adds the corresponding field.

[asierrag-amd]

Here are some comments based on the test validation.

Already incorporated the changes requested

[cfreeamd]

🤖 AI-assisted pre-review (for @cfreeamd) — follow-up after the 2026-09-02 update

Thanks for the rework — all four of my prior [Important] items are resolved:

• KFD ioctl / arg-struct: now goes through the amdgpu render-node drmIoctl(DRM_IOCTL_AMDGPU_VM) path instead of AMDKFD_IOC_AVAILABLE_MEMORY on a drm_amdgpu_vm struct.
• Query loop early-return: refactored into GetMaxPersistingL2CacheSize() (loops correctly, returns 0 when absent; member initialized to 0).
• size_t vs uint32_t ABI width: both HSA_AMD_AGENT_INFO_*_PERSISTING_L2_CACHE_SIZE queries and the HSA_AMD_AGENT_ATTRIBUTE_REQUEST_PERSISTING_L2_CACHE_SIZE setter now use size_t, matching the header docs and the backing member.
• rocrtst YAML registration: the six GL2_PersistingCache_* tests are now in both test_categories.yaml and platform_config.yaml.

The //TODO:BINCY markers and the out-of-scope amdsmi/rocm-smi-lib amdgpu_drm.h copies are gone too.

I concur with @dayatsin-amd's CHANGES_REQUESTED and defer to those items — I'm not re-posting them. Flagging the ones that are correctness/build-blocking rather than style: bump HSA_AMD_INTERFACE_VERSION_MINOR for the new API; MIT SPDX headers on the new gl2_cache_test.{h,cc}; and, per @asierrag-amd, actually wire the hsa_amd_agent_set_attribute export so it resolves at runtime (the test hit set_attr_fn == nullptr / "not found in runtime"), plus the rocprofiler-sdk registration for the new symbol (ref #10895). No new blocking items from me beyond those.

[shwetagkhatri]

In runtime/hsa-runtime/inc/hsa_api_trace_version.h (HSA_AMD_EXT_API_TABLE_STEP_VERSION):

This still reads 0x13, but adding a function to AmdExtTable needs a bump to 0x14.I think you may already have a reason to not declare it?

This is separate from the HSA_AMD_INTERFACE_VERSION_MINOR bump you already did (that one is the public header API version. This one is the ext-table ABI version):

• Your rocprofiler-sdk changes are currently dead code. Everything added in amd_ext_api_id.h, api_args.h, abi.cpp and hsa.def.cpp is wrapped in #if HSA_AMD_EXT_API_TABLE_STEP_VERSION >= 0x14. Until this is bumped, none of it compiles in, so the new API can't actually be traced.
• ABI: Also the table grew by one pointer (expected_amd_ext_table_size went 752 to 760 in hsa_api_trace.cpp).

Leaving the version at 0x13 means two different table layouts share one version number, so layers above ROCr that are built against the old 0x13 can't tell the new slot exists. The version bump is important. Please set it to 0x14 to match the >= 0x14 guards you wrote.

[cfreeamd]

🤖 AI-assisted pre-review (for @cfreeamd) — follow-up after 75e872f (2026-09-10)

All my prior [Important] findings remain resolved. dayatsin-amd approved on 2026-09-03.

Looking at the new commit relative to shwetagkhatri's CHANGES_REQUESTED (2026-09-03):

• HSA_AMD_EXT_API_TABLE_STEP_VERSION bumped 0x13 → 0x14 in hsa_api_trace_version.h ✓
• hsa_api_trace.cpp self-recursion fixed: amd_ext_api.hsa_amd_agent_set_attribute_fn = AMD::hsa_amd_agent_set_attribute (correct namespace qualifier) ✓
• KfdVirtioDriver::SetPersistingCacheSize override added (returns HSA_STATUS_ERROR_INVALID_AGENT) ✓
• rocprofiler-sdk registration files updated (VERSION, amd_ext_api_id.h, api_args.h, abi.cpp, hsa.def.cpp, enum_string.hpp) ✓
• License headers on gl2_cache_test.h and gl2_cache_test.cc ✓

All three of shwetagkhatri's items appear addressed. Pending her re-review of the latest commit.

[amd-hsivasun]

Bypassed, CI failures are unrelated.

[systems-assistant]

TheRock Submodule Bump Activity

Newest first

• 2026-10-05 — Included in TheRock via ROCm/TheRock#8732.