Add GTT to APU memory reporting - #5596
Conversation
and hipGetDeviceProperties() incorrectly report only the VRAM aperture size (~15 GiB) instead of the total usable unified memory (~96 GiB). This breaks ML frameworks like vLLM and PyTorch that check available VRAM before loading models, causing them to reject models that would otherwise fit in the APU's large unified memory pool. Root cause: globalMemSize_ is set from gpuvm_segment_ (VRAM pool) only. On APUs, when the kernel's apu_prefer_gtt is enabled (GTT > VRAM), VRAM allocation requests are transparently redirected to GTT, making both pools usable. In this case, we should report the sum of both pools. When VRAM >= GTT (large carveout), apu_prefer_gtt is disabled and only the VRAM pool is reliably used, so we report VRAM size only. This change adds the CPU agent's coarse-grained pool (GTT) size to globalMemSize_ only when GTT > VRAM, matching the kernel's allocation behavior (commit 759e764f7d58). Fixes: ROCm/hip#3892 Related: torvalds/linux@759e764
There was a problem hiding this comment.
Pull request overview
This PR updates ROCclr’s ROCm device initialization to more accurately report total usable memory on APUs with unified memory by conditionally including the CPU agent’s coarse-grained (GTT/system memory) pool when it is larger than the VRAM pool, aligning with the kernel’s apu_prefer_gtt behavior.
Changes:
- Introduce an
isApuflag derived from ROCr memory properties and use it to markhostUnifiedMemory_. - When running on an APU and GTT > VRAM, add GTT pool size (scaled by
GPU_MAX_HEAP_SIZE) toinfo_.globalMemSize_. - Add an informational log describing the VRAM/GTT/total sizes when this adjustment is applied.
💡 Add Copilot custom instructions for smarter, more guided reviews. Learn how to get started.
| uint8_t memory_properties[8]; | ||
| // Get the memory property from ROCr. | ||
| if (HSA_STATUS_SUCCESS != | ||
| Hsa::agent_get_info(bkendDevice_, (hsa_agent_info_t)HSA_AMD_AGENT_INFO_MEMORY_PROPERTIES, | ||
| memory_properties)) { | ||
| LogError("HSA_AGENT_INFO_AMD_MEMORY_PROPERTIES query failed"); | ||
| } | ||
|
|
||
| // Check if the device is APU | ||
| if (hsa_flag_isset64(memory_properties, HSA_AMD_MEMORY_PROPERTY_AGENT_IS_APU)) { | ||
| // Check if the device is APU with unified memory | ||
| const bool isApu = hsa_flag_isset64(memory_properties, HSA_AMD_MEMORY_PROPERTY_AGENT_IS_APU); | ||
|
|
||
| if (isApu) { |
There was a problem hiding this comment.
memory_properties is left uninitialized if Hsa::agent_get_info(...MEMORY_PROPERTIES...) fails, but it is still consumed by hsa_flag_isset64(...) to compute isApu. That’s undefined behavior and can incorrectly enable APU/unified-memory paths (including the new GTT addition). Initialize memory_properties to zero (e.g., {}) and/or treat the query failure as isApu = false (or return false) before using it.
| static_cast<uint64_t>(gtt_segment_size)) / 100u; | ||
| info_.globalMemSize_ += gtt_usable; | ||
| LogPrintfInfo("APU unified memory: VRAM=%zu GTT=%zu total=%llu", | ||
| global_segment_size, gtt_segment_size, info_.globalMemSize_); |
There was a problem hiding this comment.
The LogPrintfInfo format string uses %llu for info_.globalMemSize_, which is a uint64_t. On some platforms uint64_t is unsigned long (not unsigned long long), so this can cause format warnings or incorrect output. Prefer casting to unsigned long long at the callsite, or switch to PRIu64 (and include the appropriate header) for portable formatting.
| global_segment_size, gtt_segment_size, info_.globalMemSize_); | |
| global_segment_size, gtt_segment_size, | |
| static_cast<unsigned long long>(info_.globalMemSize_)); |
|
Did you mean to close this? |
Yes. HIP might not be the best place to implement this. Currently in talks with the driver team. |
Fixes ROCm/hip#3892
Motivation
On APUs with unified memory (e.g., Strix Halo gfx1151), hipMemGetInfo()
and hipGetDeviceProperties() incorrectly report only the VRAM aperture
size (~15 GiB) instead of the total usable unified memory (~96 GiB).
This breaks ML frameworks like vLLM and PyTorch that check available
VRAM before loading models, causing them to reject models that would
otherwise fit in the APU's large unified memory pool.
Technical Details
Root cause: globalMemSize_ is set from gpuvm_segment_ (VRAM pool) only.
On APUs, when the kernel's apu_prefer_gtt is enabled (GTT > VRAM), VRAM
allocation requests are transparently redirected to GTT, making both
pools usable. In this case, we should report the sum of both pools.
When VRAM >= GTT (large carveout), apu_prefer_gtt is disabled and only
the VRAM pool is reliably used, so we report VRAM size only.
This change adds the CPU agent's coarse-grained pool (GTT) size to
globalMemSize_ only when GTT > VRAM, matching the kernel's allocation
behavior (commit 759e764f7d58).
Fixes: ROCm/hip#3892
Related: torvalds/linux@759e764
JIRA ID
N/A
Test Plan
N/A
Test Result
N/A
Submission Checklist