If I'd like to support AMD, how to implement it myself easily?

[Leo-yang-1020]

目前我查看了Hami所支持的GPU厂商，没有看到对AMD的支持，在之前的ISSUE里我也看到了确实没有对AMD的支持。
我有两个问题：

1. 是否有计划原生支持AMD
2. 如何自己快速完成对新型号的AMD支持

例如，对于AMD，在不考虑虚拟化，隔离等条件下，能否快速实现调度的支持？例如哪些组件，配置可能需要更改，包括但不限于：

• device-plugin
• scheduelr-extender

在文档中我没有看到这部分的支持，感谢您的回复！

[wawa0210]

So excited that someone mentioned HAMi's support for AMD

Currently, the HAMi maintainer team is indeed seeking in-depth support for AMD, but there are some obstacles

1. Currently, we have not found AMD's related environment and manufacturer support (virtualization, isolation, etc.)
2. There are no corresponding developers

If there is an organization in the community willing to provide AMD experimental environment and original manufacturer support, we are very willing to promote this matter

[lzxzy]

So excited that someone mentioned HAMi's support for AMD

Currently, the HAMi maintainer team is indeed seeking in-depth support for AMD, but there are some obstacles

1. Currently, we have not found AMD's related environment and manufacturer support (virtualization, isolation, etc.)
2. There are no corresponding developers

If there is an organization in the community willing to provide AMD experimental environment and original manufacturer support, we are very willing to promote this matter

Hi @wawa0210 , glad to hear the team is planning to support AMD. We have both the requirement and the AMD environment available. If possible, I’d like to contribute to developing HAMi support for AMD. Could you provide us with some rough guidance or direction on how we could do to make HAMi support AMD, for example with MI250/300?

[wawa0210]

So excited that someone mentioned HAMi's support for AMD
Currently, the HAMi maintainer team is indeed seeking in-depth support for AMD, but there are some obstacles

1. Currently, we have not found AMD's related environment and manufacturer support (virtualization, isolation, etc.)
2. There are no corresponding developers

If there is an organization in the community willing to provide AMD experimental environment and original manufacturer support, we are very willing to promote this matter

Hi @wawa0210 , glad to hear the team is planning to support AMD. We have both the requirement and the AMD environment available. If possible, I’d like to contribute to developing HAMi support for AMD. Could you provide us with some rough guidance or direction on how we could do to make HAMi support AMD, for example with MI250/300?

Do you have contact information? You can try to follow HAMi WeChat public account and add assistant WeChat to contact us. We can discuss in detail.

[meng-chiang]

We are currently working on enhancing AMD GPU support within hami-core, specifically by implementing a robust GPU utilization watcher and limiter, similar to the
functionality that already exists for NVIDIA GPUs. Our goal is to accurately monitor and control the GPU compute resource usage of applications running in unprivileged
Kubernetes containers.

However, we've encountered a fundamental challenge that puts us between two unsatisfactory options, and we would greatly appreciate any advice or insights from the
community.

Challenge 1: The "Observer" Model using rocm-smi Fails Due to Permissions

Our first approach was to build an "Observer" model, similar to how the NVIDIA watcher uses NVML. The idea is to directly query the actual GPU utilization.

• Method: Periodically call rocm-smi (or use its underlying rsmi/amdsmi libraries) from within the container to get the real hardware utilization percentage.
• Problem: We have verified that rocm-smi and its libraries fail to initialize or consistently return 0% utilization when executed inside a standard, unprivileged
  container.
• Root Cause: This is because rocm-smi requires access to host-level resources (e.g., the /sys filesystem), which are correctly and necessarily restricted by the
  container security sandbox.
• Conclusion: The only way to make rocm-smi work is to run the container in privileged mode. For us, this is not a viable option, as running end-user AI/ML workloads
  with root-level host access presents an unacceptable security risk to the cluster.

Challenge 2: The "Predictor" Model using HSA Interception is Inaccurate

Since we cannot directly observe the utilization, our second approach was to build a "Predictor" model by intercepting low-level API calls.

• Method: We intercept HSA runtime API calls at the kernel dispatch level (e.g., hsa_signal_store_release). By inspecting the AQL packet, we can see the size of the
  kernel being launched (e.g., grid and workgroup dimensions).
• Problem: This method only tells us the description of the work being submitted, not the actual hardware utilization that results from it. We are forced to predict the
  GPU usage based on these parameters.
• Conclusion: This prediction has proven to be highly inaccurate. Simple models (like kernel counting) and even more advanced ones (like weighting by workgroup size) do
  not correlate well with real-world CU occupancy. This leads to either severe GPU underutilization (because the limiter is too conservative) or a complete failure to
  enforce the limits. The performance is far from the expected linear scaling.

Has anyone in the community successfully tackled this challenge? We are looking for suggestions on:

• Are there other APIs or techniques we might be missing? (e.g., Is there a way to use ROCprofiler for this without significant performance overhead, noting it's still
  in beta?)
• Are there specific driver/kernel configurations that can safely expose these metrics to an unprivileged container?
• Is there a better way to build a "Predictor" model that is more accurate than what we've tried?

Any guidance, ideas, or shared experiences would be immensely helpful for us to build a first-class AMD GPU sharing solution.

Thank you for your time and consideration.

[lzxzy]

We are currently working on enhancing AMD GPU support within hami-core, specifically by implementing a robust GPU utilization watcher and limiter, similar to the functionality that already exists for NVIDIA GPUs. Our goal is to accurately monitor and control the GPU compute resource usage of applications running in unprivileged Kubernetes containers.

However, we've encountered a fundamental challenge that puts us between two unsatisfactory options, and we would greatly appreciate any advice or insights from the community.

Challenge 1: The "Observer" Model using rocm-smi Fails Due to Permissions

Our first approach was to build an "Observer" model, similar to how the NVIDIA watcher uses NVML. The idea is to directly query the actual GPU utilization.

• Method: Periodically call rocm-smi (or use its underlying rsmi/amdsmi libraries) from within the container to get the real hardware utilization percentage.
• Problem: We have verified that rocm-smi and its libraries fail to initialize or consistently return 0% utilization when executed inside a standard, unprivileged
  container.
• Root Cause: This is because rocm-smi requires access to host-level resources (e.g., the /sys filesystem), which are correctly and necessarily restricted by the
  container security sandbox.
• Conclusion: The only way to make rocm-smi work is to run the container in privileged mode. For us, this is not a viable option, as running end-user AI/ML workloads
  with root-level host access presents an unacceptable security risk to the cluster.

Challenge 2: The "Predictor" Model using HSA Interception is Inaccurate

Since we cannot directly observe the utilization, our second approach was to build a "Predictor" model by intercepting low-level API calls.

• Method: We intercept HSA runtime API calls at the kernel dispatch level (e.g., hsa_signal_store_release). By inspecting the AQL packet, we can see the size of the
  kernel being launched (e.g., grid and workgroup dimensions).
• Problem: This method only tells us the description of the work being submitted, not the actual hardware utilization that results from it. We are forced to predict the
  GPU usage based on these parameters.
• Conclusion: This prediction has proven to be highly inaccurate. Simple models (like kernel counting) and even more advanced ones (like weighting by workgroup size) do
  not correlate well with real-world CU occupancy. This leads to either severe GPU underutilization (because the limiter is too conservative) or a complete failure to
  enforce the limits. The performance is far from the expected linear scaling.

Has anyone in the community successfully tackled this challenge? We are looking for suggestions on:

• Are there other APIs or techniques we might be missing? (e.g., Is there a way to use ROCprofiler for this without significant performance overhead, noting it's still
  in beta?)
• Are there specific driver/kernel configurations that can safely expose these metrics to an unprivileged container?
• Is there a better way to build a "Predictor" model that is more accurate than what we've tried?

Any guidance, ideas, or shared experiences would be immensely helpful for us to build a first-class AMD GPU sharing solution.

Thank you for your time and consideration.

For Challenge 1, is it possible to use amd-smi instead of rocm-smi? It seems to produce similar output to nvidia-smi, as described in AMD’s container runtime quick start guide

[meng-chiang]

As far as I know, amdsmi also needs to be in privileged mode and needs to mount /dev/kfd and other paths ? And it suddenly occurred to me that does amdsmi have the gpu usage of individual processes? According to your screenshot, it seems that only the global gpu usage is shown.

update:
ROCm/amdsmi#130

[lzxzy]

As far as I know, amdsmi also needs to be in privileged mode and needs to mount /dev/kfd and other paths ? And it suddenly occurred to me that does amdsmi have the gpu usage of individual processes? According to your screenshot, it seems that only the global gpu usage is shown.

update: ROCm/amdsmi#130

This is what I got when using amd-smi on the host:

[mesutoezdil]

Hi,

We are doing a cleanup of open issues to keep the backlog manageable. This issue was opened before 2026 and has not seen recent activity.

Please reply by 2026-07-13 to confirm this is still relevant and reproducible on the current version of HAMi. If we do not hear back, we will close this issue. You can always reopen it or file a new one if the problem persists.

Thank you for contributing to HAMi.