Skip to content
Merged
Show file tree
Hide file tree
Changes from all commits
Commits
File filter

Filter by extension

Filter by extension

Conversations
Failed to load comments.
Loading
Jump to
Jump to file
Failed to load files.
Loading
Diff view
Diff view
14 changes: 7 additions & 7 deletions docs/faq/faq.md
Original file line number Diff line number Diff line change
Expand Up @@ -178,7 +178,7 @@ If the official Device Plugin cannot provide the required information, HAMi deve

## How does HAMi enforce GPU memory and compute limits?

HAMi injects `libvgpu.so` into containers via `/etc/ld.so.preload`. The library intercepts CUDA memory allocation calls and returns OOM when the `nvidia.com/gpumem` limit is exceeded; compute limits use a token-bucket throttle on kernel launch calls. Applications that bypass the CUDA library (Docker-in-Docker, direct driver API) are not covered. For the full interception flow, see [GPU Virtualization](./core-concepts/gpu-virtualization).
HAMi injects `libvgpu.so` into containers via `/etc/ld.so.preload`. The library intercepts CUDA memory allocation calls and returns OOM when the `nvidia.com/gpumem` limit is exceeded; compute limits use a token-bucket throttle on kernel launch calls. Applications that bypass the CUDA library (Docker-in-Docker, direct driver API) are not covered. For the full interception flow, see [GPU Virtualization](../core-concepts/gpu-virtualization).

## How does HAMi vGPU differ from NVIDIA MIG? When should I use each?

Expand All @@ -194,19 +194,19 @@ HAMi vGPU is software-only with no hardware requirements. NVIDIA MIG is hardware
| Dynamic reconfiguration | Yes, no node drain needed | Requires MIG profile reconfiguration |
| Multi-tenant noise isolation | Best-effort | Strong |

Use HAMi vGPU when the GPU does not support MIG, workloads need flexible memory sizes, or dynamic repacking without node drains is needed. Use MIG when hard hardware isolation is a compliance or SLA requirement. HAMi also supports dynamic MIG via `mig-parted`; see [Dynamic MIG Support](./userguide/nvidia-device/dynamic-mig-support).
Use HAMi vGPU when the GPU does not support MIG, workloads need flexible memory sizes, or dynamic repacking without node drains is needed. Use MIG when hard hardware isolation is a compliance or SLA requirement. HAMi also supports dynamic MIG via `mig-parted`; see [Dynamic MIG Support](../userguide/nvidia-device/dynamic-mig-support).

## Why does nvidia-smi inside my container show less memory than on the host?

`libvgpu.so` intercepts `nvmlDeviceGetMemoryInfo` and related calls, returning the `nvidia.com/gpumem` limit instead of physical VRAM. This is intentional: workloads that size their allocations based on reported memory (such as vLLM) will use only their budget. The host’s `nvidia-smi` always shows physical memory. See [GPU Virtualization](./core-concepts/gpu-virtualization).
`libvgpu.so` intercepts `nvmlDeviceGetMemoryInfo` and related calls, returning the `nvidia.com/gpumem` limit instead of physical VRAM. This is intentional: workloads that size their allocations based on reported memory (such as vLLM) will use only their budget. The host’s `nvidia-smi` always shows physical memory. See [GPU Virtualization](../core-concepts/gpu-virtualization).

## Why is my nvidia.com/gpumem limit not enforced? {#why-is-my-nvidiagpumem-limit-not-enforced}

The four most common causes: `CUDA_DISABLE_CONTROL=true` is set, the workload runs inside Docker-in-Docker, the application calls the GPU driver directly (bypassing `libvgpu.so`), or `nvidia-container-runtime` is not the default runtime on the node. See [Troubleshooting](./troubleshooting) for resolution steps.
The four most common causes: `CUDA_DISABLE_CONTROL=true` is set, the workload runs inside Docker-in-Docker, the application calls the GPU driver directly (bypassing `libvgpu.so`), or `nvidia-container-runtime` is not the default runtime on the node. See [Troubleshooting](../troubleshooting/troubleshooting.md) for resolution steps.

## Does HAMi replace kube-scheduler or run alongside it?

HAMi runs alongside kube-scheduler as a [scheduler extender](https://github.com/kubernetes/design-proposals-archive/blob/main/scheduling/scheduler_extender.md) - it does not replace it. The MutatingWebhook sets `schedulerName: hami-scheduler` only on pods requesting HAMi resources; all other pods follow the default scheduler path unchanged. See [Architecture](./core-concepts/architecture).
HAMi runs alongside kube-scheduler as a [scheduler extender](https://github.com/kubernetes/design-proposals-archive/blob/main/scheduling/scheduler_extender.md) - it does not replace it. The MutatingWebhook sets `schedulerName: hami-scheduler` only on pods requesting HAMi resources; all other pods follow the default scheduler path unchanged. See [Architecture](../core-concepts/architecture).

## Does HAMi work with vLLM, and what are the known limitations for multi-GPU tensor parallelism?

Expand All @@ -221,8 +221,8 @@ devicePlugin:
enabled: false
```

DCGM Exporter is not affected and continues to report physical-level counters normally. HAMi’s per-container virtual metrics are separate; see [GPU Utilization Metrics](./developers/gpu-utilization-metrics).
DCGM Exporter is not affected and continues to report physical-level counters normally. HAMi’s per-container virtual metrics are separate; see [GPU Utilization Metrics](../developers/gpu-utilization-metrics).

## How do I set up Prometheus and Grafana monitoring for HAMi vGPU metrics?

The `hami-device-plugin` pod on each node exposes per-container vGPU metrics on port `31992` (configurable via `devicePlugin.monitorPort`). See [Grafana Dashboard](./userguide/monitoring/grafana-dashboard) for the full setup including Prometheus scrape config and dashboard import.
The `hami-device-plugin` pod on each node exposes per-container vGPU metrics on port `31992` (configurable via `devicePlugin.monitorPort`). See [Grafana Dashboard](../userguide/monitoring/grafana-dashboard) for the full setup including Prometheus scrape config and dashboard import.
14 changes: 7 additions & 7 deletions i18n/zh/docusaurus-plugin-content-docs/current/faq/faq.md
Original file line number Diff line number Diff line change
Expand Up @@ -175,7 +175,7 @@ HAMi 的调度器需要从节点获取足够的 GPU 信息来完成资源调度

## HAMi 如何强制执行 GPU 显存和算力限制?

HAMi 通过 `/etc/ld.so.preload` 将 `libvgpu.so` 注入容器。该库拦截 CUDA 显存分配调用,当超过 `nvidia.com/gpumem` 限制时返回 OOM;算力限制通过令牌桶算法对 kernel launch 调用进行节流。绕过 CUDA 库的应用(如 Docker-in-Docker、直接调用驱动 API)不受管控。完整的拦截流程参见 [GPU 虚拟化](./core-concepts/gpu-virtualization)。
HAMi 通过 `/etc/ld.so.preload` 将 `libvgpu.so` 注入容器。该库拦截 CUDA 显存分配调用,当超过 `nvidia.com/gpumem` 限制时返回 OOM;算力限制通过令牌桶算法对 kernel launch 调用进行节流。绕过 CUDA 库的应用(如 Docker-in-Docker、直接调用驱动 API)不受管控。完整的拦截流程参见 [GPU 虚拟化](../core-concepts/gpu-virtualization)。

## HAMi vGPU 与 NVIDIA MIG 有何区别?各适用于什么场景?

Expand All @@ -191,19 +191,19 @@ HAMi vGPU 是纯软件方案,无硬件要求。NVIDIA MIG 是硬件分区,
| 动态重配置 | 支持,无需排空节点 | 需要重新配置 MIG 配置文件 |
| 多租户噪声隔离 | 尽力而为 | 强隔离 |

当 GPU 不支持 MIG、工作负载需要灵活的显存大小、或需要无需排空节点的动态重打包时,使用 HAMi vGPU。当硬隔离是合规或 SLA 要求时,使用 MIG。HAMi 也通过 `mig-parted` 支持动态 MIG;参见[动态 MIG 支持](./userguide/nvidia-device/dynamic-mig-support)。
当 GPU 不支持 MIG、工作负载需要灵活的显存大小、或需要无需排空节点的动态重打包时,使用 HAMi vGPU。当硬隔离是合规或 SLA 要求时,使用 MIG。HAMi 也通过 `mig-parted` 支持动态 MIG;参见[动态 MIG 支持](../userguide/nvidia-device/dynamic-mig-support)。

## 为什么容器内 nvidia-smi 显示的显存比宿主机少?

`libvgpu.so` 拦截了 `nvmlDeviceGetMemoryInfo` 及相关调用,返回 `nvidia.com/gpumem` 限制值而非物理显存。这是预期行为:根据上报显存大小进行分配的工作负载(如 vLLM)将只使用其配额。宿主机的 `nvidia-smi` 始终显示物理显存。参见 [GPU 虚拟化](./core-concepts/gpu-virtualization)。
`libvgpu.so` 拦截了 `nvmlDeviceGetMemoryInfo` 及相关调用,返回 `nvidia.com/gpumem` 限制值而非物理显存。这是预期行为:根据上报显存大小进行分配的工作负载(如 vLLM)将只使用其配额。宿主机的 `nvidia-smi` 始终显示物理显存。参见 [GPU 虚拟化](../core-concepts/gpu-virtualization)。

## 为什么 nvidia.com/gpumem 限制未生效? {#why-is-my-nvidiagpumem-limit-not-enforced}

最常见的四种原因:设置了 `CUDA_DISABLE_CONTROL=true`、工作负载运行在 Docker-in-Docker 中、应用直接调用 GPU 驱动(绕过了 `libvgpu.so`)、或节点上 `nvidia-container-runtime` 未设为默认运行时。解决步骤参见[排障手册](./troubleshooting)。
最常见的四种原因:设置了 `CUDA_DISABLE_CONTROL=true`、工作负载运行在 Docker-in-Docker 中、应用直接调用 GPU 驱动(绕过了 `libvgpu.so`)、或节点上 `nvidia-container-runtime` 未设为默认运行时。解决步骤参见[排障手册](../troubleshooting/troubleshooting.md)。

## HAMi 是替换 kube-scheduler 还是与其并行运行?

HAMi 作为 [scheduler extender](https://github.com/kubernetes/design-proposals-archive/blob/main/scheduling/scheduler_extender.md) 与 kube-scheduler 并行运行,不会替换它。MutatingWebhook 仅在请求 HAMi 资源的 Pod 上设置 `schedulerName: hami-scheduler`;所有其他 Pod 仍走默认调度器路径。参见[架构](./core-concepts/architecture)。
HAMi 作为 [scheduler extender](https://github.com/kubernetes/design-proposals-archive/blob/main/scheduling/scheduler_extender.md) 与 kube-scheduler 并行运行,不会替换它。MutatingWebhook 仅在请求 HAMi 资源的 Pod 上设置 `schedulerName: hami-scheduler`;所有其他 Pod 仍走默认调度器路径。参见[架构](../core-concepts/architecture)。

## HAMi 是否支持 vLLM?多 GPU 张量并行有哪些已知限制?

Expand All @@ -218,8 +218,8 @@ devicePlugin:
enabled: false
```

DCGM Exporter 不受影响,继续正常上报物理级计数器。HAMi 的每容器虚拟指标是独立的;参见 [GPU 利用率指标](./developers/gpu-utilization-metrics)。
DCGM Exporter 不受影响,继续正常上报物理级计数器。HAMi 的每容器虚拟指标是独立的;参见 [GPU 利用率指标](../developers/gpu-utilization-metrics)。

## 如何为 HAMi vGPU 指标设置 Prometheus 和 Grafana 监控?

每个节点上的 `hami-device-plugin` Pod 在端口 `31992`(可通过 `devicePlugin.monitorPort` 配置)上暴露每容器 vGPU 指标。完整的设置步骤(包括 Prometheus 采集配置和 Dashboard 导入)参见 [Grafana Dashboard](./userguide/monitoring/grafana-dashboard)。
每个节点上的 `hami-device-plugin` Pod 在端口 `31992`(可通过 `devicePlugin.monitorPort` 配置)上暴露每容器 vGPU 指标。完整的设置步骤(包括 Prometheus 采集配置和 Dashboard 导入)参见 [Grafana Dashboard](../userguide/monitoring/grafana-dashboard)。
Original file line number Diff line number Diff line change
Expand Up @@ -16,7 +16,7 @@ translated: true
containerd config dump | grep default_runtime_name
```

输出必须显示 `nvidia`。如未显示,请按照[前置条件](./installation/online-installation)指南操作。
输出必须显示 `nvidia`。如未显示,请按照[前置条件](../installation/online-installation)指南操作。

- 如果在使用 NVIDIA 镜像的设备插件时不请求 vGPU,机器上的所有 GPU 可能会在容器内暴露。
- 目前,A100 MIG 仅支持 "none" 和 "mixed" 模式。
Expand Down