Skip to content
Merged
Show file tree
Hide file tree
Changes from all commits
Commits
File filter

Filter by extension

Filter by extension

Conversations
Failed to load comments.
Loading
Jump to
Jump to file
Failed to load files.
Loading
Diff view
Diff view
26 changes: 14 additions & 12 deletions blog/hami-core-adopted-by-nvidia-kai-scheduler/index.md
Original file line number Diff line number Diff line change
Expand Up @@ -9,13 +9,13 @@ tags:

> The integration target here is strictly HAMi-core, not the full HAMi platform. KAI Scheduler keeps its own scheduling capability and brings in HAMi-core to provide GPU memory isolation.

In June 2026, two core PRs were officially merged into the NVIDIA KAI Scheduler main branch. This means HAMi's GPU memory hard isolation will ship as a built-in feature in the next KAI Scheduler release. Cloud-native GPU scheduling has officially moved from "cooperative sharing" into the "hard isolation" era.
In June 2026, two core PRs were officially merged into the NVIDIA KAI Scheduler main branch. HAMi's GPU memory hard isolation shipped as a built-in feature starting with KAI Scheduler v0.16.4. Cloud-native GPU scheduling has officially moved from "cooperative sharing" into the "hard isolation" era.

<!-- truncate -->

## What is KAI Scheduler?

[KAI Scheduler](https://github.com/kai-scheduler/KAI-Scheduler) is NVIDIA's open source, Kubernetes-native scheduler for AI workloads. It grew out of the Run:ai scheduling engine. After NVIDIA acquired Run:ai in late 2024, it was open sourced under Apache 2.0 in April 2025 and is now a CNCF Sandbox project.
[KAI Scheduler](https://github.com/kai-scheduler/KAI-Scheduler) is NVIDIA's open-source, Kubernetes-native scheduler for AI workloads. It grew out of the Run:ai scheduling engine. After NVIDIA acquired Run:ai in late 2024, it was open sourced under Apache 2.0 in April 2025 and is now a CNCF Sandbox project.

The Kubernetes default scheduler was designed for stateless services and schedules GPUs the same way it schedules CPU cores: one Pod takes a whole GPU, with no gang scheduling, no team fairness, and no topology awareness. KAI Scheduler exists to solve the scheduling problems unique to AI scenarios:

Expand Down Expand Up @@ -48,7 +48,7 @@ A simple way to understand HAMi's position:
- **KAI Scheduler** decides "who uses which GPU, and when" (the scheduling layer).
- **HAMi** ensures "once allocated, that is all you get, and you cannot take more" (the isolation layer).

Only by combining the two do you get true production-grade GPU sharing. HAMi supports NVIDIA GPUs, Huawei Ascend NPUs, Cambricon MLUs, Hygon DCUs, Kunlun XPUs, and many other heterogeneous accelerators, making it the open source solution with the broadest coverage in cloud-native GPU virtualization. For the full list of supported accelerators, see the [HAMi documentation](/docs/userguide/device-supported).
Only by combining the two do you get true production-grade GPU sharing. HAMi supports NVIDIA GPUs, Huawei Ascend NPUs, Cambricon MLUs, Hygon DCUs, Kunlun XPUs, and many other heterogeneous accelerators, making it the open-source solution with the broadest coverage in cloud-native GPU virtualization. For the full list of supported accelerators, see the [HAMi documentation](/docs/userguide/device-supported).

## Integration architecture: how HAMi and KAI Scheduler work together

Expand Down Expand Up @@ -105,8 +105,9 @@ Integrating HAMi into KAI Scheduler is simple, only two steps.

```bash
helm install kai-scheduler oci://ghcr.io/nvidia/kai-scheduler \
--set scheduler.gpuSharing.enabled=true \
--set scheduler.gpuSharing.hamicoreEnabled=true \
--version v0.16.4 \
--set global.gpuSharing=true \
--set binder.plugins.hamicore.enabled=true \
--namespace kai-scheduler --create-namespace
```

Expand Down Expand Up @@ -160,9 +161,9 @@ This post covers the highlights. For the complete setup guide (prerequisites, in

:::

## Open source collaboration: from proposal to merge
## Open-source collaboration: from proposal to merge

This integration is a model of open source community collaboration. It took over a year of close work between the HAMi team and the NVIDIA KAI Scheduler team:
This integration is a model of open-source community collaboration. It took over a year of close work between the HAMi team and the NVIDIA KAI Scheduler team:

| Date | Milestone | Participants |
| --- | --- | --- |
Expand Down Expand Up @@ -209,18 +210,19 @@ For users who already use both KAI Scheduler and HAMi, this integration solves t

> "We're currently using KAI Scheduler to handle our ML workloads, but we have a new requirement: we need to enforce strict vGPU restrictions (memory/compute isolation) at the pod level. I know HAMi excels at this."

## What's next: the next KAI Scheduler release
## Available now: HAMi resource isolation in KAI Scheduler v0.16.4

Both core PRs have fully merged into the KAI Scheduler main branch and are expected to ship in the next release. At that point, users only need to enable GPU sharing and the `hamicore` plugin when installing KAI Scheduler with Helm to get HAMi resource isolation:
Both core PRs have fully merged into the KAI Scheduler main branch and shipped starting with v0.16.4. Users only need to enable GPU sharing and the `hamicore` plugin when installing KAI Scheduler with Helm to get HAMi resource isolation:

```bash
helm install kai-scheduler oci://ghcr.io/nvidia/kai-scheduler \
--set scheduler.gpuSharing.enabled=true \
--set scheduler.gpuSharing.hamicoreEnabled=true \
--version v0.16.4 \
--set global.gpuSharing=true \
--set binder.plugins.hamicore.enabled=true \
--namespace kai-scheduler --create-namespace
```

`scheduler.gpuSharing.enabled=true` turns on GPU sharing, and `scheduler.gpuSharing.hamicoreEnabled=true` activates the `hamicore` plugin, which injects the `CUDA_DEVICE_MEMORY_LIMIT` environment variable into containers that share a GPU. Combined with the node-side kai-resource-isolator that enforces it, this delivers memory hard isolation (full steps above, in "Deploy").
`global.gpuSharing=true` turns on GPU sharing, and `binder.plugins.hamicore.enabled=true` activates the `hamicore` plugin, which injects the `CUDA_DEVICE_MEMORY_LIMIT` environment variable into containers that share a GPU. Combined with the node-side kai-resource-isolator that enforces it, this delivers memory hard isolation (full steps above, in "Deploy").

### Roadmap

Expand Down
13 changes: 7 additions & 6 deletions docs/userguide/kai-scheduler/how-to-use-kai-scheduler.md
Original file line number Diff line number Diff line change
Expand Up @@ -2,9 +2,9 @@
title: How to use KAI Scheduler with HAMi
---

[KAI Scheduler](https://github.com/kai-scheduler/KAI-Scheduler) is NVIDIA's open source, Kubernetes-native scheduler for AI workloads. Starting with its next release, KAI Scheduler ships built-in GPU memory hard isolation powered by HAMi-core. You enable it with a single Helm flag, and a node-side component enforces the limit at the CUDA layer.
[KAI Scheduler](https://github.com/kai-scheduler/KAI-Scheduler) is NVIDIA's open-source, Kubernetes-native scheduler for AI workloads. As of v0.16.4, KAI Scheduler ships built-in GPU memory hard isolation powered by HAMi-core. You enable it with two Helm flags, and a node-side component enforces the limit at the CUDA layer.

For the background on why KAI Scheduler needs HAMi-core and how the two projects divide the work, see [Ecosystem Integrations](../../core-concepts/ecosystem-integrations.md). For the announcement and the open source collaboration behind it, read the blog post: [HAMi-core Adopted by NVIDIA KAI Scheduler](/blog/hami-core-adopted-by-nvidia-kai-scheduler).
For the background on why KAI Scheduler needs HAMi-core and how the two projects divide the work, see [Ecosystem Integrations](../../core-concepts/ecosystem-integrations.md). For the announcement and the open-source collaboration behind it, read the blog post: [HAMi-core Adopted by NVIDIA KAI Scheduler](/blog/hami-core-adopted-by-nvidia-kai-scheduler).

> The integration uses HAMi-core directly, not the full HAMi platform. KAI Scheduler keeps its own scheduling capability and brings in HAMi-core only for GPU memory isolation.

Expand All @@ -17,7 +17,7 @@ This turns KAI Scheduler's cooperative GPU sharing (the scheduler keeps the sum
## Prerequisites

- A Kubernetes cluster with NVIDIA GPUs and the NVIDIA GPU operator or device plugin installed.
- KAI Scheduler supported from its next release. Confirm the version exposes `scheduler.gpuSharing.hamicoreEnabled` before you rely on this guide.
- KAI Scheduler v0.16.4 or later, with `binder.plugins.hamicore.enabled` exposed.
- Helm 3.

## 1. Install KAI Scheduler with the hamicore plugin
Expand All @@ -26,12 +26,13 @@ Install KAI Scheduler with GPU sharing enabled and the `hamicore` plugin activat

```bash
helm install kai-scheduler oci://ghcr.io/nvidia/kai-scheduler \
--set scheduler.gpuSharing.enabled=true \
--set scheduler.gpuSharing.hamicoreEnabled=true \
--version v0.16.4 \
--set global.gpuSharing=true \
--set binder.plugins.hamicore.enabled=true \
--namespace kai-scheduler --create-namespace
```

`scheduler.gpuSharing.enabled=true` turns on GPU sharing. `scheduler.gpuSharing.hamicoreEnabled=true` activates the `hamicore` plugin, which injects `CUDA_DEVICE_MEMORY_LIMIT` into containers that share a GPU.
`global.gpuSharing=true` turns on GPU sharing. `binder.plugins.hamicore.enabled=true` activates the `hamicore` plugin, which injects `CUDA_DEVICE_MEMORY_LIMIT` into containers that share a GPU.

## 2. Deploy kai-resource-isolator

Expand Down
Original file line number Diff line number Diff line change
@@ -1,14 +1,14 @@
---
title: "HAMi-core 被 NVIDIA KAI Scheduler 采用:GPU 共享正式迈入硬隔离时代"
date: "2026-06-20"
description: "两项核心 PR 已合并进入 NVIDIA KAI Scheduler 主干,HAMi 的 GPU 显存硬隔离能力将成为 KAI Scheduler 下个版本的内置特性。云原生 GPU 资源调度正式从「软共享」迈入「硬隔离」时代。"
description: "两项核心 PR 已合并进入 NVIDIA KAI Scheduler 主干,HAMi 的 GPU 显存硬隔离能力已成为 KAI Scheduler v0.16.4 的内置特性。云原生 GPU 资源调度正式从「软共享」迈入「硬隔离」时代。"
authors: [hami_community]
tags: ["HAMi", "NVIDIA", "KAI Scheduler", "GPU 共享", "硬隔离", "Kubernetes", "云原生"]
---

> 本文中的集成对象严格来说是 HAMi-core,而非完整 HAMi 平台。KAI Scheduler 保留自身调度能力,引入 HAMi-core 提供 GPU Memory Isolation 能力。

2026 年 6 月,两项核心 PR 正式合并进入 NVIDIA KAI Scheduler 主干。这意味着,HAMi 的 GPU 显存硬隔离能力将作为 KAI Scheduler 下个版本的内置特性发布,云原生 GPU 资源调度正式从「软共享」迈入「硬隔离」时代。
2026 年 6 月,两项核心 PR 正式合并进入 NVIDIA KAI Scheduler 主干。HAMi 的 GPU 显存硬隔离能力已作为内置特性随 KAI Scheduler v0.16.4 发布,云原生 GPU 资源调度正式从「软共享」迈入「硬隔离」时代。

<!-- truncate -->

Expand Down Expand Up @@ -104,8 +104,9 @@ graph TD

```bash
helm install kai-scheduler oci://ghcr.io/nvidia/kai-scheduler \
--set scheduler.gpuSharing.enabled=true \
--set scheduler.gpuSharing.hamicoreEnabled=true \
--version v0.16.4 \
--set global.gpuSharing=true \
--set binder.plugins.hamicore.enabled=true \
--namespace kai-scheduler --create-namespace
```

Expand Down Expand Up @@ -155,7 +156,7 @@ Pod 启动后,容器内的 `nvidia-smi` 只会显示分配到的显存,而

:::tip

本文只讲重点。完整步骤(前置条件、安装、调度隔离 Pod、用 `nvidia-smi` 验证隔离、按需关闭)见 [如何在 KAI Scheduler 中使用 HAMi](/docs/next/userguide/kai-scheduler/how-to-use-kai-scheduler) 文档。
本文只讲重点。完整步骤(前置条件、安装、调度隔离 Pod、用 `nvidia-smi` 验证隔离、按需关闭)见 [如何在 KAI Scheduler 中使用 HAMi](/zh/docs/next/userguide/kai-scheduler/how-to-use-kai-scheduler) 文档。

:::

Expand Down Expand Up @@ -208,18 +209,19 @@ graph TD

> "We're currently using KAI Scheduler to handle our ML workloads, but we have a new requirement: we need to enforce strict vGPU restrictions (memory/compute isolation) at the pod level. I know HAMi excels at this."

## 展望:KAI Scheduler 下个版本即将发布
## 现已可用:KAI Scheduler v0.16.4 中的 HAMi 资源隔离

两项核心 PR 已全部合并进入 KAI Scheduler 主干,预计将在下个版本中正式发布。届时用户只需在 Helm 安装 KAI Scheduler 时启用 GPU 共享与 `hamicore` 插件,即可获得 HAMi 资源隔离:
两项核心 PR 已全部合并进入 KAI Scheduler 主干,并随 v0.16.4 正式发布。用户只需在 Helm 安装 KAI Scheduler 时启用 GPU 共享与 `hamicore` 插件,即可获得 HAMi 资源隔离:

```bash
helm install kai-scheduler oci://ghcr.io/nvidia/kai-scheduler \
--set scheduler.gpuSharing.enabled=true \
--set scheduler.gpuSharing.hamicoreEnabled=true \
--version v0.16.4 \
--set global.gpuSharing=true \
--set binder.plugins.hamicore.enabled=true \
--namespace kai-scheduler --create-namespace
```

`scheduler.gpuSharing.enabled=true` 开启 GPU 共享,`scheduler.gpuSharing.hamicoreEnabled=true` 激活 `hamicore` 插件,由其为共享 GPU 的容器注入 `CUDA_DEVICE_MEMORY_LIMIT` 环境变量;再配合节点侧的 kai-resource-isolator 强制执行,即可实现显存硬隔离(完整步骤见上文「部署方式」)。
`global.gpuSharing=true` 开启 GPU 共享,`binder.plugins.hamicore.enabled=true` 激活 `hamicore` 插件,由其为共享 GPU 的容器注入 `CUDA_DEVICE_MEMORY_LIMIT` 环境变量;再配合节点侧的 kai-resource-isolator 强制执行,即可实现显存硬隔离(完整步骤见上文「部署方式」)。

### 后续规划

Expand Down
Original file line number Diff line number Diff line change
Expand Up @@ -2,9 +2,9 @@
title: 如何在 KAI Scheduler 中使用 HAMi
---

[KAI Scheduler](https://github.com/kai-scheduler/KAI-Scheduler) 是 NVIDIA 开源的 Kubernetes 原生 AI 工作负载调度器。从下个版本起,KAI Scheduler 内置了由 HAMi-core 驱动的 GPU 显存硬隔离能力。只需一个 Helm 参数即可开启,再由节点侧组件在 CUDA 层强制执行显存上限。
[KAI Scheduler](https://github.com/kai-scheduler/KAI-Scheduler) 是 NVIDIA 开源的 Kubernetes 原生 AI 工作负载调度器。自 v0.16.4 起,KAI Scheduler 内置了由 HAMi-core 驱动的 GPU 显存硬隔离能力。只需两个 Helm 参数即可开启,再由节点侧组件在 CUDA 层强制执行显存上限。

关于 KAI Scheduler 为何需要 HAMi-core、两者如何分工的背景,见[生态集成](../../core-concepts/ecosystem-integrations.md)。关于本次采用的公告与背后的开源协作故事,阅读博客:[HAMi-core 被 NVIDIA KAI Scheduler 采用:GPU 共享正式迈入硬隔离时代](/blog/hami-core-adopted-by-nvidia-kai-scheduler)。
关于 KAI Scheduler 为何需要 HAMi-core、两者如何分工的背景,见[生态集成](../../core-concepts/ecosystem-integrations.md)。关于本次采用的公告与背后的开源协作故事,阅读博客:[HAMi-core 被 NVIDIA KAI Scheduler 采用:GPU 共享正式迈入硬隔离时代](/zh/blog/hami-core-adopted-by-nvidia-kai-scheduler)。

> 该集成直接使用 HAMi-core,而非完整 HAMi 平台。KAI Scheduler 保留自身调度能力,仅引入 HAMi-core 提供 GPU 显存隔离。

Expand All @@ -17,7 +17,7 @@ KAI Scheduler 负责调度 Pod,并通过其 Admission 组件向每个请求共
## 前置条件

- 一个已安装 NVIDIA GPU,并部署 NVIDIA GPU Operator 或 device plugin 的 Kubernetes 集群。
- KAI Scheduler 下个版本起支持。使用本指南前请确认所用版本已暴露 `scheduler.gpuSharing.hamicoreEnabled` 参数。
- KAI Scheduler v0.16.4 或更高版本(需暴露 `binder.plugins.hamicore.enabled` 参数)。
- Helm 3。

## 1. 安装启用 hamicore 插件的 KAI Scheduler
Expand All @@ -26,12 +26,13 @@ KAI Scheduler 负责调度 Pod,并通过其 Admission 组件向每个请求共

```bash
helm install kai-scheduler oci://ghcr.io/nvidia/kai-scheduler \
--set scheduler.gpuSharing.enabled=true \
--set scheduler.gpuSharing.hamicoreEnabled=true \
--version v0.16.4 \
--set global.gpuSharing=true \
--set binder.plugins.hamicore.enabled=true \
--namespace kai-scheduler --create-namespace
```

`scheduler.gpuSharing.enabled=true` 开启 GPU 共享,`scheduler.gpuSharing.hamicoreEnabled=true` 激活 `hamicore` 插件,由其为共享 GPU 的容器注入 `CUDA_DEVICE_MEMORY_LIMIT` 环境变量。
`global.gpuSharing=true` 开启 GPU 共享,`binder.plugins.hamicore.enabled=true` 激活 `hamicore` 插件,由其为共享 GPU 的容器注入 `CUDA_DEVICE_MEMORY_LIMIT` 环境变量。

## 2. 部署 kai-resource-isolator

Expand Down