Skip to content
Merged
Show file tree
Hide file tree
Changes from all commits
Commits
File filter

Filter by extension

Filter by extension

Conversations
Failed to load comments.
Loading
Jump to
Jump to file
Failed to load files.
Loading
Diff view
Diff view
2 changes: 1 addition & 1 deletion concept.md
Original file line number Diff line number Diff line change
Expand Up @@ -8,7 +8,7 @@ title: HAMi 虚拟化原理

于是出现了各种 GPU 共享方案。NVIDIA 官方的时间切片(Time-Slicing)可以让多个 Pod 同时被调度,但没有显存隔离,一个 Pod OOM 会拖垮整张卡上的所有任务。MIG 硬件分区有真正的隔离,但只有 A100、H100 这类数据中心级卡才支持。

HAMi 走了另一条路:**不改驱动、不改应用**,通过 CUDA API 劫持在软件层实现 GPU 虚拟化,多个 Pod 共享同一张物理 GPU,每个 Pod 只能"看到"自己申请的那部分显存,超额分配直接返回 OOM。这是一个 CNCF Sandbox 项目,前身为 `k8s-vGPU-scheduler`。
HAMi 走了另一条路:**不改驱动、不改应用**,通过 CUDA API 劫持在软件层实现 GPU 虚拟化,多个 Pod 共享同一张物理 GPU,每个 Pod 只能"看到"自己申请的那部分显存,超额分配直接返回 OOM。这是一个 CNCF 孵化项目,前身为 `k8s-vGPU-scheduler`。

本文先从 Kubernetes GPU 调度的原理讲起,理解默认模型的局限性,再深入 HAMi 的架构和实现,看它是如何绕过这些限制的。

Expand Down
2 changes: 1 addition & 1 deletion docs/contributor/contributing.md
Original file line number Diff line number Diff line change
Expand Up @@ -3,7 +3,7 @@ title: Contributing to HAMi
sidebar_label: Contributing
---

HAMi is a CNCF Sandbox project that brings GPU and AI accelerator virtualization to Kubernetes. The scheduler, device plugins, documentation, and tooling are all built and maintained by community contributors.
HAMi is a CNCF Incubating project that brings GPU and AI accelerator virtualization to Kubernetes. The scheduler, device plugins, documentation, and tooling are all built and maintained by community contributors.

This guide is the starting point for any contribution, whether you are fixing a typo or implementing a new hardware backend.

Expand Down
2 changes: 1 addition & 1 deletion docs/core-concepts/gpu-virtualization.md
Original file line number Diff line number Diff line change
Expand Up @@ -9,7 +9,7 @@ A typical inference service often only uses 20%~40% of the GPU's compute and a s

This led to various GPU sharing solutions. NVIDIA's official Time-Slicing allows multiple Pods to be scheduled concurrently, but provides no VRAM isolation - a Pod OOM can crash all tasks on the card. MIG hardware partitioning offers true isolation, but only datacenter-grade cards like A100 and H100 support it.

HAMi takes a different approach: **no driver changes, no application changes** - it achieves GPU virtualization at the software layer through CUDA API interception. Multiple Pods share the same physical GPU, and each Pod can only "see" the VRAM it requested. Over-allocation directly returns OOM. HAMi is a CNCF Sandbox project, formerly known as `k8s-vGPU-scheduler`.
HAMi takes a different approach: **no driver changes, no application changes** - it achieves GPU virtualization at the software layer through CUDA API interception. Multiple Pods share the same physical GPU, and each Pod can only "see" the VRAM it requested. Over-allocation directly returns OOM. HAMi is a CNCF Incubating project, formerly known as `k8s-vGPU-scheduler`.

This article starts with the fundamentals of Kubernetes GPU scheduling, explains the limitations of the default model, and then dives into HAMi's architecture and implementation to show how it works around these constraints.

Expand Down
2 changes: 1 addition & 1 deletion docs/core-concepts/introduction.md
Original file line number Diff line number Diff line change
Expand Up @@ -5,7 +5,7 @@ slug: /

HAMi (Heterogeneous AI Computing Virtualization Middleware) is an open source platform for managing heterogeneous AI computing devices in Kubernetes clusters. Formerly known as k8s-vGPU-scheduler, HAMi enables device sharing across multiple containers and workloads.

HAMi is a [Cloud Native Computing Foundation](https://cncf.io/) [Sandbox project](https://landscape.cncf.io/?item=orchestration-management--scheduling-orchestration--hami) and is listed in both the [CNCF Landscape](https://landscape.cncf.io/?item=orchestration-management--scheduling-orchestration--hami) and [CNAI Landscape](https://landscape.cncf.io/?group=cnai&item=orchestration-management--scheduling-orchestration--hami).
HAMi is a [Cloud Native Computing Foundation](https://cncf.io/) [Incubating project](https://landscape.cncf.io/?group=projects-and-products&project=incubating&item=orchestration-management--scheduling-orchestration--hami) and is listed in both the [CNCF Landscape](https://landscape.cncf.io/?group=projects-and-products&project=incubating&item=orchestration-management--scheduling-orchestration--hami) and [CNAI Landscape](https://landscape.cncf.io/?group=cnai&item=orchestration-management--scheduling-orchestration--hami).

## Key Features

Expand Down
Original file line number Diff line number Diff line change
Expand Up @@ -10,7 +10,7 @@ translated: true

于是出现了各种 GPU 共享方案。NVIDIA 官方的时间切片(Time-Slicing)可以让多个 Pod 同时被调度,但没有显存隔离,一个 Pod OOM 会拖垮整张卡上的所有任务。MIG 硬件分区有真正的隔离,但只有 A100、H100 这类数据中心级卡才支持。

HAMi 走了另一条路:**不改驱动、不改应用**,通过 CUDA API 劫持在软件层实现 GPU 虚拟化,多个 Pod 共享同一张物理 GPU,每个 Pod 只能"看到"自己申请的那部分显存,超额分配直接返回 OOM。这是一个 CNCF Sandbox 项目,前身为 `k8s-vGPU-scheduler`。
HAMi 走了另一条路:**不改驱动、不改应用**,通过 CUDA API 劫持在软件层实现 GPU 虚拟化,多个 Pod 共享同一张物理 GPU,每个 Pod 只能"看到"自己申请的那部分显存,超额分配直接返回 OOM。这是一个 CNCF 孵化项目,前身为 `k8s-vGPU-scheduler`。

本文先从 Kubernetes GPU 调度的原理讲起,理解默认模型的局限性,再深入 HAMi 的架构和实现,看它是如何绕过这些限制的。

Expand Down
Original file line number Diff line number Diff line change
Expand Up @@ -6,7 +6,7 @@ slug: /

HAMi(异构 AI 计算虚拟化中间件)是一个用于管理 Kubernetes 集群中异构 AI 计算设备的开源平台。前身为 k8s-vGPU-scheduler,HAMi 可在多个容器和工作负载之间实现设备共享。

HAMi 是[云原生计算基金会(CNCF)](https://cncf.io/)的 [Sandbox 项目](https://landscape.cncf.io/?item=orchestration-management--scheduling-orchestration--hami),并被收录于 [CNCF 技术全景图](https://landscape.cncf.io/?item=orchestration-management--scheduling-orchestration--hami)和 [CNAI 技术全景图](https://landscape.cncf.io/?group=cnai&item=orchestration-management--scheduling-orchestration--hami)。
HAMi 是[云原生计算基金会(CNCF)](https://cncf.io/)的 [孵化项目](https://landscape.cncf.io/?group=projects-and-products&project=incubating&item=orchestration-management--scheduling-orchestration--hami),并被收录于 [CNCF 技术全景图](https://landscape.cncf.io/?group=projects-and-products&project=incubating&item=orchestration-management--scheduling-orchestration--hami)和 [CNAI 技术全景图](https://landscape.cncf.io/?group=cnai&item=orchestration-management--scheduling-orchestration--hami)。

## 核心特性

Expand Down
Original file line number Diff line number Diff line change
Expand Up @@ -10,7 +10,7 @@ translated: true

于是出现了各种 GPU 共享方案。NVIDIA 官方的时间切片(Time-Slicing)可以让多个 Pod 同时被调度,但没有显存隔离,一个 Pod OOM 会拖垮整张卡上的所有任务。MIG 硬件分区有真正的隔离,但只有 A100、H100 这类数据中心级卡才支持。

HAMi 走了另一条路:**不改驱动、不改应用**,通过 CUDA API 劫持在软件层实现 GPU 虚拟化,多个 Pod 共享同一张物理 GPU,每个 Pod 只能"看到"自己申请的那部分显存,超额分配直接返回 OOM。这是一个 CNCF Sandbox 项目,前身为 `k8s-vGPU-scheduler`。
HAMi 走了另一条路:**不改驱动、不改应用**,通过 CUDA API 劫持在软件层实现 GPU 虚拟化,多个 Pod 共享同一张物理 GPU,每个 Pod 只能"看到"自己申请的那部分显存,超额分配直接返回 OOM。这是一个 CNCF 孵化项目,前身为 `k8s-vGPU-scheduler`。

本文先从 Kubernetes GPU 调度的原理讲起,理解默认模型的局限性,再深入 HAMi 的架构和实现,看它是如何绕过这些限制的。

Expand Down
Original file line number Diff line number Diff line change
Expand Up @@ -6,7 +6,7 @@ slug: /

HAMi(异构 AI 计算虚拟化中间件)是一个用于管理 Kubernetes 集群中异构 AI 计算设备的开源平台。前身为 k8s-vGPU-scheduler,HAMi 可在多个容器和工作负载之间实现设备共享。

HAMi 是[云原生计算基金会(CNCF)](https://cncf.io/)的 [Sandbox 项目](https://landscape.cncf.io/?item=orchestration-management--scheduling-orchestration--hami),并被收录于 [CNCF 技术全景图](https://landscape.cncf.io/?item=orchestration-management--scheduling-orchestration--hami)和 [CNAI 技术全景图](https://landscape.cncf.io/?group=cnai&item=orchestration-management--scheduling-orchestration--hami)。
HAMi 是[云原生计算基金会(CNCF)](https://cncf.io/)的 [孵化项目](https://landscape.cncf.io/?group=projects-and-products&project=incubating&item=orchestration-management--scheduling-orchestration--hami),并被收录于 [CNCF 技术全景图](https://landscape.cncf.io/?group=projects-and-products&project=incubating&item=orchestration-management--scheduling-orchestration--hami)和 [CNAI 技术全景图](https://landscape.cncf.io/?group=cnai&item=orchestration-management--scheduling-orchestration--hami)。

## 核心特性

Expand Down
2 changes: 1 addition & 1 deletion versioned_docs/version-v2.9.0/contributor/contributing.md
Original file line number Diff line number Diff line change
Expand Up @@ -3,7 +3,7 @@ title: Contributing to HAMi
sidebar_label: Contributing
---

HAMi is a CNCF Sandbox project that brings GPU and AI accelerator virtualization to Kubernetes. The scheduler, device plugins, documentation, and tooling are all built and maintained by community contributors.
HAMi is a CNCF Incubating project that brings GPU and AI accelerator virtualization to Kubernetes. The scheduler, device plugins, documentation, and tooling are all built and maintained by community contributors.

This guide is the starting point for any contribution, whether you are fixing a typo or implementing a new hardware backend.

Expand Down
Original file line number Diff line number Diff line change
Expand Up @@ -9,7 +9,7 @@ A typical inference service often only uses 20%~40% of the GPU's compute and a s

This led to various GPU sharing solutions. NVIDIA's official Time-Slicing allows multiple Pods to be scheduled concurrently, but provides no VRAM isolation - a Pod OOM can crash all tasks on the card. MIG hardware partitioning offers true isolation, but only datacenter-grade cards like A100 and H100 support it.

HAMi takes a different approach: **no driver changes, no application changes** - it achieves GPU virtualization at the software layer through CUDA API interception. Multiple Pods share the same physical GPU, and each Pod can only "see" the VRAM it requested. Over-allocation directly returns OOM. HAMi is a CNCF Sandbox project, formerly known as `k8s-vGPU-scheduler`.
HAMi takes a different approach: **no driver changes, no application changes** - it achieves GPU virtualization at the software layer through CUDA API interception. Multiple Pods share the same physical GPU, and each Pod can only "see" the VRAM it requested. Over-allocation directly returns OOM. HAMi is a CNCF Incubating project, formerly known as `k8s-vGPU-scheduler`.

This article starts with the fundamentals of Kubernetes GPU scheduling, explains the limitations of the default model, and then dives into HAMi's architecture and implementation to show how it works around these constraints.

Expand Down
Original file line number Diff line number Diff line change
Expand Up @@ -5,7 +5,7 @@ slug: /

HAMi (Heterogeneous AI Computing Virtualization Middleware) is an open source platform for managing heterogeneous AI computing devices in Kubernetes clusters. Formerly known as k8s-vGPU-scheduler, HAMi enables device sharing across multiple containers and workloads.

HAMi is a [Cloud Native Computing Foundation](https://cncf.io/) [Sandbox project](https://landscape.cncf.io/?item=orchestration-management--scheduling-orchestration--hami) and is listed in both the [CNCF Landscape](https://landscape.cncf.io/?item=orchestration-management--scheduling-orchestration--hami) and [CNAI Landscape](https://landscape.cncf.io/?group=cnai&item=orchestration-management--scheduling-orchestration--hami).
HAMi is a [Cloud Native Computing Foundation](https://cncf.io/) [Incubating project](https://landscape.cncf.io/?group=projects-and-products&project=incubating&item=orchestration-management--scheduling-orchestration--hami) and is listed in both the [CNCF Landscape](https://landscape.cncf.io/?group=projects-and-products&project=incubating&item=orchestration-management--scheduling-orchestration--hami) and [CNAI Landscape](https://landscape.cncf.io/?group=cnai&item=orchestration-management--scheduling-orchestration--hami).

## Key Features

Expand Down
Loading