Project-HAMi · hami-robot · May 29, 2026 · May 28, 2026 · May 28, 2026 · May 28, 2026
diff --git a/CHANGELOG.md b/CHANGELOG.md
@@ -548,9 +548,9 @@
 
 #### :rocket: Major features
 
-- Support dynamic mig feature, please refer to this [document](https://github.com/Project-HAMi/HAMi/blob/master/docs/dynamic-mig-support.md)
+- Support dynamic mig feature, please refer to this [document](/docs/userguide/nvidia-device/dynamic-mig-support)
 - Reinstall Hami will NOT crash GPU tasks
-- Put all configurations into a configMap, you can customize hami installation by modify its content: see [details](https://github.com/Project-HAMi/HAMi/blob/master/docs/config.md)
+- Put all configurations into a configMap, you can customize hami installation by modify its content: see [details](/docs/userguide/configure)
 
 #### :bug: Major bug fixes
 

diff --git a/blog/2024-12-31-post/index.md b/blog/2024-12-31-post/index.md
@@ -21,7 +21,7 @@ This section provides a rough walkthrough of the related code to explain the int
 
 The official documentation provides a clear overview before looking at the code:
 
-![flowchart](https://github.com/Project-HAMi/HAMi/blob/master/docs/develop/imgs/flowchart.jpeg?raw=true)
+![flowchart](/img/blog/flowchart.jpeg)
 
 The process can be broken down into three phases:
 

diff --git a/blog/hami-v2-9-0-release/index.md b/blog/hami-v2-9-0-release/index.md
@@ -70,7 +70,7 @@ In v2.9, user Pods must explicitly declare `huawei.com/vnpu-mode: 'hami-core'` i
 
 Thanks to Huawei Cloud Canada Lab and China Merchants Bank [@ashergaga](https://github.com/ashergaga) for their contributions.
 
-This release also updates the [HAMi-core performance benchmark data](https://github.com/Project-HAMi/HAMi/blob/master/docs/benchmark_cn.md). For detailed benchmark procedures, refer to the [project documentation](https://github.com/Project-HAMi/HAMi/tree/master/benchmarks).
+This release also updates the [HAMi-core performance benchmark data](/docs/userguide/benchmark). For detailed benchmark procedures, refer to the [project documentation](https://github.com/Project-HAMi/HAMi/tree/master/benchmarks).
 
 ### HAMi-DRA Solution: Lightweight HAMi
 

diff --git a/docs/contributor/e2e-testing.md b/docs/contributor/e2e-testing.md
@@ -0,0 +1,70 @@
+---
+id: e2e-testing
+title: End-to-End Testing
+sidebar_label: E2E Testing
+---
+
+# End-to-End Testing
+
+## Summary
+
+This document proposes End-to-End (E2E) testing support for HAMi, ensuring its functionality and compatibility within the Kubernetes ecosystem. It introduces mechanisms to validate the entire workflow and guarantees that the system meets production-level requirements.
+
+## Motivation
+
+E2E tests validate the complete functionality of a system, ensuring that the end-user experience aligns with developer specifications.
+
+While unit and integration tests provide valuable feedback, they are often insufficient in distributed systems. Minor changes may pass unit and integration tests but still introduce unforeseen issues at the system level.
+
+Comprehensive E2E test coverage is essential to mitigate the risks of regressions, improve reliability, and maintain confidence in the system's seamless integration with Kubernetes.
+
+### Goals
+
+- Set up the E2E testing basic environment.
+- Define the scope and scenarios for E2E testing of HAMi.
+- Implement E2E tests that cover key workflows and edge cases.
+- Ensure compatibility with Kubernetes.
+- Establish a reliable and repeatable test framework for future enhancements.
+
+### Non-Goals
+
+- Unit or integration testing of individual features (covered elsewhere).
+- Performance benchmarking beyond basic scenarios.
+
+## Proposal
+
+### Test Scope
+
+- **Core functionality:** Validate basic operations and workflows.
+- **Edge cases:** Test unusual scenarios or invalid inputs to ensure robustness.
+- **Compatibility:**
+  - Verify integration with different heterogeneous devices.
+  - Verify integration with different Kubernetes versions.
+  - Verify integration with different CUDA versions (optional).
+- **Error handling:** Ensure appropriate error messages and recovery mechanisms are in place.
+
+### Implementation Details
+
+- Tests run in a local environment.
+- Tests are written using the [Ginkgo](https://onsi.github.io/ginkgo/) framework.
+- All tests use isolated namespaces to avoid conflicts.
+- Resource cleanup is automated after each test run.
+- CI integration ensures tests run against PRs, daily builds, and releases.
+
+### User Stories
+
+**Story 1:** Automating E2E testing with Helm deployment.
+
+**Story 2:** Automating E2E testing with resource validation.
+
+**Story 3:** Automating E2E testing with Kubernetes resource deployment.
+
+## Risks and Mitigations
+
+**Resource Limitations**
+
+Testing clusters may encounter resource constraints such as insufficient CPU, memory, or storage. This can lead to test failures, degraded performance, or timeouts during deployments.
+
+**Environment Instability**
+
+Instabilities such as network latency, intermittent failures, or cluster node failures can cause tests to fail or behave inconsistently. Tests should be designed to retry on transient errors and report deterministically on persistent ones.
diff --git a/docs/contributor/roadmap.md b/docs/contributor/roadmap.md
@@ -0,0 +1,39 @@
+---
+id: roadmap
+title: Hardware Support Roadmap
+sidebar_label: Roadmap
+---
+
+# Hardware Support Roadmap
+
+## Device Support Matrix
+
+| Device Type | Manufacturer | Models | Memory Isolation | Core Isolation | Multi-Card |
+| ----------- | ------------ | ------ | :--------------: | :------------: | :--------: |
+| GPU | NVIDIA | All | Yes | Yes | Yes |
+| MLU | Cambricon | 370, 590 | Yes | Yes | No |
+| GCU | Enflame | S60 | Yes | Yes | No |
+| DCU | Hygon | Z100, Z100L | Yes | Yes | No |
+| NPU | Ascend | 310P, 910B, 910B3 | Yes | Yes | No |
+| GPU | iluvatar | All | Yes | Yes | No |
+| DPU | Teco | Checking | In progress | In progress | No |
+| GPU | Moore Threads | MTT S4000 | Yes | Yes | No |
+| GPU | Birentech | Model 110 | In progress | In progress | No |
+| GPU | MetaX | MXC500 | Yes | Yes | No |
+| XPU | Kunlunxin | P800 | Yes | Yes | No |
+| GPU | Vastai | VA16 | Yes | Yes | No |
+
+## Planned Features
+
+- [ ] Support video codec processing
+- [ ] Support Multi-Instance GPUs (MIG)
+- [ ] Support flexible scheduling policies
+  - [x] binpack
+  - [x] spread
+  - [ ] numa affinity
+- [ ] Integrated gpu-operator
+- [ ] Rich observability support
+- [ ] DRA support
+- [ ] Support Intel GPU device
+- [ ] Support AMD GPU device
+- [x] Support Enflame GCU device
diff --git a/docs/developers/dynamic-mig.md b/docs/developers/dynamic-mig.md
@@ -100,7 +100,7 @@ data:
 
 ## Structure
 
-<img src="https://github.com/Project-HAMi/HAMi/blob/master/docs/develop/imgs/hami-dynamic-mig-structure.png?raw=true" width="600" alt="HAMi dynamic MIG structure diagram showing vGPU Pool and Scheduler components" />
+<img src="/img/docs/en/dynamic-mig/hami-dynamic-mig-structure.png" width="600" alt="HAMi dynamic MIG structure diagram showing vGPU Pool and Scheduler components" />
 
 ## Examples
 
@@ -147,7 +147,7 @@ spec:
 
 The Procedure of a vGPU task which uses dynamic-mig is shown below:
 
-<img src="https://github.com/Project-HAMi/HAMi/blob/master/docs/develop/imgs/hami-dynamic-mig-procedure.png?raw=true" width="800" alt="HAMi dynamic MIG procedure flowchart showing task scheduling process" />
+<img src="/img/docs/en/dynamic-mig/hami-dynamic-mig-procedure.png" width="800" alt="HAMi dynamic MIG procedure flowchart showing task scheduling process" />
 
 After submitting a task, the deviceshare plugin iterates over templates defined in configMap `hami-scheduler-device` and finds the first available template to fit. You can always change the content of that configMap, and restart vc-scheduler to customize.
 

diff --git a/docs/developers/gpu-topology-scheduling.md b/docs/developers/gpu-topology-scheduling.md
@@ -0,0 +1,81 @@
+---
+id: gpu-topology-scheduling
+title: GPU Topology-Aware Scheduling
+sidebar_label: GPU Topology Scheduling
+---
+
+# GPU Topology-Aware Scheduling
+
+HAMi supports GPU topology-aware scheduling in vGPU environments. HAMi can optimize GPU card scheduling based on the topological relationships between GPUs, thereby improving GPU resource utilization and performance.
+
+Use `nvidia-smi topo -m` to view the topological relationships between GPUs on a node.
+
+## Enabling GPU Topology-Aware Scheduling
+
+When installing HAMi, set `scheduler.defaultSchedulerPolicy.gpuSchedulerPolicy` to `topology-aware`:
+
+```bash
+helm install hami hami-charts/hami \
+  --set scheduler.defaultSchedulerPolicy.gpuSchedulerPolicy=topology-aware \
+  -n kube-system
+```
+
+If HAMi is already installed, enable it via one of the following methods:
+
+**1. Device-plugin configuration**
+
+Set the environment variable `ENABLE_TOPOLOGY_SCORE: 'true'` in the DaemonSet `hami-device-plugin`.
+
+**2. Global scheduler settings**
+
+Add `gpu-scheduler-policy=topology-aware` when starting `hami-scheduler`.
+
+**3. Pod-level annotation**
+
+```yaml
+metadata:
+  annotations:
+    hami.io/gpu-scheduler-policy: topology-aware
+```
+
+After submitting the Pod, check the logs of `hami-scheduler` (log level must be greater than 5):
+
+```
+I0703 08:34:27.032644  1 device.go:708] "device allocate success" pod="default/testpod" best device combination={"NVIDIA":[{"Idx":7,"UUID":"GPU-dsaf","Type":"NVIDIA","Usedmem":1024,"Usedcores":0},{"Idx":5,"UUID":"GPU-gads","Type":"NVIDIA","Usedmem":1024,"Usedcores":0}]}
+```
+
+## Scheduling Strategy
+
+### Node Selection
+
+When multiple nodes meet the requirements, the node with the minimum number of GPUs that still satisfies the request is preferred.
+
+For example, given two candidate nodes:
+
+- Node1: 4 GPUs
+- Node2: 6 GPUs
+
+If the workload requires 2 GPUs, Node1 is preferred because it is the smaller node that still fits the request. This leaves Node2 available for larger workloads.
+
+### Single-GPU Allocation (One Pod, One Device)
+
+When a Pod requests only one GPU, the GPU with the **worst connectivity** to other GPUs on the node is preferred (assuming memory and compute requirements are met). This preserves high-bandwidth GPU pairs for future multi-GPU workloads.
+
+Example on a 4-GPU node:
+
+```json
+[
+  { "uuid": "gpu0", "score": { "gpu1": "100", "gpu2": "100", "gpu3": "200" } },
+  { "uuid": "gpu1", "score": { "gpu0": "100", "gpu2": "200", "gpu3": "100" } },
+  { "uuid": "gpu2", "score": { "gpu0": "100", "gpu1": "200", "gpu3": "200" } },
+  { "uuid": "gpu3", "score": { "gpu0": "200", "gpu1": "100", "gpu2": "200" } }
+]
+```
+
+`gpu0` and `gpu1` have the lowest total connectivity scores, so they are preferred for single-GPU allocation.
+
+### Multi-GPU Allocation (One Pod, Multiple Devices)
+
+When a Pod requests multiple GPUs, the set of GPUs with the **best mutual connectivity** is preferred.
+
+Using the same 4-GPU node as above, `gpu2` and `gpu3` have the highest connectivity with each other, so they are preferred for a 2-GPU request.
diff --git a/docs/developers/gpu-utilization-metrics.md b/docs/developers/gpu-utilization-metrics.md
@@ -0,0 +1,112 @@
+---
+id: gpu-utilization-metrics
+title: GPU Utilization Metrics
+sidebar_label: GPU Utilization Metrics
+---
+
+# GPU Utilization Metrics
+
+## Summary
+
+HAMi supports dividing a single NVIDIA GPU card into several vGPU cards to efficiently utilize GPU capacity. However, when a vGPU is assigned to a Pod, HAMi does not currently expose per-Pod GPU utilization metrics. This makes it impossible for users to observe how much GPU each Pod is actually consuming.
+
+This document describes the design for adding per-Pod vGPU utilization monitoring.
+
+## Motivation
+
+### Goals
+
+- Support monitoring of per-Pod vGPU utilization.
+
+### Non-Goals
+
+- Does not support monitoring GPU utilization for non-NVIDIA GPUs.
+
+## Design Details
+
+### Extend `shared_region` with a `gpu_util` Field
+
+A new `gpu_util` field is added to `shrreg_proc_slot_t` to record per-process GPU utilization:
+
+```c
+typedef struct {
+    uint64_t dec_util;
+    uint64_t enc_util;
+    uint64_t sm_util;
+} device_gpu_t;
+
+typedef struct {
+    int32_t pid;
+    int32_t hostpid;
+    device_memory_t used[CUDA_DEVICE_MAX_COUNT];
+    uint64_t monitorused[CUDA_DEVICE_MAX_COUNT];
+    int32_t status;
+    device_gpu_t gpu_util[CUDA_DEVICE_MAX_COUNT]; // new field
+} shrreg_proc_slot_t;
+
+
+int set_gpu_device_gpu_monitor(int32_t pid, int dev, unsigned int smUtil) {
+    int i;
+    ensure_initialized();
+    lock_shrreg();
+    for (i = 0; i < region_info.shared_region->proc_num; i++) {
+        if (region_info.shared_region->procs[i].hostpid == pid) {
+            region_info.shared_region->procs[i].gpu_util[dev].smUtil = smUtil;
+            break;
+        }
+    }
+    unlock_shrreg();
+    return 1;
+}
+```
+
+### Update `get_used_gpu_utilization`
+
+The `get_used_gpu_utilization` method is updated to record the GPU usage rate of the current pid:
+
+```c
+int get_used_gpu_utilization(int *userutil, int *sysprocnum) {
+    // ...
+    for (i = 0; i < processes_num; i++) {
+        set_gpu_device_memory_monitor(processes_sample[i].pid, cudadev, summonitor);
+        set_gpu_device_gpu_monitor(processes_sample[i].pid, cudadev, processes_sample[i].smUtil); // new
+    }
+    // ...
+    return 0;
+}
+```
+
+### Expose Metrics via vGPUMonitor
+
+`vGPUMonitor` is updated to read `sm_util` from the shared region and expose it as a Prometheus metric:
+
+```go
+ctrDeviceUtilizationdesc = prometheus.NewDesc(
+    "Device_utilization_desc_of_container",
+    "Container device utilization description",
+    []string{"podnamespace", "podname", "ctrname", "vdeviceid", "deviceuuid"}, nil,
+)
+
+func getTotalUtilization(usage podusage, vidx int) deviceUtilization {
+    added := deviceUtilization{decUtil: 0, encUtil: 0, smUtil: 0}
+    for _, val := range usage.sr.procs {
+        added.decUtil += val.gpuUtil[vidx].decUtil
+        added.encUtil += val.gpuUtil[vidx].encUtil
+        added.smUtil  += val.gpuUtil[vidx].smUtil
+    }
+    return added
+}
+
+utilization := getTotalUtilization(srPodList[sridx], i)
+
+ch <- prometheus.MustNewConstMetric(
+    ctrDeviceUtilizationdesc,
+    prometheus.GaugeValue,
+    float64(utilization.smUtil),
+    val.Namespace, val.Name, ctrName, fmt.Sprint(i), uuid,
+)
+```
+
+## Test Plan
+
+Deploy multiple Pods that actively use GPU on the same node and verify that HAMi exposes accurate per-Pod GPU utilization rates via the Prometheus metrics endpoint.