diff --git a/docs/troubleshooting/troubleshooting.md b/docs/troubleshooting/troubleshooting.md index ec4851f41..2a45da2cd 100644 --- a/docs/troubleshooting/troubleshooting.md +++ b/docs/troubleshooting/troubleshooting.md @@ -159,4 +159,302 @@ devicePlugin: enabled: false ``` +::: This will apply the fix automatically. + +## NVIDIA containers fail with GPU Operator 25.10+ {#nvidia-toolkit-gpu-operator-25-10} + +Use this section when the HAMi Device Plugin or a HAMi-scheduled NVIDIA workload stops starting after GPU Operator is installed or upgraded. + +### Problem 1: The HAMi Device Plugin fails to start + +#### Identify the cause + +Check the Device Plugin logs: + +```bash +kubectl logs -n kube-system \ + -l app.kubernetes.io/component=hami-device-plugin \ + --all-containers --tail=200 +``` + +Match the output to one of these errors: + +| Error in the log | Cause | +| --- | --- | +| `Incompatible strategy detected auto` | The Device Plugin cannot discover NVML devices because its container did not receive the NVIDIA driver and devices. | +| `invalid device discovery strategy` | The Device Plugin could not initialize NVIDIA device discovery, usually for the same runtime-injection reason. | +| `failed to locate libcuda.so` or `failed to locate libnvidia-ml.so` | HAMi cannot find the driver libraries under the configured driver root while generating a CDI specification. | + +Confirm the runtime and CDI configuration: + +```bash +kubectl get clusterpolicy -o yaml | grep -A 5 'cdi:' +kubectl get runtimeclass nvidia + +kubectl get pods -n kube-system \ + -l app.kubernetes.io/component=hami-device-plugin \ + -o custom-columns=NAME:.metadata.name,RUNTIMECLASS:.spec.runtimeClassName +``` + +With GPU Operator 25.10+, CDI is normally enabled, the `nvidia` RuntimeClass must exist, and the HAMi Device Plugin must show `nvidia` in the `RUNTIMECLASS` column. + +#### Solution + +Configure the `nvidia` RuntimeClass for HAMi and restart the Device Plugin: + +```bash +helm upgrade hami hami-charts/hami \ + --namespace kube-system \ + --reuse-values \ + --set devicePlugin.runtimeClassName=nvidia + +kubectl rollout restart daemonset/hami-device-plugin -n kube-system +kubectl rollout status daemonset/hami-device-plugin -n kube-system +``` + +If the logs report missing driver libraries while HAMi CDI is enabled, use the GPU Operator paths: + +```yaml +devicePlugin: + runtimeClassName: nvidia + deviceListStrategy: cdi-annotations + nvidiaDriverRoot: /run/nvidia/driver + nvidiaHookPath: /usr/local/nvidia/toolkit/nvidia-ctk +``` + +Wait until the NVIDIA driver and Toolkit DaemonSets are ready, then restart the HAMi Device Plugin. For host-installed drivers, set `devicePlugin.nvidiaDriverRoot` to `/` instead. + +### Problem 2: A HAMi-scheduled Pod fails to start + +#### Identify the cause + +Inspect the Pod events and its assigned RuntimeClass: + +```bash +kubectl describe pod -n +kubectl get pod -n \ + -o custom-columns=NAME:.metadata.name,RUNTIMECLASS:.spec.runtimeClassName +``` + +Use the error text to select the correct path: + +| Error in the Pod events | Cause | +| --- | --- | +| `libcuda.so.1: cannot open shared object file` | The container started without the NVIDIA driver libraries. | +| `unresolvable CDI devices management.nvidia.com/gpu=GPU-...` | The NVIDIA runtime selected a GPU management CDI device, but the corresponding device could not be generated or resolved. | +| `unresolvable CDI devices k8s.device-plugin.nvidia.com/gpu=GPU-...` | HAMi returned a CDI device, but the runtime cannot find a matching HAMi-generated CDI specification. | + +#### Solution + +First determine which HAMi injection mode is configured: + +```bash +helm get values hami -n kube-system | grep -A 5 'devicePlugin:' +``` + +- For the default `devicePlugin.deviceListStrategy=envvar` mode, set `devicePlugin.runtimeClassName=nvidia` by using the Helm command from Problem 1. This makes the NVIDIA runtime process the UUID returned through `NVIDIA_VISIBLE_DEVICES`. +- For `devicePlugin.deviceListStrategy=cdi-annotations`, apply all four CDI values shown in Problem 1. Then inspect `/var/run/cdi/k8s.device-plugin.nvidia.com-gpu.json` on the node and verify that it contains the allocated GPU UUID. +- For a host-installed Container Toolkit, confirm that the `nvidia` runtime is present in the active container runtime configuration. Restart the container runtime after correcting its configuration. + +Do not mix HAMi CDI annotations with a missing or stale HAMi CDI specification. See [NVIDIA CDI support](../installation/configure-cdi.md) for the complete setup and verification procedure. + +### Why this happens + +Starting with [GPU Operator 25.10.0](https://docs.nvidia.com/datacenter/cloud-native/gpu-operator/25.10/cdi.html), CDI is enabled by default and the Operator no longer makes the `nvidia` runtime the default runtime. + +Before 25.10.0, GPU Operator normally configured the NVIDIA runtime as the default. As a result, the NVIDIA runtime hook processed every Pod and injected the devices and driver libraries selected through `NVIDIA_VISIBLE_DEVICES`. + +With 25.10.0 and later, `runc` remains the default runtime and the container runtime uses native CDI for standard Device Plugin workloads. GPU management containers that access GPUs through `NVIDIA_VISIBLE_DEVICES`, including the HAMi Device Plugin, must explicitly use `runtimeClassName: nvidia`. + +HAMi supports two device-injection paths: + +| HAMi mode | Allocation result | Runtime requirement | +| --- | --- | --- | +| `envvar` (default) | HAMi writes the allocated GPU UUID to `NVIDIA_VISIBLE_DEVICES`. | On GPU Operator 25.10+, the Pod must use the `nvidia` RuntimeClass. | +| `cdi-annotations` | HAMi returns a CDI device named `k8s.device-plugin.nvidia.com/gpu=GPU-...` and generates its CDI specification on the node. | The container runtime must have CDI enabled and be able to read the current HAMi specification. | + +The HAMi chart applies `devicePlugin.runtimeClassName` both to the Device Plugin and to NVIDIA workloads mutated by the HAMi scheduler. This is why setting it to `nvidia` fixes the management container and keeps the workload runtime path consistent. + +For new clusters, GPU Operator is recommended because it provides one entry point for configuring and upgrading the driver, Container Toolkit, and monitoring components. If these components are already installed on the hosts and you maintain their runtime configuration yourself, GPU Operator is optional; follow [Prerequisites](../installation/prerequisites.md). + +:::warning + +Disable the GPU Operator Device Plugin when using HAMi. Both plugins advertise `nvidia.com/gpu` and must not run on the same nodes. + +```yaml +devicePlugin: + enabled: false +``` + +::: + +## Diagnostic Cheat Sheet + +```bash +# Check if HAMi resources are allocatable on a node +kubectl get node -o yaml | grep nvidia.com + +# Check the logs of the HAMi device plugin +kubectl logs -n kube-system -l app.kubernetes.io/name=hami + +# Verify if the libvgpu library is mounted inside a running pod +kubectl exec -it -- ls -l /usr/local/vgpu/libvgpu.so +``` +imit-not-enforced} + +If a container exceeds its `nvidia.com/gpumem` limit, check the following causes: + +- **`CUDA_DISABLE_CONTROL=true` is set** - disables HAMi-core enforcement entirely. Remove it from production workloads. +- **Docker-in-Docker (DinD)** - inner containers do not inherit the `/etc/ld.so.preload` hostPath mount. HAMi enforcement does not apply inside DinD. +- **Direct driver API usage** - workloads calling NVML or the CUDA Driver API directly bypass `libvgpu.so`. +- **`nvidia-container-runtime` not set as default** - verify with: + + ```bash + containerd config dump | grep default_runtime_name + ``` + + The output must show `nvidia`. If not, follow the [Prerequisites](./installation/online-installation) guide. + +- If you don’t explicitly request vGPUs when using the device plugin with NVIDIA images, all GPUs on the host may be exposed to your container. +- Currently, A100 MIG can be supported in only "none" and "mixed" modes. +- Tasks with the "nodeName" field cannot be scheduled at the moment; please use "nodeSelector" instead. +- Only computing tasks are currently supported; video codec processing is not supported. +- Since v2.3.10, HAMi has changed the `device-plugin` environment variable name from `NodeName` to `NODE_NAME`. If you are using an image version earlier than v2.3.10, the `device-plugin` may fail to start. + + To resolve this issue, you have two options: + + - Manually edit the DaemonSet using `kubectl edit daemonset` and update the environment variable from `NodeName` to `NODE_NAME`. + - Upgrade the `device-plugin` image to the latest version using Helm: + + ```bash + helm upgrade hami hami/hami -n kube-system + ``` + + This will apply the fix automatically. + +## NVIDIA containers fail with GPU Operator 25.10+ {#nvidia-toolkit-gpu-operator-25-10} + +Use this section when the HAMi Device Plugin or a HAMi-scheduled NVIDIA workload stops starting after GPU Operator is installed or upgraded. + +### Problem 1: The HAMi Device Plugin fails to start + +#### Identify the cause + +Check the Device Plugin logs: + +```bash +kubectl logs -n kube-system \ + -l app.kubernetes.io/component=hami-device-plugin \ + --all-containers --tail=200 +``` + +Match the output to one of these errors: + +| Error in the log | Cause | +| --- | --- | +| `Incompatible strategy detected auto` | The Device Plugin cannot discover NVML devices because its container did not receive the NVIDIA driver and devices. | +| `invalid device discovery strategy` | The Device Plugin could not initialize NVIDIA device discovery, usually for the same runtime-injection reason. | +| `failed to locate libcuda.so` or `failed to locate libnvidia-ml.so` | HAMi cannot find the driver libraries under the configured driver root while generating a CDI specification. | + +Confirm the runtime and CDI configuration: + +```bash +kubectl get clusterpolicy -o yaml | grep -A 5 'cdi:' +kubectl get runtimeclass nvidia + +kubectl get pods -n kube-system \ + -l app.kubernetes.io/component=hami-device-plugin \ + -o custom-columns=NAME:.metadata.name,RUNTIMECLASS:.spec.runtimeClassName +``` + +With GPU Operator 25.10+, CDI is normally enabled, the `nvidia` RuntimeClass must exist, and the HAMi Device Plugin must show `nvidia` in the `RUNTIMECLASS` column. + +#### Solution + +Configure the `nvidia` RuntimeClass for HAMi and restart the Device Plugin: + +```bash +helm upgrade hami hami-charts/hami \ + --namespace kube-system \ + --reuse-values \ + --set devicePlugin.runtimeClassName=nvidia + +kubectl rollout restart daemonset/hami-device-plugin -n kube-system +kubectl rollout status daemonset/hami-device-plugin -n kube-system +``` + +If the logs report missing driver libraries while HAMi CDI is enabled, use the GPU Operator paths: + +```yaml +devicePlugin: + runtimeClassName: nvidia + deviceListStrategy: cdi-annotations + nvidiaDriverRoot: /run/nvidia/driver + nvidiaHookPath: /usr/local/nvidia/toolkit/nvidia-ctk +``` + +Wait until the NVIDIA driver and Toolkit DaemonSets are ready, then restart the HAMi Device Plugin. For host-installed drivers, set `devicePlugin.nvidiaDriverRoot` to `/` instead. + +### Problem 2: A HAMi-scheduled Pod fails to start + +#### Identify the cause + +Inspect the Pod events and its assigned RuntimeClass: + +```bash +kubectl describe pod -n +kubectl get pod -n \ + -o custom-columns=NAME:.metadata.name,RUNTIMECLASS:.spec.runtimeClassName +``` + +Use the error text to select the correct path: + +| Error in the Pod events | Cause | +| --- | --- | +| `libcuda.so.1: cannot open shared object file` | The container started without the NVIDIA driver libraries. | +| `unresolvable CDI devices management.nvidia.com/gpu=GPU-...` | The NVIDIA runtime selected a GPU management CDI device, but the corresponding device could not be generated or resolved. | +| `unresolvable CDI devices k8s.device-plugin.nvidia.com/gpu=GPU-...` | HAMi returned a CDI device, but the runtime cannot find a matching HAMi-generated CDI specification. | + +#### Solution + +First determine which HAMi injection mode is configured: + +```bash +helm get values hami -n kube-system | grep -A 5 'devicePlugin:' +``` + +- For the default `devicePlugin.deviceListStrategy=envvar` mode, set `devicePlugin.runtimeClassName=nvidia` by using the Helm command from Problem 1. This makes the NVIDIA runtime process the UUID returned through `NVIDIA_VISIBLE_DEVICES`. +- For `devicePlugin.deviceListStrategy=cdi-annotations`, apply all four CDI values shown in Problem 1. Then inspect `/var/run/cdi/k8s.device-plugin.nvidia.com-gpu.json` on the node and verify that it contains the allocated GPU UUID. +- For a host-installed Container Toolkit, confirm that the `nvidia` runtime is present in the active container runtime configuration. Restart the container runtime after correcting its configuration. + +Do not mix HAMi CDI annotations with a missing or stale HAMi CDI specification. See [NVIDIA CDI support](../installation/configure-cdi.md) for the complete setup and verification procedure. + +### Why this happens + +Starting with [GPU Operator 25.10.0](https://docs.nvidia.com/datacenter/cloud-native/gpu-operator/25.10/cdi.html), CDI is enabled by default and the Operator no longer makes the `nvidia` runtime the default runtime. + +Before 25.10.0, GPU Operator normally configured the NVIDIA runtime as the default. As a result, the NVIDIA runtime hook processed every Pod and injected the devices and driver libraries selected through `NVIDIA_VISIBLE_DEVICES`. + +With 25.10.0 and later, `runc` remains the default runtime and the container runtime uses native CDI for standard Device Plugin workloads. GPU management containers that access GPUs through `NVIDIA_VISIBLE_DEVICES`, including the HAMi Device Plugin, must explicitly use `runtimeClassName: nvidia`. + +HAMi supports two device-injection paths: + +| HAMi mode | Allocation result | Runtime requirement | +| --- | --- | --- | +| `envvar` (default) | HAMi writes the allocated GPU UUID to `NVIDIA_VISIBLE_DEVICES`. | On GPU Operator 25.10+, the Pod must use the `nvidia` RuntimeClass. | +| `cdi-annotations` | HAMi returns a CDI device named `k8s.device-plugin.nvidia.com/gpu=GPU-...` and generates its CDI specification on the node. | The container runtime must have CDI enabled and be able to read the current HAMi specification. | + +The HAMi chart applies `devicePlugin.runtimeClassName` both to the Device Plugin and to NVIDIA workloads mutated by the HAMi scheduler. This is why setting it to `nvidia` fixes the management container and keeps the workload runtime path consistent. + +For new clusters, GPU Operator is recommended because it provides one entry point for configuring and upgrading the driver, Container Toolkit, and monitoring components. If these components are already installed on the hosts and you maintain their runtime configuration yourself, GPU Operator is optional; follow [Prerequisites](../installation/prerequisites.md). + +:::warning + +Disable the GPU Operator Device Plugin when using HAMi. Both plugins advertise `nvidia.com/gpu` and must not run on the same nodes. + +```yaml +devicePlugin: + enabled: false +``` + ::: diff --git a/i18n/zh/docusaurus-plugin-content-blog/hami-meetup-beijing-2025/bagualu-intelligent-computing-stack.webp b/i18n/zh/docusaurus-plugin-content-blog/hami-meetup-beijing-2025/bagualu-intelligent-computing-stack.webp index 50428b435..dfb636952 100644 Binary files a/i18n/zh/docusaurus-plugin-content-blog/hami-meetup-beijing-2025/bagualu-intelligent-computing-stack.webp and b/i18n/zh/docusaurus-plugin-content-blog/hami-meetup-beijing-2025/bagualu-intelligent-computing-stack.webp differ diff --git a/i18n/zh/docusaurus-plugin-content-blog/hami-meetup-beijing-2025/bagualu-performance-delivery.webp b/i18n/zh/docusaurus-plugin-content-blog/hami-meetup-beijing-2025/bagualu-performance-delivery.webp index a43fa3453..a5ca19309 100644 Binary files a/i18n/zh/docusaurus-plugin-content-blog/hami-meetup-beijing-2025/bagualu-performance-delivery.webp and b/i18n/zh/docusaurus-plugin-content-blog/hami-meetup-beijing-2025/bagualu-performance-delivery.webp differ diff --git a/i18n/zh/docusaurus-plugin-content-blog/hami-meetup-beijing-2025/beike-hami-partnership.webp b/i18n/zh/docusaurus-plugin-content-blog/hami-meetup-beijing-2025/beike-hami-partnership.webp index 53b2b8c28..0b74d0059 100644 Binary files a/i18n/zh/docusaurus-plugin-content-blog/hami-meetup-beijing-2025/beike-hami-partnership.webp and b/i18n/zh/docusaurus-plugin-content-blog/hami-meetup-beijing-2025/beike-hami-partnership.webp differ diff --git a/i18n/zh/docusaurus-plugin-content-blog/hami-meetup-beijing-2025/beike-vgpu-inference-cluster-practice.webp b/i18n/zh/docusaurus-plugin-content-blog/hami-meetup-beijing-2025/beike-vgpu-inference-cluster-practice.webp index 2a24d74c8..f92677140 100644 Binary files a/i18n/zh/docusaurus-plugin-content-blog/hami-meetup-beijing-2025/beike-vgpu-inference-cluster-practice.webp and b/i18n/zh/docusaurus-plugin-content-blog/hami-meetup-beijing-2025/beike-vgpu-inference-cluster-practice.webp differ diff --git a/i18n/zh/docusaurus-plugin-content-blog/hami-meetup-beijing-2025/dcu-practice-implementation.webp b/i18n/zh/docusaurus-plugin-content-blog/hami-meetup-beijing-2025/dcu-practice-implementation.webp index 13c9d3300..31e36abd4 100644 Binary files a/i18n/zh/docusaurus-plugin-content-blog/hami-meetup-beijing-2025/dcu-practice-implementation.webp and b/i18n/zh/docusaurus-plugin-content-blog/hami-meetup-beijing-2025/dcu-practice-implementation.webp differ diff --git a/i18n/zh/docusaurus-plugin-content-blog/hami-meetup-beijing-2025/dcu-virtualization-basics.webp b/i18n/zh/docusaurus-plugin-content-blog/hami-meetup-beijing-2025/dcu-virtualization-basics.webp index a689a261b..009e724f1 100644 Binary files a/i18n/zh/docusaurus-plugin-content-blog/hami-meetup-beijing-2025/dcu-virtualization-basics.webp and b/i18n/zh/docusaurus-plugin-content-blog/hami-meetup-beijing-2025/dcu-virtualization-basics.webp differ diff --git a/i18n/zh/docusaurus-plugin-content-blog/hami-meetup-beijing-2025/device-plugin-architecture.webp b/i18n/zh/docusaurus-plugin-content-blog/hami-meetup-beijing-2025/device-plugin-architecture.webp index 85122e9f9..bdee49ca8 100644 Binary files a/i18n/zh/docusaurus-plugin-content-blog/hami-meetup-beijing-2025/device-plugin-architecture.webp and b/i18n/zh/docusaurus-plugin-content-blog/hami-meetup-beijing-2025/device-plugin-architecture.webp differ diff --git a/i18n/zh/docusaurus-plugin-content-blog/hami-meetup-beijing-2025/device-plugin-new-features.webp b/i18n/zh/docusaurus-plugin-content-blog/hami-meetup-beijing-2025/device-plugin-new-features.webp index 523c61692..6d6b30568 100644 Binary files a/i18n/zh/docusaurus-plugin-content-blog/hami-meetup-beijing-2025/device-plugin-new-features.webp and b/i18n/zh/docusaurus-plugin-content-blog/hami-meetup-beijing-2025/device-plugin-new-features.webp differ diff --git a/i18n/zh/docusaurus-plugin-content-blog/hami-meetup-beijing-2025/dra-driver-practice.webp b/i18n/zh/docusaurus-plugin-content-blog/hami-meetup-beijing-2025/dra-driver-practice.webp index 00ab35ed7..686d728c0 100644 Binary files a/i18n/zh/docusaurus-plugin-content-blog/hami-meetup-beijing-2025/dra-driver-practice.webp and b/i18n/zh/docusaurus-plugin-content-blog/hami-meetup-beijing-2025/dra-driver-practice.webp differ diff --git a/i18n/zh/docusaurus-plugin-content-blog/hami-meetup-beijing-2025/hami-community.webp b/i18n/zh/docusaurus-plugin-content-blog/hami-meetup-beijing-2025/hami-community.webp index 8d81c59f4..89505356c 100644 Binary files a/i18n/zh/docusaurus-plugin-content-blog/hami-meetup-beijing-2025/hami-community.webp and b/i18n/zh/docusaurus-plugin-content-blog/hami-meetup-beijing-2025/hami-community.webp differ diff --git a/i18n/zh/docusaurus-plugin-content-blog/hami-meetup-beijing-2025/hami-core-dra-architecture.webp b/i18n/zh/docusaurus-plugin-content-blog/hami-meetup-beijing-2025/hami-core-dra-architecture.webp index b0cbdbdbf..1f2249d67 100644 Binary files a/i18n/zh/docusaurus-plugin-content-blog/hami-meetup-beijing-2025/hami-core-dra-architecture.webp and b/i18n/zh/docusaurus-plugin-content-blog/hami-meetup-beijing-2025/hami-core-dra-architecture.webp differ diff --git a/i18n/zh/docusaurus-plugin-content-blog/hami-meetup-beijing-2025/hami-meetup-beijing-banner.webp b/i18n/zh/docusaurus-plugin-content-blog/hami-meetup-beijing-2025/hami-meetup-beijing-banner.webp index c0acda3f9..18b462617 100644 Binary files a/i18n/zh/docusaurus-plugin-content-blog/hami-meetup-beijing-2025/hami-meetup-beijing-banner.webp and b/i18n/zh/docusaurus-plugin-content-blog/hami-meetup-beijing-2025/hami-meetup-beijing-banner.webp differ diff --git a/i18n/zh/docusaurus-plugin-content-blog/hami-meetup-beijing-2025/hami-new-features-capability-matrix.webp b/i18n/zh/docusaurus-plugin-content-blog/hami-meetup-beijing-2025/hami-new-features-capability-matrix.webp index f72b0f35d..50c612f6f 100644 Binary files a/i18n/zh/docusaurus-plugin-content-blog/hami-meetup-beijing-2025/hami-new-features-capability-matrix.webp and b/i18n/zh/docusaurus-plugin-content-blog/hami-meetup-beijing-2025/hami-new-features-capability-matrix.webp differ diff --git a/i18n/zh/docusaurus-plugin-content-blog/hami-meetup-beijing-2025/hami-v2.7.0-domestic-compute.webp b/i18n/zh/docusaurus-plugin-content-blog/hami-meetup-beijing-2025/hami-v2.7.0-domestic-compute.webp index 35908b1ec..19d56a868 100644 Binary files a/i18n/zh/docusaurus-plugin-content-blog/hami-meetup-beijing-2025/hami-v2.7.0-domestic-compute.webp and b/i18n/zh/docusaurus-plugin-content-blog/hami-meetup-beijing-2025/hami-v2.7.0-domestic-compute.webp differ diff --git a/i18n/zh/docusaurus-plugin-content-blog/hami-meetup-beijing-2025/kunlunxin-xpu-adapter.webp b/i18n/zh/docusaurus-plugin-content-blog/hami-meetup-beijing-2025/kunlunxin-xpu-adapter.webp index 824b7d1e1..36010cf54 100644 Binary files a/i18n/zh/docusaurus-plugin-content-blog/hami-meetup-beijing-2025/kunlunxin-xpu-adapter.webp and b/i18n/zh/docusaurus-plugin-content-blog/hami-meetup-beijing-2025/kunlunxin-xpu-adapter.webp differ diff --git a/i18n/zh/docusaurus-plugin-content-blog/hami-meetup-beijing-2025/meetup-networking-session.webp b/i18n/zh/docusaurus-plugin-content-blog/hami-meetup-beijing-2025/meetup-networking-session.webp index 6f1937a8a..6b8340dab 100644 Binary files a/i18n/zh/docusaurus-plugin-content-blog/hami-meetup-beijing-2025/meetup-networking-session.webp and b/i18n/zh/docusaurus-plugin-content-blog/hami-meetup-beijing-2025/meetup-networking-session.webp differ diff --git a/i18n/zh/docusaurus-plugin-content-blog/hami-meetup-shanghai-2025/daocloud-drun-practice.png b/i18n/zh/docusaurus-plugin-content-blog/hami-meetup-shanghai-2025/daocloud-drun-practice.png index 2685ea960..c3c5aa9cb 100644 Binary files a/i18n/zh/docusaurus-plugin-content-blog/hami-meetup-shanghai-2025/daocloud-drun-practice.png and b/i18n/zh/docusaurus-plugin-content-blog/hami-meetup-shanghai-2025/daocloud-drun-practice.png differ diff --git a/i18n/zh/docusaurus-plugin-content-blog/hami-meetup-shanghai-2025/hami-version-evolution.png b/i18n/zh/docusaurus-plugin-content-blog/hami-meetup-shanghai-2025/hami-version-evolution.png index 6a4254383..90f007cd9 100644 Binary files a/i18n/zh/docusaurus-plugin-content-blog/hami-meetup-shanghai-2025/hami-version-evolution.png and b/i18n/zh/docusaurus-plugin-content-blog/hami-meetup-shanghai-2025/hami-version-evolution.png differ diff --git a/i18n/zh/docusaurus-plugin-content-blog/hami-meetup-shanghai-2025/keith-opening-keynote.png b/i18n/zh/docusaurus-plugin-content-blog/hami-meetup-shanghai-2025/keith-opening-keynote.png index a834eec7e..e937abc71 100644 Binary files a/i18n/zh/docusaurus-plugin-content-blog/hami-meetup-shanghai-2025/keith-opening-keynote.png and b/i18n/zh/docusaurus-plugin-content-blog/hami-meetup-shanghai-2025/keith-opening-keynote.png differ diff --git a/i18n/zh/docusaurus-plugin-content-blog/hami-meetup-shanghai-2025/meetup-banner.png b/i18n/zh/docusaurus-plugin-content-blog/hami-meetup-shanghai-2025/meetup-banner.png index a5c4db4d5..864131b7d 100644 Binary files a/i18n/zh/docusaurus-plugin-content-blog/hami-meetup-shanghai-2025/meetup-banner.png and b/i18n/zh/docusaurus-plugin-content-blog/hami-meetup-shanghai-2025/meetup-banner.png differ diff --git a/i18n/zh/docusaurus-plugin-content-blog/hami-meetup-shanghai-2025/metax-sgpu-hami.png b/i18n/zh/docusaurus-plugin-content-blog/hami-meetup-shanghai-2025/metax-sgpu-hami.png index 7e35fc486..c944e3679 100644 Binary files a/i18n/zh/docusaurus-plugin-content-blog/hami-meetup-shanghai-2025/metax-sgpu-hami.png and b/i18n/zh/docusaurus-plugin-content-blog/hami-meetup-shanghai-2025/metax-sgpu-hami.png differ diff --git a/i18n/zh/docusaurus-plugin-content-blog/hami-meetup-shanghai-2025/networking-session.png b/i18n/zh/docusaurus-plugin-content-blog/hami-meetup-shanghai-2025/networking-session.png index e8de65cb3..7252039e1 100644 Binary files a/i18n/zh/docusaurus-plugin-content-blog/hami-meetup-shanghai-2025/networking-session.png and b/i18n/zh/docusaurus-plugin-content-blog/hami-meetup-shanghai-2025/networking-session.png differ diff --git a/i18n/zh/docusaurus-plugin-content-blog/hami-meetup-shanghai-2025/performance-optimization.png b/i18n/zh/docusaurus-plugin-content-blog/hami-meetup-shanghai-2025/performance-optimization.png index 8bead6ad2..d47849d14 100644 Binary files a/i18n/zh/docusaurus-plugin-content-blog/hami-meetup-shanghai-2025/performance-optimization.png and b/i18n/zh/docusaurus-plugin-content-blog/hami-meetup-shanghai-2025/performance-optimization.png differ diff --git a/i18n/zh/docusaurus-plugin-content-blog/hami-meetup-shanghai-2025/transwarp-technology.png b/i18n/zh/docusaurus-plugin-content-blog/hami-meetup-shanghai-2025/transwarp-technology.png index 9e449c2e1..f969b84b3 100644 Binary files a/i18n/zh/docusaurus-plugin-content-blog/hami-meetup-shanghai-2025/transwarp-technology.png and b/i18n/zh/docusaurus-plugin-content-blog/hami-meetup-shanghai-2025/transwarp-technology.png differ diff --git a/i18n/zh/docusaurus-plugin-content-blog/hami-meetup-shenzhen-2026/meetup-shenzhen-group-photo.webp b/i18n/zh/docusaurus-plugin-content-blog/hami-meetup-shenzhen-2026/meetup-shenzhen-group-photo.webp index 7c944a136..75d263d6d 100644 Binary files a/i18n/zh/docusaurus-plugin-content-blog/hami-meetup-shenzhen-2026/meetup-shenzhen-group-photo.webp and b/i18n/zh/docusaurus-plugin-content-blog/hami-meetup-shenzhen-2026/meetup-shenzhen-group-photo.webp differ diff --git a/i18n/zh/docusaurus-plugin-content-blog/hami-meetup-shenzhen-2026/meetup-shenzhen-video-qr.png b/i18n/zh/docusaurus-plugin-content-blog/hami-meetup-shenzhen-2026/meetup-shenzhen-video-qr.png index 0a3c837c3..9db82fd6e 100644 Binary files a/i18n/zh/docusaurus-plugin-content-blog/hami-meetup-shenzhen-2026/meetup-shenzhen-video-qr.png and b/i18n/zh/docusaurus-plugin-content-blog/hami-meetup-shenzhen-2026/meetup-shenzhen-video-qr.png differ diff --git a/sidebars-tutorials.js b/sidebars-tutorials.js index 0b44c90a7..3a8af9314 100644 --- a/sidebars-tutorials.js +++ b/sidebars-tutorials.js @@ -41,6 +41,11 @@ module.exports = { id: "labs/hami-vllm", customProps: { level: "Intermediate", duration: "about 45 minutes" }, }, + { + type: "doc", + id: "labs/hami-sglang", + customProps: { level: "Intermediate", duration: "about 45 minutes" }, + }, { type: "doc", id: "labs/hami-isolation-k3s", diff --git a/static/img/adopters/ashermed.png b/static/img/adopters/ashermed.png index cff1b7a70..3c0add592 100644 Binary files a/static/img/adopters/ashermed.png and b/static/img/adopters/ashermed.png differ diff --git a/static/img/adopters/baidu-cloud.png b/static/img/adopters/baidu-cloud.png index 4ddc67dba..b3c02d64a 100644 Binary files a/static/img/adopters/baidu-cloud.png and b/static/img/adopters/baidu-cloud.png differ diff --git a/static/img/adopters/baihai.jpg b/static/img/adopters/baihai.jpg index 6064f97f6..39febef42 100644 Binary files a/static/img/adopters/baihai.jpg and b/static/img/adopters/baihai.jpg differ diff --git a/static/img/adopters/boardware.jpg b/static/img/adopters/boardware.jpg index 0adb0b2bd..69836b809 100644 Binary files a/static/img/adopters/boardware.jpg and b/static/img/adopters/boardware.jpg differ diff --git a/static/img/adopters/bonc.png b/static/img/adopters/bonc.png index 5022d349d..20ad5cde4 100644 Binary files a/static/img/adopters/bonc.png and b/static/img/adopters/bonc.png differ diff --git a/static/img/adopters/cetc-zh.png b/static/img/adopters/cetc-zh.png index cbb8fed7e..dc71c8a4d 100644 Binary files a/static/img/adopters/cetc-zh.png and b/static/img/adopters/cetc-zh.png differ diff --git a/static/img/adopters/cetc.png b/static/img/adopters/cetc.png index dce2b1105..b6a2a275a 100644 Binary files a/static/img/adopters/cetc.png and b/static/img/adopters/cetc.png differ diff --git a/static/img/adopters/china-merchants-bank.png b/static/img/adopters/china-merchants-bank.png index 28886fbcd..9a63f563e 100644 Binary files a/static/img/adopters/china-merchants-bank.png and b/static/img/adopters/china-merchants-bank.png differ diff --git a/static/img/adopters/china-mobile.png b/static/img/adopters/china-mobile.png index 528eec82c..3b2dc2589 100644 Binary files a/static/img/adopters/china-mobile.png and b/static/img/adopters/china-mobile.png differ diff --git a/static/img/adopters/china-unicom.png b/static/img/adopters/china-unicom.png index 71cdf4793..80f24402d 100644 Binary files a/static/img/adopters/china-unicom.png and b/static/img/adopters/china-unicom.png differ diff --git a/static/img/adopters/china-university-of-mining-and-technology.jpg b/static/img/adopters/china-university-of-mining-and-technology.jpg index 8c9b84ac1..6ee6aba65 100644 Binary files a/static/img/adopters/china-university-of-mining-and-technology.jpg and b/static/img/adopters/china-university-of-mining-and-technology.jpg differ diff --git a/static/img/adopters/coocaa.png b/static/img/adopters/coocaa.png index e05232369..7a454c359 100644 Binary files a/static/img/adopters/coocaa.png and b/static/img/adopters/coocaa.png differ diff --git a/static/img/adopters/cyintell.png b/static/img/adopters/cyintell.png index 7d60d0ee6..2a461ecb9 100644 Binary files a/static/img/adopters/cyintell.png and b/static/img/adopters/cyintell.png differ diff --git a/static/img/adopters/deeproute.png b/static/img/adopters/deeproute.png index 97d4b2930..65894902d 100644 Binary files a/static/img/adopters/deeproute.png and b/static/img/adopters/deeproute.png differ diff --git a/static/img/adopters/dialo.png b/static/img/adopters/dialo.png index 1d01ea26e..9781b6428 100644 Binary files a/static/img/adopters/dialo.png and b/static/img/adopters/dialo.png differ diff --git a/static/img/adopters/donghua-university.png b/static/img/adopters/donghua-university.png index 339bcb65b..809a030aa 100644 Binary files a/static/img/adopters/donghua-university.png and b/static/img/adopters/donghua-university.png differ diff --git a/static/img/adopters/ecloud.jpg b/static/img/adopters/ecloud.jpg index b97fcabe8..733bcab56 100644 Binary files a/static/img/adopters/ecloud.jpg and b/static/img/adopters/ecloud.jpg differ diff --git a/static/img/adopters/empathy.png b/static/img/adopters/empathy.png index 50b5b796a..4b0fd5f94 100644 Binary files a/static/img/adopters/empathy.png and b/static/img/adopters/empathy.png differ diff --git a/static/img/adopters/ghostcloud.png b/static/img/adopters/ghostcloud.png index 5926de18f..87fb164f5 100644 Binary files a/static/img/adopters/ghostcloud.png and b/static/img/adopters/ghostcloud.png differ diff --git a/static/img/adopters/gsafety.jpg b/static/img/adopters/gsafety.jpg index e84d20c58..5b9d9ac98 100644 Binary files a/static/img/adopters/gsafety.jpg and b/static/img/adopters/gsafety.jpg differ diff --git a/static/img/adopters/guangdong-university-of-technology.png b/static/img/adopters/guangdong-university-of-technology.png index 54e8a9b9d..208d45cc5 100644 Binary files a/static/img/adopters/guangdong-university-of-technology.png and b/static/img/adopters/guangdong-university-of-technology.png differ diff --git a/static/img/adopters/guangzhou-pingao.jpg b/static/img/adopters/guangzhou-pingao.jpg index 77a592cdb..bc8c6bcec 100644 Binary files a/static/img/adopters/guangzhou-pingao.jpg and b/static/img/adopters/guangzhou-pingao.jpg differ diff --git a/static/img/adopters/h3c.png b/static/img/adopters/h3c.png index 06f5a0903..acfe50124 100644 Binary files a/static/img/adopters/h3c.png and b/static/img/adopters/h3c.png differ diff --git a/static/img/adopters/hangzhou-lianhui.png b/static/img/adopters/hangzhou-lianhui.png index 00d5a60cb..d553ae856 100644 Binary files a/static/img/adopters/hangzhou-lianhui.png and b/static/img/adopters/hangzhou-lianhui.png differ diff --git a/static/img/adopters/haofan.jpg b/static/img/adopters/haofan.jpg index 4e92eb780..a6f934295 100644 Binary files a/static/img/adopters/haofan.jpg and b/static/img/adopters/haofan.jpg differ diff --git a/static/img/adopters/harbin-institute-of-technology.png b/static/img/adopters/harbin-institute-of-technology.png index 6c7ff920f..c1960fc24 100644 Binary files a/static/img/adopters/harbin-institute-of-technology.png and b/static/img/adopters/harbin-institute-of-technology.png differ diff --git a/static/img/adopters/i-tudou.png b/static/img/adopters/i-tudou.png index cfdaefbbe..21ad15278 100644 Binary files a/static/img/adopters/i-tudou.png and b/static/img/adopters/i-tudou.png differ diff --git a/static/img/adopters/infervision.png b/static/img/adopters/infervision.png index 320ecb75a..518eca4c6 100644 Binary files a/static/img/adopters/infervision.png and b/static/img/adopters/infervision.png differ diff --git a/static/img/adopters/institute-of-information-engineering-cas.png b/static/img/adopters/institute-of-information-engineering-cas.png index 7461fdc15..f08b70c55 100644 Binary files a/static/img/adopters/institute-of-information-engineering-cas.png and b/static/img/adopters/institute-of-information-engineering-cas.png differ diff --git a/static/img/adopters/kylinsoft.png b/static/img/adopters/kylinsoft.png index 415ec82d9..8ca7fc062 100644 Binary files a/static/img/adopters/kylinsoft.png and b/static/img/adopters/kylinsoft.png differ diff --git a/static/img/adopters/linkedin.png b/static/img/adopters/linkedin.png index 4e4f8b134..6c92b895e 100644 Binary files a/static/img/adopters/linkedin.png and b/static/img/adopters/linkedin.png differ diff --git a/static/img/adopters/linklogis.jpg b/static/img/adopters/linklogis.jpg index 7c65f84fd..53f0d1e06 100644 Binary files a/static/img/adopters/linklogis.jpg and b/static/img/adopters/linklogis.jpg differ diff --git a/static/img/adopters/miaoyun.png b/static/img/adopters/miaoyun.png index 3c804c33d..08a382f12 100644 Binary files a/static/img/adopters/miaoyun.png and b/static/img/adopters/miaoyun.png differ diff --git a/static/img/adopters/msxf.png b/static/img/adopters/msxf.png index ef194d98d..33aaa684f 100644 Binary files a/static/img/adopters/msxf.png and b/static/img/adopters/msxf.png differ diff --git a/static/img/adopters/nankai-university.png b/static/img/adopters/nankai-university.png index 768006491..589579c52 100644 Binary files a/static/img/adopters/nankai-university.png and b/static/img/adopters/nankai-university.png differ diff --git a/static/img/adopters/northsoft.png b/static/img/adopters/northsoft.png index 42a0fc086..6eccf66cd 100644 Binary files a/static/img/adopters/northsoft.png and b/static/img/adopters/northsoft.png differ diff --git a/static/img/adopters/pingan-bank.png b/static/img/adopters/pingan-bank.png index 953bb5c56..5af0f9be3 100644 Binary files a/static/img/adopters/pingan-bank.png and b/static/img/adopters/pingan-bank.png differ diff --git a/static/img/adopters/pingan-securities.png b/static/img/adopters/pingan-securities.png index 1df89a0ed..a00d6064e 100644 Binary files a/static/img/adopters/pingan-securities.png and b/static/img/adopters/pingan-securities.png differ diff --git a/static/img/adopters/ppio.png b/static/img/adopters/ppio.png index e98bbc93a..399abc6b9 100644 Binary files a/static/img/adopters/ppio.png and b/static/img/adopters/ppio.png differ diff --git a/static/img/adopters/ragehealth.png b/static/img/adopters/ragehealth.png index 84b2d1ba0..43065a5c8 100644 Binary files a/static/img/adopters/ragehealth.png and b/static/img/adopters/ragehealth.png differ diff --git a/static/img/adopters/sangfor.jpg b/static/img/adopters/sangfor.jpg index 9f2fccca2..3dd74b225 100644 Binary files a/static/img/adopters/sangfor.jpg and b/static/img/adopters/sangfor.jpg differ diff --git a/static/img/adopters/sap.png b/static/img/adopters/sap.png index 647e917f7..59015a45c 100644 Binary files a/static/img/adopters/sap.png and b/static/img/adopters/sap.png differ diff --git a/static/img/adopters/si-tech.png b/static/img/adopters/si-tech.png index f23179582..e625d5308 100644 Binary files a/static/img/adopters/si-tech.png and b/static/img/adopters/si-tech.png differ diff --git a/static/img/adopters/sinochem.png b/static/img/adopters/sinochem.png index 8fea9ca72..cded2bcec 100644 Binary files a/static/img/adopters/sinochem.png and b/static/img/adopters/sinochem.png differ diff --git a/static/img/adopters/snow.png b/static/img/adopters/snow.png index 35b561891..99326496b 100644 Binary files a/static/img/adopters/snow.png and b/static/img/adopters/snow.png differ diff --git a/static/img/adopters/southeast-university.png b/static/img/adopters/southeast-university.png index 459a7375f..266b035c4 100644 Binary files a/static/img/adopters/southeast-university.png and b/static/img/adopters/southeast-university.png differ diff --git a/static/img/adopters/szzt.png b/static/img/adopters/szzt.png index 5541a289f..bfbd7e971 100644 Binary files a/static/img/adopters/szzt.png and b/static/img/adopters/szzt.png differ diff --git a/static/img/adopters/tongcheng-travel.png b/static/img/adopters/tongcheng-travel.png index 4476c9ca9..61c1f2f38 100644 Binary files a/static/img/adopters/tongcheng-travel.png and b/static/img/adopters/tongcheng-travel.png differ diff --git a/static/img/adopters/tum.png b/static/img/adopters/tum.png index 1fd6c0c8a..4e059436b 100644 Binary files a/static/img/adopters/tum.png and b/static/img/adopters/tum.png differ diff --git a/static/img/adopters/ucloud.png b/static/img/adopters/ucloud.png index c439d0ac2..0374f7caa 100644 Binary files a/static/img/adopters/ucloud.png and b/static/img/adopters/ucloud.png differ diff --git a/static/img/adopters/unicdata.png b/static/img/adopters/unicdata.png index 226c6c0ee..6f8326755 100644 Binary files a/static/img/adopters/unicdata.png and b/static/img/adopters/unicdata.png differ diff --git a/static/img/adopters/viettel.png b/static/img/adopters/viettel.png index d1139ce7f..aa7168486 100644 Binary files a/static/img/adopters/viettel.png and b/static/img/adopters/viettel.png differ diff --git a/static/img/adopters/weibo.png b/static/img/adopters/weibo.png index 1384972e6..509d94e86 100644 Binary files a/static/img/adopters/weibo.png and b/static/img/adopters/weibo.png differ diff --git a/static/img/adopters/weidian.png b/static/img/adopters/weidian.png index 3cfc40172..99c728e89 100644 Binary files a/static/img/adopters/weidian.png and b/static/img/adopters/weidian.png differ diff --git a/static/img/adopters/woqu.png b/static/img/adopters/woqu.png index 51e50959c..6bde99e9e 100644 Binary files a/static/img/adopters/woqu.png and b/static/img/adopters/woqu.png differ diff --git a/static/img/adopters/xuanyuan-network.png b/static/img/adopters/xuanyuan-network.png index e19d7b328..7629a9514 100644 Binary files a/static/img/adopters/xuanyuan-network.png and b/static/img/adopters/xuanyuan-network.png differ diff --git a/static/img/adopters/zstack.png b/static/img/adopters/zstack.png index 8d376029e..7c96b2f7a 100644 Binary files a/static/img/adopters/zstack.png and b/static/img/adopters/zstack.png differ diff --git a/static/img/blog-hami-at-kubecon-eu-2026-booth.png b/static/img/blog-hami-at-kubecon-eu-2026-booth.png index bfa7c0814..ab5a11192 100644 Binary files a/static/img/blog-hami-at-kubecon-eu-2026-booth.png and b/static/img/blog-hami-at-kubecon-eu-2026-booth.png differ diff --git a/static/img/blog-hami-at-kubecon-eu-2026-kubecon.png b/static/img/blog-hami-at-kubecon-eu-2026-kubecon.png index 77274f4f3..bafaee420 100644 Binary files a/static/img/blog-hami-at-kubecon-eu-2026-kubecon.png and b/static/img/blog-hami-at-kubecon-eu-2026-kubecon.png differ diff --git a/static/img/blog/flowchart.jpeg b/static/img/blog/flowchart.jpeg index 1cbe0a590..34fd31bc2 100644 Binary files a/static/img/blog/flowchart.jpeg and b/static/img/blog/flowchart.jpeg differ diff --git a/static/img/blog/hami-pod-scheduler-preparation.png b/static/img/blog/hami-pod-scheduler-preparation.png index 307779b64..35a4bf8c6 100644 Binary files a/static/img/blog/hami-pod-scheduler-preparation.png and b/static/img/blog/hami-pod-scheduler-preparation.png differ diff --git a/static/img/case-studies/ke-holdings.png b/static/img/case-studies/ke-holdings.png index 163159b9d..22dddb2dc 100644 Binary files a/static/img/case-studies/ke-holdings.png and b/static/img/case-studies/ke-holdings.png differ diff --git a/static/img/case-studies/snow-corp.webp b/static/img/case-studies/snow-corp.webp index 4214264cc..e758b57b2 100644 Binary files a/static/img/case-studies/snow-corp.webp and b/static/img/case-studies/snow-corp.webp differ diff --git a/static/img/community/wechat-assistant-qr.jpg b/static/img/community/wechat-assistant-qr.jpg index 65e77d75b..a6ff8dfbc 100644 Binary files a/static/img/community/wechat-assistant-qr.jpg and b/static/img/community/wechat-assistant-qr.jpg differ diff --git a/static/img/community/wechat-official-account-qr.jpg b/static/img/community/wechat-official-account-qr.jpg index 2487afca5..00787ad07 100644 Binary files a/static/img/community/wechat-official-account-qr.jpg and b/static/img/community/wechat-official-account-qr.jpg differ diff --git a/static/img/community/wechat-video-channel-qr.jpg b/static/img/community/wechat-video-channel-qr.jpg index 8b53e95cd..8fbd46375 100644 Binary files a/static/img/community/wechat-video-channel-qr.jpg and b/static/img/community/wechat-video-channel-qr.jpg differ diff --git a/static/img/contributors/alauda.jpg b/static/img/contributors/alauda.jpg index 3a1ea05d2..7dca7808e 100644 Binary files a/static/img/contributors/alauda.jpg and b/static/img/contributors/alauda.jpg differ diff --git a/static/img/contributors/bit.png b/static/img/contributors/bit.png index 5b2c426af..99fe1846d 100644 Binary files a/static/img/contributors/bit.png and b/static/img/contributors/bit.png differ diff --git a/static/img/contributors/caih.png b/static/img/contributors/caih.png index ed88cfb89..24f79e067 100644 Binary files a/static/img/contributors/caih.png and b/static/img/contributors/caih.png differ diff --git a/static/img/contributors/glovo.png b/static/img/contributors/glovo.png index 2a2b4de86..c0fdf3d76 100644 Binary files a/static/img/contributors/glovo.png and b/static/img/contributors/glovo.png differ diff --git a/static/img/contributors/hygon.png b/static/img/contributors/hygon.png index 74b432881..108322645 100644 Binary files a/static/img/contributors/hygon.png and b/static/img/contributors/hygon.png differ diff --git a/static/img/contributors/iluvatar.png b/static/img/contributors/iluvatar.png index 5fbc8327b..03e9c7df3 100644 Binary files a/static/img/contributors/iluvatar.png and b/static/img/contributors/iluvatar.png differ diff --git a/static/img/contributors/iqiy.png b/static/img/contributors/iqiy.png index 107f6b0e7..1ab17bcf6 100644 Binary files a/static/img/contributors/iqiy.png and b/static/img/contributors/iqiy.png differ diff --git a/static/img/contributors/jd-zh.png b/static/img/contributors/jd-zh.png index 64d5e9a76..fccb2b389 100644 Binary files a/static/img/contributors/jd-zh.png and b/static/img/contributors/jd-zh.png differ diff --git a/static/img/contributors/kunlunxin.jpg b/static/img/contributors/kunlunxin.jpg index 62f1c744f..d308da39d 100644 Binary files a/static/img/contributors/kunlunxin.jpg and b/static/img/contributors/kunlunxin.jpg differ diff --git a/static/img/contributors/memverge.png b/static/img/contributors/memverge.png index 40b14de89..0d526593e 100644 Binary files a/static/img/contributors/memverge.png and b/static/img/contributors/memverge.png differ diff --git a/static/img/contributors/metax.png b/static/img/contributors/metax.png index 45d580e2e..6eba87044 100644 Binary files a/static/img/contributors/metax.png and b/static/img/contributors/metax.png differ diff --git a/static/img/contributors/mthread.png b/static/img/contributors/mthread.png index 7d3c5d9bb..d8f8f783f 100644 Binary files a/static/img/contributors/mthread.png and b/static/img/contributors/mthread.png differ diff --git a/static/img/contributors/phancy.png b/static/img/contributors/phancy.png index b004ae263..a3d517e64 100644 Binary files a/static/img/contributors/phancy.png and b/static/img/contributors/phancy.png differ diff --git a/static/img/vllm-meetup-shanghai-2026-recap/li-mengxuan-speaking.webp b/static/img/vllm-meetup-shanghai-2026-recap/li-mengxuan-speaking.webp index 7a780067c..b32a97726 100644 Binary files a/static/img/vllm-meetup-shanghai-2026-recap/li-mengxuan-speaking.webp and b/static/img/vllm-meetup-shanghai-2026-recap/li-mengxuan-speaking.webp differ diff --git a/static/img/vllm-meetup-shanghai-2026-recap/phase0-code.webp b/static/img/vllm-meetup-shanghai-2026-recap/phase0-code.webp index 943d77310..f3dc18cbc 100644 Binary files a/static/img/vllm-meetup-shanghai-2026-recap/phase0-code.webp and b/static/img/vllm-meetup-shanghai-2026-recap/phase0-code.webp differ diff --git a/static/img/vllm-meetup-shanghai-2026-recap/phase0-overview.webp b/static/img/vllm-meetup-shanghai-2026-recap/phase0-overview.webp index 88022fa88..9e36a24b6 100644 Binary files a/static/img/vllm-meetup-shanghai-2026-recap/phase0-overview.webp and b/static/img/vllm-meetup-shanghai-2026-recap/phase0-overview.webp differ diff --git a/static/img/vllm-meetup-shanghai-2026-recap/phase1-k8s.webp b/static/img/vllm-meetup-shanghai-2026-recap/phase1-k8s.webp index 52681195d..bc1b1bae8 100644 Binary files a/static/img/vllm-meetup-shanghai-2026-recap/phase1-k8s.webp and b/static/img/vllm-meetup-shanghai-2026-recap/phase1-k8s.webp differ diff --git a/static/img/vllm-meetup-shanghai-2026-recap/phase1-overview.webp b/static/img/vllm-meetup-shanghai-2026-recap/phase1-overview.webp index a69dee6d5..6f7d20f30 100644 Binary files a/static/img/vllm-meetup-shanghai-2026-recap/phase1-overview.webp and b/static/img/vllm-meetup-shanghai-2026-recap/phase1-overview.webp differ diff --git a/static/img/vllm-meetup-shanghai-2026-recap/phase2-llmd-detail.webp b/static/img/vllm-meetup-shanghai-2026-recap/phase2-llmd-detail.webp index 35fbc44bf..bff28f237 100644 Binary files a/static/img/vllm-meetup-shanghai-2026-recap/phase2-llmd-detail.webp and b/static/img/vllm-meetup-shanghai-2026-recap/phase2-llmd-detail.webp differ diff --git a/static/img/vllm-meetup-shanghai-2026-recap/phase2-llmd.webp b/static/img/vllm-meetup-shanghai-2026-recap/phase2-llmd.webp index f5fb6cdd6..a91727b17 100644 Binary files a/static/img/vllm-meetup-shanghai-2026-recap/phase2-llmd.webp and b/static/img/vllm-meetup-shanghai-2026-recap/phase2-llmd.webp differ diff --git a/static/img/vllm-meetup-shanghai-2026-recap/phase2-mooncake-code.webp b/static/img/vllm-meetup-shanghai-2026-recap/phase2-mooncake-code.webp index 8248a26cc..9de5cb5b7 100644 Binary files a/static/img/vllm-meetup-shanghai-2026-recap/phase2-mooncake-code.webp and b/static/img/vllm-meetup-shanghai-2026-recap/phase2-mooncake-code.webp differ diff --git a/static/img/vllm-meetup-shanghai-2026-recap/phase2-mooncake.webp b/static/img/vllm-meetup-shanghai-2026-recap/phase2-mooncake.webp index b91ac08af..c2a1f20a4 100644 Binary files a/static/img/vllm-meetup-shanghai-2026-recap/phase2-mooncake.webp and b/static/img/vllm-meetup-shanghai-2026-recap/phase2-mooncake.webp differ diff --git a/static/img/vllm-meetup-shanghai-2026-recap/phase2-nccl.webp b/static/img/vllm-meetup-shanghai-2026-recap/phase2-nccl.webp index 0a86d912f..4794a7c39 100644 Binary files a/static/img/vllm-meetup-shanghai-2026-recap/phase2-nccl.webp and b/static/img/vllm-meetup-shanghai-2026-recap/phase2-nccl.webp differ diff --git a/static/img/vllm-meetup-shanghai-2026-recap/phase2-pd.webp b/static/img/vllm-meetup-shanghai-2026-recap/phase2-pd.webp index 7f56f8457..fc0b755c9 100644 Binary files a/static/img/vllm-meetup-shanghai-2026-recap/phase2-pd.webp and b/static/img/vllm-meetup-shanghai-2026-recap/phase2-pd.webp differ diff --git a/static/img/vllm-meetup-shanghai-2026-recap/phase3-architecture.webp b/static/img/vllm-meetup-shanghai-2026-recap/phase3-architecture.webp index 7881f3a81..dbb023e14 100644 Binary files a/static/img/vllm-meetup-shanghai-2026-recap/phase3-architecture.webp and b/static/img/vllm-meetup-shanghai-2026-recap/phase3-architecture.webp differ diff --git a/static/img/vllm-meetup-shanghai-2026-recap/phase3-overview.webp b/static/img/vllm-meetup-shanghai-2026-recap/phase3-overview.webp index 417d00ca8..97a84e0d8 100644 Binary files a/static/img/vllm-meetup-shanghai-2026-recap/phase3-overview.webp and b/static/img/vllm-meetup-shanghai-2026-recap/phase3-overview.webp differ diff --git a/static/img/vllm-meetup-shanghai-2026-recap/phase3-result.webp b/static/img/vllm-meetup-shanghai-2026-recap/phase3-result.webp index 4771af23a..00fbc7369 100644 Binary files a/static/img/vllm-meetup-shanghai-2026-recap/phase3-result.webp and b/static/img/vllm-meetup-shanghai-2026-recap/phase3-result.webp differ diff --git a/static/img/vllm-meetup-shanghai-2026-recap/title.webp b/static/img/vllm-meetup-shanghai-2026-recap/title.webp index 4a4775762..f7ea94b59 100644 Binary files a/static/img/vllm-meetup-shanghai-2026-recap/title.webp and b/static/img/vllm-meetup-shanghai-2026-recap/title.webp differ diff --git a/static/img/vllm-meetup-shanghai-2026-recap/vllm-meetup-group-photo.webp b/static/img/vllm-meetup-shanghai-2026-recap/vllm-meetup-group-photo.webp index d3f48ac12..6514b9b37 100644 Binary files a/static/img/vllm-meetup-shanghai-2026-recap/vllm-meetup-group-photo.webp and b/static/img/vllm-meetup-shanghai-2026-recap/vllm-meetup-group-photo.webp differ diff --git a/tutorials/labs/hami-sglang.md b/tutorials/labs/hami-sglang.md new file mode 100644 index 000000000..d88f29e50 --- /dev/null +++ b/tutorials/labs/hami-sglang.md @@ -0,0 +1,186 @@ +--- +title: "Lab 7: Run SGLang on HAMi GPU Shares" +description: "Install HAMi on a GPU cluster and schedule SGLang inference services with GPU partitioning." +sidebar_label: "Lab 7: SGLang Inference" +lab: + level: Intermediate + duration: about 45 minutes + environment: Kubernetes cluster with NVIDIA GPUs +tags: + - sglang + - inference +toc_max_heading_level: 2 +--- + +This lab demonstrates how to deploy [SGLang](https://sgl-project.github.io/), a high-performance LLM serving framework optimized for RadixAttention, on a Kubernetes cluster using HAMi for GPU memory and compute isolation. Upon completion, you will have an OpenAI-compatible model service running SGLang on a partitioned GPU. + +## Learning Objectives + +- Understand the benefits of SGLang for high-throughput inference +- Run SGLang using HAMi's `nvidia.com/gpu`, `nvidia.com/gpumem`, and `nvidia.com/gpucores` resources +- Test the SGLang OpenAI-compatible API via port forwarding + +## Lab Overview + +```mermaid +%% title: HAMi + SGLang Lab Flowchart +flowchart LR + Step1["Step 1
Check GPU Cluster"] --> Step2["Step 2
Install HAMi"] + Step2 --> Step3["Step 3
Deploy SGLang"] + Step3 --> Step4["Step 4
Test Inference"] + Step4 --> Step5["Step 5
Cleanup"] +``` + +## Deployment Architecture + +```mermaid +%% title: HAMi + SGLang Deployment Architecture +flowchart TB + Client["Client
curl / OpenAI SDK"] --> SVC["SGLang Service
port 30000"] + SVC --> P1["SGLang Pod 1
Meta-Llama-3-8B-Instruct
1 GPU slot / 24 GiB"] + SVC --> P2["SGLang Pod 2
Meta-Llama-3-8B-Instruct
1 GPU slot / 24 GiB"] + + subgraph K8S["Kubernetes GPU Cluster"] + HAMI_S["hami-scheduler"] + HAMI_D["hami-device-plugin"] + P1 + P2 + N1["GPU Node 1
NVIDIA A100"] + end + + HAMI_S --> P1 + HAMI_S --> P2 + HAMI_D --> N1 + P1 --> N1 + P2 --> N1 +``` + +## Prerequisites + +- A working Kubernetes cluster with HAMi installed +- At least 1 NVIDIA GPU node with sufficient memory for the target model +- `kubectl` connected to the cluster +- The cluster can pull SGLang images (`lmsysorg/sglang:latest`) and access Hugging Face models + +## Step 1: Create the SGLang Deployment + +We will create a Kubernetes Deployment for SGLang, requesting a specific partition of the GPU using HAMi's extended resources. + +Save the following YAML as `sglang-deployment.yaml`: + +```yaml +apiVersion: apps/v1 +kind: Deployment +metadata: + name: sglang-llama3 + labels: + app: sglang +spec: + replicas: 1 + selector: + matchLabels: + app: sglang + template: + metadata: + labels: + app: sglang + spec: + containers: + - name: sglang + image: lmsysorg/sglang:latest + command: + - python3 + - "-m" + - "sglang.launch_server" + - "--model-path" + - "meta-llama/Meta-Llama-3-8B-Instruct" + - "--host" + - "0.0.0.0" + - "--port" + - "30000" + ports: + - containerPort: 30000 + resources: + limits: + nvidia.com/gpu: 1 + nvidia.com/gpumem: 24000 + nvidia.com/gpucores: 50 + env: + - name: HUGGING_FACE_HUB_TOKEN + value: "YOUR_HF_TOKEN" # Replace with your Hugging Face token +``` + +**Key Resource Constraints:** + +- `nvidia.com/gpu: 1`: Requests 1 virtual GPU slot. +- `nvidia.com/gpumem: 24000`: Allocates exactly 24GB of GPU memory to the SGLang worker. +- `nvidia.com/gpucores: 50`: Allocates 50% of the physical GPU's compute capacity. + +Apply the deployment: + +```bash +kubectl apply -f sglang-deployment.yaml +``` + +## Step 2: Create the Service + +Expose the SGLang deployment internally using a ClusterIP service. + +Save as `sglang-service.yaml`: + +```yaml +apiVersion: v1 +kind: Service +metadata: + name: sglang-service +spec: + selector: + app: sglang + ports: + - protocol: TCP + port: 30000 + targetPort: 30000 +``` + +Apply the service: + +```bash +kubectl apply -f sglang-service.yaml +``` + +## Step 3: Verify Inference + +Wait for the SGLang pod to become `Running`. Since it needs to download the Llama 3 weights, this might take several minutes depending on your network speed. + +```bash +kubectl get pods -l app=sglang -w +``` + +Once running, port-forward the service to your local machine: + +```bash +kubectl port-forward svc/sglang-service 30000:30000 +``` + +Open a new terminal and test the OpenAI-compatible endpoint: + +```bash +curl http://localhost:30000/v1/chat/completions \ + -H "Content-Type: application/json" \ + -d '{ + "model": "meta-llama/Meta-Llama-3-8B-Instruct", + "messages": [{"role": "user", "content": "Explain GPU virtualization in one sentence."}], + "temperature": 0.7 + }' +``` + +If successful, SGLang will return a generated response powered by your HAMi-partitioned GPU! + +## Step 4: Cleanup + +To remove the lab resources from your cluster, run: + +```bash +kubectl delete -f sglang-service.yaml +kubectl delete -f sglang-deployment.yaml +```