Skip to content

Added AWS EC2 GPU Instructions to Improve Multi-Cloud Hami Tutorial Lab 1 Documentation #737

New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

Sign up for GitHub

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Closed
Creativeklvn wants to merge 5 commits into from
Closed

Added AWS EC2 GPU Instructions to Improve Multi-Cloud Hami Tutorial Lab 1 Documentation #737

Changes from all commits
Commits
Show all changes
5 commits
Select commit Hold shift + click to select a range
59d3ac7
AWS EC2 GPU Instructions to Improve Multi-Cloud HAMi Lab Documentation
Creativeklvn Aug 7, 2026
e569c64
fix: AWS EC2 GPU Instructions to Improve Multi-Cloud HAMi Lab Documen…
Creativeklvn Aug 7, 2026
aa31bbd
fix: AWS EC2 GPU Instructions to Improve Multi-Cloud HAMi Lab Documen…
Creativeklvn Aug 7, 2026
a0f0d1c
fix: resolve merge conflicts
Creativeklvn Aug 13, 2026
d884fb3
fix: resolve merge conflicts
Creativeklvn Aug 13, 2026
File filter

Filter by extension

Filter by extension
Conversations
Failed to load comments.
Loading
Jump to
Jump to file
Failed to load files.
Loading
Diff view
Diff view
242 changes: 229 additions & 13 deletions tutorials/labs/online-install.md
View file
Open in desktop
Original file line number Diff line number Diff line change
Expand Up @@ -5,19 +5,22 @@ sidebar_label: "Lab 1: Online Install"
lab:
level: Beginner
duration: about 60 minutes
environment: GCP VM with one NVIDIA T4
environment: AWS or GCP VM with one NVIDIA T4
cost: about $1 in VM time
authors:
- rootsongjc
- saiyam1814
- creativeklvn
verified: "2026-06-04"
Comment thread
Creativeklvn marked this conversation as resolved.
tags:
- installation
- nvidia
toc_max_heading_level: 2
---

This lab walks you through building a Kubernetes cluster from scratch on a Google Cloud GPU virtual machine and installing HAMi online, resulting in a complete GPU virtualization runtime environment.
import Tabs from '@theme/Tabs'; import TabItem from '@theme/TabItem';

This lab walks you through building a Kubernetes cluster from scratch on an AWS EC2 GPU or Google Cloud GPU virtual machine and installing HAMi online, resulting in a complete GPU virtualization runtime environment.

## What You'll Get

Expand All @@ -30,7 +33,7 @@ The entire installation process is divided into 6 steps, each solving a specific
```mermaid
%% title: HAMi Installation Overview
flowchart LR
Step1["Step 1<br/>Create GCP VM"] --> Step2["Step 2<br/>Install Helm"]
Step1["Step 1<br/>Create AWS or GCP VM"] --> Step2["Step 2<br/>Install Helm"]
Step2 --> Step3["Step 3<br/>Install Kubernetes"]
Step3 --> Step4["Step 4<br/>Install Prometheus"]
Step4 --> Step5["Step 5<br/>Install GPU Operator"]
Expand All @@ -39,7 +42,7 @@ flowchart LR

| Step | Purpose | What Problem It Solves |
| --- | --- | --- |
| Create GCP VM | Provision a Linux server with a GPU | Kubernetes needs GPU hardware to schedule GPU workloads |
| Create AWS or GCP VM | Provision a Linux server with a GPU | Kubernetes needs GPU hardware to schedule GPU workloads |
| Install Helm | Kubernetes package manager | All subsequent components are installed via Helm, similar to apt/yum |
| Install Kubernetes | Container orchestration platform | HAMi runs on top of Kubernetes; all GPU resources are managed by K8s |
| Install Prometheus | Monitoring system | HAMi and GPU Operator depend on Prometheus to collect and store metrics |
Expand All @@ -48,19 +51,131 @@ flowchart LR

## Prerequisites

<Tabs groupId="cloud-provider">
<TabItem value="aws" label="AWS">

- AWS account that can run a [G-instance type](https://aws.amazon.com/ec2/instance-types/g4/). To use this instance, take the following steps:
1. Search for the "Service Quota" service in the AWS console search bar and select it.
2. At the right of the screen, under "Manage quotas", search for "Amazon Elastic Compute Cloud" quotas and select "View quotas".
3. Search for "All G and VT Spot Instance Requests" in the quota search bar and select it.
4. At the top right of the service page, click "Request increase at account level" and request for 4 vCPU.
Comment thread
Creativeklvn marked this conversation as resolved.

@mesutoezdil mesutoezdil Aug 20, 2026

Copy link
Copy Markdown
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

aws login is not a command. it is aws configure or aws sso login.

- `AWS` CLI installed and authenticated (`aws login`)
- **Instance type**: `g4dn.xlarge` instance type because it supports the nvidia-tesla-t4.
- **Operating System**: `Ubuntu 24.04 LTS`
- **Kubernetes version**: `1.33`
- **Kernel version**: `AWS Kernel v6.8` (The installation step is in this tutorial.)

</TabItem>
<TabItem value="gcp" label="GCP">

@mesutoezdil mesutoezdil Aug 20, 2026

Copy link
Copy Markdown
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

add a blank line after the tabitem tag, mdx may not render this list otherwise. the aws tab has one.

- Google Cloud account with Compute Engine API enabled
- `gcloud` CLI installed and authenticated (`gcloud auth login`)
- NVIDIA T4 GPU quota available in your GCP project
- **Operating System**: Ubuntu 22.04 LTS
- **Kubernetes version**: `1.34`

Comment thread
Creativeklvn marked this conversation as resolved.
> Cost note: the `n1-standard-4` + T4 VM costs about $0.55 per hour. [Lab 3](./gpu-partitioning.md) and [Lab 4](./hami-dra.md) continue on this same cluster, so one session covers all three labs. Delete the VM when you finish.

## Step 1: Create a GCP Virtual Machine
</TabItem>
</Tabs>

### Purpose
> To get the full list of supported OS, Kubernetes version, and VM Kernel version supported by the NVIDIA GPU Operator v25.3 used in this tutorial, visit [NVIDIA GPU Operator Platform Support v25.3](https://docs.nvidia.com/datacenter/cloud-native/gpu-operator/25.3/platform-support.html#supported-operating-systems-and-kubernetes-platforms).

---

## Step 1: Create a Virtual Machine
Comment thread
coderabbitai[bot] marked this conversation as resolved.

<Tabs groupId="cloud-provider">
<TabItem value="aws" label="AWS">

### AWS Purpose

Create a virtual machine with a GPU to serve as the foundation for the entire lab. HAMi requires physical GPU hardware (or pass-through virtual GPU) to function, it does not emulate GPUs; instead, it partitions and shares real GPUs.

### Instructions
### AWS Instructions

#### 1.1 Export EC2 Configuration Variables

```bash
export INSTANCE_TYPE="g4dn.xlarge"
export AMI_ID="<YOUR_AWS_AMI_ID>"
export KEY_NAME="hami-eks"
export VOLUME_SIZE="50"
export REGION="<YOUR_AWS_REGION>"
```

#### 1.2 Create the EC2 Spot Instance

```bash

@mesutoezdil mesutoezdil Aug 20, 2026

Copy link
Copy Markdown
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

no security group is created or referenced. a default sg blocks ssh and port 3000, so step 1.3 and the webui step fail. add an sg with 22 and 3000 restricted to the reader ip.

aws ec2 run-instances \
--instance-type "$INSTANCE_TYPE" \
--image-id "$AMI_ID" \
--key-name "$KEY_NAME" \
--block-device-mappings "[{\"DeviceName\":\"/dev/sda1\",\"Ebs\":{\"VolumeSize\":$VOLUME_SIZE,\"VolumeType\":\"gp3\",\"DeleteOnTermination\":true}}]" \
--instance-market-options '{"MarketType":"spot"}' \
--region "$REGION"
Comment thread
Creativeklvn marked this conversation as resolved.
```

#### 1.3 SSH Into the Instance

```bash
export NODE_PUBLIC_IP=<your-vm-public-ip>
ssh -i <your-pem-key-file-path> ubuntu@$NODE_PUBLIC_IP
```

After logging in, switch to root:

```bash
sudo -i
```

#### 1.4 Downgrade to the v6.8 AWS Kernel, Use It as the Default, and Reboot

The GPU operator installation in Step 5, runs v25.3.0, and this version [only supports a specific kernel version](https://docs.nvidia.com/datacenter/cloud-native/gpu-operator/25.3/platform-support.html#supported-precompiled-drivers) depending on the operating system being used. AWS ships with a different version on default, this step changes it to the supported one.

```bash
# Install the AWS 6.8 kernel and its headers
apt install linux-image-6.8.0-1008-aws linux-headers-6.8.0-1008-aws

@mesutoezdil mesutoezdil Aug 20, 2026

Copy link
Copy Markdown
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

this grub uuid is from your own disk. the sed will not match on any other machine and the kernel silently stays on default. use a generic method, for example grub-set-default with the menu entry name, and add -y to the apt install above.

# Configure GRUB to boot the 6.8 AWS kernel by default
sed -i 's|^GRUB_DEFAULT=.*|GRUB_DEFAULT="gnulinux-advanced-0ef35759-eb42-4358-9a2a-1f74696d7007>gnulinux-6.8.0-1008-aws-advanced-0ef35759-eb42-4358-9a2a-1f74696d7007"|' /etc/default/grub

# Regenerate the GRUB boot configuration
update-grub

# Verify that GRUB_DEFAULT is set correctly
grep '^GRUB_DEFAULT' /etc/default/grub
Comment thread
Creativeklvn marked this conversation as resolved.

# Reboot the EC2 instance using the new default kernel
reboot
```

#### 1.5 SSH Into the Instance Again and Confirm Kernel Version

```bash
export NODE_PUBLIC_IP=<your-vm-public-ip>
ssh -i <your-pem-key-file-path> ubuntu@$NODE_PUBLIC_IP

# After logging in, switch to the root user:
sudo -i

# Check the currently running Linux kernel version
uname -r
```

The output is similar to the following:

```plaintext
6.8.0-1008-aws
```

</TabItem>
<TabItem value="gcp" label="GCP">

### GCP Purpose

Create a virtual machine with a GPU to serve as the foundation for the entire lab. HAMi requires physical GPU hardware (or pass-through virtual GPU) to function, it does not emulate GPUs; instead, it partitions and shares real GPUs.

### GCP Instructions

Set environment variables:

Expand Down Expand Up @@ -104,6 +219,9 @@ After logging in, switch to root:
sudo su -
```

</TabItem>
</Tabs>

## Step 2: Install Helm

### Purpose
Expand Down Expand Up @@ -171,7 +289,7 @@ EOF
sysctl --system
```

#### 3.4 Install containerd
#### 3.4 Install Containerd

containerd is the default container runtime for Kubernetes, responsible for actually creating and running containers. Docker is no longer the default runtime since Kubernetes 1.24.

Expand All @@ -189,7 +307,7 @@ systemctl restart containerd
systemctl enable containerd
```

#### 3.5 Install kubeadm, kubelet, and kubectl
#### 3.5 Install Kubeadm, Kubelet, and Kubectl

The relationship between these three tools:

Expand All @@ -205,6 +323,28 @@ flowchart LR
- **kubelet**: A daemon process responsible for creating and destroying Pods on the local node
- **kubectl**: The command-line tool used for day-to-day operations

<Tabs groupId="cloud-provider">
<TabItem value="aws" label="AWS">

```bash
apt-get install -y apt-transport-https ca-certificates curl gpg

mkdir -p /etc/apt/keyrings

curl -fsSL https://pkgs.k8s.io/core:/stable:/v1.33/deb/Release.key | \
gpg --dearmor -o /etc/apt/keyrings/kubernetes-apt-keyring.gpg

echo 'deb [signed-by=/etc/apt/keyrings/kubernetes-apt-keyring.gpg] https://pkgs.k8s.io/core:/stable:/v1.33/deb/ /' | \
tee /etc/apt/sources.list.d/kubernetes.list

apt-get update
apt-get install -y kubelet kubeadm kubectl
apt-mark hold kubelet kubeadm kubectl
```

</TabItem>
<TabItem value="gcp" label="GCP">

```bash
apt-get install -y apt-transport-https ca-certificates curl gpg

Expand All @@ -221,6 +361,9 @@ apt-get install -y kubelet kubeadm kubectl
apt-mark hold kubelet kubeadm kubectl
```

</TabItem>
</Tabs>

> `apt-mark hold` prevents these packages from being automatically upgraded. Kubernetes component versions need to be managed manually.

#### 3.6 Initialize the Cluster
Expand Down Expand Up @@ -256,6 +399,29 @@ Wait for the Calico Pods to be ready:
kubectl get pods -n calico-system
```

The output is similar to the following:

```plaintext
NAME READY STATUS RESTARTS AGE
calico-kube-controllers-7566c4cd97-f8jpp 1/1 Running 0 50s
calico-node-skfxl 1/1 Running 0 50s
calico-typha-5b5969dcf9-mwb99 1/1 Running 0 51s
csi-node-driver-nppmm 2/2 Running 0 50s
```

#### 3.8 Verify Cluster Status

```bash

@mesutoezdil mesutoezdil Aug 20, 2026

Copy link
Copy Markdown
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

two sections are numbered 3.8 now. renumber this or the next one.

kubectl get nodes
```

Expected output (STATUS of Ready indicates the cluster is ready):

```plaintext
NAME STATUS ROLES AGE VERSION
hami-workshop Ready control-plane 2m v1.34.8
```

@mesutoezdil mesutoezdil Aug 20, 2026

Copy link
Copy Markdown
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

this shared section shows v1.34.8 but the aws path installs 1.33. either tab the outputs or drop the version detail.

#### 3.8 Allow Master Node to Schedule Pods

In a single-node cluster, this node serves as both the control plane and the worker node. By default, Kubernetes does not schedule workloads on Master nodes. You need to manually remove this restriction:
Expand Down Expand Up @@ -304,6 +470,29 @@ helm install prometheus prometheus-community/kube-prometheus-stack \
>
> `serviceMonitorSelectorNilUsesHelmValues=false` makes Prometheus pick up ServiceMonitors from all namespaces regardless of labels. Without it, Prometheus only selects ServiceMonitors labeled `release: prometheus`, silently ignores the one the GPU Operator creates for dcgm-exporter, and you end up with no GPU metrics at all.

<Tabs groupId="cloud-provider">
<TabItem value="aws" label="AWS">

Verify Prometheus component status:

```bash
kubectl get po -n monitoring
```

All Pods should have a status of `Running`:

```plaintext
NAME READY STATUS RESTARTS AGE
alertmanager-prometheus-kube-prometheus-alertmanager-0 2/2 Running 0 28s
prometheus-kube-prometheus-operator-58fcd77f9d-zm2w5 1/1 Running 0 35s
prometheus-kube-state-metrics-6f8b5cc99-6p9zf 1/1 Running 0 35s
prometheus-prometheus-kube-prometheus-prometheus-0 2/2 Running 0 28s
prometheus-prometheus-node-exporter-5vp4b 1/1 Running 0 35s
```

</TabItem>
<TabItem value="gcp" label="GCP">

Verify Prometheus component status:

```bash
Expand All @@ -320,6 +509,9 @@ prometheus-prometheus-kube-prometheus-prometheus-0 2/2 Running 0
prometheus-prometheus-node-exporter-xxxxx 1/1 Running 0 2m
```

</TabItem>
</Tabs>

> If the installation fails, uninstall first before retrying: `helm uninstall -n monitoring prometheus`

## Step 5: Install GPU Operator
Expand Down Expand Up @@ -392,6 +584,20 @@ The expected output includes GPU information (driver version, CUDA version, GPU
+-----------------------------------------------------------------------------------------+
```

### Get The GPU Node Name

```bash
kubectl get nodes
```

The output is similar to the following:

```plaintext
ip-172-31-6-1 Ready control-plane 13m v1.33.13

@mesutoezdil mesutoezdil Aug 20, 2026

Copy link
Copy Markdown
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

aws node name and v1.33.13 in a shared section, gcp readers see something else. same mixing as above.

```

---

## Step 6: Install HAMi

### Purpose
Expand Down Expand Up @@ -438,6 +644,10 @@ NODE_NAME=$(kubectl get nodes -o jsonpath='{.items[0].metadata.name}')
kubectl label nodes ${NODE_NAME} gpu=on
```

```bash
kubectl get nodes --show-labels | grep -i gpu=on
```

The device plugin starts on the labeled node:

```bash
Expand All @@ -452,10 +662,10 @@ hami-scheduler-6d659887fc-j5ngc 2/2 Running 0 95s
Verify GPU registration information:

```bash
kubectl get node ${NODE_NAME} -o jsonpath='{.metadata.annotations.hami\.io/node-nvidia-register}'
kubectl get node ${NODE_NAME} -o jsonpath='{.metadata.annotations.hami\.io/node-nvidia-register}' | jq
```
Comment thread
Creativeklvn marked this conversation as resolved.

Expected output is one JSON object per GPU:
The output is similar to the following JSON object per GPU:

@mesutoezdil mesutoezdil Aug 20, 2026

Copy link
Copy Markdown
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

jq is not installed anywhere in the lab. add it to prereqs or drop the pipe.


```json
[
Expand Down Expand Up @@ -502,10 +712,16 @@ helm install my-hami-webui hami-webui/hami-webui \

> `--set dcgm-exporter.enabled=false` because the GPU Operator already installed dcgm-exporter, avoiding duplicate deployment.

Check the pod is running:

```bash
kubectl get pods -n kube-system -l app.kubernetes.io/name=hami-webui
```

Access the WebUI via port forwarding:

```bash
kubectl port-forward service/my-hami-webui 3000:3000 --namespace=kube-system
kubectl port-forward --address 0.0.0.0 service/my-hami-webui 3000:3000 --namespace=kube-system

@mesutoezdil mesutoezdil Aug 20, 2026

Copy link
Copy Markdown
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

binding 0.0.0.0 plus a public ip exposes the webui to the internet with no auth. add a warning and point to the sg rule. this also changes the gcp flow which used localhost.

```

Visit `http://localhost:3000` to open the HAMi WebUI.
Visit `http://<your-vm-public-ip-address>:3000` to open the HAMi WebUI.
Comment thread
coderabbitai[bot] marked this conversation as resolved.