docs(developers): add missing task dispatch section and diagram to protocol.md #811
There are no files selected for viewing
| Original file line number | Diff line number | Diff line change |
|---|---|---|
|
|
@@ -38,3 +38,25 @@ Since system clock on scheduler node and 'device' node may not align properly, s | |
| ```text | ||
| hami.io/node-handshake-\{device-type\}: Requesting_{scheduler_node_current_timestamp} | ||
| ``` | ||
|
|
||
| ## Task Dispatch & Scheduling Decisions | ||
|
|
||
| During the `bind` process, `kube-scheduler` invokes the device plugin to mount the device, but only provides the device `UUID`. In GPU sharing scenarios, the device plugin cannot natively obtain the workload's requested device specifications, such as GPU memory and compute core limits. | ||
|
|
||
| Therefore, HAMi uses a protocol for the scheduler to communicate task allocation metadata to the device plugin. The scheduler passes this information by patching allocation annotations onto the Pod, which the device plugin reads during container setup, as shown below: | ||
|
Comment on lines
+44
to
+46
There was a problem hiding this comment. 🎯 Functional Correctness | 🟠 Major | ⚡ Quick win 🧩 Analysis chain🏁 Script executed: #!/bin/bash
set -eu
for file in docs/developers/protocol.md versioned_docs/version-v2.9.0/developers/protocol.md; do
echo "=== $file ==="
sed -n '38,62p' "$file"
done
echo "=== related Kubernetes terms in documentation ==="
rg -n -i 'kube-scheduler|kubelet|device plugin|bind|allocation annotations|patching allocation' \
docs/developers/protocol.md versioned_docs/version-v2.9.0/developers/protocol.mdRepository: Project-HAMi/website Length of output: 6428 🌐 Web query:
💡 Result: In Kubernetes, the Device Plugin framework enables the kubelet to manage and allocate specialized hardware (e.g., GPUs, FPGAs) to containers without modifying core Kubernetes code [1][2]. The process involves several key components and steps: Registration and Discovery Device plugins operate as gRPC servers [1][3]. Upon startup, a plugin registers itself with the kubelet via a Unix socket (typically located at /var/lib/kubelet/device-plugins/) [1][4]. After registration, the plugin uses the ListAndWatch method to report the list of available devices and their status to the kubelet [3][4]. The kubelet then advertises these devices as extended resources to the Kubernetes API server [4]. Scheduling and Allocation 1. Scheduling: When a Pod requests an extended resource (e.g., nvidia.com/gpu), the Kubernetes scheduler identifies a node that has sufficient capacity of that resource [5][2]. 2. Admission and Preparation: Once the Pod is bound to a node, the kubelet on that node initiates the resource assignment [2]. The kubelet's Device Manager interacts with the plugin during the pod admission phase [6][7]. 3. GetPreferredAllocation (Optional): Before final allocation, the Device Manager may call the plugin's GetPreferredAllocation method [7]. This allows the plugin to suggest an optimal set of devices (e.g., based on topology constraints) [1][7]. 4. Allocate: When creating the container, the kubelet calls the plugin's Allocate gRPC method [1][3]. The plugin performs device-specific operations (such as resetting hardware, GPU cleanup, or initializing variables) and returns an AllocateResponse [1][4]. 5. Container Configuration: This response contains information (e.g., device paths, environment variables) that the kubelet passes to the container runtime to make the device accessible within the container [1][2]. For more sophisticated requirements—such as dynamic sharing between pods or custom resource parameters—Kubernetes also supports Dynamic Resource Allocation (DRA), which uses a distinct set of APIs (resource.k8s.io) and controllers [8]. Top Results: [1][9][3][2][7] Citations:
🏁 Script executed: #!/bin/bash
set -eu
echo "=== protocol-related files ==="
fd -t f . | rg '(^|/)(protocol|scheduler|device.?plugin|devicemanager|allocate|bind)' | head -200
echo "=== HAMi protocol and annotation references ==="
rg -n -i 'vgpu-devices-(allocated|to-allocate)|bind-time|task dispatch|device plugin|Allocate\(|Mount|Bind\(' \
--glob '!node_modules/**' --glob '!build/**' --glob '!dist/**' . | head -300
echo "=== task dispatch diagram metadata ==="
fd -t f . | rg 'task-dispatch\.(png|svg|drawio|md)$' || trueRepository: Project-HAMi/website Length of output: 50376 🌐 Web query:
💡 Result: In Kubernetes, the Allocate gRPC method is a critical component of the device plugin framework, invoked by the kubelet during the container creation process [1]. When a pod requests a device managed by a plugin, the kubelet’s device manager calls the Allocate method to perform necessary device-specific operations [1]. These operations may include tasks such as GPU cleanup, hardware initialization, or resetting a device to prepare it for use within the container [1]. After the device plugin performs these operations, it returns an AllocateResponse [1]. This response contains specific container runtime configurations—such as environment variables, mount points, or device nodes—required for the container to access the allocated devices [1]. The kubelet then passes these configurations to the container runtime to ensure the device is correctly exposed to the container environment [1]. The Allocate process is part of a larger workflow where the device plugin also supports other methods, such as PreStartContainer (called before each container start for final preparations) and GetPreferredAllocation (which helps the kubelet make informed decisions when multiple devices are available) [1]. Successful execution of these methods requires the device plugin to be registered and actively serving gRPC requests [1]. Citations: Use the correct Kubernetes component for device allocation. The scheduler binds the Pod. The node's kubelet calls the device plugin's
📍 Affects 2 files
🤖 Prompt for AI Agents |
||
|
|
||
| <img src="/img/docs/common/developers/protocol/task-dispatch.png" width="600px" alt="HAMi task dispatch protocol diagram showing scheduler and device-plugin interaction" /> | ||
|
|
||
| During this process, three annotations are managed: | ||
|
|
||
| - `hami.io/bind-time`: Timestamp when the scheduling decision was made. | ||
| - `hami.io/vgpu-devices-allocated`: The devices and specifications allocated by the scheduler. | ||
| - `hami.io/vgpu-devices-to-allocate`: The devices pending allocation. When the scheduler creates the pod annotations, `hami.io/vgpu-devices-to-allocate` contains the target devices. The device plugin determines the allocation based on this annotation, and once allocation is complete, removes the allocated devices. When the task is successfully running, `hami.io/vgpu-devices-to-allocate` becomes empty. | ||
|
Comment on lines
+52
to
+54
There was a problem hiding this comment. 🗄️ Data Integrity & Integration | 🟠 Major | ⚡ Quick win Keep the annotation lifecycle identical to Both pages incorrectly describe when
📍 Affects 2 files
🤖 Prompt for AI Agents |
||
|
|
||
| An example of a GPU task requesting 3000 MiB of device memory generates the following annotations on the Pod: | ||
|
|
||
| ```yaml | ||
| hami.io/bind-time: "1716199325" | ||
| hami.io/vgpu-devices-allocated: GPU-0fc3eda5-e98b-a25b-5b0d-cf5c855d1448,NVIDIA,3000,0:; | ||
| hami.io/vgpu-devices-to-allocate: ; | ||
| ``` | ||
There was a problem hiding this comment.
Choose a reason for hiding this comment
The reason will be displayed to describe this comment to others. Learn more.
#719 syncs the other way, it deletes this exact section from the zh page because en lacked it, and #779 rewrites this same file. the three prs contradict each other, maintainers must pick one direction first.
There was a problem hiding this comment.
Choose a reason for hiding this comment
The reason will be displayed to describe this comment to others. Learn more.
Task Dispatch is live architecture..it is missing from the English docs (also it is noted in docs/diagrams-inventory.md ) so according to me it should be added rather than removed like in #719..also #779 updates the Device Registration section and adds node troubleshooting, so the Task Dispatch section is needed regardless