feat: add host PID broker server and device plugin integration - #2417
feat: add host PID broker server and device plugin integration#2417iemAnshuman wants to merge 21 commits into
Conversation
Signed-off-by: iemAnshuman <asquare567@gmail.com>
Signed-off-by: iemAnshuman <asquare567@gmail.com>
Signed-off-by: iemAnshuman <asquare567@gmail.com>
Signed-off-by: iemAnshuman <asquare567@gmail.com>
Signed-off-by: iemAnshuman <asquare567@gmail.com>
|
[APPROVALNOTIFIER] This PR is NOT APPROVED This pull-request has been approved by: iemAnshuman The full list of commands accepted by this bot can be found here. DetailsNeeds approval from an approver in each of these files:Approvers can indicate their approval by writing |
|
Note Reviews pausedIt looks like this branch is under active development. To avoid overwhelming you with review comments due to an influx of new commits, CodeRabbit has automatically paused this review. You can configure this behavior by changing the Use the following commands to manage reviews:
Use the checkboxes below for quick actions:
📝 WalkthroughWalkthroughThe PR adds an optional host PID broker for the NVIDIA device plugin. It defines a Unix-socket protocol, implements Linux broker lifecycle and safety controls, integrates socket access into allocation responses, and adds Helm configuration, tests, and documentation. ChangesHost PID broker
Estimated code review effort: 4 (Complex) | ~60 minutes Possibly related issues
Possibly related PRs
Suggested reviewers: Poem
🚥 Pre-merge checks | ✅ 4 | ❌ 1❌ Failed checks (1 warning)
✅ Passed checks (4 passed)
✨ Finishing Touches🧪 Generate unit tests (beta)
Thanks for using CodeRabbit! It's free for OSS, and your support helps us grow. If you like it, consider giving us a shout-out. Comment |
Signed-off-by: iemAnshuman <asquare567@gmail.com>
Signed-off-by: iemAnshuman <asquare567@gmail.com>
3963c56 to
a83c20f
Compare
There was a problem hiding this comment.
Actionable comments posted: 5
🧹 Nitpick comments (2)
pkg/device-plugin/nvidiadevice/nvinternal/hostpid/broker_linux.go (1)
256-280: 📐 Maintainability & Code Quality | 🔵 Trivial | ⚡ Quick winConsider logging dropped transactions.
handlereturns without a response when the read fails, when the deadline expires, or whenpeerPIDfails. The broker emits no log for these paths. Field diagnosis of a failing client then depends only on HAMi-core falling back to NVML.Add a rate-limited debug log for the read error and the
peerPIDerror paths.🤖 Prompt for AI Agents
Verify each finding against current code. Fix only still-valid issues, skip the rest with a brief reason, keep changes minimal, and validate. In `@pkg/device-plugin/nvidiadevice/nvinternal/hostpid/broker_linux.go` around lines 256 - 280, Add rate-limited debug logging in Broker.handle for errors returned by io.ReadFull and peerPID, including the relevant error details, while preserving the existing early-return behavior and response handling. Do not add logging for unrelated paths.pkg/device-plugin/nvidiadevice/nvinternal/hostpid/broker_linux_test.go (1)
576-586: 📐 Maintainability & Code Quality | 🔵 Trivial | ⚡ Quick winAssert the recorded identity, not just the stat.
TestBrokerSocketIdentityUsesDeviceAndInodediscards the broker at Line 577 and only checks that the stat cast succeeds andstat.Inois non-zero. It never compares againstbroker.socket, so it does not coverreadSocketIdentity. Compare the device and inode with the recorded values.💚 Proposed test
- _, socketPath := startTestBroker(t) + broker, socketPath := startTestBroker(t) info, err := os.Lstat(socketPath) if err != nil { t.Fatal(err) } stat, ok := info.Sys().(*syscall.Stat_t) if !ok || stat.Ino == 0 { t.Fatalf("invalid socket stat: %#v", info.Sys()) } + if uint64(stat.Dev) != broker.socket.device || + stat.Ino != broker.socket.inode { + t.Fatalf("recorded identity dev=%d ino=%d, want dev=%d ino=%d", + broker.socket.device, broker.socket.inode, stat.Dev, stat.Ino) + }🤖 Prompt for AI Agents
Verify each finding against current code. Fix only still-valid issues, skip the rest with a brief reason, keep changes minimal, and validate. In `@pkg/device-plugin/nvidiadevice/nvinternal/hostpid/broker_linux_test.go` around lines 576 - 586, Update TestBrokerSocketIdentityUsesDeviceAndInode to retain the broker returned by startTestBroker and compare broker.socket’s recorded device and inode values with the device and inode from the Lstat result, thereby exercising readSocketIdentity.
🤖 Prompt for all review comments with AI agents
Verify each finding against current code. Fix only still-valid issues, skip the
rest with a brief reason, keep changes minimal, and validate.
Inline comments:
In `@cmd/device-plugin/nvidia/main.go`:
- Around line 275-291: Update the startup flow around startHostPIDBroker and
p.Start so host PID broker health is validated before the device-plugin gRPC
server starts accepting requests. Await broker readiness and surface any broker
failure before invoking p.Start, or ensure a startup failure cancels/stops both
the broker and server; preserve hostPIDBrokerFailureReported and deferred
cleanup behavior for failures after startup.
In `@pkg/device-plugin/nvidiadevice/nvinternal/hostpid/broker_linux_test.go`:
- Around line 199-221: Update TestBrokerTimesOutPartialRequest to set a
client-side read deadline on connection immediately before reading into buffer.
Keep the existing assertion that the broker closes the connection, while
ensuring an open connection causes Read to return promptly and fail the test
rather than hang.
In `@pkg/device-plugin/nvidiadevice/nvinternal/hostpid/broker_linux.go`:
- Around line 225-238: Update Broker.Serve to retry temporary AcceptUnix errors
with a bounded backoff instead of immediately returning, while preserving the
existing clean return when broker.closing is set. Return only non-temporary
permanent errors, and reset the backoff after a successful connection is
accepted.
In `@pkg/device-plugin/nvidiadevice/nvinternal/plugin/hostpid_broker_test.go`:
- Around line 9-15: Reorder the imports in the hostpid broker test: keep the
standard-library testing import first, place testify and kubelet imports in the
external group next, and move the local
github.com/Project-HAMi/HAMi/pkg/device-plugin/.../hostpid import to the final
group.
In `@pkg/device-plugin/nvidiadevice/nvinternal/plugin/hostpid_broker.go`:
- Around line 9-14: Reorder the imports in hostpid_broker.go so standard-library
imports remain first, external k8s.io imports follow, and the local
github.com/Project-HAMi/HAMi/pkg/device-plugin/nvidiadevice/nvinternal/hostpid
import is placed last.
---
Nitpick comments:
In `@pkg/device-plugin/nvidiadevice/nvinternal/hostpid/broker_linux_test.go`:
- Around line 576-586: Update TestBrokerSocketIdentityUsesDeviceAndInode to
retain the broker returned by startTestBroker and compare broker.socket’s
recorded device and inode values with the device and inode from the Lstat
result, thereby exercising readSocketIdentity.
In `@pkg/device-plugin/nvidiadevice/nvinternal/hostpid/broker_linux.go`:
- Around line 256-280: Add rate-limited debug logging in Broker.handle for
errors returned by io.ReadFull and peerPID, including the relevant error
details, while preserving the existing early-return behavior and response
handling. Do not add logging for unrelated paths.
🪄 Autofix
Fix all unresolved CodeRabbit comments on this PR:
- Push a commit to this branch (recommended)
- Create a new PR with the fixes
ℹ️ Review info
⚙️ Run configuration
Configuration used: Organization UI
Review profile: CHILL
Plan: Pro Plus
Run ID: a5588f6f-7e0b-4306-9854-aaeb57e348fd
📒 Files selected for processing (18)
charts/hami/README.mdcharts/hami/templates/device-plugin/daemonsetnvidia.yamlcharts/hami/values.yamlcmd/device-plugin/nvidia/hostpid_broker.gocmd/device-plugin/nvidia/hostpid_broker_test.gocmd/device-plugin/nvidia/main.godocs/develop/hostpid-broker.mdgo.modpkg/device-plugin/nvidiadevice/nvinternal/hostpid/broker_linux.gopkg/device-plugin/nvidiadevice/nvinternal/hostpid/broker_linux_test.gopkg/device-plugin/nvidiadevice/nvinternal/hostpid/broker_unsupported.gopkg/device-plugin/nvidiadevice/nvinternal/hostpid/config.gopkg/device-plugin/nvidiadevice/nvinternal/hostpid/protocol.gopkg/device-plugin/nvidiadevice/nvinternal/hostpid/protocol_test.gopkg/device-plugin/nvidiadevice/nvinternal/plugin/hostpid_broker.gopkg/device-plugin/nvidiadevice/nvinternal/plugin/hostpid_broker_test.gopkg/device-plugin/nvidiadevice/nvinternal/plugin/server.gopkg/device-plugin/nvidiadevice/nvinternal/plugin/server_test.go
Signed-off-by: iemAnshuman <asquare567@gmail.com>
Signed-off-by: iemAnshuman <asquare567@gmail.com>
Signed-off-by: iemAnshuman <asquare567@gmail.com>
Signed-off-by: iemAnshuman <asquare567@gmail.com>
Signed-off-by: iemAnshuman <asquare567@gmail.com>
Signed-off-by: iemAnshuman <asquare567@gmail.com>
Signed-off-by: iemAnshuman <asquare567@gmail.com>
Signed-off-by: iemAnshuman <asquare567@gmail.com>
|
/lgtm |
|
cc @maverick123123 |
|
@iemAnshuman This is really well done — the broker design is clean, the security validations are thorough (root-owned socket, SO_PEERCRED, read-only mount, 500ms deadline), and the feature gate with A couple of requests for the remaining validation:
Looking forward to seeing this land. |
Prepare /tmp/vgpulock as a root owned sticky directory before allocation. Canonicalize the parent and broker mounts, keep the parent before the read only child, and remove stale broker settings when the gate is off. Reject allocation when the parent cannot be prepared safely. Signed-off-by: iemAnshuman <asquare567@gmail.com>
|
New changes are detected. LGTM label has been removed. |
There was a problem hiding this comment.
Actionable comments posted: 2
🧹 Nitpick comments (3)
pkg/device-plugin/nvidiadevice/nvinternal/plugin/hostpid_broker.go (2)
40-48: 📐 Maintainability & Code Quality | 🔵 Trivial | 💤 Low valueUse
errors.Isfor theEEXISTcheck.
createHostPIDLockParentWithaccepts an injectedmkdiratfunction. A wrappedEEXISTerror then fails the direct comparison and the function reports a failure for an existing directory.errors.Ishandles both raw and wrapped errno values.♻️ Proposed refactor
err := mkdirat(parentFD, baseName, hostPIDLockParentCreateMode) - if err != nil && err != unix.EEXIST { + if err != nil && !errors.Is(err, unix.EEXIST) { return err } return nilAdd
"errors"to the standard-library import group.🤖 Prompt for AI Agents
Verify each finding against current code. Fix only still-valid issues, skip the rest with a brief reason, keep changes minimal, and validate. In `@pkg/device-plugin/nvidiadevice/nvinternal/plugin/hostpid_broker.go` around lines 40 - 48, Update createHostPIDLockParentWith to use errors.Is when checking whether mkdirat returned unix.EEXIST, and add the standard-library errors import. Preserve returning other errors while treating raw or wrapped EEXIST as success.
74-82: 📐 Maintainability & Code Quality | 🔵 Trivial | 💤 Low valueSeparate the directory-type check from the ownership check.
The condition combines three distinct failures. A non-directory or a failed
Stat_tassertion produces the message "parent directory is not owned by trusted UID". The message then misleads the operator during triage of a security check failure.♻️ Proposed refactor
parentStat, ok := parentInfo.Sys().(*syscall.Stat_t) parentMode := parentInfo.Mode() - if !ok || !parentInfo.IsDir() || parentStat.Uid != trustedOwner { + if !ok || !parentInfo.IsDir() { + return fmt.Errorf("parent path is not a real directory") + } + if parentStat.Uid != trustedOwner { return fmt.Errorf("parent directory is not owned by trusted UID %d", trustedOwner) }🤖 Prompt for AI Agents
Verify each finding against current code. Fix only still-valid issues, skip the rest with a brief reason, keep changes minimal, and validate. In `@pkg/device-plugin/nvidiadevice/nvinternal/plugin/hostpid_broker.go` around lines 74 - 82, In the parent-directory validation flow, separate the `Stat_t` assertion and `IsDir()` checks from the `parentStat.Uid != trustedOwner` check. Return an error specific to an invalid or non-directory parent before evaluating ownership, while preserving the existing trusted-UID error for ownership failures and the subsequent permission checks.pkg/device-plugin/nvidiadevice/nvinternal/plugin/server_test.go (1)
1095-1100: 📐 Maintainability & Code Quality | 🔵 Trivial | ⚡ Quick winAssert the allocation-failure bookkeeping on the error path.
Allocatecalls the pod-allocation-failure hook before it returns the preparation error. The test stubs that hook with an empty function and never checks it. A regression that skips the node-lock release would still pass.🧪 Proposed addition
+ failureCalls := 0 + podAllocationFailed = func(string, *corev1.Pod, string) { failureCalls++ } prepareHostPIDLockParentForAllocation = func() error { return errors.New("parent preparation fixture") } failedResponse, err := plugin.Allocate(context.Background(), request) require.Nil(t, failedResponse) require.ErrorContains(t, err, "failed to prepare host PID lock parent") + require.Equal(t, 1, failureCalls)🤖 Prompt for AI Agents
Verify each finding against current code. Fix only still-valid issues, skip the rest with a brief reason, keep changes minimal, and validate. In `@pkg/device-plugin/nvidiadevice/nvinternal/plugin/server_test.go` around lines 1095 - 1100, Update the allocation failure test around plugin.Allocate to stub the pod-allocation-failure hook with an observable signal, then assert that the signal is triggered when prepareHostPIDLockParentForAllocation returns its preparation error. Keep the existing failedResponse and error assertions, and ensure the test specifically verifies the node-lock release bookkeeping before returning.
🤖 Prompt for all review comments with AI agents
Verify each finding against current code. Fix only still-valid issues, skip the
rest with a brief reason, keep changes minimal, and validate.
Inline comments:
In `@docs/develop/hostpid-broker.md`:
- Around line 48-50: Update the documentation’s compound modifiers to use
hyphens: change “descriptor based” to “descriptor-based,” “read only” to
“read-only,” and the corresponding “path equivalent” and “root owned” usages to
“path-equivalent” and “root-owned” in the affected sections.
In `@pkg/device-plugin/nvidiadevice/nvinternal/plugin/hostpid_broker_test.go`:
- Around line 107-113: Update the "owner" subtest in hostpid_broker_test.go to
assert the specific error returned by prepareHostPIDLockParent, or adjust the
fixture so the parent is trusted and only directory has an untrusted owner.
Ensure the test explicitly exercises and validates the intended directory
ownership-check branch rather than merely asserting any error.
---
Nitpick comments:
In `@pkg/device-plugin/nvidiadevice/nvinternal/plugin/hostpid_broker.go`:
- Around line 40-48: Update createHostPIDLockParentWith to use errors.Is when
checking whether mkdirat returned unix.EEXIST, and add the standard-library
errors import. Preserve returning other errors while treating raw or wrapped
EEXIST as success.
- Around line 74-82: In the parent-directory validation flow, separate the
`Stat_t` assertion and `IsDir()` checks from the `parentStat.Uid !=
trustedOwner` check. Return an error specific to an invalid or non-directory
parent before evaluating ownership, while preserving the existing trusted-UID
error for ownership failures and the subsequent permission checks.
In `@pkg/device-plugin/nvidiadevice/nvinternal/plugin/server_test.go`:
- Around line 1095-1100: Update the allocation failure test around
plugin.Allocate to stub the pod-allocation-failure hook with an observable
signal, then assert that the signal is triggered when
prepareHostPIDLockParentForAllocation returns its preparation error. Keep the
existing failedResponse and error assertions, and ensure the test specifically
verifies the node-lock release bookkeeping before returning.
🪄 Autofix
Fix all unresolved CodeRabbit comments on this PR:
- Push a commit to this branch (recommended)
- Create a new PR with the fixes
ℹ️ Review info
⚙️ Run configuration
Configuration used: Organization UI
Review profile: CHILL
Plan: Pro Plus
Run ID: 5190264b-5643-4065-9876-e7e493bfd6e9
📒 Files selected for processing (5)
docs/develop/hostpid-broker.mdpkg/device-plugin/nvidiadevice/nvinternal/plugin/hostpid_broker.gopkg/device-plugin/nvidiadevice/nvinternal/plugin/hostpid_broker_test.gopkg/device-plugin/nvidiadevice/nvinternal/plugin/server.gopkg/device-plugin/nvidiadevice/nvinternal/plugin/server_test.go
🚧 Files skipped from review as they are similar to previous changes (1)
- pkg/device-plugin/nvidiadevice/nvinternal/plugin/server.go
thanks. I agree with both requirements. The current evidence is not the complete report yet. Local and linux root checks cover lower level broker behavior and /tmp/vgpulock path safety. Real kubelet allocation, rollout, rollback, and performance runs are still pending. I will post one report here after those runs with the source revisions, environment, enabled and disabled results, unreachable socket behavior, rollout and rollback results, and raw performance data. I will keep the feature marked incomplete until both paths pass and the disabled path shows no regression. |
Signed-off-by: iemAnshuman <asquare567@gmail.com>
Signed-off-by: iemAnshuman <asquare567@gmail.com>
Signed-off-by: iemAnshuman <asquare567@gmail.com>
Interim host PID broker validation reportI am posting the Linux root, A100, fallback, and package results now. The Kubernetes gate is still open, so this is not the completed test report requested above. Recorded 11 August 2026 for HAMi issue 1662, RFC 2244, HAMi-core PR 251, and HAMi PR 2417. Source boundaries
Each result below keeps its recorded source boundary. Linux root validationThe root run completed on Ubuntu 24.04 arm64 with Linux 6.8.0. It recorded real and effective UID 0. Before creating a private mount namespace, the initial user namespace mapping was The source contract binds HAMi-core base
That patch is the tree at core commit It also binds private HAMi base
All 28 recorded checks passed. They cover the normal and sanitizer CTest runs, cache deadlines, owner death, permission and object replacement cases, reverse order recovery, Go race and vet checks, root parent creation, nonroot fallback access, supported read only filesystems, and rejection of unsafe paths. Both CTest runs completed two tests with zero failures. Linux result archive SHA256:
A100
|
| Processes | Gate unset | Gate enabled | Change |
|---|---|---|---|
| 128 | 56.373 seconds | 22.501 seconds | 60.085% lower |
| 300 | 130.662 seconds | 50.288 seconds | 61.513% lower |
All 16 cells completed with zero failed workers. The 107 file result manifest SHA256 is:
f2f2129e239e3c7ef6e9cca94ee3f559f8c17f9057029a52d6267971aa777100
This run measures balanced cuInit(). It does not measure first primary context completion or Kubernetes deployment.
Primary context and fallback results
Job 181845 completed the broker condition at N=128 in 155.151656 seconds and at N=240 in 792.572612 seconds. N=300 reached its 903.11 second deadline.
Supplemental job 182609 recorded all 300 broker queries at N=300 with zero broker failures. The timeout therefore occurred after the broker query. This result does not establish that the broker alone fixes application ready latency.
Job 182607 tested the cross cache fallback correction at revision b0df70a7670ed9f8ccd2503f58ac40fa0463e97c.
| Layout | Own PID | Clear failure | Peer PID | External PID |
|---|---|---|---|---|
| N=128, one GPU | 57 | 71 | 0 | 0 |
| N=128, eight GPUs | 12 | 116 | 0 | 0 |
| N=300 | 10 | 290 | 0 | 0 |
The candidate accepted no peer or external PID. Clear failures remained failures.
Fallback archive SHA256:
b0908fa732086f9244cd495a4c53c9529032faebbda89420aae316b480e29e7c
Package verification
The package verifier was rerun on 11 August 2026. It reports:
46 verified, 0 pending, 0 failures
Generated decision table SHA256:
a1fab16d4a7b25ec8b35962ea3d88f270f936ee09088cc735c6fcdb03dc9e314
Package ledger SHA256:
294637ae0abbe69115c9f900442029fa0ccb335ec08ad3e8982d17f8dea1448a
Kubernetes status
The controlled allocation and Helm bundles cover enabled and disabled response construction, lifecycle handling, mount configuration, chart lint, and render checks. They do not use a real kubelet or CUDA workload.
The following checks remain:
- Real kubelet allocation with the broker enabled and disabled.
- DaemonSet rollout and device plugin process restart.
- Degradation when the broker socket is unavailable.
- Helm rollback followed by a new CUDA allocation.
The available Rostam A100 environment provides Slurm user access. It does not provide the cgroup delegation, cluster administration, or node root access required by the runbook. Rootless Podman and Apptainer do not cover that boundary.
No Kubernetes result is claimed here. I will add the sealed Kubernetes result when a suitable isolated NVIDIA Kubernetes node is available.
Disclosure: I use AI assistance in my workflow. The code archaeology and measurements above are my own, and I am happy to walk through any part of them.
|
This is being closed because it does not comply with the contribution guidelines. |
|
Hi @mesutoezdil, i read the contribution guidelines again after this pr was closed. The interim test report were generated with ai assistance rather than writthen by me. I take responsibility for posting that material and I now understand that it does not comply with the contribution guidelines. Could you tell me which requirements caused the closure? Besides the Interim report? would like to know whether the size of the pr or the incomplete kubelet allocation and rollout tests were also factors. |
|
@iemAnshuman Please resolve the conflict. |
Signed-off-by: iemAnshuman <asquare567@gmail.com>
resolved, merged master in. |
Related issue: #1662
Design discussion: #2244
Companion client PR: Project-HAMi/HAMi-core#251
Current companion client head:
4421b0970c172e343c7fb59491952b78ea9d86ebWhat this changes
This adds an optional host PID broker to the NVIDIA device plugin. The broker obtains each connecting process's host PID from Linux
SO_PEERCREDand returns it to the HAMi-core client.The client never supplies a PID. The design does not expose or mount host procfs.
The feature is disabled by default. Enable it with:
The chart rejects configurations that enable the broker while
devicePlugin.hostPIDis false.When enabled:
The device plugin serves
/var/run/hami/hostpid/broker.sock.Compatible workload allocations receive
LIBVGPU_HOSTPID_BROKER=1.The broker directory is mounted read only at
/tmp/vgpulock/hostpid.HAMi-core uses its NVML discovery path if the broker request fails and the trusted fallback mount is available.
Disabled configurations retain the current allocation response.
Before returning a non-MIG allocation, the device plugin prepares
/tmp/vgpulockas a directory owned by root with mode01777. Allocation fails if the parent cannot be prepared safely. This check rejects a symlink, the wrong object type, an unsafe owner or mode, and replacement during preparation.The allocation response contains one writable parent mount at
/tmp/vgpulock. When the broker is enabled, it also contains one read only mount at/tmp/vgpulock/hostpid. The plugin removes duplicate or path equivalent entries and places the parent before the nested broker mount. When the gate is disabled, it clears the reserved broker environment value and removes stale broker mounts.Protocol and failure behavior
Protocol version 1 uses an 8 byte request and a 12 byte response. The request contains the magic, version, and command. The response adds the host PID obtained from the kernel.
The server requires root for its default path, protects its directory with a root owned lock file, rejects unsafe or active socket paths, limits active handlers to 512, and applies one 500 millisecond transaction deadline. Excess or incomplete connections are closed so the client can use its fallback.
A broker startup failure prevents the device plugin from serving allocations. An unexpected broker exit stops the device plugin so Kubernetes can restart it.
Validation
Current server head
146ee027b8b0ac44c8f8b3d48581f2cf69a485e9passes the GitHub Go analysis, CodeQL, compile, unit, chart lint, end to end, lint, packaging, and DCO checks. Focused tests cover disabled and enabled startup, listener and serve failures, shutdown, plugin cleanup, restart after a plugin start failure, parent ownership and mode checks, canonical mount ordering, stale configuration cleanup, and allocation failure handling.The production C client to Go server contract passed with 300 concurrent clients and no failures.
Exact public heads
8d59bf9and935a6decompleted all 16 balanced eight GPUcuInit()cells in job181726. Mean API wall time changed from 56.373 to 22.501 seconds at N=128 and from 130.662 to 50.288 seconds at N=300. Every worker completed.Helm 3.8.1 lint and render checks pass for disabled and enabled configurations. The invalid
hostPID: falseconfiguration is rejected with the named prerequisite.Allocation response tests verify the exact environment value and the read only broker mount.
The Linux root path hardening run completed 28 recorded checks with zero failures. Its archive SHA256 is
941d0a7c3de6e4602a8c57f3e71b8c21ad3acc60cef127d07599ef8ed6f7783a. The archive covers the server experiment at private revision261567a97d2ad157094f07b8f0a007587d4481bdand the companion client through607d7e83c282f8c494b39a0c2d92e16a83bdf965. It does not bind either current public head byte for byte.The package verifier reports 46 verified artifacts, 0 pending artifacts, and 0 failures. The package ledger SHA256 is
294637ae0abbe69115c9f900442029fa0ccb335ec08ad3e8982d17f8dea1448a.The interim validation report records the source boundaries, A100 results, fallback result, Linux archive, and remaining Kubernetes checks.
Remaining work
Real kubelet allocation with the broker enabled and disabled, DaemonSet rollout and process restart, rollback, and degradation when the socket is unavailable remain untested on an isolated NVIDIA Kubernetes node. Additional container runtimes and other GPU and driver versions also remain untested.
Primary context testing shows that removing PID discovery does not remove the CUDA driver contention after
cuInit().Driver admission is not part of this public change. It remains a separate disabled follow up pending a maintainer decision.
User facing change
This adds
devicePlugin.hostPIDBroker.enabled. Its default value isfalse.AI assistance
I used AI assistance for implementation, tests, and evidence checks. I reviewed the code and results and can explain the design and failure cases.