fix: address issue #5489

[jason-ma-nv]

Summary

Addresses issue #5489.

Related Issue

Fixes #5489

Changes

• Automated Claude Code fix selected by auto_fix/auto_fix_recent_issues.py.
• See the commits on this branch for the exact file-level changes.

Type of Change

• Code change (feature, bug fix, or refactor)
• Code change with doc updates
• Doc only (prose changes, no code sample modifications)
• Doc only (includes code sample changes)

Verification

• Git hooks passed during commit and push, or npx prek run --from-ref main --to-ref HEAD passes
• Targeted tests pass for changed behavior
• Full npm test passes (broad runtime changes only)
• Tests added or updated for new or changed behavior
• No secrets, API keys, or credentials committed
• Docs updated for user-facing behavior changes
• npm run docs builds without warnings (doc changes only)
• Doc pages follow the style guide (doc changes only)
• New doc pages include SPDX header and frontmatter (new pages only)

Verification details reported by automation:

• Verification (local; host unreachable): build=ok; repro tests on fix: PASS.
• Codex review (independent, advisory) posted as a PR comment.
• Addressed from codex review: Restricted the lspci hardware fallback in detectNvidiaGpuHardware to display-class PCI devices (VGA compatible controller / 3D controller / Display controller) that are also NVIDIA, instead of matching any line containing 'nvidia'. This prevents NVIDIA/Mellanox NICs and other non-GPU NVIDIA PCI devices from being marked hasNvidiaGpu and forced through blocking CDI/toolkit remediation. Added a preflight-cdi test asserting a host with only NVIDIA/Mellanox NICs is not flagged as having a GPU; the existing [Linux][Onboard] preflight skips install_nvidia_container_toolkit remediation when toolkit absent, Docker CDI configured, and nvidia-smi unavailable #5489 GPU repro still passes.
• Run focused tests for the touched module, then npm test when feasible.

---

Signed-off-by: Jason Ma jama@nvidia.com

Summary by CodeRabbit

• Bug Fixes
  • Improved NVIDIA GPU detection by falling back to PCI hardware probing when software-based checks are missing or return no results.
  • Refined CDI NVIDIA GPU spec enforcement so remediation is triggered only when specs are missing or unhealthy, and is skipped only when the operator explicitly disables GPU passthrough (not when it’s disabled implicitly).
• Tests
  • Added coverage for enforcement gating and for NVIDIA GPU classification across different lspci outcomes, including scenarios where nvidia-smi returns empty results.

[jason-ma-nv]

Codex review (automated, advisory)

• [high] src/lib/onboard/preflight.ts:395 — lspci fallback treats any line containing “nvidia” as a GPU, so hosts with NVIDIA/Mellanox NICs or other non-GPU NVIDIA PCI devices can be marked hasNvidiaGpu and hit blocking CDI/toolkit remediation.

Verdict: Needs changes; the fallback should restrict detection to GPU/display PCI classes.

Addressed in this PR

Restricted the lspci hardware fallback in detectNvidiaGpuHardware to display-class PCI devices (VGA compatible controller / 3D controller / Display controller) that are also NVIDIA, instead of matching any line containing 'nvidia'. This prevents NVIDIA/Mellanox NICs and other non-GPU NVIDIA PCI devices from being marked hasNvidiaGpu and forced through blocking CDI/toolkit remediation. Added a preflight-cdi test asserting a host with only NVIDIA/Mellanox NICs is not flagged as having a GPU; the existing #5489 GPU repro still passes.

[coderabbitai]

Note

Reviews paused

It looks like this branch is under active development. To avoid overwhelming you with review comments due to an influx of new commits, CodeRabbit has automatically paused this review. You can configure this behavior by changing the reviews.auto_review.auto_pause_after_reviewed_commits setting.

Use the following commands to manage reviews:

• @coderabbitai resume to resume automatic reviews.
• @coderabbitai review to trigger a single review.

Use the checkboxes below for quick actions:

• ▶️ Resume reviews
• 🔍 Trigger review

📝 Walkthrough

Walkthrough

Adds lspci-based NVIDIA GPU hardware detection as a fallback when nvidia-smi is absent or returns empty output, filtering to display-class PCI devices. Introduces shouldEnforceCdiNvidiaGpuSpec to centralize CDI enforcement decisions. Updates assertCdiNvidiaGpuSpecPresent and its caller to enforce only when the operator explicitly opts out, removing the prior auto-disable bypass. Adds corresponding tests for lspci detection and enforcement branches.

Changes

NVIDIA GPU lspci fallback and CDI enforcement fix

Layer / File(s)	Summary
lspci-based GPU detection and CDI enforcement helper src/lib/onboard/preflight.ts	Adds import of printRemediationActions; implements PCI parsing with display-class filtering via detectNvidiaGpuHardware; updates detectNvidiaGpu to fall back to lspci when nvidia-smi is absent or empty. Introduces exported shouldEnforceCdiNvidiaGpuSpec returning false on explicit opt-out and true when CDI spec is missing or needs repair. Updates assertCdiNvidiaGpuSpecPresent to call this helper and print remediation before exiting.
CDI enforcement gate update src/lib/onboard.ts	Removes local assertCdiNvidiaGpuSpecPresent helper. In preflight, explicitlyOptedOutGpuPassthrough is computed strictly from optedOutGpuPassthrough/noGpu inputs, removing the prior !sandboxGpuConfig.sandboxGpuEnabled auto-disable bypass. Dependency wiring is updated to source assertCdiNvidiaGpuSpecPresent from preflightUtils.
Tests for lspci detection and enforcement gate src/lib/onboard/preflight-cdi.test.ts	Adds test module import of shouldEnforceCdiNvidiaGpuSpec; adds two assessHost/planHostRemediation cases exercising the lspci path (display-class NVIDIA GPU triggers install_nvidia_container_toolkit; non-GPU NVIDIA PCI devices do not). Adds dedicated shouldEnforceCdiNvidiaGpuSpec suite covering all enforcement branches.

Estimated code review effort

🎯 3 (Moderate) | ⏱️ ~20 minutes

Poem

🐇 When nvidia-smi goes dark and quiet,
I peek at lspci — no need to riot!
Display-class GPUs? I'll find them, no doubt.
Explicit opt-out only — not auto-snuffed out.
CDI enforced, the toolkit in place,
This bunny fixed onboard at record pace! 🎉

🚥 Pre-merge checks | ✅ 4 | ❌ 2

❌ Failed checks (1 warning, 1 inconclusive)

Check name	Status	Explanation	Resolution
Docstring Coverage	⚠️ Warning	Docstring coverage is 20.00% which is insufficient. The required threshold is 80.00%.	Write docstrings for the functions missing them to satisfy the coverage threshold.
Title check	❓ Inconclusive	The title 'fix: address issue #5489' is vague and generic; it does not convey the specific nature of the changes (GPU detection fallback and enforcement logic).	Use a more descriptive title like 'fix: add lspci GPU detection fallback and enforce CDI toolkit when driver missing' to clarify the core changes.

✅ Passed checks (4 passed)

Check name	Status	Explanation
Description Check	✅ Passed	Check skipped - CodeRabbit’s high-level summary is enabled.
Linked Issues check	✅ Passed	The PR successfully addresses all requirements from issue #5489: implements lspci-based hardware detection fallback when nvidia-smi is unavailable, restricts detection to display-class NVIDIA GPUs to prevent false positives, and introduces enforcement logic to block and emit remediation when CDI toolkit is missing.
Out of Scope Changes check	✅ Passed	All changes are directly scoped to resolving issue #5489: GPU detection fallback implementation, enforcement gate logic, test coverage, and remediation action wiring remain focused on the documented problem.
Description check	✅ Passed	The PR objectives provide comprehensive context: the problem statement is clear, the solution approach is well-documented, and the rationale for both the lspci fallback and enforcement gap fixes is thoroughly explained with verification details.

_{✏️ Tip: You can configure your own custom pre-merge checks in the settings.}

✨ Finishing Touches

🧪 Generate unit tests (beta)

• Create PR with unit tests
• Commit unit tests in branch auto/fix-5489-linux-onboard-preflight-skips-install-nvidia

---

_{Comment @coderabbitai help to get the list of available commands.}

[github-code-quality]

Code Coverage Overview

Languages: TypeScript

TypeScript / code-coverage/plugin

The overall coverage in the auto/fix-5489-linux-... branch is 96%. Coverage data for the main branch is not yet available.

Show a code coverage summary of the most covered files.

File	main	auto/fix-5489-linux-... 9fbd21e	+/-
nemoclaw/src/se...cret-scanner.ts	—	100%	—
nemoclaw/src/commands/slash.ts	—	100%	—
nemoclaw/src/li...bprocess-env.ts	—	100%	—
nemoclaw/src/bl...eprint/state.ts	—	98%	—
nemoclaw/src/onboard/config.ts	—	98%	—
nemoclaw/src/bl...int/snapshot.ts	—	97%	—
nemoclaw/src/bl...print/runner.ts	—	95%	—
nemoclaw/src/co...ration-state.ts	—	94%	—
nemoclaw/src/bl...ate-networks.ts	—	94%	—
nemoclaw/src/index.ts	—	94%	—

TypeScript / code-coverage/cli

The overall coverage in the auto/fix-5489-linux-... branch is 47%. Coverage data for the main branch is not yet available.

Show a code coverage summary of the most covered files.

File	main	auto/fix-5489-linux-... 9fbd21e	+/-
src/lib/state/o...oard-session.ts	—	91%	—
src/lib/inference/local.ts	—	76%	—
src/lib/sandbox/config.ts	—	72%	—
src/lib/actions...dbox/rebuild.ts	—	69%	—
src/lib/onboard/preflight.ts	—	62%	—
src/lib/actions...licy-channel.ts	—	58%	—
src/lib/state/sandbox.ts	—	55%	—
src/lib/onboard...er-gpu-patch.ts	—	50%	—
src/lib/policy/index.ts	—	49%	—
src/lib/onboard.ts	—	18%	—

---

_{Updated June 24, 2026 16:44 UTC
Code Coverage is in Public Preview. Learn more and provide us with your feedback.}

[github-actions]

E2E Advisor Recommendation

Required E2E: cloud-onboard-e2e, onboard-negative-paths-e2e, gpu-e2e
Optional E2E: onboard-resume-e2e, onboard-repair-e2e

Dispatch hint: cloud-onboard-e2e,onboard-negative-paths-e2e,gpu-e2e

Workflow run

Full advisor summary

E2E Recommendation Advisor

Base: origin/main
Head: HEAD
Confidence: high

Required E2E

• cloud-onboard-e2e: Required because live onboarding preflight behavior changed. This validates a clean install plus full hosted non-interactive onboarding path and catches import/export, remediation, or preflight regressions that would block normal CPU hosted onboarding.
• onboard-negative-paths-e2e: Required because the PR changes a blocking preflight/remediation decision path. This job exercises onboarding validation and friendly failure behavior for negative/edge paths, providing coverage that failures remain controlled and user-facing rather than crashing.
• gpu-e2e: Required because GPU host detection and NVIDIA CDI enforcement affect real GPU passthrough and sandbox startup. This job validates the real NVIDIA GPU onboarding path with Ollama/local inference on the GPU runner.

Optional E2E

• onboard-resume-e2e: Optional adjacent confidence because src/lib/onboard.ts wires the changed CDI assertion through the initial flow phases used by resume-capable onboarding. Not strictly required by the resume compatibility rule because this PR does not modify src/lib/onboard/machine resume state orchestration.
• onboard-repair-e2e: Optional adjacent confidence for repair-mode onboarding after preflight/gateway state changes, but the PR does not directly modify repair state-machine policy.

New E2E recommendations

• gpu-cdi-preflight-negative (high): Existing GPU E2E requires nvidia-smi to work, so it does not cover the [Linux][Onboard] preflight skips install_nvidia_container_toolkit remediation when toolkit absent, Docker CDI configured, and nvidia-smi unavailable #5489 scenario this PR targets: NVIDIA GPU hardware visible via lspci, nvidia-smi unavailable or empty, Docker CDISpecDirs configured, and nvidia-container-toolkit/CDI spec missing. Add a hermetic E2E or extend onboard-negative-paths-e2e to stub host probes and assert onboarding blocks with toolkit/CDI remediation, while explicit --no-gpu bypasses enforcement.
  • Suggested test: Add a CDI hardware-fallback negative onboarding E2E scenario under onboard-negative-paths-e2e or a new targeted preflight-CDI E2E job.

Dispatch hint

• Workflow: nightly-e2e.yaml
• jobs input: cloud-onboard-e2e,onboard-negative-paths-e2e,gpu-e2e

[github-actions]

Vitest E2E Scenario Recommendation

Required Vitest E2E scenarios: onboard-resume-vitest, onboard-repair-vitest, gpu-e2e-vitest
Optional Vitest E2E scenarios: gpu-double-onboard-vitest

Dispatch required Vitest E2E scenarios:

• gh workflow run e2e-vitest-scenarios.yaml --ref <pr-head-ref> --field jobs=onboard-resume-vitest
• gh workflow run e2e-vitest-scenarios.yaml --ref <pr-head-ref> --field jobs=onboard-repair-vitest
• gh workflow run e2e-vitest-scenarios.yaml --ref <pr-head-ref> --field jobs=gpu-e2e-vitest

Workflow run

Full Vitest E2E advisor summary

Vitest E2E Scenario Advisor

Base: origin/main
Head: HEAD
Confidence: high

Required Vitest E2E scenarios

• onboard-resume-vitest: The PR changes onboarding preflight enforcement and the dependency passed into the initial onboard flow, which can affect resume-mode preflight/backstop behavior. The onboarding resume compatibility rule requires the dedicated live Vitest resume job.
  • Dispatch: gh workflow run e2e-vitest-scenarios.yaml --ref <pr-head-ref> --field jobs=onboard-resume-vitest
• onboard-repair-vitest: The same preflight/CDI enforcement path can affect repair/backstop execution from persisted onboarding sessions, so repair is required alongside resume for this state-machine-adjacent change.
  • Dispatch: gh workflow run e2e-vitest-scenarios.yaml --ref <pr-head-ref> --field jobs=onboard-repair-vitest
• gpu-e2e-vitest: The PR changes NVIDIA GPU detection, Docker CDI spec enforcement, and GPU opt-out gating. The GPU live Vitest job is the smallest wired path that exercises the real GPU/CDI onboarding surface.
  • Dispatch: gh workflow run e2e-vitest-scenarios.yaml --ref <pr-head-ref> --field jobs=gpu-e2e-vitest

Optional Vitest E2E scenarios

• gpu-double-onboard-vitest: Optional adjacent GPU coverage for repeated onboarding on a CDI-capable GPU runner; useful if reviewers want extra confidence beyond the primary GPU E2E path.
  • Dispatch: gh workflow run e2e-vitest-scenarios.yaml --ref <pr-head-ref> --field jobs=gpu-double-onboard-vitest

Relevant changed files

• src/lib/onboard.ts
• src/lib/onboard/preflight.ts

[github-actions]

PR Review Advisor — Blocked

Merge posture: Do not merge until addressed
Primary next action: Fix PRA-2: Cached resume can still bypass NVIDIA CDI/toolkit enforcement; then add or justify PRA-T1.
Open items: 1 required · 2 warnings · 0 suggestions · 8 test follow-ups
Since last review: 0 prior items resolved · 3 still apply · 0 new items found

Action checklist

• PRA-2 Fix: Cached resume can still bypass NVIDIA CDI/toolkit enforcement in src/lib/onboard/machine/handlers/preflight.ts:145
• PRA-1 Resolve or justify: Source-of-truth review needed: CDI enforcement opt-out source across fresh preflight and cached resume
• PRA-3 Resolve or justify: Add preflight-level exit and remediation coverage for the [Linux][Onboard] preflight skips install_nvidia_container_toolkit remediation when toolkit absent, Docker CDI configured, and nvidia-smi unavailable #5489 host state in src/lib/onboard/preflight-cdi.test.ts:338
• PRA-T1 Add or justify test follow-up: Runtime validation
• PRA-T2 Add or justify test follow-up: Runtime validation
• PRA-T3 Add or justify test follow-up: Runtime validation
• PRA-T4 Add or justify test follow-up: Runtime validation
• PRA-T5 Add or justify test follow-up: Add preflight-level exit and remediation coverage for the [Linux][Onboard] preflight skips install_nvidia_container_toolkit remediation when toolkit absent, Docker CDI configured, and nvidia-smi unavailable #5489 host state
• PRA-T6 Add or justify test follow-up: Acceptance clause
• PRA-T7 Add or justify test follow-up: Acceptance clause
• PRA-T8 Add or justify test follow-up: Acceptance clause

Findings index

ID	Severity	Category	Location	Required action
PRA-1	Resolve/justify	architecture	—	Identify the invalid state, source boundary, source-fix constraint, regression test, and removal condition before merging the localized behavior.
PRA-2	Required	security	src/lib/onboard/machine/handlers/preflight.ts:145	Update the resume path to pass an explicit opt-out boolean matching the fresh preflight contract: `noGpu` and any truly explicit operator opt-out only. Do not include auto-disabled sandbox GPU state or a recorded `session.gpuPassthrough === false` unless that value is known to have come from an explicit user opt-out.
PRA-3	Resolve/justify	tests	src/lib/onboard/preflight-cdi.test.ts:338	Add a narrow preflight-level regression with mocked host/GPU/probe boundaries for the #5489 host state. Assert `process.exit(1)` occurs before bridge/DNS/gateway/provider checks, and assert the printed `install_nvidia_container_toolkit` remediation includes the full apt/toolkit/CDI command sequence named by the issue.

🚨 Required before merge

Address these before merging unless a maintainer explicitly overrides the advisor with rationale.

PRA-2 Required — Cached resume can still bypass NVIDIA CDI/toolkit enforcement

• Location: src/lib/onboard/machine/handlers/preflight.ts:145
• Category: security
• Problem: The PR updates fresh preflight to pass only explicit GPU opt-out state into `assertCdiNvidiaGpuSpecPresent()`, and `shouldEnforceCdiNvidiaGpuSpec()` documents that sandbox GPU auto-disable must not suppress the CDI/toolkit gate. The cached resume path still computes `resumeOptedOutGpuPassthrough` from `noGpu || (!gpuRequested && session?.gpuPassthrough === false) || !resumeSandboxGpuConfig.sandboxGpuEnabled`, then passes that value to the same guard.
• Impact: A resumed onboarding session can advance past the host CDI/toolkit policy gate when NVIDIA hardware is present but `nvidia-smi` is unavailable and sandbox GPU auto-disables, recreating the unsafe side-effect ordering this PR is intended to prevent.
• Required action: Update the resume path to pass an explicit opt-out boolean matching the fresh preflight contract: `noGpu` and any truly explicit operator opt-out only. Do not include auto-disabled sandbox GPU state or a recorded `session.gpuPassthrough === false` unless that value is known to have come from an explicit user opt-out.
• Expected follow-up: Fix before merge or get explicit maintainer override.
• Verification: Read `src/lib/onboard/preflight.ts:654-681` for the explicit opt-out helper/guard, `src/lib/onboard.ts:1626-1631` for the fresh preflight caller, then read `src/lib/onboard/machine/handlers/preflight.ts:145-158`; the resume caller still derives opt-out from `session?.gpuPassthrough === false` and `!resumeSandboxGpuConfig.sandboxGpuEnabled`.
• Missing regression test: Add or update `src/lib/onboard/machine/handlers/preflight.test.ts` with `handlePreflightState resumes cached preflight and passes explicitOptOut=false to CDI guard when sandbox GPU auto-disables and --no-gpu was not supplied`; assert the CDI guard receives `false` and downstream bridge/DNS/gateway validation is not reached if that guard exits or throws. Add a companion `handlePreflightState resume with --no-gpu passes explicitOptOut=true to CDI guard` assertion.
• Done when: The required change is committed and verification passes: Read `src/lib/onboard/preflight.ts:654-681` for the explicit opt-out helper/guard, `src/lib/onboard.ts:1626-1631` for the fresh preflight caller, then read `src/lib/onboard/machine/handlers/preflight.ts:145-158`; the resume caller still derives opt-out from `session?.gpuPassthrough === false` and `!resumeSandboxGpuConfig.sandboxGpuEnabled`.
• Evidence: `shouldEnforceCdiNvidiaGpuSpec()` says auto-disable from unavailable `nvidia-smi` must not skip enforcement, but `handlePreflightState()` still passes `true` when `!resumeSandboxGpuConfig.sandboxGpuEnabled`. The nearby existing resume test currently expects `assertCdiNvidiaGpuSpecPresent(..., true, ...)` for `session.gpuPassthrough = false`.

Review findings by urgency: 1 required fix, 2 items to resolve/justify, 0 in-scope improvements

⚠️ Resolve or justify before merge

Investigate these in the current review; either fix them, explain why they are not applicable, or document the accepted risk.

PRA-1 Resolve/justify — Source-of-truth review needed: CDI enforcement opt-out source across fresh preflight and cached resume

• Location: not file-specific
• Category: architecture
• Problem: The advisor marked localized patch analysis as needs_followup.
• Impact: A localized workaround can preserve or hide an invalid state when the source boundary is unclear.
• Recommended action: Identify the invalid state, source boundary, source-fix constraint, regression test, and removal condition before merging the localized behavior.
• Expected follow-up: Resolve in this PR or explain why the risk is acceptable.
• Verification: Inspect the localized patch and source-of-truth review fields for a concrete invalid state, source boundary, source-fix constraint, regression test, and removal condition.
• Missing regression test: Missing cached-resume tests should assert explicitOptOut=false when sandbox GPU auto-disables without `--no-gpu`, and explicitOptOut=true when `--no-gpu` is supplied.
• Done when: The risk is fixed or explicitly justified in the PR. Verification: Inspect the localized patch and source-of-truth review fields for a concrete invalid state, source boundary, source-fix constraint, regression test, and removal condition.
• Evidence: `src/lib/onboard.ts:1626-1631` passes explicit opt-out only, but `src/lib/onboard/machine/handlers/preflight.ts:145-158` still passes `noGpu || (!gpuRequested && session?.gpuPassthrough === false) || !resumeSandboxGpuConfig.sandboxGpuEnabled`.

PRA-3 Resolve/justify — Add preflight-level exit and remediation coverage for the #5489 host state

• Location: src/lib/onboard/preflight-cdi.test.ts:338
• Category: tests
• Problem: The added tests cover `assessHost()`, `planHostRemediation()`, the `lspci` fallback, non-GPU NVIDIA PCI negatives, and the pure enforcement helper. They do not exercise the actual onboarding preflight path that must print the full remediation and exit before bridge/DNS/gateway/provider side effects.
• Impact: A future refactor could preserve the unit-level remediation action while accidentally moving, bypassing, or suppressing the fatal preflight exit, causing onboarding to continue to [2/8] or provider selection without toolkit/CDI remediation.
• Recommended action: Add a narrow preflight-level regression with mocked host/GPU/probe boundaries for the [Linux][Onboard] preflight skips install_nvidia_container_toolkit remediation when toolkit absent, Docker CDI configured, and nvidia-smi unavailable #5489 host state. Assert `process.exit(1)` occurs before bridge/DNS/gateway/provider checks, and assert the printed `install_nvidia_container_toolkit` remediation includes the full apt/toolkit/CDI command sequence named by the issue.
• Expected follow-up: Resolve in this PR or explain why the risk is acceptable.
• Verification: Read `src/lib/onboard.ts:1609-1633` for fresh preflight ordering and `src/lib/onboard/preflight-cdi.test.ts:338-378`; current coverage stops at detection, remediation planning, and helper behavior rather than invoking `preflight()` or the initial flow ordering path.
• Missing regression test: Add `preflight exits before bridge and gateway checks when lspci sees an NVIDIA display GPU, nvidia-smi is unavailable, CDI dirs are configured, and nvidia-ctk is absent`; assert non-zero exit, assert no downstream bridge/gateway/provider probes ran, and assert keyring setup, apt repo setup, `sudo apt-get update`, `sudo apt-get install -y nvidia-container-toolkit`, `sudo nvidia-ctk cdi generate --output=/etc/cdi/nvidia.yaml`, and `nvidia-ctk cdi list` are printed.
• Done when: The risk is fixed or explicitly justified in the PR. Verification: Read `src/lib/onboard.ts:1609-1633` for fresh preflight ordering and `src/lib/onboard/preflight-cdi.test.ts:338-378`; current coverage stops at detection, remediation planning, and helper behavior rather than invoking `preflight()` or the initial flow ordering path.
• Evidence: The new [Linux][Onboard] preflight skips install_nvidia_container_toolkit remediation when toolkit absent, Docker CDI configured, and nvidia-smi unavailable #5489 test asserts the `install_nvidia_container_toolkit` action exists and checks the install command plus generate prefix, but it does not invoke `preflight()` and does not assert the named keyring/repo/update/list commands or the before-[2/8] exit behavior.

💡 In-scope improvements

These are lower-risk, not throwaway. Prefer fixing them in this PR when they are local to changed code; defer only with rationale or a linked follow-up.

• None.

Test follow-ups to resolve or justify

If these cover changed behavior, prefer adding them in this PR; otherwise state why existing coverage is enough or link the follow-up.

• PRA-T1 Runtime validation — handlePreflightState resumes cached preflight and passes explicitOptOut=false to CDI guard when sandbox GPU auto-disables and --no-gpu was not supplied. The changed behavior controls a host preflight policy gate that must stop onboarding before gateway, bridge/DNS, sandbox, and provider side effects. Unit tests cover detection and remediation planning, but caller/callee and preflight-level ordering coverage are still needed.
• PRA-T2 Runtime validation — handlePreflightState resume with --no-gpu passes explicitOptOut=true to CDI guard. The changed behavior controls a host preflight policy gate that must stop onboarding before gateway, bridge/DNS, sandbox, and provider side effects. Unit tests cover detection and remediation planning, but caller/callee and preflight-level ordering coverage are still needed.
• PRA-T3 Runtime validation — preflight exits with code 1 before bridge and gateway checks when lspci sees an NVIDIA display GPU, nvidia-smi is unavailable, CDI dirs are configured, and nvidia-ctk is absent. The changed behavior controls a host preflight policy gate that must stop onboarding before gateway, bridge/DNS, sandbox, and provider side effects. Unit tests cover detection and remediation planning, but caller/callee and preflight-level ordering coverage are still needed.
• PRA-T4 Runtime validation — preflight [Linux][Onboard] preflight skips install_nvidia_container_toolkit remediation when toolkit absent, Docker CDI configured, and nvidia-smi unavailable #5489 remediation output includes keyring setup, apt repo setup, sudo apt-get update, sudo apt-get install -y nvidia-container-toolkit, sudo nvidia-ctk cdi generate --output=/etc/cdi/nvidia.yaml, and nvidia-ctk cdi list. The changed behavior controls a host preflight policy gate that must stop onboarding before gateway, bridge/DNS, sandbox, and provider side effects. Unit tests cover detection and remediation planning, but caller/callee and preflight-level ordering coverage are still needed.
• PRA-T5 Add preflight-level exit and remediation coverage for the [Linux][Onboard] preflight skips install_nvidia_container_toolkit remediation when toolkit absent, Docker CDI configured, and nvidia-smi unavailable #5489 host state — Add a narrow preflight-level regression with mocked host/GPU/probe boundaries for the [Linux][Onboard] preflight skips install_nvidia_container_toolkit remediation when toolkit absent, Docker CDI configured, and nvidia-smi unavailable #5489 host state. Assert `process.exit(1)` occurs before bridge/DNS/gateway/provider checks, and assert the printed `install_nvidia_container_toolkit` remediation includes the full apt/toolkit/CDI command sequence named by the issue.
• PRA-T6 Acceptance clause — NemoClaw v0.0.64 preflight [1/8] does not emit the `install_nvidia_container_toolkit` remediation block (apt repo bootstrap + install commands) when Docker CDI dirs are configured, `nvidia-container-toolkit` is absent, and the NVIDIA driver is not loaded (`nvidia-smi` unavailable). — add test evidence or identify existing coverage. `detectNvidiaGpu()` now falls back to display-class `lspci` hardware probing when `nvidia-smi` is missing/empty, and the new [Linux][Onboard] preflight skips install_nvidia_container_toolkit remediation when toolkit absent, Docker CDI configured, and nvidia-smi unavailable #5489 unit test drives Docker CDI dirs plus absent toolkit to an `install_nvidia_container_toolkit` action. Cached resume still has a bypass path covered by the blocker finding.
• PRA-T7 Acceptance clause — Preflight prints a single informational note and continues to [2/8] without halting. Onboard should block and provide step-by-step toolkit remediation in this scenario. — add test evidence or identify existing coverage. Fresh `preflight()` now calls `assertCdiNvidiaGpuSpecPresent()` before bridge/DNS/gateway work using explicit opt-out only, but there is no preflight-level test proving non-zero exit before [2/8], and cached resume can still skip the guard.
• PRA-T8 Acceptance clause — Run: ```bash nemoclaw onboard ``` — add test evidence or identify existing coverage. Changed tests exercise `assessHost()` and `planHostRemediation()` directly, not a `nemoclaw onboard` or `preflight()` path.

Since last review details

Current findings, using the urgency labels above:

PRA-1 Resolve/justify — Source-of-truth review needed: CDI enforcement opt-out source across fresh preflight and cached resume

• Location: not file-specific
• Category: architecture
• Problem: The advisor marked localized patch analysis as needs_followup.
• Impact: A localized workaround can preserve or hide an invalid state when the source boundary is unclear.
• Recommended action: Identify the invalid state, source boundary, source-fix constraint, regression test, and removal condition before merging the localized behavior.
• Expected follow-up: Resolve in this PR or explain why the risk is acceptable.
• Verification: Inspect the localized patch and source-of-truth review fields for a concrete invalid state, source boundary, source-fix constraint, regression test, and removal condition.
• Missing regression test: Missing cached-resume tests should assert explicitOptOut=false when sandbox GPU auto-disables without `--no-gpu`, and explicitOptOut=true when `--no-gpu` is supplied.
• Done when: The risk is fixed or explicitly justified in the PR. Verification: Inspect the localized patch and source-of-truth review fields for a concrete invalid state, source boundary, source-fix constraint, regression test, and removal condition.
• Evidence: `src/lib/onboard.ts:1626-1631` passes explicit opt-out only, but `src/lib/onboard/machine/handlers/preflight.ts:145-158` still passes `noGpu || (!gpuRequested && session?.gpuPassthrough === false) || !resumeSandboxGpuConfig.sandboxGpuEnabled`.

PRA-2 Required — Cached resume can still bypass NVIDIA CDI/toolkit enforcement

• Location: src/lib/onboard/machine/handlers/preflight.ts:145
• Category: security
• Problem: The PR updates fresh preflight to pass only explicit GPU opt-out state into `assertCdiNvidiaGpuSpecPresent()`, and `shouldEnforceCdiNvidiaGpuSpec()` documents that sandbox GPU auto-disable must not suppress the CDI/toolkit gate. The cached resume path still computes `resumeOptedOutGpuPassthrough` from `noGpu || (!gpuRequested && session?.gpuPassthrough === false) || !resumeSandboxGpuConfig.sandboxGpuEnabled`, then passes that value to the same guard.
• Impact: A resumed onboarding session can advance past the host CDI/toolkit policy gate when NVIDIA hardware is present but `nvidia-smi` is unavailable and sandbox GPU auto-disables, recreating the unsafe side-effect ordering this PR is intended to prevent.
• Required action: Update the resume path to pass an explicit opt-out boolean matching the fresh preflight contract: `noGpu` and any truly explicit operator opt-out only. Do not include auto-disabled sandbox GPU state or a recorded `session.gpuPassthrough === false` unless that value is known to have come from an explicit user opt-out.
• Expected follow-up: Fix before merge or get explicit maintainer override.
• Verification: Read `src/lib/onboard/preflight.ts:654-681` for the explicit opt-out helper/guard, `src/lib/onboard.ts:1626-1631` for the fresh preflight caller, then read `src/lib/onboard/machine/handlers/preflight.ts:145-158`; the resume caller still derives opt-out from `session?.gpuPassthrough === false` and `!resumeSandboxGpuConfig.sandboxGpuEnabled`.
• Missing regression test: Add or update `src/lib/onboard/machine/handlers/preflight.test.ts` with `handlePreflightState resumes cached preflight and passes explicitOptOut=false to CDI guard when sandbox GPU auto-disables and --no-gpu was not supplied`; assert the CDI guard receives `false` and downstream bridge/DNS/gateway validation is not reached if that guard exits or throws. Add a companion `handlePreflightState resume with --no-gpu passes explicitOptOut=true to CDI guard` assertion.
• Done when: The required change is committed and verification passes: Read `src/lib/onboard/preflight.ts:654-681` for the explicit opt-out helper/guard, `src/lib/onboard.ts:1626-1631` for the fresh preflight caller, then read `src/lib/onboard/machine/handlers/preflight.ts:145-158`; the resume caller still derives opt-out from `session?.gpuPassthrough === false` and `!resumeSandboxGpuConfig.sandboxGpuEnabled`.
• Evidence: `shouldEnforceCdiNvidiaGpuSpec()` says auto-disable from unavailable `nvidia-smi` must not skip enforcement, but `handlePreflightState()` still passes `true` when `!resumeSandboxGpuConfig.sandboxGpuEnabled`. The nearby existing resume test currently expects `assertCdiNvidiaGpuSpecPresent(..., true, ...)` for `session.gpuPassthrough = false`.

PRA-3 Resolve/justify — Add preflight-level exit and remediation coverage for the #5489 host state

• Location: src/lib/onboard/preflight-cdi.test.ts:338
• Category: tests
• Problem: The added tests cover `assessHost()`, `planHostRemediation()`, the `lspci` fallback, non-GPU NVIDIA PCI negatives, and the pure enforcement helper. They do not exercise the actual onboarding preflight path that must print the full remediation and exit before bridge/DNS/gateway/provider side effects.
• Impact: A future refactor could preserve the unit-level remediation action while accidentally moving, bypassing, or suppressing the fatal preflight exit, causing onboarding to continue to [2/8] or provider selection without toolkit/CDI remediation.
• Recommended action: Add a narrow preflight-level regression with mocked host/GPU/probe boundaries for the [Linux][Onboard] preflight skips install_nvidia_container_toolkit remediation when toolkit absent, Docker CDI configured, and nvidia-smi unavailable #5489 host state. Assert `process.exit(1)` occurs before bridge/DNS/gateway/provider checks, and assert the printed `install_nvidia_container_toolkit` remediation includes the full apt/toolkit/CDI command sequence named by the issue.
• Expected follow-up: Resolve in this PR or explain why the risk is acceptable.
• Verification: Read `src/lib/onboard.ts:1609-1633` for fresh preflight ordering and `src/lib/onboard/preflight-cdi.test.ts:338-378`; current coverage stops at detection, remediation planning, and helper behavior rather than invoking `preflight()` or the initial flow ordering path.
• Missing regression test: Add `preflight exits before bridge and gateway checks when lspci sees an NVIDIA display GPU, nvidia-smi is unavailable, CDI dirs are configured, and nvidia-ctk is absent`; assert non-zero exit, assert no downstream bridge/gateway/provider probes ran, and assert keyring setup, apt repo setup, `sudo apt-get update`, `sudo apt-get install -y nvidia-container-toolkit`, `sudo nvidia-ctk cdi generate --output=/etc/cdi/nvidia.yaml`, and `nvidia-ctk cdi list` are printed.
• Done when: The risk is fixed or explicitly justified in the PR. Verification: Read `src/lib/onboard.ts:1609-1633` for fresh preflight ordering and `src/lib/onboard/preflight-cdi.test.ts:338-378`; current coverage stops at detection, remediation planning, and helper behavior rather than invoking `preflight()` or the initial flow ordering path.
• Evidence: The new [Linux][Onboard] preflight skips install_nvidia_container_toolkit remediation when toolkit absent, Docker CDI configured, and nvidia-smi unavailable #5489 test asserts the `install_nvidia_container_toolkit` action exists and checks the install command plus generate prefix, but it does not invoke `preflight()` and does not assert the named keyring/repo/update/list commands or the before-[2/8] exit behavior.

Workflow run details

This is an automated, non-binding review; it still expects maintainers and agents to respond to each required or warning item. Treat suggestions as current-PR improvements when they touch changed code; defer only with maintainer rationale or a linked follow-up. A human maintainer must make the final merge decision.

[coderabbitai]

Actionable comments posted: 1

🤖 Prompt for all review comments with AI agents

Verify each finding against current code. Fix only still-valid issues, skip the
rest with a brief reason, keep changes minimal, and validate.

Inline comments:
In `@auto_fix_result.json`:
- Around line 1-3: The auto_fix_result.json file is an internal automation
artifact that should not be included in the codebase. Remove this file entirely
from the commit to ensure it is not shipped with the final deliverable.

🪄 Autofix (Beta)

Fix all unresolved CodeRabbit comments on this PR:

• Push a commit to this branch (recommended)
• Create a new PR with the fixes

---

ℹ️ Review info

⚙️ Run configuration

Configuration used: Path: .coderabbit.yaml

Review profile: CHILL

Plan: Enterprise

Run ID: c98b26d2-fecb-49b6-ba0f-5db37683e717

📥 Commits

Reviewing files that changed from the base of the PR and between 6c0fb04 and d60ea3b.

📒 Files selected for processing (3)

• auto_fix_result.json
• src/lib/onboard/preflight-cdi.test.ts
• src/lib/onboard/preflight.ts

[jason-ma-nv]

Update: fixed the enforcement gap (detection alone didn't fix #5489)

Runtime-verified the prior state of this PR on a real NVIDIA H100 host (recreated #5489: nvidia-smi hidden so the driver looks unloaded, nvidia-container-toolkit removed, Docker CDI configured in daemon.json, CDI spec wiped — H100 still visible via lspci). Result: onboard still did NOT block — [1/8] printed the soft note and continued to [2/8], exactly the reported bug.

Root cause: the lspci detection here correctly sets assessHost().hasNvidiaGpu and planHostRemediation() returns the blocking install_nvidia_container_toolkit action (which is what the existing unit test asserted) — but onboard's [1/8] gate assertCdiNvidiaGpuSpecPresent() early-returns whenever sandbox GPU passthrough is disabled, including the auto-disable that happens when nvidia-smi is unavailable. So the remediation was computed but never enforced. The unit test passed because it called planHostRemediation(assessHost()) directly, never driving the onboard gate.

Pushed e8da69ce0:

• The gate now keys off an explicit GPU opt-out only (--no-gpu), not the auto-disable, via a new pure shouldEnforceCdiNvidiaGpuSpec() helper in preflight.ts (unit-tested in preflight-cdi.test.ts).
• An explicit --no-gpu still skips enforcement, so a host with an unusable GPU can onboard CPU-only.

Re-verified on the same H100: onboard now blocks at [1/8] with the full install_nvidia_container_toolkit remediation (GPG keyring → apt repo → install → nvidia-ctk cdi generate → nvidia-ctk cdi list) and exits non-zero before [2/8] — matching the issue's Expected Result. (Host restored afterward.)

[coderabbitai]

Actionable comments posted: 1

🤖 Prompt for all review comments with AI agents

Verify each finding against current code. Fix only still-valid issues, skip the
rest with a brief reason, keep changes minimal, and validate.

Inline comments:
In `@src/lib/onboard.ts`:
- Around line 1594-1599: The detailed explanatory comments about issue `#5489` in
src/lib/onboard.ts at lines 1594-1599 and lines 1661-1664 are causing the file
to exceed the growth guardrail limit. Either significantly trim these comment
blocks to be more concise (removing the detailed rationale about sandbox GPU
passthrough, auto-disable behavior, and the specifics of how the CDI gate
contract works), or move the detailed `#5489` rationale explanation to
src/lib/onboard/preflight.ts or to test files, keeping only a brief reference
comment in the main onboard.ts file if needed. This will allow
src/lib/onboard.ts to remain net-neutral in size as required by CI.

🪄 Autofix (Beta)

Fix all unresolved CodeRabbit comments on this PR:

• Push a commit to this branch (recommended)
• Create a new PR with the fixes

---

ℹ️ Review info

⚙️ Run configuration

Configuration used: Path: .coderabbit.yaml

Review profile: CHILL

Plan: Enterprise

Run ID: 04a3c12e-e6bc-424f-bd90-d38ecc121175

📥 Commits

Reviewing files that changed from the base of the PR and between d60ea3b and e8da69c.

📒 Files selected for processing (3)

• src/lib/onboard.ts
• src/lib/onboard/preflight-cdi.test.ts
• src/lib/onboard/preflight.ts

🚧 Files skipped from review as they are similar to previous changes (1)

• src/lib/onboard/preflight.ts

[prekshivyas]

Both fixes are correct. lspciLineIsNvidiaGpu anchoring on PCI class label before checking vendor prevents Mellanox NICs and other non-GPU NVIDIA PCI devices from triggering GPU detection — the regex approach is standard. Separating explicitlyOptedOutGpuPassthrough from the auto-disable path and extracting shouldEnforceCdiNvidiaGpuSpec correctly fixes the bypass where auto-disable was masking enforcement.

Test coverage is thorough: three new assessHost integration cases and four unit cases for shouldEnforceCdiNvidiaGpuSpec cover the primary bug, the false-positive NIC case, the explicit --no-gpu escape hatch, and the healthy-spec no-op.

One thing to note: this is an automated fix and several verification boxes are unchecked (hooks, secrets scan, docs). Please confirm npx prek run --from-ref main --to-ref HEAD passes and no secrets were introduced before merge.

Approve pending hook verification.

[github-actions]

Selective E2E Results — ✅ All requested jobs passed

Run: 28204392989
Target ref: auto/fix-5489-linux-onboard-preflight-skips-install-nvidia
Requested jobs: cloud-onboard-e2e,onboard-negative-paths-e2e,gpu-e2e
Summary: 2 passed, 0 failed, 0 cancelled, 1 skipped

Job	Result
cloud-onboard-e2e	✅ success
gpu-e2e	⏭️ skipped
onboard-negative-paths-e2e	✅ success