fix(onboard): bind gateway to 0.0.0.0 on Docker Desktop WSL (#5513) - #5534
fix(onboard): bind gateway to 0.0.0.0 on Docker Desktop WSL (#5513)#5534abhi-0906 wants to merge 3 commits into
Conversation
|
No actionable comments were generated in the recent review. 🎉 ℹ️ Recent review info⚙️ Run configurationConfiguration used: Path: .coderabbit.yaml Review profile: CHILL Plan: Enterprise Run ID: 📒 Files selected for processing (4)
🚧 Files skipped from review as they are similar to previous changes (4)
📝 WalkthroughWalkthrough
ChangesDynamic Gateway Bind Address for Docker Desktop WSL
Sequence Diagram(s)sequenceDiagram
participant Onboard as nemoclaw onboard
participant GatewayEnv as docker-driver-gateway-env
participant WSLProbe as wsl-docker-desktop-gpu
participant DockerAdapter as adapters/docker (lazy)
Onboard->>GatewayEnv: resolveGatewayBindAddress(deps?)
GatewayEnv->>GatewayEnv: check NEMOCLAW_GATEWAY_BIND_ADDRESS in env
alt env override present
GatewayEnv-->>Onboard: return env override address
else no override
GatewayEnv->>WSLProbe: resolveWslDockerDesktopStatus()
WSLProbe->>DockerAdapter: require('../adapters/docker').dockerInfoFormat()
DockerAdapter-->>WSLProbe: docker info output
WSLProbe-->>GatewayEnv: "docker-desktop" | other
alt WSL docker-desktop
GatewayEnv-->>Onboard: return 0.0.0.0 (wildcard)
else native Linux
GatewayEnv-->>Onboard: return 127.0.0.1 (default)
end
end
Onboard->>GatewayEnv: getGatewayStartNetworkEnv(deps?)
GatewayEnv-->>Onboard: { OPENSHELL_BIND_ADDRESS: resolvedAddr, ... }
Onboard->>GatewayEnv: getGatewayPortCheckOptions(deps?)
GatewayEnv-->>Onboard: { host: resolvedAddr }
Estimated code review effort🎯 3 (Moderate) | ⏱️ ~25 minutes Suggested labels
Suggested reviewers
Poem
🚥 Pre-merge checks | ✅ 4 | ❌ 1❌ Failed checks (1 warning)
✅ Passed checks (4 passed)
✏️ Tip: You can configure your own custom pre-merge checks in the settings. ✨ Finishing Touches🧪 Generate unit tests (beta)
Comment |
b61603e to
a3590d1
Compare
|
Opened a small follow-up, #5536, for the secondary orphaned-gateway symptom in #5513: on a genuine sandbox-bridge probe failure, onboard now tears down the gateway it started (via the existing The two are complementary: this PR (#5534) keeps the probe from failing on Docker Desktop WSL in the first place; #5536 handles cleanup for any genuine probe failure on any host. They touch different files ( |
…VIDIA#5513) When onboard's [2/8] sandbox-bridge reachability probe fails, NemoClaw aborts via process.exit(1) without stopping the OpenShell gateway it started (or reused/adopted) earlier in the same run. The gateway is left running, bound to the loopback address, so the accompanying "restart Docker and re-run" hint is misleading: the stale listener survives a Docker restart and collides with the next attempt. Add an onUnreachable hook to verifySandboxBridgeGatewayReachableOrExit that fires only on a genuine unreachable result (not the soft probe_unavailable skip or a successful probe), and wire it at the three host-mode gateway paths in startDockerDriverGateway (fresh start, reuse, adopt) to tear the gateway down via the existing stopDockerDriverGatewayProcess(). That helper reads the pid file written in every path and only terminates a verified gateway process, so it is a safe no-op otherwise. Follow-up to the bind-address fix in NVIDIA#5534: that change keeps the probe from failing on Docker Desktop WSL in the first place, while this ensures any genuine probe failure no longer orphans the gateway. Signed-off-by: Abhimanyu Kumar <abhimanyukumar7290@gmail.com>
|
@abhi-0906 can you add a DCO "Signed-off-by: ..." line to the PR description, please? |
|
Thanks @cv — added the Signed-off-by line to the PR description. |
prekshivyas
left a comment
There was a problem hiding this comment.
Correct. `resolveGatewayBindAddress` gives explicit `NEMOCLAW_GATEWAY_BIND_ADDRESS` priority; the WSL detection is memoized; the lazy `require()` in `wsl-docker-desktop-gpu.ts` keeps the Docker adapter out of importers that only need the cheap detection. `OPENSHELL_SSH_GATEWAY_HOST` stays on `127.0.0.1` so client behaviour is unchanged. Tests inject `detectStatus` stubs that bypass the cache, so no teardown needed — the cache reset export is for other suites that hit the real probe.
) On Docker Desktop + WSL2, onboard [2/8] starts the host-mode OpenShell gateway bound to NemoClaw's default 127.0.0.1, but sandbox containers reach it via Docker's host-gateway route, which Docker Desktop maps to its own bridge IP rather than the WSL distro loopback. The [2/8] sandbox-bridge reachability probe then fails with Connection refused, 100% of the time, blocking onboarding on every Docker Desktop WSL host. Resolve the effective bind address with Docker Desktop WSL awareness: with no explicit NEMOCLAW_GATEWAY_BIND_ADDRESS override, bind 0.0.0.0 on Docker Desktop WSL so the host-gateway route can reach the gateway. This mirrors the containerized compat path, which already binds 0.0.0.0 for the same reason. An explicit override still wins, and the existing wildcard-bind warning now fires for the auto-widened case too. The decision is computed in the gateway env layer so it flows consistently into the launch, the runtime drift identity/marker, the preflight port check, and the systemd env file. To keep the detector out of source-level unit tests, wsl-docker-desktop-gpu now lazy-loads the Docker adapter only on an actual probe. Signed-off-by: Abhimanyu Kumar <abhimanyukumar7290@gmail.com>
a3590d1 to
c65d6cf
Compare
|
✨ Thanks for the proposed fix addressing the OpenShell gateway bind address on Docker Desktop WSL2. This proposes a way to resolve the effective bind address with Docker Desktop WSL awareness so sandbox containers can reach the gateway via host-gateway routing. Related open issues: |
|
I can't reproduce this issue on my machine with windows 11 + WSL version: 2.7.8.0 |
|
The fix only changes behavior when To reproduce on the unfixed build:
It surfaces at onboard Could you confirm whether your Docker is the Docker Desktop WSL integration or a native |
|
@abhi-0906 , |
|
@abhi-0906 , Log from my machine. The second retry worked. |
|
Reproduced your finding on a second Docker Desktop WSL setup (Win 11, Docker Desktop 29.3.1, Ubuntu-24.04 / WSL2 kernel 6.6.87). Bound a listener to
|
On Docker Desktop + WSL2, onboard's [6/8] Docker GPU patch recreates the sandbox with `--device nvidia.com/gpu=all` (CDI) and fails with "CDI device injection failed: unresolvable CDI devices nvidia.com/gpu=all", even though preflight already commits to the `--gpus` compatibility path. Docker Desktop advertises CDI spec directories, so dockerReportsNvidiaCdiDevices() returns true and buildDockerGpuModeCandidates offers CDI first; the create-only probe passes but the real recreate fails because the WSL distro exposes no usable nvidia.com/gpu spec. Thread the existing Docker Desktop WSL detection (isDockerDesktopWslRuntime, already used to gate the patch) through selectDockerGpuPatchMode into buildDockerGpuModeCandidates, and skip the CDI candidate when on Docker Desktop WSL so the patch uses `--gpus all`. Native Docker-CDI hosts are unaffected and still prefer CDI (preserving the NVIDIA#4948 gateway supervisor-wiring contract). Reached only after the [2/8] gateway-bind issue (NVIDIA#5513 / NVIDIA#5534). A follow-up is still needed for the orphaned `*-nemoclaw-gpu-backup-*` container left behind on an early patch failure. Signed-off-by: Abhimanyu Kumar <abhimanyukumar7290@gmail.com>
…VIDIA#5513) When onboard's [2/8] sandbox-bridge reachability probe fails, NemoClaw aborts via process.exit(1) without stopping the OpenShell gateway it started (or reused/adopted) earlier in the same run. The gateway is left running, bound to the loopback address, so the accompanying "restart Docker and re-run" hint is misleading: the stale listener survives a Docker restart and collides with the next attempt. Add an onUnreachable hook to verifySandboxBridgeGatewayReachableOrExit that fires only on a genuine unreachable result (not the soft probe_unavailable skip or a successful probe), and wire it at the three host-mode gateway paths in startDockerDriverGateway (fresh start, reuse, adopt) to tear the gateway down via the existing stopDockerDriverGatewayProcess(). That helper reads the pid file written in every path and only terminates a verified gateway process, so it is a safe no-op otherwise. Follow-up to the bind-address fix in NVIDIA#5534: that change keeps the probe from failing on Docker Desktop WSL in the first place, while this ensures any genuine probe failure no longer orphans the gateway. Signed-off-by: Abhimanyu Kumar <abhimanyukumar7290@gmail.com>
|
@abhi-0906 Thanks for confirmation. Closing this PR. I will create a new PR to fix the timing issue. |
## Summary
On Docker Desktop + WSL2 with an NVIDIA GPU, onboard's `[6/8]` Docker
GPU patch recreates the sandbox container with `--device
nvidia.com/gpu=all` (CDI syntax) and fails:
```
CDI device injection failed: unresolvable CDI devices nvidia.com/gpu=all
```
even though preflight already logs that it will use the `--gpus`
compatibility path. The only workaround today is `--no-gpu` /
`NEMOCLAW_SANDBOX_GPU=0`, which disables GPU entirely.
## Root cause
Docker Desktop advertises CDI spec **directories**, so
`dockerReportsNvidiaCdiDevices()` returns true and
`buildDockerGpuModeCandidates()` offers CDI as the first candidate. The
create-only probe (`docker create … true`) passes, but the real recreate
fails because the WSL distro exposes **no usable `nvidia.com/gpu`
spec**. The Docker Desktop WSL status was detected at preflight but
never reached the mode selector — `selectDockerGpuPatchMode` only
received `{image, device, backend}`.
PR #5198 (which closed #5180) added the CDI-injection failure
classification, the `--no-gpu` recovery hint, and the warning that
`NEMOCLAW_DOCKER_GPU_PATCH=0` is ignored on this runtime — but it did
not change mode selection. This is the unaddressed root cause.
## Fix
Thread the existing Docker Desktop WSL detection
(`isDockerDesktopWslRuntime()`, already used to gate the patch) through
`selectDockerGpuPatchMode` into `buildDockerGpuModeCandidates`, and skip
the CDI candidate when on Docker Desktop WSL so the patch uses `--gpus
all` — the path preflight already commits to.
- Native Docker-CDI hosts are **unaffected**: they still prefer CDI,
preserving the gateway supervisor-wiring contract from #4948.
- The flag is resolved via the cached detector in
`docker-gpu-sandbox-create.ts`, so no change to `onboard.ts` and no
extra `docker info` calls.
## Testing
- New unit tests in `docker-gpu-patch-wsl.test.ts`: CDI is skipped
(first candidate is `--gpus all`) when `dockerDesktopWsl` is true even
with CDI advertised, and CDI is still preferred otherwise.
- `tsc -p tsconfig.src.json` clean; GPU-patch suites pass (remaining
failures are pre-existing Windows-only `/etc/cdi` path tests, identical
on `main`).
## Notes / follow-up
- This step is only reached after the `[2/8]` gateway-bind issue (#5513,
fix in #5534).
- Separate latent bug still open: on an **early** patch failure the
original sandbox is already renamed to
`*-nemoclaw-gpu-backup-<timestamp>` before container creation, and only
the new container is removed — leaving an orphan backup. Happy to follow
up with a focused PR for that cleanup.
Fixes #5512.
<!-- This is an auto-generated comment: release notes by coderabbit.ai
-->
## Summary by CodeRabbit
* **Bug Fixes**
* Enhanced GPU configuration for Docker Desktop on Windows Subsystem for
Linux (WSL). The system now properly detects WSL runtime environments
and automatically selects GPU acceleration modes that work reliably on
Docker Desktop WSL, avoiding GPU modes that may not be available or
incompatible within that specific environment.
<!-- end of auto-generated comment: release notes by coderabbit.ai -->
Signed-off-by: Abhimanyu Kumar <abhimanyukumar7290@gmail.com>
Signed-off-by: Abhimanyu Kumar <abhimanyukumar7290@gmail.com>
…#5512) (#5541) ## Summary Follow-up to #5537 addressing the orphan-backup symptom in #5512. When the Docker GPU patch's recreate `docker run` fails **after** the original sandbox was already renamed to `*-nemoclaw-gpu-backup-<timestamp>`, the early-failure path removed only the failed *new* container and left the backup orphaned — stranding the sandbox with no live original, and colliding with `*-nemoclaw-gpu-backup-*` on the next retry (as reported in #5512). The supervisor-reconnect failure path already rolls back to the backup; this path didn't. ## Fix Reuse the existing rollback primitive (`rollbackToBackupContainer`) on the early-failure path: remove the failed new container, rename the backup back to the original name, and start it — restoring the pre-patch sandbox instead of leaking a backup container. - Adds `rollbackDockerGpuPatchOnRecreateFailure(refs, deps)` to `docker-gpu-patch-finalize.ts`, which resolves the real `docker start` / `docker rename` defaults (the recreate call path only carries a deps subset, so `dockerStart` would otherwise be unset). - Records `context.rolledBack` for failure diagnostics, matching the reconnect-failure path. - No `onboard.ts` change; all edits are under `src/lib/onboard/`. ## Testing - New composed test in `docker-gpu-patch-rollback.test.ts`: when `dockerRunDetached` fails, the backup is renamed back to the original and started, and is never left as an orphaned container. - `tsc -p tsconfig.src.json` clean; rollback / finalize / sandbox-create suites pass (18/18). ## Relationship to the WSL Docker Desktop chain - #5534 — gateway bind at `[2/8]` - #5536 — gateway cleanup on probe failure - #5537 — skip CDI GPU mode at `[6/8]` (makes the patch succeed on Docker Desktop WSL, so this early-failure path is no longer hit there) - this PR — restore the pre-patch sandbox for any *other* early GPU-recreate failure Refs #5512. <!-- This is an auto-generated comment: release notes by coderabbit.ai --> ## Summary by CodeRabbit * **Bug Fixes** * Improved recovery for Docker GPU patch recreation failures: if the GPU-enabled recreate step fails after the original container is renamed, the system now performs a reliable rollback to the pre-patch sandbox state and cleans up the failed recreate attempt. * **Tests** * Added a rollback-path test for a recreate-phase `docker run --detach` failure, verifying restoration of the original container name, restart behavior, and correct cleanup (including ignoring the failed recreated container). <!-- end of auto-generated comment: release notes by coderabbit.ai --> Signed-off-by: Abhimanyu Kumar <abhimanyukumar7290@gmail.com> --------- Signed-off-by: Abhimanyu Kumar <abhimanyukumar7290@gmail.com> Signed-off-by: Preksha Vyas <prekshiv@nvidia.com> Co-authored-by: Prekshi Vyas <prekshiv@nvidia.com> Co-authored-by: Claude Sonnet 4.6 <noreply@anthropic.com> Co-authored-by: Julie Yaunches <jyaunches@nvidia.com>
…VIDIA#5513) When onboard's [2/8] sandbox-bridge reachability probe fails, NemoClaw aborts via process.exit(1) without stopping the OpenShell gateway it started (or reused/adopted) earlier in the same run. The gateway is left running, bound to the loopback address, so the accompanying "restart Docker and re-run" hint is misleading: the stale listener survives a Docker restart and collides with the next attempt. Add an onUnreachable hook to verifySandboxBridgeGatewayReachableOrExit that fires only on a genuine unreachable result (not the soft probe_unavailable skip or a successful probe), and wire it at the three host-mode gateway paths in startDockerDriverGateway (fresh start, reuse, adopt) to tear the gateway down via the existing stopDockerDriverGatewayProcess(). That helper reads the pid file written in every path and only terminates a verified gateway process, so it is a safe no-op otherwise. Follow-up to the bind-address fix in NVIDIA#5534: that change keeps the probe from failing on Docker Desktop WSL in the first place, while this ensures any genuine probe failure no longer orphans the gateway. Signed-off-by: Abhimanyu Kumar <abhimanyukumar7290@gmail.com>
…VIDIA#5513) When onboard's [2/8] sandbox-bridge reachability probe fails, NemoClaw aborts via process.exit(1) without stopping the OpenShell gateway it started (or reused/adopted) earlier in the same run. The gateway is left running, bound to the loopback address, so the accompanying "restart Docker and re-run" hint is misleading: the stale listener survives a Docker restart and collides with the next attempt. Add an onUnreachable hook to verifySandboxBridgeGatewayReachableOrExit that fires only on a genuine unreachable result (not the soft probe_unavailable skip or a successful probe), and wire it at the three host-mode gateway paths in startDockerDriverGateway (fresh start, reuse, adopt) to tear the gateway down via the existing stopDockerDriverGatewayProcess(). That helper reads the pid file written in every path and only terminates a verified gateway process, so it is a safe no-op otherwise. Follow-up to the bind-address fix in NVIDIA#5534: that change keeps the probe from failing on Docker Desktop WSL in the first place, while this ensures any genuine probe failure no longer orphans the gateway. Signed-off-by: Abhimanyu Kumar <abhimanyukumar7290@gmail.com>
…VIDIA#5513) When onboard's [2/8] sandbox-bridge reachability probe fails, NemoClaw aborts via process.exit(1) without stopping the OpenShell gateway it started (or reused/adopted) earlier in the same run. The gateway is left running, bound to the loopback address, so the accompanying "restart Docker and re-run" hint is misleading: the stale listener survives a Docker restart and collides with the next attempt. Add an onUnreachable hook to verifySandboxBridgeGatewayReachableOrExit that fires only on a genuine unreachable result (not the soft probe_unavailable skip or a successful probe), and wire it at the three host-mode gateway paths in startDockerDriverGateway (fresh start, reuse, adopt) to tear the gateway down via the existing stopDockerDriverGatewayProcess(). That helper reads the pid file written in every path and only terminates a verified gateway process, so it is a safe no-op otherwise. Follow-up to the bind-address fix in NVIDIA#5534: that change keeps the probe from failing on Docker Desktop WSL in the first place, while this ensures any genuine probe failure no longer orphans the gateway. Signed-off-by: Abhimanyu Kumar <abhimanyukumar7290@gmail.com>
…VIDIA#5513) When onboard's [2/8] sandbox-bridge reachability probe fails, NemoClaw aborts via process.exit(1) without stopping the OpenShell gateway it started (or reused/adopted) earlier in the same run. The gateway is left running, bound to the loopback address, so the accompanying "restart Docker and re-run" hint is misleading: the stale listener survives a Docker restart and collides with the next attempt. Add an onUnreachable hook to verifySandboxBridgeGatewayReachableOrExit that fires only on a genuine unreachable result (not the soft probe_unavailable skip or a successful probe), and wire it at the three host-mode gateway paths in startDockerDriverGateway (fresh start, reuse, adopt) to tear the gateway down via the existing stopDockerDriverGatewayProcess(). That helper reads the pid file written in every path and only terminates a verified gateway process, so it is a safe no-op otherwise. Follow-up to the bind-address fix in NVIDIA#5534: that change keeps the probe from failing on Docker Desktop WSL in the first place, while this ensures any genuine probe failure no longer orphans the gateway. Signed-off-by: Abhimanyu Kumar <abhimanyukumar7290@gmail.com>
…VIDIA#5513) When onboard's [2/8] sandbox-bridge reachability probe fails, NemoClaw aborts via process.exit(1) without stopping the OpenShell gateway it started (or reused/adopted) earlier in the same run. The gateway is left running, bound to the loopback address, so the accompanying "restart Docker and re-run" hint is misleading: the stale listener survives a Docker restart and collides with the next attempt. Add an onUnreachable hook to verifySandboxBridgeGatewayReachableOrExit that fires only on a genuine unreachable result (not the soft probe_unavailable skip or a successful probe), and wire it at the three host-mode gateway paths in startDockerDriverGateway (fresh start, reuse, adopt) to tear the gateway down via the existing stopDockerDriverGatewayProcess(). That helper reads the pid file written in every path and only terminates a verified gateway process, so it is a safe no-op otherwise. Follow-up to the bind-address fix in NVIDIA#5534: that change keeps the probe from failing on Docker Desktop WSL in the first place, while this ensures any genuine probe failure no longer orphans the gateway. Signed-off-by: Abhimanyu Kumar <abhimanyukumar7290@gmail.com>
…VIDIA#5513) When onboard's [2/8] sandbox-bridge reachability probe fails, NemoClaw aborts via process.exit(1) without stopping the OpenShell gateway it started (or reused/adopted) earlier in the same run. The gateway is left running, bound to the loopback address, so the accompanying "restart Docker and re-run" hint is misleading: the stale listener survives a Docker restart and collides with the next attempt. Add an onUnreachable hook to verifySandboxBridgeGatewayReachableOrExit that fires only on a genuine unreachable result (not the soft probe_unavailable skip or a successful probe), and wire it at the three host-mode gateway paths in startDockerDriverGateway (fresh start, reuse, adopt) to tear the gateway down via the existing stopDockerDriverGatewayProcess(). That helper reads the pid file written in every path and only terminates a verified gateway process, so it is a safe no-op otherwise. Follow-up to the bind-address fix in NVIDIA#5534: that change keeps the probe from failing on Docker Desktop WSL in the first place, while this ensures any genuine probe failure no longer orphans the gateway. Signed-off-by: Abhimanyu Kumar <abhimanyukumar7290@gmail.com>
…VIDIA#5513) When onboard's [2/8] sandbox-bridge reachability probe fails, NemoClaw aborts via process.exit(1) without stopping the OpenShell gateway it started (or reused/adopted) earlier in the same run. The gateway is left running, bound to the loopback address, so the accompanying "restart Docker and re-run" hint is misleading: the stale listener survives a Docker restart and collides with the next attempt. Add an onUnreachable hook to verifySandboxBridgeGatewayReachableOrExit that fires only on a genuine unreachable result (not the soft probe_unavailable skip or a successful probe), and wire it at the three host-mode gateway paths in startDockerDriverGateway (fresh start, reuse, adopt) to tear the gateway down via the existing stopDockerDriverGatewayProcess(). That helper reads the pid file written in every path and only terminates a verified gateway process, so it is a safe no-op otherwise. Follow-up to the bind-address fix in NVIDIA#5534: that change keeps the probe from failing on Docker Desktop WSL in the first place, while this ensures any genuine probe failure no longer orphans the gateway. Signed-off-by: Abhimanyu Kumar <abhimanyukumar7290@gmail.com>
…VIDIA#5513) When onboard's [2/8] sandbox-bridge reachability probe fails, NemoClaw aborts via process.exit(1) without stopping the OpenShell gateway it started (or reused/adopted) earlier in the same run. The gateway is left running, bound to the loopback address, so the accompanying "restart Docker and re-run" hint is misleading: the stale listener survives a Docker restart and collides with the next attempt. Add an onUnreachable hook to verifySandboxBridgeGatewayReachableOrExit that fires only on a genuine unreachable result (not the soft probe_unavailable skip or a successful probe), and wire it at the three host-mode gateway paths in startDockerDriverGateway (fresh start, reuse, adopt) to tear the gateway down via the existing stopDockerDriverGatewayProcess(). That helper reads the pid file written in every path and only terminates a verified gateway process, so it is a safe no-op otherwise. Follow-up to the bind-address fix in NVIDIA#5534: that change keeps the probe from failing on Docker Desktop WSL in the first place, while this ensures any genuine probe failure no longer orphans the gateway. Signed-off-by: Abhimanyu Kumar <abhimanyukumar7290@gmail.com>
…VIDIA#5513) When onboard's [2/8] sandbox-bridge reachability probe fails, NemoClaw aborts via process.exit(1) without stopping the OpenShell gateway it started (or reused/adopted) earlier in the same run. The gateway is left running, bound to the loopback address, so the accompanying "restart Docker and re-run" hint is misleading: the stale listener survives a Docker restart and collides with the next attempt. Add an onUnreachable hook to verifySandboxBridgeGatewayReachableOrExit that fires only on a genuine unreachable result (not the soft probe_unavailable skip or a successful probe), and wire it at the three host-mode gateway paths in startDockerDriverGateway (fresh start, reuse, adopt) to tear the gateway down via the existing stopDockerDriverGatewayProcess(). That helper reads the pid file written in every path and only terminates a verified gateway process, so it is a safe no-op otherwise. Follow-up to the bind-address fix in NVIDIA#5534: that change keeps the probe from failing on Docker Desktop WSL in the first place, while this ensures any genuine probe failure no longer orphans the gateway. Signed-off-by: Abhimanyu Kumar <abhimanyukumar7290@gmail.com>
…VIDIA#5537) ## Summary On Docker Desktop + WSL2 with an NVIDIA GPU, onboard's `[6/8]` Docker GPU patch recreates the sandbox container with `--device nvidia.com/gpu=all` (CDI syntax) and fails: ``` CDI device injection failed: unresolvable CDI devices nvidia.com/gpu=all ``` even though preflight already logs that it will use the `--gpus` compatibility path. The only workaround today is `--no-gpu` / `NEMOCLAW_SANDBOX_GPU=0`, which disables GPU entirely. ## Root cause Docker Desktop advertises CDI spec **directories**, so `dockerReportsNvidiaCdiDevices()` returns true and `buildDockerGpuModeCandidates()` offers CDI as the first candidate. The create-only probe (`docker create … true`) passes, but the real recreate fails because the WSL distro exposes **no usable `nvidia.com/gpu` spec**. The Docker Desktop WSL status was detected at preflight but never reached the mode selector — `selectDockerGpuPatchMode` only received `{image, device, backend}`. PR NVIDIA#5198 (which closed NVIDIA#5180) added the CDI-injection failure classification, the `--no-gpu` recovery hint, and the warning that `NEMOCLAW_DOCKER_GPU_PATCH=0` is ignored on this runtime — but it did not change mode selection. This is the unaddressed root cause. ## Fix Thread the existing Docker Desktop WSL detection (`isDockerDesktopWslRuntime()`, already used to gate the patch) through `selectDockerGpuPatchMode` into `buildDockerGpuModeCandidates`, and skip the CDI candidate when on Docker Desktop WSL so the patch uses `--gpus all` — the path preflight already commits to. - Native Docker-CDI hosts are **unaffected**: they still prefer CDI, preserving the gateway supervisor-wiring contract from NVIDIA#4948. - The flag is resolved via the cached detector in `docker-gpu-sandbox-create.ts`, so no change to `onboard.ts` and no extra `docker info` calls. ## Testing - New unit tests in `docker-gpu-patch-wsl.test.ts`: CDI is skipped (first candidate is `--gpus all`) when `dockerDesktopWsl` is true even with CDI advertised, and CDI is still preferred otherwise. - `tsc -p tsconfig.src.json` clean; GPU-patch suites pass (remaining failures are pre-existing Windows-only `/etc/cdi` path tests, identical on `main`). ## Notes / follow-up - This step is only reached after the `[2/8]` gateway-bind issue (NVIDIA#5513, fix in NVIDIA#5534). - Separate latent bug still open: on an **early** patch failure the original sandbox is already renamed to `*-nemoclaw-gpu-backup-<timestamp>` before container creation, and only the new container is removed — leaving an orphan backup. Happy to follow up with a focused PR for that cleanup. Fixes NVIDIA#5512. <!-- This is an auto-generated comment: release notes by coderabbit.ai --> ## Summary by CodeRabbit * **Bug Fixes** * Enhanced GPU configuration for Docker Desktop on Windows Subsystem for Linux (WSL). The system now properly detects WSL runtime environments and automatically selects GPU acceleration modes that work reliably on Docker Desktop WSL, avoiding GPU modes that may not be available or incompatible within that specific environment. <!-- end of auto-generated comment: release notes by coderabbit.ai --> Signed-off-by: Abhimanyu Kumar <abhimanyukumar7290@gmail.com> Signed-off-by: Abhimanyu Kumar <abhimanyukumar7290@gmail.com>
…NVIDIA#5512) (NVIDIA#5541) ## Summary Follow-up to NVIDIA#5537 addressing the orphan-backup symptom in NVIDIA#5512. When the Docker GPU patch's recreate `docker run` fails **after** the original sandbox was already renamed to `*-nemoclaw-gpu-backup-<timestamp>`, the early-failure path removed only the failed *new* container and left the backup orphaned — stranding the sandbox with no live original, and colliding with `*-nemoclaw-gpu-backup-*` on the next retry (as reported in NVIDIA#5512). The supervisor-reconnect failure path already rolls back to the backup; this path didn't. ## Fix Reuse the existing rollback primitive (`rollbackToBackupContainer`) on the early-failure path: remove the failed new container, rename the backup back to the original name, and start it — restoring the pre-patch sandbox instead of leaking a backup container. - Adds `rollbackDockerGpuPatchOnRecreateFailure(refs, deps)` to `docker-gpu-patch-finalize.ts`, which resolves the real `docker start` / `docker rename` defaults (the recreate call path only carries a deps subset, so `dockerStart` would otherwise be unset). - Records `context.rolledBack` for failure diagnostics, matching the reconnect-failure path. - No `onboard.ts` change; all edits are under `src/lib/onboard/`. ## Testing - New composed test in `docker-gpu-patch-rollback.test.ts`: when `dockerRunDetached` fails, the backup is renamed back to the original and started, and is never left as an orphaned container. - `tsc -p tsconfig.src.json` clean; rollback / finalize / sandbox-create suites pass (18/18). ## Relationship to the WSL Docker Desktop chain - NVIDIA#5534 — gateway bind at `[2/8]` - NVIDIA#5536 — gateway cleanup on probe failure - NVIDIA#5537 — skip CDI GPU mode at `[6/8]` (makes the patch succeed on Docker Desktop WSL, so this early-failure path is no longer hit there) - this PR — restore the pre-patch sandbox for any *other* early GPU-recreate failure Refs NVIDIA#5512. <!-- This is an auto-generated comment: release notes by coderabbit.ai --> ## Summary by CodeRabbit * **Bug Fixes** * Improved recovery for Docker GPU patch recreation failures: if the GPU-enabled recreate step fails after the original container is renamed, the system now performs a reliable rollback to the pre-patch sandbox state and cleans up the failed recreate attempt. * **Tests** * Added a rollback-path test for a recreate-phase `docker run --detach` failure, verifying restoration of the original container name, restart behavior, and correct cleanup (including ignoring the failed recreated container). <!-- end of auto-generated comment: release notes by coderabbit.ai --> Signed-off-by: Abhimanyu Kumar <abhimanyukumar7290@gmail.com> --------- Signed-off-by: Abhimanyu Kumar <abhimanyukumar7290@gmail.com> Signed-off-by: Preksha Vyas <prekshiv@nvidia.com> Co-authored-by: Prekshi Vyas <prekshiv@nvidia.com> Co-authored-by: Claude Sonnet 4.6 <noreply@anthropic.com> Co-authored-by: Julie Yaunches <jyaunches@nvidia.com>
Summary
On Docker Desktop + WSL2,
nemoclaw onboardstep[2/8]starts the host-mode OpenShell gateway bound to NemoClaw's default127.0.0.1. Sandbox containers reach the gateway via Docker'shost-gatewayroute, which Docker Desktop maps to its own bridge IP rather than the WSL distro loopback — so the[2/8]sandbox-bridge reachability probe fails withConnection refused, 100% of the time, blocking onboarding on every Docker Desktop WSL host.Root cause
NemoClaw chooses the gateway bind address and passes it to OpenShell via
OPENSHELL_BIND_ADDRESS. In host mode that value defaulted to127.0.0.1(src/lib/core/gateway-address.ts,src/lib/onboard/docker-driver-gateway-env.ts) with no Docker Desktop WSL branch. The containerized compatibility path already binds0.0.0.0for exactly this reason; the host-mode gateway (used on modern-glibc WSL distros) did not.Fix
Resolve the effective bind address with Docker Desktop WSL awareness:
NEMOCLAW_GATEWAY_BIND_ADDRESSoverride → bind0.0.0.0on Docker Desktop WSL so the host-gateway route reaches the gateway;127.0.0.1everywhere else.127.0.0.1back on).getGatewayConnectHostmaps0.0.0.0→127.0.0.1); only the listen surface widens, and the existing wildcard-bind warning now fires for the auto-widened case.The decision is computed in the gateway env layer so it flows consistently into the launch, the runtime drift identity/marker, the preflight port check, and the systemd env file (no drift-driven restart loop). Detection is memoized so onboard runs
docker infoat most once. To keep the detector out of source-level unit tests,wsl-docker-desktop-gpunow lazy-loads the Docker adapter only on an actual probe.Testing
docker-driver-gateway-env.test.tscover wildcard bind on Docker Desktop WSL, loopback on native Linux, explicit-override precedence in both directions, and the start-env / port-check / warning wiring.tsc -p tsconfig.src.jsonclean; gateway/onboard suites pass (remaining failures are pre-existing Windows-onlychmod/path tests, identical onmain).Fixes #5513.
Summary by CodeRabbit
Release Notes
New Features
NEMOCLAW_GATEWAY_BIND_ADDRESSoverrides (including explicit wildcard binding rules).Tests
Refactor
Signed-off-by: Abhimanyu Kumar abhimanyukumar7290@gmail.com