Skip to content

fix(onboard): probe-and-pick WSL2 host IP candidates for local inference #1864

New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

Sign up for GitHub

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Closed
gburachas wants to merge 3 commits into from
Closed

fix(onboard): probe-and-pick WSL2 host IP candidates for local inference #1864

Show file tree
Hide file tree
Changes from all commits
Commits
Show all changes
3 commits
Select commit Hold shift + click to select a range
92d111e
fix(onboard): resolve local inference timeout on WSL2 + Docker Deskto…
gburachas Apr 11, 2026
5800da8
fix(onboard): probe-and-pick WSL2 host IP candidates for local inference
gburachas Apr 14, 2026
768b4f4
docs(troubleshooting): add Windows-hosted Ollama prerequisites for WSL2
gburachas Apr 14, 2026
File filter

Filter by extension

Filter by extension
Conversations
Failed to load comments.
Loading
Jump to
Jump to file
Failed to load files.
Loading
Diff view
Diff view
49 changes: 42 additions & 7 deletions bin/lib/onboard.js
View file
Open in desktop
Original file line number Diff line number Diff line change
Expand Up @@ -3233,6 +3233,11 @@ async function setupInference(
step(4, 8, "Setting up inference provider");
runOpenshell(["gateway", "select", GATEWAY_NAME], { ignoreError: true });

// Populated by local-inference branches on WSL2 + Docker Desktop;
// persisted to the registry so later commands / diagnostics can see
// which host IP was injected into `host.openshell.internal`.
let resolvedHostIp = null;

if (
provider === "nvidia-prod" ||
provider === "nvidia-nim" ||
Expand Down Expand Up @@ -3311,12 +3316,25 @@ async function setupInference(
process.exit(applyResult.status || 1);
}
} else if (provider === "vllm-local") {
const validation = validateLocalProvider(provider, runCapture);
const platformOpts = {
isWsl: isWsl(),
isDockerDesktop: getContainerRuntime() === "docker-desktop",
};
const validation = validateLocalProvider(provider, runCapture, platformOpts);
if (!validation.ok) {
console.error(` ${validation.message}`);
process.exit(1);
const answer = (await prompt(" Continue anyway? Inference may fail at runtime. [y/N]: "))
.trim()
.toLowerCase();
if (answer !== "y") {
process.exit(1);
}
Comment on lines 3324 to +3331

@coderabbitai coderabbitai Bot Apr 14, 2026

Copy link
Copy Markdown
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

⚠️ Potential issue | 🟠 Major

Keep local-provider probe failures non-interactive in --non-interactive mode.

These branches now call prompt() unconditionally on validation failure. If nemoclaw onboard --non-interactive selects ollama or vllm, a bad probe turns into a hang instead of the hard failure the rest of the wizard uses.

Suggested fix 
 if (!validation.ok) {
   console.error(`  ${validation.message}`);
+  if (isNonInteractive()) {
+    process.exit(1);
+  }
   const answer = (await prompt("  Continue anyway? Inference may fail at runtime. [y/N]: "))
     .trim()
     .toLowerCase();
   if (answer !== "y") {
     process.exit(1);
   }
 }

Apply the same guard in both local-provider branches.

Also applies to: 3362-3369

🤖 Prompt for AI Agents 
Verify each finding against the current code and only fix it if needed.

In `@bin/lib/onboard.js` around lines 3324 - 3331, On validation failure in the
local-provider probe branches (the blocks that check validation.ok, call
prompt(), and call process.exit(1)), respect the global non-interactive flag
instead of always invoking prompt(): if the non-interactive mode flag (the same
boolean used elsewhere in the wizard, e.g., nonInteractive or
flags.nonInteractive) is set then log the validation.message and immediately
call process.exit(1) (no prompt), otherwise keep the existing interactive prompt
flow; apply this same guard to both places that call prompt() (the shown block
and the other branch around lines 3362-3369).

}
resolvedHostIp = validation.resolvedHostIp || null;
if (resolvedHostIp) {
console.log(` Resolved WSL2 host IP for container access: ${resolvedHostIp}`);
}
const baseUrl = getLocalProviderBaseUrl(provider);
const baseUrl = getLocalProviderBaseUrl(provider, resolvedHostIp ?? undefined);
const providerResult = upsertProvider("vllm-local", "openai", "OPENAI_API_KEY", baseUrl, {
OPENAI_API_KEY: "dummy",
});
Expand All @@ -3336,13 +3354,26 @@ async function setupInference(
String(LOCAL_INFERENCE_TIMEOUT_SECS),
]);
} else if (provider === "ollama-local") {
const validation = validateLocalProvider(provider, runCapture);
const platformOpts = {
isWsl: isWsl(),
isDockerDesktop: getContainerRuntime() === "docker-desktop",
};
const validation = validateLocalProvider(provider, runCapture, platformOpts);
if (!validation.ok) {
console.error(` ${validation.message}`);
console.error(" On macOS, local inference also depends on OpenShell host routing support.");
process.exit(1);
const answer = (await prompt(" Continue anyway? Inference may fail at runtime. [y/N]: "))
.trim()
.toLowerCase();
if (answer !== "y") {
process.exit(1);
}
}
resolvedHostIp = validation.resolvedHostIp || null;
if (resolvedHostIp) {
console.log(` Resolved WSL2 host IP for container access: ${resolvedHostIp}`);
}
const baseUrl = getLocalProviderBaseUrl(provider);
const baseUrl = getLocalProviderBaseUrl(provider, resolvedHostIp ?? undefined);
const providerResult = upsertProvider("ollama-local", "openai", "OPENAI_API_KEY", baseUrl, {
OPENAI_API_KEY: "ollama",
});
Expand Down Expand Up @@ -3371,7 +3402,11 @@ async function setupInference(
}

verifyInferenceRoute(provider, model);
registry.updateSandbox(sandboxName, { model, provider });
registry.updateSandbox(sandboxName, {
model,
provider,
resolvedHostIp: resolvedHostIp || null,
});
Comment on lines +3405 to +3409

@coderabbitai coderabbitai Bot Apr 14, 2026

Copy link
Copy Markdown
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

⚠️ Potential issue | 🟠 Major

This registry write never reaches the real sandbox entry.

setupInference() still runs before createSandbox(), and onboard() passes GATEWAY_NAME here (Line 4415), not the eventual sandbox name. That makes registry.updateSandbox(...) return false and silently drop model, provider, and resolvedHostIp for normal onboard runs.

Please move this persistence until after the sandbox has been registered, or store it in session state and apply it once the real sandbox entry exists.

🤖 Prompt for AI Agents 
Verify each finding against the current code and only fix it if needed.

In `@bin/lib/onboard.js` around lines 3405 - 3409, The call to
registry.updateSandbox(...) is writing using the temporary GATEWAY_NAME before
the real sandbox exists (setupInference runs before createSandbox), so its
return is false and the data is lost; fix by deferring the persistence until
after the sandbox is registered (i.e., after createSandbox completes inside
onboard) or by caching the {model, provider, resolvedHostIp} in session state
and applying them when registry.createSandbox / registry.updateSandbox is called
for the real sandbox name; specifically modify the flow around setupInference(),
createSandbox(), and the registry.updateSandbox call so you either move the
registry.updateSandbox invocation to post-createSandbox or add a session/cache
write-read that registry.updateSandbox consumes once the real sandbox entry
exists.

console.log(` ✓ Inference route set: ${provider} / ${model}`);
return { ok: true };
}
Expand Down
4 changes: 4 additions & 0 deletions docs/CONTRIBUTING.md
View file
Open in desktop
Original file line number Diff line number Diff line change
Expand Up @@ -32,6 +32,10 @@ These generated skills let AI agents walk users through NemoClaw tasks (installa
Always edit pages in `docs/`.
Never edit generated skill files under `.agents/skills/nemoclaw-user-*/`. Your changes will be overwritten on the next run.

:::{note}
**For AI coding assistants:** Do not `git add` any file under `.agents/skills/nemoclaw-user-*/` — not even when `git status` shows it as modified. The pre-commit hook regenerates and stages those files automatically from `docs/`. Staging them manually makes the commit diff harder to review and can mask out-of-date hand edits. If you changed user-facing behavior, update the matching page under `docs/` and stage only `docs/**/*.md`; the hook does the rest.
:::

### Generated skills

The current generated skills and their source pages are:
Expand Down
109 changes: 109 additions & 0 deletions docs/reference/troubleshooting.md
View file
Open in desktop
Original file line number Diff line number Diff line change
Expand Up @@ -155,6 +155,115 @@ $ nemoclaw onboard
Podman is not a tested runtime.
If onboarding or sandbox lifecycle fails, switch to a tested runtime (Docker Desktop, Colima, or Docker Engine) and rerun onboarding.

### Local inference on WSL2 + Docker Desktop

On WSL2 with Docker Desktop, the conventional `host.openshell.internal`
gateway hostname (backed by Docker's `host-gateway`) often resolves to
an IPv6 ULA or an un-routable gateway IP. That can hang the onboard
container reachability probe (step 4/8) and break inference routing to
a host-side Ollama or vLLM.

Onboarding now detects this combination and probes a list of candidate
host IPs in order:

1. The WSL distro's outbound IPv4 (`ip -4 -o route get 1.1.1.1`) —
correct when Ollama or vLLM runs **inside WSL**.
2. The WSL2 default gateway (`ip -4 -o route show default`) — correct
when Ollama or vLLM runs on the **Windows host** in NAT networking
mode.
3. Other interface addresses from `hostname -I`.

The first candidate whose container-side probe succeeds is injected
into both `OPENAI_BASE_URL` and the reachability check, and persisted
to the sandbox registry entry as `resolvedHostIp`. No manual override
is needed for either Ollama placement.
Comment on lines +176 to +179

@coderabbitai coderabbitai Bot Apr 14, 2026

Copy link
Copy Markdown
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

⚠️ Potential issue | 🟠 Major

Avoid absolute wording about resolvedHostIp persistence.

This states persistence as guaranteed, but current behavior is best-effort in onboarding flow and has a known persistence-ordering follow-up. Please soften this to "attempts to persist" to avoid misleading troubleshooting expectations.

🤖 Prompt for AI Agents 
Verify each finding against the current code and only fix it if needed.

In `@docs/reference/troubleshooting.md` around lines 176 - 179, The documentation
currently states that the first successful container-side probe is definitively
"injected into both `OPENAI_BASE_URL` and the reachability check, and persisted
to the sandbox registry entry as `resolvedHostIp`"; change this absolute
language to reflect best-effort behavior by replacing claims of guaranteed
persistence with phrases like "attempts to persist" or "is attempted to be
persisted" for `resolvedHostIp`, and clarify that `OPENAI_BASE_URL` and the
reachability check receive the candidate when the probe succeeds, noting this
occurs during the onboarding flow and that persistence ordering/guarantees are
not strict.


In WSL **mirrored** networking mode, `host.openshell.internal` already
reaches the shared network stack directly, so no override is applied.

#### Host-side prerequisites for Windows-hosted Ollama

If Ollama runs on the **Windows host** (not inside WSL), NemoClaw's
detection only helps once the host itself is actually reachable from
WSL. Run the following checks in the indicated shell.

**1. Bind Ollama to all interfaces (run in PowerShell, as Administrator):**

```powershell
# Persist across reboots; Machine scope so services also inherit it.
[System.Environment]::SetEnvironmentVariable('OLLAMA_HOST','0.0.0.0:11434','Machine')

# Stop Ollama (tray + server) and start it in a new shell so it picks
# up the new env var. Open a NEW PowerShell window first, then:
Get-Process | Where-Object { $_.ProcessName -like 'ollama*' } | Stop-Process -Force
ollama serve
```
Comment on lines +192 to +200

@coderabbitai coderabbitai Bot Apr 14, 2026

Copy link
Copy Markdown
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

⚠️ Potential issue | 🟡 Minor

Use console fenced blocks with $ prompts for CLI commands.

These CLI examples are tagged as powershell/bash, but the docs rule requires console blocks with $ prompt prefixes for command examples.

Suggested formatting adjustment 
-```powershell
+```console
+$ [System.Environment]::SetEnvironmentVariable('OLLAMA_HOST','0.0.0.0:11434','Machine')
 ...
-Get-Process | Where-Object { $_.ProcessName -like 'ollama*' } | Stop-Process -Force
-ollama serve
+$ Get-Process | Where-Object { $_.ProcessName -like 'ollama*' } | Stop-Process -Force
+$ ollama serve

</details>

  
As per coding guidelines, "CLI code blocks must use the `console` language tag with `$` prompt prefix. Flag ```bash or ```shell for CLI examples."


Also applies to: 204-207, 211-215, 232-234, 238-241, 255-257

<details>
<summary>🤖 Prompt for AI Agents</summary>

Verify each finding against the current code and only fix it if needed.

In @docs/reference/troubleshooting.md around lines 192 - 200, The CLI examples
currently use powershell/bash fenced blocks without the required $ prompt;
update the fenced code blocks to use the console language tag and prefix each
command line with a $ prompt (e.g., for the PowerShell snippet replace the

[System.Environment]::SetEnvironmentVariable('OLLAMA_HOST'...) and the
Get-Process | Where-Object ... | Stop-Process -Force and ollama serve lines with
a ```console block and add `$ ` before each command), and apply the same
transformation to the other referenced blocks (lines 204-207, 211-215, 232-234,
238-241, 255-257) so all CLI examples follow the docs guideline.


Verify the bind address (run in PowerShell):

```powershell
Get-NetTCPConnection -LocalPort 11434 -State Listen | Select LocalAddress, LocalPort
# Expect: 0.0.0.0 or [::] — NOT 127.0.0.1
```

**2. Allow inbound TCP 11434 in Windows Defender Firewall (PowerShell, Administrator):**

```powershell
New-NetFirewallRule -DisplayName "Ollama 11434 (WSL)" `
-Direction Inbound -Protocol TCP -LocalPort 11434 `
-Action Allow -Profile Any
```

**3. Switch WSL2 to mirrored networking mode.** On recent Windows 11,
WSL2 in NAT mode routes traffic through a separate Hyper-V firewall
layer that ignores standard inbound rules (you will see
`NATInboundRuleNotApplicable` on `Get-NetFirewallHyperVRule`). Mirrored
mode makes WSL share the Windows network stack directly.

Edit `%USERPROFILE%\.wslconfig` (PowerShell or Notepad on Windows):

```ini
[wsl2]
networkingMode=mirrored
```

Then apply (run in PowerShell):

```powershell
wsl --shutdown
```

Reopen your WSL terminal and verify Ollama is reachable (run in WSL):

```bash
curl --max-time 5 http://127.0.0.1:11434/api/tags
# Expect: JSON list of installed models.
```

#### If the container reachability check still fails

If onboarding still reports that the container reachability check
failed for `http://host.openshell.internal:11434`:

- Double-check the bind address (PowerShell): Ollama shows
`127.0.0.1` in `Get-NetTCPConnection` until the env var reaches the
process from a fresh shell.
- Confirm the firewall rule is enabled (PowerShell):
`Get-NetFirewallRule -DisplayName "Ollama 11434 (WSL)" | Select Enabled, Profile`.
- If you cannot switch to mirrored mode, manually set the base URL
using the WSL2 default gateway (run in WSL to find it):
```bash
ip route show default | awk '/default/ {print $3}'
```
Comment on lines +255 to +257

@coderabbitai coderabbitai Bot Apr 14, 2026

Copy link
Copy Markdown
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

⚠️ Potential issue | 🟡 Minor

Add blank lines around the nested fenced code block.

This block trips MD031 (blanks-around-fences) in the list item. Insert blank lines before and after the fence to satisfy markdownlint and keep rendering stable.

As per coding guidelines, "Follow style guide in docs/CONTRIBUTING.md for documentation."

🧰 Tools
🪛 markdownlint-cli2 (0.22.0)

[warning] 255-255: Fenced code blocks should be surrounded by blank lines

(MD031, blanks-around-fences)

[warning] 257-257: Fenced code blocks should be surrounded by blank lines

(MD031, blanks-around-fences)

🤖 Prompt for AI Agents 
Verify each finding against the current code and only fix it if needed.

In `@docs/reference/troubleshooting.md` around lines 255 - 257, The fenced code
block containing "```bash" and "ip route show default | awk '/default/ {print
$3}'" needs a blank line immediately before the opening ``` and a blank line
immediately after the closing ``` to satisfy MD031 (blanks-around-fences);
update the nested fenced block in the markdown list so there is an empty line
above and below the triple-backtick fence.

then export that IP as `OPENAI_BASE_URL=http://<gateway-ip>:11434/v1`.
- Last resort: install Docker Engine directly inside WSL2 instead of
Docker Desktop — `host-gateway` works reliably there.

Sandbox pod → host egress is a separate path from the onboard probe. If
inference calls still fail from inside a running sandbox, verify the
sandbox's `HTTP_PROXY` / `ALL_PROXY` env vars include the resolved host
IP in `NO_PROXY`.

### Invalid sandbox name

Sandbox names must follow RFC 1123 subdomain rules: lowercase alphanumeric characters and hyphens only, and must start and end with an alphanumeric character.
Expand Down
Loading