Skip to content
Merged
Show file tree
Hide file tree
Changes from all commits
Commits
File filter

Filter by extension

Filter by extension

Conversations
Failed to load comments.
Loading
Jump to
Jump to file
Failed to load files.
Loading
Diff view
Diff view
14 changes: 10 additions & 4 deletions docs/inference/switch-models.mdx
Original file line number Diff line number Diff line change
Expand Up @@ -61,10 +61,16 @@ To validate a new model before replacing the current sandbox, onboard it under a
## Account for Shared Gateways

OpenShell exposes one inference route per gateway.
Every registered sandbox on the same gateway, including stopped sandboxes, must record the same provider and model.
NemoClaw exits before changing state when the requested model conflicts with another sandbox on the gateway.

Align the sandboxes to one route, remove a sandbox you no longer need, or onboard a sandbox with another `NEMOCLAW_GATEWAY_PORT`.
NemoClaw records each sandbox's intended model, but only one recorded route is live at a time.
Onboarding a sandbox with another valid model warns immediately before it re-points the shared route.
The warning names each affected sandbox and its recorded provider and model.
When another registered sandbox uses the same provider name, `connect` verifies the custom endpoint and API family when applicable, plus the credential environment-variable name.
It then warns and re-points the route to that sandbox's recorded provider and model.
A different custom endpoint, API family, or credential environment-variable name remains fail-closed because `connect` cannot restore that provider metadata with a provider/model-only route change.
Remove or re-onboard the conflicting sandbox before applying a different provider-global identity.

`$$nemoclaw <name> status` shows the recorded model and adds an explicit drift warning when another sandbox's route is live.
Runtime `inference set` remains fail-closed when the requested change conflicts with another registered sandbox or when route metadata is incomplete.
Comment thread
coderabbitai[bot] marked this conversation as resolved.

## Verify the Change

Expand Down
17 changes: 11 additions & 6 deletions docs/inference/switch-providers.mdx
Original file line number Diff line number Diff line change
Expand Up @@ -100,12 +100,17 @@ A rebuild reuses the recorded endpoint and cannot change it.

## Account for Shared Gateways

Every sandbox registered on one OpenShell gateway must use the same provider and model.
Custom routes must also use the same normalized endpoint and API family.
NemoClaw includes stopped sandboxes because they depend on that route when restarted.

When a route conflicts, NemoClaw exits before changing the gateway, agent configuration, or host registry and names the affected sandboxes.
Align the routes, remove a conflicting sandbox, or onboard it with another `NEMOCLAW_GATEWAY_PORT`.
OpenShell exposes one live inference route per gateway, while NemoClaw records each sandbox's intended provider and model.
Onboarding a sandbox with a different provider or model warns immediately before it re-points the shared route.
When another sandbox uses the same provider name, its custom endpoint, API family, and credential environment-variable name must match.
Onboarding refuses before provider or route mutation when those provider-global fields differ.
The warning names every affected registered sandbox and its recorded provider and model.
Those sandboxes use the new live route until a later onboarding run or, when endpoint, API-family, and credential environment-variable identity already match, `$$nemoclaw <name> connect` warns and re-points its provider and model again.
`connect` refuses a different custom endpoint, API family, or credential environment-variable name because its provider/model-only route change cannot safely reconstruct that metadata.

`$$nemoclaw <name> status` continues to show that sandbox's recorded provider and model and prints an explicit warning when the live shared route differs.
Incomplete custom-route metadata, provider-global identity conflicts, and invalid gateway bindings remain hard errors because NemoClaw cannot identify or restore those routes safely.
Runtime `inference set` also refuses a change that would leave another registered sandbox with a different recorded route.

Do not run `openshell inference set` directly on a shared NemoClaw gateway because that bypasses registry compatibility checks.
Comment thread
coderabbitai[bot] marked this conversation as resolved.

Expand Down
6 changes: 5 additions & 1 deletion docs/inference/view-active-inference-route.mdx
Original file line number Diff line number Diff line change
Expand Up @@ -65,7 +65,11 @@ Run the sandbox status command when you also need service, messaging, and infere
$$nemoclaw <name> status
```

The status output includes the configured provider, model, and endpoint with the rest of the sandbox state.
The status output shows the provider and model recorded for that sandbox.
When the gateway's live shared route differs, status also prints an explicit warning with both the live and recorded routes.
The JSON report exposes them separately as `recordedRoute` and `liveRoute` and repeats both under `routeDrift` when they differ.
The existing schema-version `1` `provider` and `model` fields remain live-first when the gateway route is readable and fall back to the recorded route otherwise.
When `routeDrift.canConnect` is `false`, incomplete route metadata, an invalid gateway binding, or a provider-global identity conflict prevents `connect` from restoring the recorded route safely.
Use the route verification workflow when you need to prove that an inference request succeeds through the sandbox path.

## Related Topics
Expand Down
27 changes: 21 additions & 6 deletions docs/reference/commands.mdx
Original file line number Diff line number Diff line change
Expand Up @@ -819,10 +819,12 @@ If another terminal is already connected to the sandbox, `connect` prints a note
An unknown model slug, malformed extra-args JSON, or a gated model (for example `deepseek-r1-distill-70b`) with no `HF_TOKEN` or `HUGGING_FACE_HUB_TOKEN` exits non-zero with the same error the installer would emit, before any sandbox readiness probe or SSH attach.
Unset the managed-vLLM variable, or fix the value, before retrying.

When the live OpenShell gateway inference route differs from the route recorded in the NemoClaw registry, `connect` checks every registered sandbox on that gateway before attempting a repair.
It realigns the route only when those registry entries are compatible with the requested provider and model.
If another sandbox records a conflicting route, `connect` exits non-zero without changing the gateway and names the affected sandboxes.
Use `$$nemoclaw inference set --provider <provider> --model <model>` to make an intentional compatible route change.
Before reading or changing the live OpenShell gateway inference route, `connect` verifies that every same-name provider has matching custom endpoint and API-family metadata when applicable, plus a matching credential environment-variable name.
When the live route differs from the route recorded for the target sandbox, `connect` then warns and re-points the shared route to that sandbox's recorded provider and model.
This switch lets registered sandboxes with different valid routes time-share the gateway.
Different custom endpoints, API families, or credential environment-variable names, incomplete route metadata, and invalid gateway bindings remain hard errors because a provider/model-only connect-time change cannot reconstruct or prove that identity.
Remove or re-onboard the conflicting sandbox before applying a different provider-global identity.
Use `$$nemoclaw inference set --provider <provider> --model <model>` to make an intentional compatible route change outside the connect flow.
Before it opens SSH, `connect` probes `https://inference.local/v1/models` from inside the sandbox with the selected agent's trusted CA and proxy context.
HTTP `200` through `499` confirms that the route is reachable.
When the probe returns a recognized broken result, `connect` attempts DNS or route repair and verifies the route again.
Expand Down Expand Up @@ -1160,7 +1162,11 @@ Use this form when you care about a specific sandbox's live OpenShell state, age
Do not pass a sandbox name to `$$nemoclaw status`; that command is the global all-sandbox/service overview.

Pass `--json` to emit a structured per-sandbox report instead of the text renderer.
The JSON output includes at least `schemaVersion`, `name`, `found`, `agent`, `agentDisplayName`, `agentRuntime`, `dcodeAutoApprovalMode`, `model`, `provider`, `phase`, `gatewayState`, `inferenceHealth`, `rpcIssue`, `hostGpuDetected`, `sandboxGpuEnabled`, `sandboxGpuMode`, `sandboxGpuDevice`, `openshellDriver`, `openshellVersion`, `policies`, `failureLayer`, `terminalRuntimeHealth`, and `dockerPaused`.
The JSON output includes at least `schemaVersion`, `name`, `found`, `agent`, `agentDisplayName`, `agentRuntime`, `dcodeAutoApprovalMode`, `model`, `provider`, `recordedRoute`, `liveRoute`, `routeDrift`, `phase`, `gatewayState`, `inferenceHealth`, `rpcIssue`, `hostGpuDetected`, `sandboxGpuEnabled`, `sandboxGpuMode`, `sandboxGpuDevice`, `openshellDriver`, `openshellVersion`, `policies`, `failureLayer`, `terminalRuntimeHealth`, and `dockerPaused`.
The schema-version `1` `model` and `provider` fields keep their established live-route meaning when the gateway route is readable.
Use `recordedRoute` for the sandbox's durable provider and model and `liveRoute` for the gateway-global route.
When the live shared route differs, text output prints both routes and JSON output sets `routeDrift.live`, `routeDrift.recorded`, and `routeDrift.canConnect`.
When `routeDrift.canConnect` is `false`, `connect` cannot safely restore the recorded route because provider-global identity differs or required route or gateway metadata is incomplete.
`openshellDriver` and `openshellVersion` are always strings (falling back to `"unknown"` when the registry has no value), so consumers can rely on `typeof` checks.
`failureLayer` is `null` when no preflight failure was detected and otherwise one of `docker_unreachable`, `sandbox_container_stopped`, or `sandbox_dashboard_port_conflict`; when set, `inferenceHealth` is suppressed to `null` so automation does not see a stale remote-provider healthy status during a local outage.
`dockerPaused` is `true` when NemoClaw detects that the Docker-driver sandbox container is paused.
Expand Down Expand Up @@ -2860,8 +2866,17 @@ Run `$$nemoclaw <name> shields down`, apply the inference change, then run `$$ne
Each OpenShell gateway exposes one inference route to every sandbox registered on that gateway.
Before changing the route, NemoClaw compares the requested provider and model with every same-gateway registry entry, including stopped sandboxes.
Custom compatible routes must also have matching normalized endpoint URLs and API families.
Provider-global credential environment-variable names must also match for the same provider name.
If a route conflicts or a legacy custom route lacks enough endpoint or API-family metadata to prove compatibility, the command exits non-zero before changing the OpenShell route, agent config, or host registry and names the conflicting sandboxes.
Align those sandboxes to the same route, remove the conflicting sandbox, or onboard it with another `NEMOCLAW_GATEWAY_PORT`.
Align those sandboxes to the same route or remove a conflicting sandbox that you no longer need.

Onboarding allows provider/model time-sharing but does not replace provider-global configuration used by another registered sandbox.
When another sandbox records a different provider or model, NemoClaw warns immediately before re-pointing the shared route.
For the same provider name, any custom endpoint, API family, and credential environment-variable name must match.
When those provider-global fields differ, onboarding exits before updating the provider or inference route.
The warning names each affected sandbox and its recorded provider and model.
Connecting to a sandbox warns before re-pointing the route to that sandbox's recorded provider and model.
`$$nemoclaw <name> status` shows the recorded route and reports any live-route drift explicitly.
Comment thread
coderabbitai[bot] marked this conversation as resolved.

```bash
$$nemoclaw inference set --provider <provider> --model <model> [--sandbox <name>] [--no-verify] [--endpoint-url <url>] [--credential-env <ENV>] [--inference-api <api>]
Expand Down
40 changes: 17 additions & 23 deletions docs/reference/troubleshooting.mdx
Original file line number Diff line number Diff line change
Expand Up @@ -356,19 +356,23 @@ $$nemoclaw list
`$$nemoclaw list` prints the recorded dashboard URL for each sandbox.

OpenShell currently exposes one inference route per gateway.
Sandboxes that share a gateway must record the same provider and model.
Custom compatible routes must also record the same normalized endpoint and API family.
An OpenClaw sandbox using the native Anthropic Messages frontend and a Hermes sandbox using the managed OpenAI frontend need separate gateway ports, even when they use the same custom endpoint and model.
NemoClaw includes stopped sandboxes in this check because they use the gateway route when restarted.

If onboarding, `inference set`, or connect-time route repair reports a shared-gateway conflict, choose one of these recovery paths:

- Align every named sandbox to the same route.
- Remove a conflicting sandbox with `$$nemoclaw <name> destroy` when you no longer need it.
- Onboard the sandbox against another gateway port when it needs a different route.

An aligned group cannot move to a wholly new route one sandbox at a time because the first change would conflict with the remaining registry rows.
Back up each workspace, remove or migrate the peer sandboxes, switch the last sandbox on the original gateway, then re-onboard the peers with that same route or with separate gateway ports.
NemoClaw records each sandbox's intended provider and model, but the gateway can expose only one of those routes at a time.
Onboarding with a different valid route warns immediately before changing the live route.
The warning names every affected registered sandbox and its recorded provider and model.
The affected sandboxes use that route until another onboarding run or `$$nemoclaw <name> connect` warns and re-points the provider and model.
When another sandbox uses the same provider name, its custom endpoint, API family, and credential environment-variable name must match before onboarding or `connect` can re-point the route.
Onboarding exits before provider or route mutation when that provider-global identity differs.
`connect` refuses the same conflict because its provider/model-only route change cannot safely reconstruct that metadata.

Run `$$nemoclaw <name> status` to compare the sandbox's recorded route with the live shared route.
The normal Model and Provider lines remain the sandbox's recorded configuration.
A separate warning shows the live and recorded routes when they differ.
The JSON status report exposes `recordedRoute` and `liveRoute` explicitly and repeats them in `routeDrift` when they differ.
Schema-version `1` `model` and `provider` remain live-first when the gateway route is readable and fall back to the recorded route otherwise.
When `routeDrift.canConnect` is `false`, the route cannot be restored safely because provider-global identity differs or required route or gateway metadata is incomplete.

If `inference set` reports a valid shared-route conflict, align the named sandbox records or remove a sandbox you no longer need.
If onboarding or `connect` reports a provider-global identity conflict, align the same-name provider's custom endpoint, API family, and credential environment-variable name across the named sandboxes, or remove a conflicting sandbox you no longer need.

If the error names incomplete legacy custom-route metadata, back up and remove the affected sandbox, then re-onboard it with an explicit custom endpoint and API family.
For an OpenAI-compatible route, replace the example endpoint, model, and sandbox name in this recovery sequence:
Expand All @@ -385,16 +389,6 @@ NEMOCLAW_PROVIDER=custom \
If the error names an invalid gateway binding, restore the affected row's known-good `gatewayName` and `gatewayPort` metadata from a trusted backup; otherwise back up and remove the sandbox, then re-onboard it.
Do not guess or copy a binding from another sandbox because lifecycle commands use it to select the gateway.

For example, keep the first sandbox on the default gateway and assign a separate gateway to the second sandbox:

```bash
$$nemoclaw onboard --name first-sandbox
NEMOCLAW_GATEWAY_PORT=8990 $$nemoclaw onboard --name second-sandbox
```

NemoClaw derives a separate gateway name and state directory from the second port.
Using a different dashboard port alone does not create a separate inference route.

</AgentOnly>

## Onboarding
Expand Down
43 changes: 37 additions & 6 deletions src/lib/actions/sandbox/connect-inference-gateway.ts
Original file line number Diff line number Diff line change
@@ -1,11 +1,39 @@
// SPDX-FileCopyrightText: Copyright (c) 2026 NVIDIA CORPORATION & AFFILIATES. All rights reserved.
// SPDX-License-Identifier: Apache-2.0

import { assertGatewayRouteCompatibility } from "../../inference/gateway-route-compatibility";
import {
checkGatewayRouteCompatibility,
GatewayRouteConflictError,
isAdvisoryProviderModelRouteConflict,
} from "../../inference/gateway-route-compatibility";
import { LOCAL_INFERENCE_TIMEOUT_SECS } from "../../onboard/env";
import type { SandboxEntry } from "../../state/registry";
import * as registry from "../../state/registry";

function sandboxGatewayRouteCompatibility(
sandboxName: string,
sb: SandboxEntry,
gatewayName: string,
sandboxes: readonly SandboxEntry[],
) {
return checkGatewayRouteCompatibility({
gatewayName,
sandboxName,
route: sb,
sandboxes,
});
}

export function canSandboxGatewayRouteRealign(
sandboxName: string,
sb: SandboxEntry,
gatewayName: string,
sandboxes: readonly SandboxEntry[] = registry.listSandboxes().sandboxes,
): boolean {
const result = sandboxGatewayRouteCompatibility(sandboxName, sb, gatewayName, sandboxes);
return result.ok || isAdvisoryProviderModelRouteConflict(result);
}

export function buildGatewayInferenceGetArgs(gatewayName: string): string[] {
return ["inference", "get", "-g", gatewayName];
}
Expand Down Expand Up @@ -37,10 +65,13 @@ export function assertSandboxGatewayRouteCompatible(
sb: SandboxEntry,
gatewayName: string,
): void {
assertGatewayRouteCompatibility({
gatewayName,
const result = sandboxGatewayRouteCompatibility(
sandboxName,
route: sb,
sandboxes: registry.listSandboxes().sandboxes,
});
sb,
gatewayName,
registry.listSandboxes().sandboxes,
);
if (!result.ok && !isAdvisoryProviderModelRouteConflict(result)) {
throw new GatewayRouteConflictError(result);
}
}
Loading
Loading