Skip to content
Merged
Show file tree
Hide file tree
Changes from all commits
Commits
File filter

Filter by extension

Filter by extension

Conversations
Failed to load comments.
Loading
Jump to
Jump to file
Failed to load files.
Loading
Diff view
Diff view
48 changes: 16 additions & 32 deletions docs/inference/switch-inference-providers.md
Original file line number Diff line number Diff line change
Expand Up @@ -34,48 +34,47 @@ No restart is required.

## Switch to a Different Model

Switching happens through the OpenShell inference route.
Use the provider and model that match the upstream you want to use.
This is one of the cases where a NemoClaw workflow intentionally uses `openshell`; see [CLI Selection Guide](../reference/cli-selection-guide.md) for the general boundary.
Use `nemoclaw inference set` with the provider and model that match the upstream you want to use.
The command updates the OpenShell inference route and synchronizes the running OpenClaw config so `agents.defaults.model.primary` continues to match the routed model.

Pass `--sandbox <name>` when you do not want to use the default registered sandbox.

### NVIDIA Endpoints

```console
$ openshell inference set --provider nvidia-prod --model nvidia/nemotron-3-super-120b-a12b
$ nemoclaw inference set --provider nvidia-prod --model nvidia/nemotron-3-super-120b-a12b
```

### OpenAI

```console
$ openshell inference set --provider openai-api --model gpt-5.4
$ nemoclaw inference set --provider openai-api --model gpt-5.4
```

### Anthropic

```console
$ openshell inference set --provider anthropic-prod --model claude-sonnet-4-6
$ nemoclaw inference set --provider anthropic-prod --model claude-sonnet-4-6
```

### Google Gemini

```console
$ openshell inference set --provider gemini-api --model gemini-2.5-flash
$ nemoclaw inference set --provider gemini-api --model gemini-2.5-flash
```

### Compatible Endpoints

If you onboarded a custom compatible endpoint, switch models with the provider created for that endpoint:

```console
$ openshell inference set --provider compatible-endpoint --model <model-name>
$ nemoclaw inference set --provider compatible-endpoint --model <model-name>
```

```console
$ openshell inference set --provider compatible-anthropic-endpoint --model <model-name>
$ nemoclaw inference set --provider compatible-anthropic-endpoint --model <model-name>
```

If the provider itself needs to change, rerun `nemoclaw onboard`.

#### Switching from Responses API to Chat Completions

If onboarding selected `/v1/responses` but the agent fails at runtime (for
Expand Down Expand Up @@ -111,28 +110,14 @@ session and the baked image.

## Cross-Provider Switching

Switching to a different provider family (for example, from NVIDIA Endpoints to Anthropic) requires updating both the gateway route and the sandbox config.

Set the gateway route on the host:
Switching to a different provider family (for example, from NVIDIA Endpoints to Anthropic) also uses `nemoclaw inference set`.
The command updates both the gateway route and the OpenClaw provider namespace in the running sandbox config.

```console
$ openshell inference set --provider anthropic-prod --model claude-sonnet-4-6 --no-verify
$ nemoclaw inference set --provider anthropic-prod --model claude-sonnet-4-6 --no-verify
```

Then set the override env vars and recreate the sandbox so they take effect at startup:

```console
$ export NEMOCLAW_MODEL_OVERRIDE="anthropic/claude-sonnet-4-6"
$ export NEMOCLAW_INFERENCE_API_OVERRIDE="anthropic-messages"
$ nemoclaw onboard --resume --recreate-sandbox
```

The entrypoint patches `openclaw.json` at container startup with the override values.
You do not need to rebuild the image.
Remove the env vars and recreate the sandbox to revert to the original model.

`NEMOCLAW_INFERENCE_API_OVERRIDE` accepts `openai-completions` (for NVIDIA, OpenAI, Gemini, compatible endpoints) or `anthropic-messages` (for Anthropic and Anthropic-compatible endpoints).
This variable is only needed when switching between provider families.
Use `--no-verify` only when OpenShell cannot verify the provider at switch time but you have already confirmed the provider and credential.

## Tune Model Metadata

Expand Down Expand Up @@ -203,9 +188,8 @@ The output includes the active provider, model, and endpoint.

- The host keeps provider credentials.
- The sandbox continues to use `inference.local`.
- Same-provider model switches take effect immediately via the gateway route alone.
- Cross-provider switches also require `NEMOCLAW_MODEL_OVERRIDE` (and `NEMOCLAW_INFERENCE_API_OVERRIDE`) plus a sandbox recreate so the entrypoint patches the config at startup.
- Overrides are applied at container startup. Changing or removing env vars requires a sandbox recreate to take effect.
- `nemoclaw inference set` patches the selected running OpenClaw sandbox config and recomputes its config hash.
- Use `nemoclaw onboard --resume --recreate-sandbox` for build-time settings such as context window, max tokens, reasoning mode, heartbeat cadence, or image contents.
- Local Ollama and local vLLM routes use local provider tokens rather than `OPENAI_API_KEY`. Rebuilds of older local-inference sandboxes clear the stale OpenAI credential requirement automatically.

## Related Topics
Expand Down
4 changes: 2 additions & 2 deletions docs/inference/use-local-inference.md
Original file line number Diff line number Diff line change
Expand Up @@ -347,10 +347,10 @@ Refer to [Switch Inference Models](switch-inference-providers.md) for the full p
For compatible endpoints, the command is:

```console
$ openshell inference set --provider compatible-endpoint --model <model-name>
$ nemoclaw inference set --provider compatible-endpoint --model <model-name>
```

If the provider itself needs to change (for example, switching from vLLM to a cloud API), rerun `nemoclaw onboard`.
If the provider itself needs to change (for example, switching from vLLM to a cloud API), pass the new provider to `nemoclaw inference set`.

## Next Steps

Expand Down
2 changes: 1 addition & 1 deletion docs/manage-sandboxes/lifecycle.md
Original file line number Diff line number Diff line change
Expand Up @@ -137,7 +137,7 @@ Recover from a misconfigured sandbox without re-running the full onboard wizard
Change the active model or provider at runtime without rebuilding the sandbox:

```console
$ openshell inference set -g nemoclaw --model <model> --provider <provider>
$ nemoclaw inference set --model <model> --provider <provider>
```

Refer to [Switch Inference Providers](../inference/switch-inference-providers.md) for provider-specific model IDs and API compatibility notes.
Expand Down
9 changes: 4 additions & 5 deletions docs/reference/cli-selection-guide.md
Original file line number Diff line number Diff line change
Expand Up @@ -99,10 +99,9 @@ Use `openshell` when the docs explicitly call for a live OpenShell gateway opera
$ openshell term
```

- Change the live gateway inference route:
- Inspect the live gateway inference route:

```console
$ openshell inference set -g nemoclaw --provider <provider> --model <model>
$ openshell inference get -g nemoclaw
```

Expand Down Expand Up @@ -182,13 +181,13 @@ Approved endpoints are session-scoped unless you also add them to the policy thr

### Change Models or Providers

For a same-provider model switch, change the live OpenShell inference route:
Use the NemoClaw command for model or provider switches so the OpenShell route and the running OpenClaw config stay consistent:

```console
$ openshell inference set -g nemoclaw --provider nvidia-prod --model nvidia/nemotron-3-super-120b-a12b
$ nemoclaw inference set --provider nvidia-prod --model nvidia/nemotron-3-super-120b-a12b
```

For a provider-family change or a build-time OpenClaw setting change, rerun onboarding so the sandbox configuration is recreated consistently:
For a build-time OpenClaw setting change, rerun onboarding so the sandbox configuration is recreated consistently:

```console
$ nemoclaw onboard --resume --recreate-sandbox
Expand Down
15 changes: 15 additions & 0 deletions docs/reference/commands.md
Original file line number Diff line number Diff line change
Expand Up @@ -865,6 +865,21 @@ $ nemoclaw status
$ nemoclaw status --json
```

### `nemoclaw inference set`

Switch the active inference provider or model for a NemoClaw-managed OpenClaw sandbox.
The command updates the OpenShell gateway route, patches the selected running OpenClaw config so the agent's primary model identity matches the route, recomputes the config hash, and updates the NemoClaw registry.

By default, the command syncs the default registered sandbox.
Pass `--sandbox <name>` to target a different OpenClaw sandbox.

```console
$ nemoclaw inference set --provider <provider> --model <model> [--sandbox <name>] [--no-verify]
```

Supported provider names are `nvidia-prod`, `nvidia-nim`, `nvidia-router`, `openai-api`, `anthropic-prod`, `compatible-anthropic-endpoint`, `gemini-api`, `compatible-endpoint`, `ollama-local`, and `vllm-local`.
Use `--no-verify` only when OpenShell cannot verify the provider at switch time but you have already confirmed the provider and credential.

### `nemoclaw setup`

:::{warning}
Expand Down
2 changes: 1 addition & 1 deletion docs/security/best-practices.md
Original file line number Diff line number Diff line change
Expand Up @@ -119,7 +119,7 @@ flowchart TB
* - Inference
- Credential exposure, unauthorized model access, cost overruns.
- OpenShell gateway
- Yes. Use `openshell inference set`.
- Yes. Use `nemoclaw inference set`.

:::

Expand Down
87 changes: 87 additions & 0 deletions scripts/nemoclaw-start.sh
Original file line number Diff line number Diff line change
Expand Up @@ -480,6 +480,91 @@ PYOVERRIDE
[ "$_write_rc" -eq 0 ] || return "$_write_rc"
}

# ── Agent identity reconciliation with provider routing ───────────
# After the host-side `openshell inference set` swaps the gateway's
# inference provider entry, agents.defaults.model.primary in
# openclaw.json can drift from models.providers.<key>.models[0].name.
# When that happens the gateway routes requests to the new model but
# the agent self-reports the old one. Realign the two on every
# sandbox start so the next session boots with a consistent identity.
# Runs after apply_model_override so explicit NEMOCLAW_MODEL_OVERRIDE
# values still win. No-op when already in sync.
# Ref: https://github.com/NVIDIA/NemoClaw/issues/3175

reconcile_agent_model_with_provider() {
if [ "$(id -u)" -ne 0 ]; then
return 0
fi

local config_file="/sandbox/.openclaw/openclaw.json"
local hash_file="/sandbox/.openclaw/.config-hash"

[ -f "$config_file" ] || return 0

if [ -L "$config_file" ] || [ -L "$hash_file" ]; then
return 0
fi

local provider_model_ref
provider_model_ref="$(
python3 - "$config_file" <<'PYRECONCILE_READ'
import json, sys
try:
with open(sys.argv[1]) as f:
cfg = json.load(f)
except Exception:
sys.exit(0)
primary = cfg.get("agents", {}).get("defaults", {}).get("model", {}).get("primary")
provider = cfg.get("models", {}).get("providers", {}).get("inference", {})
models = provider.get("models") if isinstance(provider, dict) else None
if not isinstance(models, list) or not models:
sys.exit(0)
first = models[0]
if not isinstance(first, dict):
sys.exit(0)
provider_ref = first.get("name")
if not isinstance(provider_ref, str) or not provider_ref:
provider_id = first.get("id")
if not isinstance(provider_id, str) or not provider_id:
sys.exit(0)
provider_ref = provider_id if provider_id.startswith("inference/") else f"inference/{provider_id}"
if not isinstance(primary, str) or primary == provider_ref:
sys.exit(0)
print(provider_ref)
PYRECONCILE_READ
)"

if [ -z "$provider_model_ref" ]; then
return 0
fi

printf '[config] Reconciling agent identity with provider model: %s (#3175)\n' "$provider_model_ref" >&2

prepare_openclaw_config_for_write "$config_file" "$hash_file"
local _write_rc=0

python3 - "$config_file" "$provider_model_ref" <<'PYRECONCILE_WRITE' || _write_rc=$?
import json, sys
config_file, provider_model = sys.argv[1], sys.argv[2]
with open(config_file) as f:
cfg = json.load(f)
cfg.setdefault("agents", {}).setdefault("defaults", {}).setdefault("model", {})["primary"] = provider_model
with open(config_file, "w") as f:
json.dump(cfg, f, indent=2)
PYRECONCILE_WRITE

if [ "$_write_rc" -eq 0 ]; then
if (cd /sandbox/.openclaw && sha256sum openclaw.json >"$hash_file"); then
printf '[SECURITY] Config hash recomputed after agent identity reconciliation\n' >&2
else
_write_rc=$?
fi
fi

restore_openclaw_config_after_write "$config_file" "$hash_file"
[ "$_write_rc" -eq 0 ] || return "$_write_rc"
}

# ── Runtime CORS origin override ──────────────────────────────────
# Adds a browser origin to gateway.controlUi.allowedOrigins at startup
# without rebuilding the sandbox image. Useful for custom domains/ports.
Expand Down Expand Up @@ -1572,6 +1657,7 @@ if [ "$(id -u)" -ne 0 ]; then
fi
normalize_mutable_config_perms
apply_model_override
reconcile_agent_model_with_provider
apply_cors_override
export_gateway_token
write_runtime_shell_env
Expand Down Expand Up @@ -1664,6 +1750,7 @@ fi
verify_config_integrity_if_locked /sandbox/.openclaw
normalize_mutable_config_perms
apply_model_override
reconcile_agent_model_with_provider
apply_cors_override
export_gateway_token
write_runtime_shell_env
Expand Down
16 changes: 16 additions & 0 deletions src/commands/inference/set.ts
Original file line number Diff line number Diff line change
@@ -0,0 +1,16 @@
// SPDX-FileCopyrightText: Copyright (c) 2026 NVIDIA CORPORATION & AFFILIATES. All rights reserved.
// SPDX-License-Identifier: Apache-2.0

import Command from "../../lib/commands/inference/set";
import { withCommandDisplay } from "../../lib/cli/command-display";

export default withCommandDisplay(Command, [
{
usage: "nemoclaw inference set",
description: "Switch inference and sync OpenClaw model identity",
flags: "--provider <provider> --model <model> [--sandbox <name>] [--no-verify]",
group: "Services",
scope: "global",
order: 37,
},
]);
Loading
Loading