Skip to content
Merged
Show file tree
Hide file tree
Changes from all commits
Commits
File filter

Filter by extension

Filter by extension

Conversations
Failed to load comments.
Loading
Jump to
Jump to file
Failed to load files.
Loading
Diff view
Diff view
8 changes: 7 additions & 1 deletion ci/platform-matrix.json
Original file line number Diff line number Diff line change
Expand Up @@ -83,6 +83,12 @@
"endpoint_type": "OpenAI-compatible",
"notes": "Hosted models on integrate.api.nvidia.com"
},
{
"name": "OpenRouter",
"status": "tested",
"endpoint_type": "OpenAI-compatible",
"notes": "First-class onboarding route for OpenClaw, Hermes, and LangChain Deep Agents Code. NemoClaw registers the `openrouter-api` provider through OpenShell's `openai` profile with `OPENAI_BASE_URL=https://openrouter.ai/api/v1`; host-side validation and catalog probes send the default OpenRouter attribution headers."
},
{
"name": "OpenAI",
"status": "tested",
Expand All @@ -93,7 +99,7 @@
"name": "Other OpenAI-compatible endpoint",
"status": "caveated",
"endpoint_type": "Custom OpenAI-compatible",
"notes": "Adapter path validated against OpenRouter as the `compatible-endpoint` provider with `openrouter/auto` (see `src/lib/inference/config.test.ts:208`); the onboarding prompt that surfaces OpenRouter as the worked example is in `handleRemoteProviderSelection` in `src/lib/onboard.ts`. Behavior on other OpenAI-compatible proxies, gateways, and self-hosted implementations may vary; this row claims the adapter, not the universe of compatible endpoints."
"notes": "Custom base-URL adapter for servers that implement OpenAI-compatible `/v1/chat/completions` or `/v1/responses`. Behavior on OpenAI-compatible proxies, gateways, and self-hosted implementations may vary; this row claims the adapter, not the universe of compatible endpoints."
},
{
"name": "Anthropic",
Expand Down
2 changes: 1 addition & 1 deletion ci/test-file-size-budget.json
Original file line number Diff line number Diff line change
Expand Up @@ -9,7 +9,7 @@
"test/install-preflight.test.ts": 3934,
"test/nemoclaw-start.test.ts": 4826,
"test/onboard-messaging.test.ts": 2049,
"test/onboard-selection.test.ts": 4771,
"test/onboard-selection.test.ts": 4769,
"test/onboard.test.ts": 4039,
"test/policies.test.ts": 2243
}
Expand Down
18 changes: 10 additions & 8 deletions docs/get-started/quickstart.mdx
Original file line number Diff line number Diff line change
Expand Up @@ -131,13 +131,14 @@ The inference provider prompt presents a numbered list.

```text
1) NVIDIA Endpoints
2) OpenAI
3) Other OpenAI-compatible endpoint
4) Anthropic
5) Other Anthropic-compatible endpoint
6) Google Gemini
7) Local Ollama (localhost:11434)
8) Model Router (experimental)
2) OpenRouter
3) OpenAI
4) Other OpenAI-compatible endpoint
5) Anthropic
6) Other Anthropic-compatible endpoint
7) Google Gemini
8) Local Ollama (localhost:11434)
9) Model Router (experimental)
Choose [1]:
```

Expand All @@ -162,8 +163,9 @@ For Ollama, vLLM, NIM, and compatible local servers, refer to [Use a Local Infer
| Option | Use when | Credential variable |
|---|---|---|
| NVIDIA Endpoints | You want hosted models from `build.nvidia.com`, including hosted Nemotron models. | `NVIDIA_INFERENCE_API_KEY` |
| OpenRouter | You want OpenRouter as a managed hosted OpenAI-compatible provider. | `OPENROUTER_API_KEY` |
| OpenAI | You want the OpenAI API at `https://api.openai.com/v1`. | `OPENAI_API_KEY` |
| Other OpenAI-compatible endpoint | You have OpenRouter, LocalAI, llama.cpp, vLLM, NIM, SGLang, an enterprise gateway, or another `/v1/chat/completions` endpoint. | `COMPATIBLE_API_KEY` |
| Other OpenAI-compatible endpoint | You have LocalAI, llama.cpp, vLLM, NIM, SGLang, an enterprise gateway, or another `/v1/chat/completions` endpoint. | `COMPATIBLE_API_KEY` |
| Anthropic | You want the Anthropic Messages API. | `ANTHROPIC_API_KEY` |
| Other Anthropic-compatible endpoint | You have a Claude proxy, Bedrock-compatible gateway, or self-hosted `/v1/messages` endpoint. | `COMPATIBLE_ANTHROPIC_API_KEY` |
| Google Gemini | You want Google's OpenAI-compatible Gemini endpoint. | `GEMINI_API_KEY` |
Expand Down
24 changes: 15 additions & 9 deletions docs/inference/inference-options.mdx
Original file line number Diff line number Diff line change
Expand Up @@ -64,8 +64,9 @@ This setting is generated when NemoClaw builds or recreates the sandbox image.
| Provider | Status | Endpoint type | Notes |
|----------|--------|---------------|-------|
| NVIDIA Endpoints | Tested | OpenAI-compatible | Hosted models on integrate.api.nvidia.com |
| OpenRouter | Tested | OpenAI-compatible | First-class onboarding route for OpenClaw, Hermes, and LangChain Deep Agents Code. NemoClaw registers the `openrouter-api` provider through OpenShell's `openai` profile with `OPENAI_BASE_URL=https://openrouter.ai/api/v1`; host-side validation and catalog probes send the default OpenRouter attribution headers. |
| OpenAI | Tested | Native OpenAI-compatible | Uses OpenAI model IDs |
| Other OpenAI-compatible endpoint | Tested with limitations | Custom OpenAI-compatible | Adapter path validated against OpenRouter as the `compatible-endpoint` provider with `openrouter/auto` (see `src/lib/inference/config.test.ts:208`); the onboarding prompt that surfaces OpenRouter as the worked example is in `handleRemoteProviderSelection` in `src/lib/onboard.ts`. Behavior on other OpenAI-compatible proxies, gateways, and self-hosted implementations may vary; this row claims the adapter, not the universe of compatible endpoints. |
| Other OpenAI-compatible endpoint | Tested with limitations | Custom OpenAI-compatible | Custom base-URL adapter for servers that implement OpenAI-compatible `/v1/chat/completions` or `/v1/responses`. Behavior on OpenAI-compatible proxies, gateways, and self-hosted implementations may vary; this row claims the adapter, not the universe of compatible endpoints. |
| Anthropic | Tested | Native Anthropic | Uses anthropic-messages |
| Other Anthropic-compatible endpoint | Tested with limitations | Custom Anthropic-compatible | Adapter path validated with AWS Bedrock (`src/lib/onboard/bedrock-runtime.ts`). Behavior on other Anthropic-compatible proxies and gateways may vary; this row claims the adapter, not the universe of compatible endpoints. |
| Google Gemini | Tested | OpenAI-compatible | Uses Google's OpenAI-compatible endpoint |
Expand All @@ -79,12 +80,14 @@ This setting is generated when NemoClaw builds or recreates the sandbox image.
## Provider Options

The onboard wizard presents the following provider options by default.
The first six are always available.
The first seven are always available.
Ollama appears when you have installed or started it on the host.
Local vLLM appears when NemoClaw detects a running vLLM server.
The managed install/start vLLM entry appears by default on DGX Spark and DGX Station, and appears on generic Linux NVIDIA GPU hosts after opt-in.
During interactive NVIDIA Endpoints onboarding, NemoClaw loads NVIDIA's public featured model catalog once per onboarding session and reports progress before displaying the model picker.
It excludes retired or unsafe choices and corrects known catalog lag before displaying the result.
OpenRouter uses the same catalog-backed picker flow with its own OpenRouter featured model catalog, provider route, and credential check.
NVIDIA Endpoints excludes NVIDIA-retired or unsafe choices and corrects known catalog lag before displaying the result.
OpenRouter does not inherit the NVIDIA-only retired-model filter, so a model can appear in OpenRouter onboarding after it leaves NVIDIA Endpoints.
If the catalog is unavailable, malformed, or contains no safe model IDs, the wizard warns you and uses the bundled fallback list shown below.
Nemotron 3 Super remains the shared default for OpenClaw and Hermes when it is present.
LangChain Deep Agents Code uses Nemotron 3 Ultra as its NVIDIA Endpoints default.
Expand All @@ -96,8 +99,9 @@ NemoClaw neither displays nor accepts an unsafe `NEMOCLAW_MODEL` value as the ma
| Option | Description | Curated models |
|--------|-------------|----------------|
| NVIDIA Endpoints | Routes to models hosted on [build.nvidia.com](https://build.nvidia.com). You can also enter any model ID from the catalog. Set `NVIDIA_INFERENCE_API_KEY`. | Nemotron 3 Ultra 550B, Nemotron 3 Super 120B, Minimax M3 |
| OpenRouter | Routes OpenClaw, Hermes, and Deep Agents Code through OpenRouter's OpenAI-compatible Chat Completions endpoint. Set `OPENROUTER_API_KEY`; valid keys begin with `sk-or-`. NemoClaw validates manual model entries with OpenRouter and keeps the sandbox on `https://inference.local/v1`. | OpenRouter featured catalog models, or any OpenRouter model ID you enter manually. |
| OpenAI | Routes to the OpenAI API. Set `OPENAI_API_KEY`. | `gpt-5.4`, `gpt-5.4-mini`, `gpt-5.4-nano`, `gpt-5.4-pro-2026-03-05` |
| Other OpenAI-compatible endpoint | Routes to any server that implements `/v1/chat/completions`. NemoClaw uses `/v1/chat/completions` at runtime by default; set `NEMOCLAW_PREFERRED_API=openai-responses` to allow `/v1/responses` for proxies that implement it, such as some llama.cpp builds. The wizard prompts for a base URL and model name. The adapter is validated against OpenRouter (refer to the status table above); behavior on other OpenAI-compatible proxies, gateways, and self-hosted implementations such as LocalAI or llama.cpp may vary. When you enable Telegram messaging, onboarding also runs a bounded sandbox-side smoke check through `https://inference.local/v1/chat/completions`. Set `COMPATIBLE_API_KEY`. | You provide the model name. |
| Other OpenAI-compatible endpoint | Routes to any server that implements `/v1/chat/completions`. NemoClaw uses `/v1/chat/completions` at runtime by default; set `NEMOCLAW_PREFERRED_API=openai-responses` to allow `/v1/responses` for proxies that implement it, such as some llama.cpp builds. The wizard prompts for a base URL and model name. Behavior on OpenAI-compatible proxies, gateways, and self-hosted implementations such as LocalAI or llama.cpp may vary. When you enable Telegram messaging, onboarding also runs a bounded sandbox-side smoke check through `https://inference.local/v1/chat/completions`. Set `COMPATIBLE_API_KEY`. | You provide the model name. |
| Anthropic | Routes to the Anthropic Messages API. Set `ANTHROPIC_API_KEY`. | `claude-sonnet-4-6`, `claude-haiku-4-5`, `claude-opus-4-6` |
| Other Anthropic-compatible endpoint | Routes agents that support Anthropic Messages, including OpenClaw, to `/v1/messages`. For Hermes and agents that only support OpenAI-compatible inference, NemoClaw instead requires `/v1/chat/completions`, which is the surface it validates and uses at runtime. The adapter is validated against AWS Bedrock (refer to the status table above); behavior on other Anthropic-compatible proxies and gateways may vary. The wizard prompts for a base URL and model name. Set `COMPATIBLE_ANTHROPIC_API_KEY`. | You provide the model name. |
| Google Gemini | Routes to Google's OpenAI-compatible chat-completions endpoint. NemoClaw skips the Responses-API probe because Gemini does not support `/v1/responses`. Set `GEMINI_API_KEY`. | `gemini-3.1-pro-preview`, `gemini-3.1-flash-lite-preview`, `gemini-3-flash-preview`, `gemini-2.5-pro`, `gemini-2.5-flash`, `gemini-2.5-flash-lite` |
Expand All @@ -116,7 +120,7 @@ Managed provider defaults that do not supply an explicit custom endpoint through
NemoClaw accepts `http://host.openshell.internal:<port>` only with an explicit port from `1024` through `65535`; this narrow exception supports NemoClaw's sandbox-to-host inference routes and is not a general private-endpoint bypass.

<Note>
NVIDIA Endpoints and Hermes Provider use independent model catalogs, so a model can remain available through one provider after it leaves the other's curated list.
NVIDIA Endpoints, OpenRouter, and Hermes Provider use independent model catalogs, so a model can remain available through one provider after it leaves another provider's curated list.
Curated-list updates affect new onboarding choices and do not rewrite existing sandbox configurations.
Use [Switch Inference Providers](switch-inference-providers) to move an existing sandbox before its configured model becomes unavailable.
</Note>
Expand Down Expand Up @@ -156,12 +160,12 @@ NVIDIA Nemotron models expose OpenAI-compatible APIs across every supported depl
| Nemotron Host | Onboard Wizard Option | Why |
|---|---|---|
| `build.nvidia.com` (NVIDIA-hosted) | **Option 1: NVIDIA Endpoints** | NemoClaw sets the base URL to `https://integrate.api.nvidia.com/v1` for you and validates the model against the build catalog. |
| Self-hosted NIM container | **Option 3: Other OpenAI-compatible endpoint** | NIM exposes an OpenAI-compatible `/v1/chat/completions` route. Point the base URL at your NIM service and enter the Nemotron model ID. |
| Enterprise NVIDIA AI Enterprise gateway | **Option 3: Other OpenAI-compatible endpoint** | Enterprise gateways front Nemotron with the same OpenAI-compatible contract. Use the gateway's base URL and your enterprise token. |
| vLLM, SGLang, or TRT-LLM serving Nemotron weights | **Option 3: Other OpenAI-compatible endpoint** | Each runtime exposes Nemotron through `/v1/chat/completions`. Use the runtime's base URL and the model ID it reports. |
| Self-hosted NIM container | **Other OpenAI-compatible endpoint** | NIM exposes an OpenAI-compatible `/v1/chat/completions` route. Point the base URL at your NIM service and enter the Nemotron model ID. |
| Enterprise NVIDIA AI Enterprise gateway | **Other OpenAI-compatible endpoint** | Enterprise gateways front Nemotron with the same OpenAI-compatible contract. Use the gateway's base URL and your enterprise token. |
| vLLM, SGLang, or TRT-LLM serving Nemotron weights | **Other OpenAI-compatible endpoint** | Each runtime exposes Nemotron through `/v1/chat/completions`. Use the runtime's base URL and the model ID it reports. |
| Local NIM started by the wizard | **Local NVIDIA NIM** (experimental) | Requires `NEMOCLAW_EXPERIMENTAL=1` and a NIM-capable GPU. NemoClaw pulls and manages the container for you. |

For Option 3, the API key environment variable is `COMPATIBLE_API_KEY`. Set it to whatever credential your endpoint expects, or any non-empty placeholder if your endpoint does not require auth.
For Other OpenAI-compatible endpoint, the API key environment variable is `COMPATIBLE_API_KEY`. Set it to whatever credential your endpoint expects, or any non-empty placeholder if your endpoint does not require auth.

## Model Router

Expand Down Expand Up @@ -262,11 +266,13 @@ If credential validation fails, the wizard asks whether to re-enter the API key,
The wizard retries transient upstream validation failures before it reports a provider failure.
The `nvapi-` prefix check applies only to `NVIDIA_INFERENCE_API_KEY`.
Other provider credentials, such as `OPENAI_API_KEY`, `ANTHROPIC_API_KEY`, `GEMINI_API_KEY`, and compatible endpoint keys, use provider-aware validation during retry.
OpenRouter credentials are validated as non-empty `sk-or-` keys.

| Provider type | Validation method |
|---|---|
| OpenAI | Tries `/responses` first, then `/chat/completions`. |
| NVIDIA Endpoints | Validates through `/v1/chat/completions` only; NemoClaw skips the `/v1/responses` probe because NVIDIA Build does not expose `/v1/responses` (returns 404 for every model). |
| OpenRouter | Validates through `/v1/chat/completions` only. NemoClaw skips the `/v1/responses` probe and uses managed Chat Completions for OpenClaw, Hermes, and Deep Agents Code. |
| Google Gemini | Validates through Gemini's OpenAI-compatible chat-completions path only; NemoClaw skips the `/v1/responses` probe because Gemini does not support the Responses API. |
| Other OpenAI-compatible endpoint | Tries `/v1/responses` first with a tool-calling probe; falls back to `/v1/chat/completions`. Selected runtime API defaults to `/v1/chat/completions`; set `NEMOCLAW_PREFERRED_API=openai-responses` to allow `/v1/responses` at runtime when validation succeeds. |
| Other Anthropic-compatible endpoint | For agents that support Anthropic Messages, including OpenClaw, tries `/v1/messages` with a non-streaming request, then repeats the request with `stream: true` and validates the SSE event sequence. Set `NEMOCLAW_REASONING=true` to skip the streaming check for reasoning-only endpoints. For Hermes and OpenAI-compatible-only agents, validates `/v1/chat/completions`, the surface used by the managed OpenAI frontend. |
Expand Down
6 changes: 6 additions & 0 deletions docs/inference/switch-inference-providers.mdx
Original file line number Diff line number Diff line change
Expand Up @@ -122,6 +122,12 @@ $$nemoclaw inference set --provider nvidia-prod --model nvidia/nemotron-3-super-
$$nemoclaw inference set --provider openai-api --model gpt-5.4
```

### OpenRouter

```bash
$$nemoclaw inference set --provider openrouter-api --model moonshotai/kimi-k2.6
```

### Anthropic

```bash
Expand Down
4 changes: 2 additions & 2 deletions docs/reference/commands.mdx
Original file line number Diff line number Diff line change
Expand Up @@ -338,7 +338,7 @@ After successful recovery, the installer skips generic onboarding.
For a manually prepared upgrade, set `NEMOCLAW_OPENSHELL_UPGRADE_PREPARED=1` only after preserving every registered sandbox and retiring the old gateway.

The wizard prompts for a provider first, then collects the provider credential if needed.
Supported non-experimental choices include NVIDIA Endpoints, OpenAI, Anthropic, Google Gemini, and compatible OpenAI or Anthropic endpoints.
Supported non-experimental choices include NVIDIA Endpoints, OpenRouter, OpenAI, Anthropic, Google Gemini, and compatible OpenAI or Anthropic endpoints.
Credentials are registered with the OpenShell gateway and never persisted to host disk.
Refer to [Credential Storage](../security/credential-storage) for details on inspection, rotation, and migration from earlier releases.
The legacy `$$nemoclaw setup` command is deprecated; use `$$nemoclaw onboard` instead.
Expand Down Expand Up @@ -3116,7 +3116,7 @@ Set them before running `$$nemoclaw onboard`.

| Variable | Format | Effect |
|----------|--------|--------|
| `NEMOCLAW_PROVIDER` | provider key (e.g. `build`, `openai`, `anthropic`, `anthropicCompatible`, `gemini`, `ollama`, `custom`, `vllm`, `nim-local`, `routed`, `hermes-provider`, `install-vllm`, `install-ollama`, `install-windows-ollama`, `start-windows-ollama`) | Selects the inference provider during onboarding. The wizard skips the provider menu in both interactive and non-interactive runs when this is set. Aliases: `cloud` → `build`, `nim` → `nim-local`, `hermes` / `nous` / `nous-portal` → `hermes-provider`, `anthropiccompatible` → `anthropicCompatible`. Invalid values fail fast with the list of accepted keys. |
| `NEMOCLAW_PROVIDER` | provider key (e.g. `build`, `openrouter`, `openai`, `anthropic`, `anthropicCompatible`, `gemini`, `ollama`, `custom`, `vllm`, `nim-local`, `routed`, `hermes-provider`, `install-vllm`, `install-ollama`, `install-windows-ollama`, `start-windows-ollama`) | Selects the inference provider during onboarding. The wizard skips the provider menu in both interactive and non-interactive runs when this is set. Aliases: `cloud` → `build`, `open-router` / `openrouterai` → `openrouter`, `nim` → `nim-local`, `hermes` / `nous` / `nous-portal` → `hermes-provider`, `anthropiccompatible` → `anthropicCompatible`. Invalid values fail fast with the list of accepted keys. |
| `NEMOCLAW_TOOL_DISCLOSURE` | `progressive` or `direct` | Selects progressive tool discovery or the prior direct-exposure behavior. Defaults to `progressive`; `--tool-disclosure` takes precedence when both are set. |
| `NEMOCLAW_ENDPOINT_URL` | URL | Custom endpoint URL. Used together with `NEMOCLAW_PROVIDER=custom` for OpenAI-compatible endpoints or `NEMOCLAW_PROVIDER=anthropicCompatible` for Anthropic-compatible endpoints. |
| `NEMOCLAW_PREFERRED_API` | `completions` (currently the only honored value) | Forces the validation probe to use the `/v1/chat/completions` API path instead of the newer `/v1/responses` API. |
Expand Down
Loading
Loading