Skip to content
Merged
Show file tree
Hide file tree
Changes from 1 commit
Commits
File filter

Filter by extension

Filter by extension

Conversations
Failed to load comments.
Loading
Jump to
Jump to file
Failed to load files.
Loading
Diff view
Diff view
7 changes: 5 additions & 2 deletions docs/inference/inference-options.mdx
Original file line number Diff line number Diff line change
Expand Up @@ -64,7 +64,7 @@ The managed install/start vLLM entry appears by default on DGX Spark and DGX Sta

| Option | Description | Curated models |
|--------|-------------|----------------|
| NVIDIA Endpoints | Routes to models hosted on [build.nvidia.com](https://build.nvidia.com). You can also enter any model ID from the catalog. Set `NVIDIA_INFERENCE_API_KEY`. | Nemotron 3 Super 120B, Nemotron 3 Ultra 550B, GLM-5.1, MiniMax M2.7, GPT-OSS 120B, DeepSeek V4 Pro |
| NVIDIA Endpoints | Routes to models hosted on [build.nvidia.com](https://build.nvidia.com). You can also enter any model ID from the catalog. Set `NVIDIA_INFERENCE_API_KEY`. | Nemotron 3 Super 120B, Nemotron 3 Ultra 550B, MiniMax M2.7, GPT-OSS 120B, DeepSeek V4 Pro |
| OpenAI | Routes to the OpenAI API. Set `OPENAI_API_KEY`. | `gpt-5.4`, `gpt-5.4-mini`, `gpt-5.4-nano`, `gpt-5.4-pro-2026-03-05` |
| Other OpenAI-compatible endpoint | Routes to any server that implements `/v1/chat/completions`. NemoClaw uses `/v1/chat/completions` at runtime by default; set `NEMOCLAW_PREFERRED_API=openai-responses` to allow `/v1/responses` for proxies that implement it, such as some llama.cpp builds. The wizard prompts for a base URL and model name. The adapter is validated against OpenRouter (refer to the status table above); behavior on other OpenAI-compatible proxies, gateways, and self-hosted implementations such as LocalAI or llama.cpp may vary. When you enable Telegram messaging, onboarding also runs a bounded sandbox-side smoke check through `https://inference.local/v1/chat/completions`. Set `COMPATIBLE_API_KEY`. | You provide the model name. |
| Anthropic | Routes to the Anthropic Messages API. Set `ANTHROPIC_API_KEY`. | `claude-sonnet-4-6`, `claude-haiku-4-5`, `claude-opus-4-6` |
Expand All @@ -74,6 +74,10 @@ The managed install/start vLLM entry appears by default on DGX Spark and DGX Sta
| Local Ollama | Routes to a local Ollama instance on `localhost:11434`. NemoClaw detects installed models, offers starter models if none are present, pulls and warms the selected model, and validates it. | Selected during onboarding. For more information, refer to [Use a Local Inference Server](use-local-inference). |
| Model Router | Starts a host-side router on port `4000`, registers it as an OpenAI-compatible provider, and keeps the sandbox pointed at `inference.local`. Set `NEMOCLAW_PROVIDER=routed` for non-interactive setup. | The router pool defines the model names. |

NVIDIA Endpoints and Hermes Provider use independent model catalogs, so a model can remain available through one provider after it leaves the other's curated list.
Curated-list updates affect new onboarding choices and do not rewrite existing sandbox configurations.
Use [Switch Inference Providers](switch-inference-providers) to move an existing sandbox before its configured model becomes unavailable.

## Model Task-Fit Guide

Use this table as starter guidance when selecting a curated cloud model during onboarding.
Expand All @@ -85,7 +89,6 @@ The relative labels below are qualitative and compare models within the curated
| `nvidia/nemotron-3-super-120b-a12b` | Default hosted agent work, multi-step planning, and tool-heavy shell workflows | Medium | Strong default for OpenClaw tool loops | Large agent context | Medium |
| `nvidia/nemotron-3-ultra-550b-a55b` | Quality-sensitive reasoning, careful synthesis, and complex reviews | Higher | Strong for complex tool plans | Large agent context | Higher |
| `nvidia/nemotron-3-nano-omni-30b-a3b-reasoning` | Reasoning-first and multimodal experiments where a compact hosted model is enough | Medium | Good after the smoke probe confirms final-answer content | Large agent context | Medium |
| `z-ai/glm-5.1` | General chat, multilingual text work, and fast iteration | Low-to-medium | Good for straightforward tool loops | Large agent context | Low-to-medium |
| `minimaxai/minimax-m2.7` | Long-form writing, multi-turn assistant work, and broad instruction following | Medium | Good for structured assistant turns | Large agent context | Medium |
| `moonshotai/kimi-k2.6` | Coding tasks and shell-heavy agent trajectories | Medium | Strong with NemoClaw's Kimi tool-call compatibility path | Large-context friendly | Medium |
| `openai/gpt-oss-120b` | Hosted open-weight style experimentation and cost-aware general agents | Medium | Good when provider-side tool calling is enabled | Large agent context | Medium |
Expand Down
6 changes: 5 additions & 1 deletion src/lib/inference/config.test.ts
Original file line number Diff line number Diff line change
Expand Up @@ -29,7 +29,6 @@ describe("inference selection config", () => {
"nvidia/nemotron-3-super-120b-a12b",
"nvidia/nemotron-3-ultra-550b-a55b",
"nvidia/nemotron-3-nano-omni-30b-a3b-reasoning",
"z-ai/glm-5.1",
"minimaxai/minimax-m2.7",
"moonshotai/kimi-k2.6",
"openai/gpt-oss-120b",
Expand All @@ -54,6 +53,11 @@ describe("inference selection config", () => {
expect(HERMES_PROVIDER_MODEL_OPTIONS.length).toBeGreaterThan(10);
});

it("retires GLM 5.1 only from the NVIDIA Endpoints picker", () => {
expect(CLOUD_MODEL_OPTIONS.map((option) => option.id)).not.toContain("z-ai/glm-5.1");
expect(HERMES_PROVIDER_MODEL_OPTIONS).toContain("z-ai/glm-5.1");
});

it("maps ollama-local to the sandbox inference route and default model", () => {
// Local Ollama uses a dedicated credential env so the sandbox-side
// config never points at OPENAI_API_KEY (GH #2519).
Expand Down
1 change: 0 additions & 1 deletion src/lib/inference/config.ts
Original file line number Diff line number Diff line change
Expand Up @@ -56,7 +56,6 @@ export const CLOUD_MODEL_OPTIONS = [
{ id: "nvidia/nemotron-3-super-120b-a12b", label: "Nemotron 3 Super 120B" },
{ id: "nvidia/nemotron-3-ultra-550b-a55b", label: "Nemotron 3 Ultra 550B" },
{ id: "nvidia/nemotron-3-nano-omni-30b-a3b-reasoning", label: "Nemotron 3 Nano Omni 30B" },
{ id: "z-ai/glm-5.1", label: "GLM-5" },
{ id: "minimaxai/minimax-m2.7", label: "MiniMax M2.7" },
{ id: "moonshotai/kimi-k2.6", label: "Kimi K2.6" },
{ id: "openai/gpt-oss-120b", label: "GPT-OSS 120B" },
Expand Down
2 changes: 1 addition & 1 deletion src/lib/inference/model-prompts.test.ts
Original file line number Diff line number Diff line change
Expand Up @@ -34,7 +34,7 @@ describe("model prompt helpers", () => {
});

it("returns DeepSeek V4 Pro from the default cloud model menu", async () => {
const promptFn = promptSequence(["8"]);
const promptFn = promptSequence(["7"]);
const result = await promptCloudModel({
promptFn,
writeLine: vi.fn(),
Expand Down
2 changes: 1 addition & 1 deletion test/e2e/live/hermes-inference-switch-helpers.ts
Original file line number Diff line number Diff line change
Expand Up @@ -34,7 +34,7 @@ export const SWITCH_PROVIDER =
(USE_COMPATIBLE_HOSTED ? "compatible-endpoint" : "nvidia-prod");
export const SWITCH_MODEL =
process.env.NEMOCLAW_SWITCH_MODEL ??
(USE_COMPATIBLE_HOSTED ? DEFAULT_COMPAT_MODEL : "z-ai/glm-5.1");
(USE_COMPATIBLE_HOSTED ? DEFAULT_COMPAT_MODEL : "nvidia/nemotron-3-super-120b-a12b");
export const SWITCH_API = process.env.NEMOCLAW_SWITCH_INFERENCE_API ?? "openai-completions";
const SWITCH_MOCK_ANTHROPIC = process.env.NEMOCLAW_SWITCH_MOCK_ANTHROPIC ?? "0";
const SWITCH_MOCK_PORT = Number.parseInt(process.env.NEMOCLAW_SWITCH_MOCK_PORT ?? "0", 10);
Expand Down
2 changes: 1 addition & 1 deletion test/e2e/live/openclaw-inference-switch.test.ts
Original file line number Diff line number Diff line change
Expand Up @@ -44,7 +44,7 @@ const SWITCH_PROVIDER =
(USE_COMPATIBLE_HOSTED ? "compatible-endpoint" : "nvidia-prod");
const SWITCH_MODEL =
process.env.NEMOCLAW_SWITCH_MODEL ??
(USE_COMPATIBLE_HOSTED ? DEFAULT_COMPAT_MODEL : "z-ai/glm-5.1");
(USE_COMPATIBLE_HOSTED ? DEFAULT_COMPAT_MODEL : "nvidia/nemotron-3-super-120b-a12b");
const SWITCH_INFERENCE_API = process.env.NEMOCLAW_SWITCH_INFERENCE_API ?? "openai-completions";
const SWITCH_MOCK_ANTHROPIC = process.env.NEMOCLAW_SWITCH_MOCK_ANTHROPIC ?? "0";
const SWITCH_MOCK_PORT = parsePortEnv("NEMOCLAW_SWITCH_MOCK_PORT", 0);
Expand Down
10 changes: 5 additions & 5 deletions test/onboard-selection.test.ts
Original file line number Diff line number Diff line change
Expand Up @@ -553,7 +553,7 @@ printf '%s' "$status"
const credentials = require(${credentialsPath});
const runner = require(${runnerPath});

const answers = ["1", "8"];
const answers = ["1", "7"];
const messages = [];

credentials.prompt = async (message) => {
Expand Down Expand Up @@ -656,7 +656,7 @@ printf '%s' "$status"
const credentials = require(${credentialsPath});
const runner = require(${runnerPath});

const answers = ["1", "9", "custom/provider-model"];
const answers = ["1", "8", "custom/provider-model"];
const messages = [];

credentials.prompt = async (message) => {
Expand Down Expand Up @@ -742,7 +742,7 @@ while [ "$#" -gt 0 ]; do
esac
done
if echo "$url" | grep -q '/v1/models$'; then
body='{"data":[{"id":"nvidia/nemotron-3-super-120b-a12b"},{"id":"z-ai/glm-5.1"}]}'
body='{"data":[{"id":"nvidia/nemotron-3-super-120b-a12b"},{"id":"custom/provider-model"}]}'
fi
printf '%s' "$body" > "$outfile"
printf '%s' "$status"
Expand All @@ -754,7 +754,7 @@ printf '%s' "$status"
const credentials = require(${credentialsPath});
const runner = require(${runnerPath});

const answers = ["1", "9", "bad/model", "z-ai/glm-5.1"];
const answers = ["1", "8", "bad/model", "custom/provider-model"];
const messages = [];

credentials.prompt = async (message) => {
Expand Down Expand Up @@ -806,7 +806,7 @@ const { setupNim } = require(${onboardPath});

assert.equal(result.status, 0, result.stderr);
const payload = JSON.parse(result.stdout.trim());
assert.equal(payload.result.model, "z-ai/glm-5.1");
assert.equal(payload.result.model, "custom/provider-model");
assert.equal(
payload.messages.filter((message: string) => /NVIDIA Endpoints model id:/.test(message))
.length,
Expand Down
Loading