Skip to content
Merged
Show file tree
Hide file tree
Changes from all commits
Commits
File filter

Filter by extension

Filter by extension

Conversations
Failed to load comments.
Loading
Jump to
Jump to file
Failed to load files.
Loading
Diff view
Diff view
9 changes: 7 additions & 2 deletions docs/inference/inference-options.mdx
Original file line number Diff line number Diff line change
Expand Up @@ -64,7 +64,7 @@ The managed install/start vLLM entry appears by default on DGX Spark and DGX Sta

| Option | Description | Curated models |
|--------|-------------|----------------|
| NVIDIA Endpoints | Routes to models hosted on [build.nvidia.com](https://build.nvidia.com). You can also enter any model ID from the catalog. Set `NVIDIA_INFERENCE_API_KEY`. | Nemotron 3 Super 120B, Nemotron 3 Ultra 550B, GLM-5.1, MiniMax M2.7, GPT-OSS 120B, DeepSeek V4 Pro |
| NVIDIA Endpoints | Routes to models hosted on [build.nvidia.com](https://build.nvidia.com). You can also enter any model ID from the catalog. Set `NVIDIA_INFERENCE_API_KEY`. | Nemotron 3 Super 120B, Nemotron 3 Ultra 550B, MiniMax M2.7, GPT-OSS 120B, DeepSeek V4 Pro |
| OpenAI | Routes to the OpenAI API. Set `OPENAI_API_KEY`. | `gpt-5.4`, `gpt-5.4-mini`, `gpt-5.4-nano`, `gpt-5.4-pro-2026-03-05` |
| Other OpenAI-compatible endpoint | Routes to any server that implements `/v1/chat/completions`. NemoClaw uses `/v1/chat/completions` at runtime by default; set `NEMOCLAW_PREFERRED_API=openai-responses` to allow `/v1/responses` for proxies that implement it, such as some llama.cpp builds. The wizard prompts for a base URL and model name. The adapter is validated against OpenRouter (refer to the status table above); behavior on other OpenAI-compatible proxies, gateways, and self-hosted implementations such as LocalAI or llama.cpp may vary. When you enable Telegram messaging, onboarding also runs a bounded sandbox-side smoke check through `https://inference.local/v1/chat/completions`. Set `COMPATIBLE_API_KEY`. | You provide the model name. |
| Anthropic | Routes to the Anthropic Messages API. Set `ANTHROPIC_API_KEY`. | `claude-sonnet-4-6`, `claude-haiku-4-5`, `claude-opus-4-6` |
Expand All @@ -74,6 +74,12 @@ The managed install/start vLLM entry appears by default on DGX Spark and DGX Sta
| Local Ollama | Routes to a local Ollama instance on `localhost:11434`. NemoClaw detects installed models, offers starter models if none are present, pulls and warms the selected model, and validates it. | Selected during onboarding. For more information, refer to [Use a Local Inference Server](use-local-inference). |
| Model Router | Starts a host-side router on port `4000`, registers it as an OpenAI-compatible provider, and keeps the sandbox pointed at `inference.local`. Set `NEMOCLAW_PROVIDER=routed` for non-interactive setup. | The router pool defines the model names. |

<Note>
NVIDIA Endpoints and Hermes Provider use independent model catalogs, so a model can remain available through one provider after it leaves the other's curated list.
Curated-list updates affect new onboarding choices and do not rewrite existing sandbox configurations.
Use [Switch Inference Providers](switch-inference-providers) to move an existing sandbox before its configured model becomes unavailable.
</Note>

## Model Task-Fit Guide

Use this table as starter guidance when selecting a curated cloud model during onboarding.
Expand All @@ -85,7 +91,6 @@ The relative labels below are qualitative and compare models within the curated
| `nvidia/nemotron-3-super-120b-a12b` | Default hosted agent work, multi-step planning, and tool-heavy shell workflows | Medium | Strong default for OpenClaw tool loops | Large agent context | Medium |
| `nvidia/nemotron-3-ultra-550b-a55b` | Quality-sensitive reasoning, careful synthesis, and complex reviews | Higher | Strong for complex tool plans | Large agent context | Higher |
| `nvidia/nemotron-3-nano-omni-30b-a3b-reasoning` | Reasoning-first and multimodal experiments where a compact hosted model is enough | Medium | Good after the smoke probe confirms final-answer content | Large agent context | Medium |
| `z-ai/glm-5.1` | General chat, multilingual text work, and fast iteration | Low-to-medium | Good for straightforward tool loops | Large agent context | Low-to-medium |
| `minimaxai/minimax-m2.7` | Long-form writing, multi-turn assistant work, and broad instruction following | Medium | Good for structured assistant turns | Large agent context | Medium |
| `moonshotai/kimi-k2.6` | Coding tasks and shell-heavy agent trajectories | Medium | Strong with NemoClaw's Kimi tool-call compatibility path | Large-context friendly | Medium |
| `openai/gpt-oss-120b` | Hosted open-weight style experimentation and cost-aware general agents | Medium | Good when provider-side tool calling is enabled | Large agent context | Medium |
Expand Down
6 changes: 5 additions & 1 deletion src/lib/inference/config.test.ts
Original file line number Diff line number Diff line change
Expand Up @@ -29,7 +29,6 @@ describe("inference selection config", () => {
"nvidia/nemotron-3-super-120b-a12b",
"nvidia/nemotron-3-ultra-550b-a55b",
"nvidia/nemotron-3-nano-omni-30b-a3b-reasoning",
"z-ai/glm-5.1",
"minimaxai/minimax-m2.7",
"moonshotai/kimi-k2.6",
"openai/gpt-oss-120b",
Expand All @@ -54,6 +53,11 @@ describe("inference selection config", () => {
expect(HERMES_PROVIDER_MODEL_OPTIONS.length).toBeGreaterThan(10);
});

it("retires GLM 5.1 only from the NVIDIA Endpoints picker", () => {
expect(CLOUD_MODEL_OPTIONS.map((option) => option.id)).not.toContain("z-ai/glm-5.1");
expect(HERMES_PROVIDER_MODEL_OPTIONS).toContain("z-ai/glm-5.1");
});

it("maps ollama-local to the sandbox inference route and default model", () => {
// Local Ollama uses a dedicated credential env so the sandbox-side
// config never points at OPENAI_API_KEY (GH #2519).
Expand Down
1 change: 0 additions & 1 deletion src/lib/inference/config.ts
Original file line number Diff line number Diff line change
Expand Up @@ -56,7 +56,6 @@ export const CLOUD_MODEL_OPTIONS = [
{ id: "nvidia/nemotron-3-super-120b-a12b", label: "Nemotron 3 Super 120B" },
{ id: "nvidia/nemotron-3-ultra-550b-a55b", label: "Nemotron 3 Ultra 550B" },
{ id: "nvidia/nemotron-3-nano-omni-30b-a3b-reasoning", label: "Nemotron 3 Nano Omni 30B" },
{ id: "z-ai/glm-5.1", label: "GLM-5" },
{ id: "minimaxai/minimax-m2.7", label: "MiniMax M2.7" },
{ id: "moonshotai/kimi-k2.6", label: "Kimi K2.6" },
{ id: "openai/gpt-oss-120b", label: "GPT-OSS 120B" },
Expand Down
2 changes: 1 addition & 1 deletion src/lib/inference/model-prompts.test.ts
Original file line number Diff line number Diff line change
Expand Up @@ -34,7 +34,7 @@ describe("model prompt helpers", () => {
});

it("returns DeepSeek V4 Pro from the default cloud model menu", async () => {
const promptFn = promptSequence(["8"]);
const promptFn = promptSequence(["7"]);
const result = await promptCloudModel({
promptFn,
writeLine: vi.fn(),
Expand Down
2 changes: 1 addition & 1 deletion test/e2e/live/hermes-inference-switch-helpers.ts
Original file line number Diff line number Diff line change
Expand Up @@ -34,7 +34,7 @@ export const SWITCH_PROVIDER =
(USE_COMPATIBLE_HOSTED ? "compatible-endpoint" : "nvidia-prod");
export const SWITCH_MODEL =
process.env.NEMOCLAW_SWITCH_MODEL ??
(USE_COMPATIBLE_HOSTED ? DEFAULT_COMPAT_MODEL : "z-ai/glm-5.1");
(USE_COMPATIBLE_HOSTED ? DEFAULT_COMPAT_MODEL : "nvidia/nemotron-3-super-120b-a12b");
export const SWITCH_API = process.env.NEMOCLAW_SWITCH_INFERENCE_API ?? "openai-completions";
const SWITCH_MOCK_ANTHROPIC = process.env.NEMOCLAW_SWITCH_MOCK_ANTHROPIC ?? "0";
const SWITCH_MOCK_PORT = Number.parseInt(process.env.NEMOCLAW_SWITCH_MOCK_PORT ?? "0", 10);
Expand Down
2 changes: 1 addition & 1 deletion test/e2e/live/openclaw-inference-switch.test.ts
Original file line number Diff line number Diff line change
Expand Up @@ -44,7 +44,7 @@ const SWITCH_PROVIDER =
(USE_COMPATIBLE_HOSTED ? "compatible-endpoint" : "nvidia-prod");
const SWITCH_MODEL =
process.env.NEMOCLAW_SWITCH_MODEL ??
(USE_COMPATIBLE_HOSTED ? DEFAULT_COMPAT_MODEL : "z-ai/glm-5.1");
(USE_COMPATIBLE_HOSTED ? DEFAULT_COMPAT_MODEL : "nvidia/nemotron-3-super-120b-a12b");
const SWITCH_INFERENCE_API = process.env.NEMOCLAW_SWITCH_INFERENCE_API ?? "openai-completions";
const SWITCH_MOCK_ANTHROPIC = process.env.NEMOCLAW_SWITCH_MOCK_ANTHROPIC ?? "0";
const SWITCH_MOCK_PORT = parsePortEnv("NEMOCLAW_SWITCH_MOCK_PORT", 0);
Expand Down
24 changes: 21 additions & 3 deletions test/inference-options-docs.test.ts
Original file line number Diff line number Diff line change
Expand Up @@ -121,8 +121,26 @@ describe("inference options model task-fit docs (#4755)", () => {
expect(section).not.toMatch(/\bTBD\b|\bTODO\b/i);
expect(section).not.toContain("Very large context");

for (const modelId of readCuratedOnboardingModelIds()) {
expect(section).toContain(`| \`${modelId}\` |`);
}
const documentedModelIds = Array.from(
section.matchAll(/^\| `([^`]+)` \|/gm),
(match) => match[1],
);
expect(documentedModelIds).toEqual(readCuratedOnboardingModelIds());
});

it("keeps GLM 5.1 scoped to the independent Hermes Provider catalog", () => {
const markdown = fs.readFileSync(inferenceOptionsPath, "utf8");
const start = markdown.indexOf("## Provider Options");
const end = markdown.indexOf("## Model Task-Fit Guide", start);
expect(start).toBeGreaterThanOrEqual(0);
expect(end).toBeGreaterThan(start);
const section = markdown.slice(start, end);
const lines = section.split("\n");
const nvidiaRow = lines.find((line) => line.startsWith("| NVIDIA Endpoints |"));
const hermesRow = lines.find((line) => line.startsWith("| Hermes Provider |"));

expect(nvidiaRow).toBeDefined();
expect(nvidiaRow).not.toMatch(/GLM-?5\.1|z-ai\/glm-5\.1/i);
expect(hermesRow).toContain("`z-ai/glm-5.1`");
});
});
10 changes: 5 additions & 5 deletions test/onboard-selection.test.ts
Original file line number Diff line number Diff line change
Expand Up @@ -553,7 +553,7 @@ printf '%s' "$status"
const credentials = require(${credentialsPath});
const runner = require(${runnerPath});

const answers = ["1", "8"];
const answers = ["1", "7"];
const messages = [];

credentials.prompt = async (message) => {
Expand Down Expand Up @@ -656,7 +656,7 @@ printf '%s' "$status"
const credentials = require(${credentialsPath});
const runner = require(${runnerPath});

const answers = ["1", "9", "custom/provider-model"];
const answers = ["1", "8", "custom/provider-model"];
const messages = [];

credentials.prompt = async (message) => {
Expand Down Expand Up @@ -742,7 +742,7 @@ while [ "$#" -gt 0 ]; do
esac
done
if echo "$url" | grep -q '/v1/models$'; then
body='{"data":[{"id":"nvidia/nemotron-3-super-120b-a12b"},{"id":"z-ai/glm-5.1"}]}'
body='{"data":[{"id":"nvidia/nemotron-3-super-120b-a12b"},{"id":"custom/provider-model"}]}'
fi
printf '%s' "$body" > "$outfile"
printf '%s' "$status"
Expand All @@ -754,7 +754,7 @@ printf '%s' "$status"
const credentials = require(${credentialsPath});
const runner = require(${runnerPath});

const answers = ["1", "9", "bad/model", "z-ai/glm-5.1"];
const answers = ["1", "8", "bad/model", "custom/provider-model"];
const messages = [];

credentials.prompt = async (message) => {
Expand Down Expand Up @@ -806,7 +806,7 @@ const { setupNim } = require(${onboardPath});

assert.equal(result.status, 0, result.stderr);
const payload = JSON.parse(result.stdout.trim());
assert.equal(payload.result.model, "z-ai/glm-5.1");
assert.equal(payload.result.model, "custom/provider-model");
assert.equal(
payload.messages.filter((message: string) => /NVIDIA Endpoints model id:/.test(message))
.length,
Expand Down
Loading