Skip to content

docs: refresh for v0.0.20 and document context window, max tokens, and reasoning env vars #2108

New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

Sign up for GitHub

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Merged
ericksoa merged 2 commits into from
Apr 20, 2026
Merged

docs: refresh for v0.0.20 and document context window, max tokens, and reasoning env vars #2108

Show file tree
Hide file tree
Changes from all commits
Commits
Show all changes
2 commits
Select commit Hold shift + click to select a range
7209849
docs(inference): document context window, max tokens, and reasoning e…
miyoungc Apr 20, 2026
cf94963
docs: bump preferred docs version to 0.0.20
miyoungc Apr 20, 2026
File filter

Filter by extension

Filter by extension
Conversations
Failed to load comments.
Loading
Jump to
Jump to file
Failed to load files.
Loading
Diff view
Diff view
47 changes: 37 additions & 10 deletions .agents/skills/nemoclaw-user-configure-inference/SKILL.md
View file
Open in desktop
Original file line number Diff line number Diff line change
Expand Up @@ -184,7 +184,34 @@ Remove the env vars and recreate the sandbox to revert to the original model.
`NEMOCLAW_INFERENCE_API_OVERRIDE` accepts `openai-completions` (for NVIDIA, OpenAI, Gemini, compatible endpoints) or `anthropic-messages` (for Anthropic and Anthropic-compatible endpoints).
This variable is only needed when switching between provider families.

## Step 3: Verify the Active Model
## Step 3: Tune Model Metadata

The sandbox image bakes model metadata (context window, max output tokens, and reasoning mode) into `openclaw.json` at build time.
To change these values, set the corresponding environment variables before running `nemoclaw onboard` so they patch into the Dockerfile before the image builds.

| Variable | Values | Default |
|---|---|---|
| `NEMOCLAW_CONTEXT_WINDOW` | Positive integer (tokens) | `131072` |
| `NEMOCLAW_MAX_TOKENS` | Positive integer (tokens) | `4096` |
| `NEMOCLAW_REASONING` | `true` or `false` | `false` |

Invalid values are ignored, and the default bakes into the image.

```console
$ export NEMOCLAW_CONTEXT_WINDOW=65536
$ export NEMOCLAW_MAX_TOKENS=8192
$ export NEMOCLAW_REASONING=true
$ nemoclaw onboard
```

These variables are build-time settings.
If you change them on an existing sandbox, recreate the sandbox so the new values bake into the image:

```console
$ nemoclaw onboard --resume --recreate-sandbox
```

## Step 4: Verify the Active Model

Run the status command to confirm the change:

Expand All @@ -200,7 +227,7 @@ $ nemoclaw <name> status --json

The output includes the active provider, model, and endpoint.

## Step 4: Notes
## Step 5: Notes

- The host keeps provider credentials.
- The sandbox continues to use `inference.local`.
Expand All @@ -217,7 +244,7 @@ All approaches use the same `inference.local` routing model.
The agent inside the sandbox never connects to your model server directly.
OpenShell intercepts inference traffic and forwards it to the local endpoint you configure.

## Step 5: Ollama
## Step 6: Ollama

Ollama is the default local inference option.
The onboard wizard detects Ollama automatically when it is installed or running on the host.
Expand Down Expand Up @@ -280,7 +307,7 @@ If `NEMOCLAW_MODEL` is not set, NemoClaw selects a default model based on availa
| `NEMOCLAW_PROVIDER` | Set to `ollama`. |
| `NEMOCLAW_MODEL` | Ollama model tag to use. Optional. |

## Step 6: OpenAI-Compatible Server
## Step 7: OpenAI-Compatible Server

This option works with any server that implements `/v1/chat/completions`, including vLLM, TensorRT-LLM, llama.cpp, LocalAI, and others.
For compatible endpoints, NemoClaw uses `/v1/chat/completions` by default.
Expand Down Expand Up @@ -356,7 +383,7 @@ If you already onboarded and the sandbox is failing at runtime, re-run
into the image.
Refer to Switch Inference Models (see the `nemoclaw-user-configure-inference` skill) for details.

## Step 7: Anthropic-Compatible Server
## Step 8: Anthropic-Compatible Server

If your local server implements the Anthropic Messages API (`/v1/messages`), choose **Other Anthropic-compatible endpoint** during onboarding instead.

Expand All @@ -374,7 +401,7 @@ $ NEMOCLAW_PROVIDER=anthropicCompatible \
nemoclaw onboard --non-interactive
```

## Step 8: vLLM Auto-Detection (Experimental)
## Step 9: vLLM Auto-Detection (Experimental)

When vLLM is already running on `localhost:8000`, NemoClaw can detect it automatically and query the `/v1/models` endpoint to determine the loaded model.

Expand All @@ -401,7 +428,7 @@ $ NEMOCLAW_EXPERIMENTAL=1 \
NemoClaw auto-detects the model from the running vLLM instance.
To override the model, set `NEMOCLAW_MODEL`.

## Step 9: NVIDIA NIM (Experimental)
## Step 10: NVIDIA NIM (Experimental)

NemoClaw can pull, start, and manage a NIM container on hosts with a NIM-capable NVIDIA GPU.

Expand All @@ -427,7 +454,7 @@ $ NEMOCLAW_EXPERIMENTAL=1 \

To select a specific model, set `NEMOCLAW_MODEL`.

## Step 10: Timeout Configuration
## Step 11: Timeout Configuration

Local inference requests use a default timeout of 180 seconds.
Large prompts on hardware such as DGX Spark can exceed shorter timeouts, so NemoClaw sets a higher default for local providers (Ollama, vLLM, NIM).
Expand All @@ -443,7 +470,7 @@ The value is in seconds.
This setting is baked into the sandbox at build time.
Changing it after onboarding requires re-running `nemoclaw onboard`.

## Step 11: Verify the Configuration
## Step 12: Verify the Configuration

After onboarding completes, confirm the active provider and model.

Expand All @@ -453,7 +480,7 @@ $ nemoclaw <name> status

The output shows the provider label (for example, "Local vLLM" or "Other OpenAI-compatible endpoint") and the active model.

## Step 12: Switch Models at Runtime
## Step 13: Switch Models at Runtime

You can change the model without re-running onboard.
Refer to Switch Inference Models (see the `nemoclaw-user-configure-inference` skill) for the full procedure.
Expand Down
2 changes: 1 addition & 1 deletion .agents/skills/nemoclaw-user-reference/references/troubleshooting.md
View file
Open in desktop
Original file line number Diff line number Diff line change
Expand Up @@ -360,7 +360,7 @@ Instead:

1. Upgrade to a NemoClaw release that includes the newer `openclaw` version.
2. If you build NemoClaw from source, bump the pinned `openclaw` version in `Dockerfile.base` and rebuild the sandbox base image.
3. Back up any workspace files you need, then recreate the sandbox so it uses the rebuilt image.
3. Run `nemoclaw <name> rebuild` to recreate the sandbox with the updated image. The rebuild command automatically backs up workspace state before destroying the old sandbox and restores it afterward.

### Inference requests time out

Expand Down
27 changes: 27 additions & 0 deletions docs/inference/switch-inference-providers.md
View file
Open in desktop
Original file line number Diff line number Diff line change
Expand Up @@ -131,6 +131,33 @@ Remove the env vars and recreate the sandbox to revert to the original model.
`NEMOCLAW_INFERENCE_API_OVERRIDE` accepts `openai-completions` (for NVIDIA, OpenAI, Gemini, compatible endpoints) or `anthropic-messages` (for Anthropic and Anthropic-compatible endpoints).
This variable is only needed when switching between provider families.

## Tune Model Metadata

The sandbox image bakes model metadata (context window, max output tokens, and reasoning mode) into `openclaw.json` at build time.
To change these values, set the corresponding environment variables before running `nemoclaw onboard` so they patch into the Dockerfile before the image builds.

| Variable | Values | Default |
|---|---|---|
| `NEMOCLAW_CONTEXT_WINDOW` | Positive integer (tokens) | `131072` |
| `NEMOCLAW_MAX_TOKENS` | Positive integer (tokens) | `4096` |
| `NEMOCLAW_REASONING` | `true` or `false` | `false` |

Invalid values are ignored, and the default bakes into the image.
Comment on lines +139 to +145

@coderabbitai coderabbitai Bot Apr 20, 2026

Copy link
Copy Markdown
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

⚠️ Potential issue | 🟡 Minor

🧩 Analysis chain

🏁 Script executed:

#!/bin/bash
# Verify that the validation code does not log warnings for invalid values
rg -nP -A5 -B5 'NEMOCLAW_CONTEXT_WINDOW|NEMOCLAW_MAX_TOKENS|NEMOCLAW_REASONING' src/lib/onboard.ts | rg -i 'console\.(warn|error|log)'

Repository: NVIDIA/NemoClaw

Length of output: 41

🏁 Script executed:

# Search for where these environment variables are used/validated in the codebase
rg -n 'NEMOCLAW_CONTEXT_WINDOW|NEMOCLAW_MAX_TOKENS|NEMOCLAW_REASONING' --type ts --type js src/

Repository: NVIDIA/NemoClaw

Length of output: 807

🏁 Script executed:

# Search more broadly including config and setup files
rg -n 'NEMOCLAW_CONTEXT_WINDOW|NEMOCLAW_MAX_TOKENS|NEMOCLAW_REASONING' --type ts --type js --type json --type yaml

Repository: NVIDIA/NemoClaw

Length of output: 2211

🏁 Script executed:

# Read the validation implementation in onboard.ts
sed -n '1235,1270p' src/lib/onboard.ts

Repository: NVIDIA/NemoClaw

Length of output: 1623

🏁 Script executed:

# Read the test to understand how validation errors are checked
sed -n '361,378p' test/nemoclaw-start.test.ts

Repository: NVIDIA/NemoClaw

Length of output: 817

🏁 Script executed:

# Find where apply_model_override function is defined
rg -n 'apply_model_override' --type ts --type sh --type dockerfile src/ test/

Repository: NVIDIA/NemoClaw

Length of output: 93

🏁 Script executed:

# Search for where these error messages are generated
rg -n 'must be a positive integer|must be "true" or "false"' src/

Repository: NVIDIA/NemoClaw

Length of output: 41

🏁 Script executed:

# Search for apply_model_override without specifying dockerfile type
rg -n 'apply_model_override' --type ts --type sh src/ test/

Repository: NVIDIA/NemoClaw

Length of output: 2793

🏁 Script executed:

# Look at more context in the test file to understand what 'src' is
sed -n '335,365p' test/nemoclaw-start.test.ts

Repository: NVIDIA/NemoClaw

Length of output: 1247

🏁 Script executed:

# Look at the start of the test file to understand what 'src' is
head -50 test/nemoclaw-start.test.ts

Repository: NVIDIA/NemoClaw

Length of output: 2310

🏁 Script executed:

# Also look for where the apply_model_override function is actually generated
rg -n 'apply_model_override.*{' src/ --type ts -A 10 | head -40

Repository: NVIDIA/NemoClaw

Length of output: 168

🏁 Script executed:

# Search for where apply_model_override is generated in the codebase
rg -n 'apply_model_override' src/ --type ts -B 2 -A 2

Repository: NVIDIA/NemoClaw

Length of output: 41

🏁 Script executed:

# Let's look at the actual script that the test is checking
head -100 scripts/nemoclaw-start.sh | tail -50

Repository: NVIDIA/NemoClaw

Length of output: 2239

🏁 Script executed:

# Find where the error messages are generated in the TypeScript code
rg -n '".*must be.*"' src/lib/ --type ts

Repository: NVIDIA/NemoClaw

Length of output: 866

🏁 Script executed:

# Look for where the shell script is generated/written
rg -n 'apply_model_override' src/lib/ --type ts -B 5 -A 5

Repository: NVIDIA/NemoClaw

Length of output: 41

🏁 Script executed:

# Search for the error messages in shell script templates
rg -n 'must be a positive integer' --type sh --type ts

Repository: NVIDIA/NemoClaw

Length of output: 758

🏁 Script executed:

# Look for where apply_model_override function might be generated or templated
rg -n 'apply_model_override' src/lib/onboard.ts -B 5 -A 10

Repository: NVIDIA/NemoClaw

Length of output: 41

🏁 Script executed:

# Check if nemoclaw-start.sh is a static file or generated
ls -la scripts/nemoclaw-start.sh
file scripts/nemoclaw-start.sh
head -20 scripts/nemoclaw-start.sh

Repository: NVIDIA/NemoClaw

Length of output: 1429

🏁 Script executed:

# Look at the actual apply_model_override function in the shell script
sed -n '250,310p' scripts/nemoclaw-start.sh

Repository: NVIDIA/NemoClaw

Length of output: 2583

🏁 Script executed:

# Check for NEMOCLAW_REASONING validation in the shell script
sed -n '280,300p' scripts/nemoclaw-start.sh

Repository: NVIDIA/NemoClaw

Length of output: 937

Consider clarifying validation behavior — values are not silently ignored at runtime.

The current text states "Invalid values are ignored, and the default bakes into the image," which accurately describes build-time behavior (in src/lib/onboard.ts). However, this may mislead readers about runtime validation: at sandbox startup, the apply_model_override() function validates these environment variables and logs detailed errors to stderr if they are invalid, rather than silently ignoring them.

For clarity, update the explanation to distinguish build-time and runtime:

  • Build-time (image creation): Invalid values are silently ignored; defaults from the Dockerfile apply.
  • Runtime (sandbox startup): Invalid values trigger validation errors logged to stderr; startup may fail depending on how the error is handled.

You may also clarify the exact validation rules: NEMOCLAW_CONTEXT_WINDOW and NEMOCLAW_MAX_TOKENS must be positive integers; NEMOCLAW_REASONING must be exactly "true" or "false".

🤖 Prompt for AI Agents 
Verify each finding against the current code and only fix it if needed.

In `@docs/inference/switch-inference-providers.md` around lines 139 - 145, Update
the docs to distinguish build-time vs runtime validation: state that during
image build (as in onboard logic) invalid env values are silently ignored and
Dockerfile defaults apply, while at sandbox startup the function
apply_model_override() performs runtime validation, writing detailed errors to
stderr and potentially causing startup to fail; explicitly list the validation
rules for NEMOCLAW_CONTEXT_WINDOW and NEMOCLAW_MAX_TOKENS (must be positive
integers) and NEMOCLAW_REASONING (must be exactly "true" or "false"), and remove
the phrasing that implies invalid values are silently ignored at runtime.


```console
$ export NEMOCLAW_CONTEXT_WINDOW=65536
$ export NEMOCLAW_MAX_TOKENS=8192
$ export NEMOCLAW_REASONING=true
$ nemoclaw onboard
```

These variables are build-time settings.
If you change them on an existing sandbox, recreate the sandbox so the new values bake into the image:

```console
$ nemoclaw onboard --resume --recreate-sandbox
```

## Verify the Active Model

Run the status command to confirm the change:
Expand Down
2 changes: 1 addition & 1 deletion docs/project.json
View file
Open in desktop
Original file line number Diff line number Diff line change
@@ -1 +1 @@
{"name": "nemoclaw", "version": "0.0.18"}
{"name": "nemoclaw", "version": "0.0.20"}
8 changes: 8 additions & 0 deletions docs/versions1.json
View file
Open in desktop
Original file line number Diff line number Diff line change
@@ -1,6 +1,14 @@
[
{
"preferred": true,
"version": "0.0.20",
"url": "https://docs.nvidia.com/nemoclaw/0.0.20/"
},
{
"version": "0.0.19",
"url": "https://docs.nvidia.com/nemoclaw/0.0.19/"
},
{
"version": "0.0.18",
"url": "https://docs.nvidia.com/nemoclaw/0.0.18/"
},
Expand Down
Loading