Skip to content
Merged
Show file tree
Hide file tree
Changes from 5 commits
Commits
File filter

Filter by extension

Filter by extension

Conversations
Failed to load comments.
Loading
Jump to
Jump to file
Failed to load files.
Loading
Diff view
Diff view
15 changes: 9 additions & 6 deletions docs/inference/choose-local-inference-server.mdx
Original file line number Diff line number Diff line change
Expand Up @@ -27,7 +27,7 @@ Accept the N1x Express prompt or set `NEMOCLAW_PROVIDER=install-vllm` to provide
| Option | When to use it | Availability | Runtime API |
|---|---|---|---|
| Ollama | You want the default local option and want NemoClaw to install, start, or use Ollama on supported hosts. | Appears when Ollama is installed or running, and the wizard can offer installation on supported hosts. | Ollama through the managed local route. |
| Existing vLLM | You already run vLLM on `localhost:8000`. | Unavailable on N1x. On other hosts, it appears when NemoClaw detects the server. | `/v1/chat/completions`. |
| Existing vLLM | You already run vLLM on `localhost:${NEMOCLAW_VLLM_PORT:-8000}`. | Unavailable on N1x. On other hosts, it appears when NemoClaw detects the server. | `/v1/chat/completions`. |
| Managed vLLM | You want NemoClaw to pull an image, download model weights, and manage the server container. | Appears by default on DGX Spark and DGX Station. N1x offers this route as its only admitted provider through a Deferred preview. Generic Linux NVIDIA GPU hosts require `NEMOCLAW_EXPERIMENTAL=1` or `NEMOCLAW_PROVIDER=install-vllm`. | `/v1/chat/completions`. |
| Fixed vLLM profile | You need the fixed catalog-selected vLLM model and serving configuration. | Appears as option 2 in DGX Spark Express. Direct installation can use the dedicated installer flag. The profile does not appear in the provider menu. | `/v1/chat/completions`. |
| Existing llama.cpp | You already operate an authenticated llama.cpp server on loopback port `8081`. | Experimental. Always available for explicit selection and attaches only after cooperative fingerprinting succeeds. | `/v1/chat/completions`. |
Expand All @@ -38,7 +38,7 @@ Accept the N1x Express prompt or set `NEMOCLAW_PROVIDER=install-vllm` to provide
<AgentOnly variant="deepagents">
| Option | When to use it | Availability | Runtime API |
|---|---|---|---|
| Existing vLLM | You already run vLLM on `localhost:8000`. | Unavailable on N1x. On other hosts, it appears when NemoClaw detects the server. | `/v1/chat/completions`. |
| Existing vLLM | You already run vLLM on `localhost:${NEMOCLAW_VLLM_PORT:-8000}`. | Unavailable on N1x. On other hosts, it appears when NemoClaw detects the server. | `/v1/chat/completions`. |
| Managed vLLM | You want NemoClaw to pull an image, download model weights, and manage the server container. | Appears by default on DGX Spark and DGX Station. N1x offers this route as its only admitted provider through a Deferred preview. Generic Linux NVIDIA GPU hosts require `NEMOCLAW_EXPERIMENTAL=1` or `NEMOCLAW_PROVIDER=install-vllm`. | `/v1/chat/completions`. |
| Fixed vLLM profile | You need the fixed catalog-selected vLLM model and serving configuration. | Appears as option 2 in DGX Spark Express. Direct installation can use the dedicated installer flag. The profile does not appear in the provider menu. | `/v1/chat/completions`. |
| Existing llama.cpp | You already operate an authenticated llama.cpp server on loopback port `8081`. | Experimental. Always available for explicit selection and attaches only after cooperative fingerprinting succeeds. | `/v1/chat/completions`. |
Expand Down Expand Up @@ -113,13 +113,15 @@ curl -fsSL https://www.nvidia.com/nemoclaw.sh | \
The flag enables the dedicated vLLM profile gate, disables Express selection, and selects the matching catalog recipe.
Selecting Express option 2 enables the same profile gate and selects the vLLM runtime.
Do not combine this path with `NEMOCLAW_PROVIDER` or `NEMOCLAW_MODEL`.
The profile rejects `NEMOCLAW_VLLM_MODEL`, `NEMOCLAW_VLLM_PORT`, and `NEMOCLAW_VLLM_EXTRA_ARGS_JSON` before installation.
The profile rejects `NEMOCLAW_VLLM_MODEL` and `NEMOCLAW_VLLM_EXTRA_ARGS_JSON` before installation.
Set `NEMOCLAW_VLLM_PORT` before installation to use another host listener port.
Before installation, ensure no other process or container owns `${NEMOCLAW_VLLM_PORT:-8000}` on `127.0.0.1` or on the private IPv4 gateway of `openshell-docker`.

The profile performs these actions:

- Uses the pinned runtime image and fixed serving command from the catalog.
- Stores model files in the host Hugging Face cache.
- Publishes the authenticated server on `127.0.0.1:8000` and the exact private IPv4 gateway of the `openshell-docker` bridge, such as `172.18.0.1:8000`.
- Publishes the authenticated server on `127.0.0.1:<port>` and the same port on the exact private IPv4 gateway of the `openshell-docker` bridge, such as `172.18.0.1:<port>`. The default is `8000`.
Comment thread
prekshivyas marked this conversation as resolved.
Resolved
- Never publishes the Docker host port on `0.0.0.0` or `::`.
- Reuses the owner-only host-global managed-vLLM API key.

Expand All @@ -139,16 +141,17 @@ The `healthy` state means the route served one inference request; it does not es
Verify the bounded host publication:

```bash
VLLM_PORT="${NEMOCLAW_VLLM_PORT:-8000}"
docker network inspect \
--format '{{json .IPAM.Config}}' \
openshell-docker
docker container inspect \
--format '{{json (index .NetworkSettings.Ports "8000/tcp")}}' \
--format "{{json (index .NetworkSettings.Ports \"${VLLM_PORT}/tcp\")}}" \
nemoclaw-vllm
```

The bridge result must contain exactly one private IPv4 `Gateway` value.
The container result must contain exactly two bindings with `HostPort` set to `8000`: one `HostIp` value of `127.0.0.1` and one that exactly matches the private bridge gateway.
The container result must contain exactly two bindings with `HostPort` set to `${NEMOCLAW_VLLM_PORT:-8000}`: one `HostIp` value of `127.0.0.1` and one that exactly matches the private bridge gateway.
Reject the runtime if a binding uses `0.0.0.0`, `::`, an empty address, another host address, or an additional entry.

If installation stops after a model download, rerun the same installer command.
Expand Down
20 changes: 10 additions & 10 deletions docs/inference/set-up-vllm.mdx
Original file line number Diff line number Diff line change
Expand Up @@ -20,12 +20,12 @@ The vLLM `/v1/responses` endpoint does not run the configured tool-call parser,

<Warning>
Local vLLM does not require authentication by default.
Existing-server and single-host managed-vLLM paths need port `8000` on host loopback for validation and on the OpenShell Docker bridge for sandbox traffic.
Existing-server and single-host managed-vLLM paths use `NEMOCLAW_VLLM_PORT`, which defaults to `8000`, on host loopback for validation and on the OpenShell Docker bridge for sandbox traffic.
Use a host firewall with default-deny inbound rules.
Allow TCP port `8000` only from the OpenShell Docker subnet to its gateway address, keep loopback access, and deny the port on every other interface.
Allow the configured TCP port only from the OpenShell Docker subnet to its gateway address, keep loopback access, and deny the port on every other interface.
Do not expose the port to your LAN or the internet.

An authenticated fixed profile publishes Docker port `8000` on exactly two host addresses: `127.0.0.1` and the exact private IPv4 gateway of the `openshell-docker` bridge.
An authenticated fixed profile publishes the configured host port on exactly two host addresses: `127.0.0.1` and the exact private IPv4 gateway of the `openshell-docker` bridge. The container continues to listen on its recipe-owned internal port `8000`.
It never publishes that host port on `0.0.0.0` or `::`.
The provider route remains `https://inference.local` inside the sandbox, and OpenShell reaches the server through the private bridge binding.
The managed Muse Glimmer profile uses the same bearer-protected listener and two-address publication boundary.
Expand All @@ -36,7 +36,7 @@ Follow [Set Up vLLM on Two DGX Sparks](set-up-vllm-on-two-dgx-sparks) or [Set Up

## Use an Existing Server

On hosts other than N1x, when vLLM is already running on `localhost:8000`, NemoClaw detects it and queries `/v1/models` for the loaded model.
On hosts other than N1x, when vLLM is already running on `localhost:${NEMOCLAW_VLLM_PORT:-8000}`, NemoClaw detects it and queries `/v1/models` for the loaded model.

Run onboarding.

Expand Down Expand Up @@ -135,13 +135,13 @@ Legacy sessions that predate profile provenance still resume with their original
In an interactive run, the managed path lists the supported model registry for your host profile.
Press **Enter** to use the profile default or select another validated model with its registered `vllm serve` arguments.

The single-host path pulls the vLLM image, downloads model weights into `~/.cache/huggingface`, and starts the `nemoclaw-vllm` container on `localhost:8000`.
The single-host path pulls the vLLM image, downloads model weights into `~/.cache/huggingface`, and starts the `nemoclaw-vllm` container on `localhost:<port>` (`8000` by default).
The distributed DGX pages document their separate container names, endpoints, and lifecycle checks.
The container carries NemoClaw ownership and immutable catalog labels so later onboarding, recovery, and uninstall runs can distinguish it from a bring-your-own server without trusting the container name alone.
NemoClaw also records an owner-only, credential-free runtime receipt that binds those labels to the exact container ID and API-key fingerprint.

If an existing `nemoclaw-vllm` container does not carry that label, NemoClaw preserves it and asks you to remove or rename it before installing managed vLLM.
On hosts other than N1x, if another vLLM server already occupies port `8000`, NemoClaw selects the existing-server path and applies the model-identity checks described above.
On hosts other than N1x, if another vLLM server already occupies the configured vLLM port, NemoClaw selects the existing-server path and applies the model-identity checks described above.
On N1x, NemoClaw stops managed onboarding and requires you to stop that server before trying the Deferred preview again.
If another process holds the configured single-host port, NemoClaw stops the managed install before it persists a bearer key or records the selected model.
NemoClaw also stops before storage prompts, cache creation, image pull, or container start.
Expand Down Expand Up @@ -377,10 +377,10 @@ On a host that still meets the N1x identity checks, `rebuild` reuses the recorde

- The sandbox registry records provider `vllm-local`.
- The registry records model `nvidia/Qwen3.6-35B-A3B-NVFP4`.
- If the registry records `endpointUrl` as `null`, NemoClaw derives `http://host.openshell.internal:8000/v1`; otherwise, `endpointUrl` must equal that URL.
- If the registry records `endpointUrl` as `null`, NemoClaw derives `http://host.openshell.internal:${NEMOCLAW_VLLM_PORT:-8000}/v1`; otherwise, `endpointUrl` must equal that URL.
- The registry records `endpointSource` as `onboard` and `openshellDriver` as `docker`.
- The replacement keeps the recorded provider and model, then re-derives the canonical local endpoint.
- `hostLocalInferenceReceipt` is absent or `null`, or contains a canonical vLLM receipt for `host.openshell.internal:8000` and the same model.
- `hostLocalInferenceReceipt` is absent or `null`, or contains a canonical vLLM receipt for `host.openshell.internal:${NEMOCLAW_VLLM_PORT:-8000}` and the same model.

NemoClaw v0.0.109 can omit `hostLocalInferenceReceipt` or record it as `null`.
Matching sandboxes from that release remain eligible only when every condition above passes.
Expand All @@ -401,8 +401,8 @@ Refer to [Set Up vLLM on Two DGX Stations](set-up-vllm-on-two-dgx-stations) for

On DGX Spark, Express keeps automatic managed-vLLM serving-profile selection as option 1, the default.
It offers the fixed local vLLM profile as option 2.
When you select option 2, the serving catalog supplies the pinned model, runtime image, port, and serve arguments.
The installer rejects provider and model overrides for that fixed profile, and the dedicated onboarder rejects vLLM model, port, and serve-argument overrides before it starts the managed container.
When you select option 2, the serving catalog supplies the pinned model, runtime image, container port, and serve arguments.
The installer rejects provider and model overrides for that fixed profile, and the dedicated onboarder rejects vLLM model and serve-argument overrides before it starts the managed container. `NEMOCLAW_VLLM_PORT` may select another host listener.
Explicit `NEMOCLAW_MODEL` or `NEMOCLAW_VLLM_MODEL` intent retains the customizable managed-vLLM path and skips the fixed-profile choice.
If onboarding stops after you select option 2, `$$nemoclaw onboard --resume` reuses the recorded fixed profile instead of returning to automatic serving-profile selection.
Refer to [Choose a Local Inference Server](choose-local-inference-server#install-a-fixed-vllm-profile) for fixed-profile installation and verification.
Expand Down
5 changes: 3 additions & 2 deletions docs/reference/commands.mdx
Original file line number Diff line number Diff line change
Expand Up @@ -142,8 +142,9 @@ curl -fsSL https://www.nvidia.com/nemoclaw.sh | \
```

The profile selects a fixed catalog model and serving command from the managed-inference catalog.
The hosted installer rejects `NEMOCLAW_PROVIDER`, `NEMOCLAW_MODEL`, and a vLLM `NEMOCLAW_VLLM_PORT` override before onboarding.
The dedicated vLLM onboarder rejects `NEMOCLAW_VLLM_MODEL`, `NEMOCLAW_VLLM_PORT`, and `NEMOCLAW_VLLM_EXTRA_ARGS_JSON` before it installs vLLM.
The hosted installer rejects `NEMOCLAW_PROVIDER` and `NEMOCLAW_MODEL` before onboarding.
The dedicated vLLM onboarder rejects `NEMOCLAW_VLLM_MODEL` and `NEMOCLAW_VLLM_EXTRA_ARGS_JSON` before it installs vLLM.
Set `NEMOCLAW_VLLM_PORT` before installation to publish the fixed serving recipe on another host port.

The hosted installer's equivalent environment-variable form requires both `NEMOCLAW_ENABLE_LOCAL_MODEL_PROFILE=1` and `NEMOCLAW_LOCAL_MODEL_RUNTIME`.
Use the installer flag unless an automation boundary cannot pass installer arguments.
Expand Down
4 changes: 2 additions & 2 deletions docs/resources/prompt-assets/dgx-spark.md
Original file line number Diff line number Diff line change
Expand Up @@ -28,8 +28,8 @@ For option 1:
For option 2:

- Set `NEMOCLAW_ENABLE_LOCAL_MODEL_PROFILE=1` and `NEMOCLAW_LOCAL_MODEL_RUNTIME=vllm`.
- Leave `NEMOCLAW_PROVIDER`, `NEMOCLAW_MODEL`, `NEMOCLAW_VLLM_MODEL`, `NEMOCLAW_VLLM_PORT`, and `NEMOCLAW_VLLM_EXTRA_ARGS_JSON` unset.
- Explain that the serving catalog selects the fixed model, runtime image, port, and vLLM arguments.
- Leave `NEMOCLAW_PROVIDER`, `NEMOCLAW_MODEL`, `NEMOCLAW_VLLM_MODEL`, and `NEMOCLAW_VLLM_EXTRA_ARGS_JSON` unset. Preserve an existing `NEMOCLAW_VLLM_PORT` host-port override.
- Explain that the serving catalog selects the fixed model, runtime image, container port, and vLLM arguments. `NEMOCLAW_VLLM_PORT` may select another host listener.
- Explain that the fixed profile serves `nvidia/Qwen3.6-35B-A3B-NVFP4`.

For either accepted Express option:
Expand Down
6 changes: 3 additions & 3 deletions docs/resources/starter-prompt.md
Original file line number Diff line number Diff line change
Expand Up @@ -79,9 +79,9 @@ Set `NEMOCLAW_AGENT=langchain-deepagents-code` for Deep Agents, or use `nemo-dee

After the readiness check, load exactly one matching instruction asset before provider selection:

- Confirmed DGX Spark: [DGX Spark Express instructions](https://raw.githubusercontent.com/NVIDIA/NemoClaw/4394858b3bae38b04768619f99b9614161f1b565/docs/resources/prompt-assets/dgx-spark.md).
- Confirmed DGX Station: [DGX Station installation instructions](https://raw.githubusercontent.com/NVIDIA/NemoClaw/4394858b3bae38b04768619f99b9614161f1b565/docs/resources/prompt-assets/dgx-station.md).
- Officially detected Windows WSL: [Windows WSL Express instructions](https://raw.githubusercontent.com/NVIDIA/NemoClaw/4394858b3bae38b04768619f99b9614161f1b565/docs/resources/prompt-assets/windows-wsl.md).
- Confirmed DGX Spark: [DGX Spark Express instructions](https://raw.githubusercontent.com/NVIDIA/NemoClaw/66b532695db0ae38b74725ce7c57e4c91be24b19/docs/resources/prompt-assets/dgx-spark.md).
- Confirmed DGX Station: [DGX Station installation instructions](https://raw.githubusercontent.com/NVIDIA/NemoClaw/66b532695db0ae38b74725ce7c57e4c91be24b19/docs/resources/prompt-assets/dgx-station.md).
- Officially detected Windows WSL: [Windows WSL Express instructions](https://raw.githubusercontent.com/NVIDIA/NemoClaw/66b532695db0ae38b74725ce7c57e4c91be24b19/docs/resources/prompt-assets/windows-wsl.md).

Read the matching raw Markdown file completely and follow it before continuing.
Do not load a platform asset for any other computer.
Expand Down
16 changes: 5 additions & 11 deletions scripts/install.sh
Original file line number Diff line number Diff line change
Expand Up @@ -5190,9 +5190,9 @@ activate_express_install() {
export NEMOCLAW_SANDBOX_NAME="${NEMOCLAW_SANDBOX_NAME:-my-assistant}"
if [ "${_SPARK_EXPRESS_INFERENCE_SELECTION:-managed-vllm}" = "fixed-vllm" ]; then
if [ -n "${NEMOCLAW_PROVIDER:-}" ] || [ -n "${NEMOCLAW_MODEL:-}" ] \
|| [ -n "${NEMOCLAW_VLLM_MODEL:-}" ] || [ -n "${NEMOCLAW_VLLM_PORT:-}" ] \
|| [ -n "${NEMOCLAW_VLLM_MODEL:-}" ] \
|| [ -n "${NEMOCLAW_VLLM_EXTRA_ARGS_JSON:-}" ]; then
error "The fixed DGX Spark vLLM profile does not accept provider, model, port, or serve-argument overrides."
error "The fixed DGX Spark vLLM profile does not accept provider, model, or serve-argument overrides."
fi
unset NEMOCLAW_PROVIDER
export NEMOCLAW_ENABLE_LOCAL_MODEL_PROFILE=1
Expand Down Expand Up @@ -5292,7 +5292,7 @@ station_managed_dual_head_running() {
"$running" == "true" &&
"$managed" == "true" &&
"$role" == "head" &&
"$schema" == "2" &&
("$schema" == "2" || "$schema" == "3") &&
"$cluster" =~ ^[a-f0-9]{64}$ &&
"$launch_contract" =~ ^[a-f0-9]{64}$ &&
"$api_fingerprint" =~ ^[a-f0-9]{64}$ &&
Expand Down Expand Up @@ -5334,7 +5334,7 @@ ensure_station_express_host() {
# the post-Node coordinator revalidates the reciprocal physical pair before
# the existing lifecycle is allowed to reuse it.
_STATION_EXPRESS_DEFERRED_MANAGED_PAIR=1
info "Found a complete running NemoClaw-managed dual-Station head candidate; deferring host preparation until reciprocal pair and lifecycle validation."
info "Found a complete running NemoClaw-managed dual-Station head candidate; deferring host preparation until reciprocal pair and lifecycle validation, including any required launch-schema replacement."
return 0
fi
if station_dual_model_requested && station_migratable_legacy_single_head_running; then
Expand Down Expand Up @@ -5645,7 +5645,7 @@ describe_express_install() {
"DGX Spark")
if [ "${_SPARK_EXPRESS_INFERENCE_SELECTION:-managed-vllm}" = "fixed-vllm" ]; then
inference_summary="Qwen3.6 35B-A3B NVFP4 with the fixed catalog-backed vLLM profile"
inference_disclosure="The serving catalog owns the model, image, port, and vLLM arguments. The installer rejects provider and model overrides, and the dedicated local-model onboarder rejects vLLM model, port, and serve-argument overrides before starting its managed container."
inference_disclosure="The serving catalog owns the model, image, and vLLM arguments. The installer rejects provider and model overrides, and the dedicated local-model onboarder rejects vLLM model and serve-argument overrides before starting its managed container. NEMOCLAW_VLLM_PORT may override the host listener."
elif [ -n "${NEMOCLAW_VLLM_MODEL:-}" ]; then
inference_summary="managed local vLLM with model ${NEMOCLAW_VLLM_MODEL}"
inference_disclosure="The explicit model remains authoritative, so this run keeps the existing single-host DGX Spark profile. Managed vLLM pulls the configured image/model and runs only its dedicated container."
Expand Down Expand Up @@ -6034,12 +6034,6 @@ main() {
&& { [ -n "${NEMOCLAW_PROVIDER:-}" ] || [ -n "${NEMOCLAW_MODEL:-}" ]; }; then
error "The local model profile does not accept NEMOCLAW_PROVIDER or NEMOCLAW_MODEL overrides."
fi
if [ "${NEMOCLAW_ENABLE_LOCAL_MODEL_PROFILE:-}" = "1" ] \
&& [ "${NEMOCLAW_LOCAL_MODEL_RUNTIME:-}" = "vllm" ] \
&& [ -n "${NEMOCLAW_VLLM_PORT:-}" ]; then
error "The vLLM local model profile uses fixed port 8000 and does not accept NEMOCLAW_VLLM_PORT."
fi

# If the user explicitly accepted the third-party-software notice, treat
# that as non-interactive intent for the rest of the run too — show_usage_notice
# is only one of several phase-3 steps that need a TTY or --non-interactive
Expand Down
Loading
Loading