Skip to content
Merged
Show file tree
Hide file tree
Changes from all commits
Commits
Show all changes
21 commits
Select commit Hold shift + click to select a range
07503c4
fix(vllm): guard managed downloads by storage capacity
sandl99 Jul 13, 2026
4248cd0
test(vllm): cover architecture-specific image mappings
sandl99 Jul 13, 2026
e52d18d
test(vllm): keep architecture setup linear
sandl99 Jul 13, 2026
878fac5
test(vllm): cover inconclusive storage probes
sandl99 Jul 13, 2026
e52905f
fix(vllm): use exact pinned image sizes
prekshivyas Jul 13, 2026
cab3598
test(vllm): guard managed image docs
prekshivyas Jul 13, 2026
bfbcfc5
fix(vllm): harden Docker storage preflight
prekshivyas Jul 13, 2026
be534eb
test(vllm): cover conservative storage paths
prekshivyas Jul 13, 2026
e216ceb
test(vllm): trace Docker context through install
prekshivyas Jul 13, 2026
a3e8d1f
fix(vllm): validate cached-image Docker host
prekshivyas Jul 13, 2026
97e8eab
Merge branch 'main' into fix/6757-vllm-storage-preflight
cv Jul 13, 2026
feaea68
fix(vllm): preserve Docker TLS client state
prekshivyas Jul 13, 2026
20ba412
fix(vllm): honor explicit Docker env source
prekshivyas Jul 13, 2026
b9c4f34
fix(vllm): reject unverifiable Docker sockets
prekshivyas Jul 13, 2026
fe02076
fix(vllm): reject ambiguous client mount namespaces
prekshivyas Jul 13, 2026
4482a79
fix(vllm): prove Docker bind identity
prekshivyas Jul 13, 2026
c4681e3
fix(vllm): keep bind sentinel secret
prekshivyas Jul 13, 2026
a05fae6
test(vllm): guard bind token secrecy
prekshivyas Jul 13, 2026
f248258
fix(vllm): block ambiguous pre-pull namespaces
prekshivyas Jul 13, 2026
02f6c65
Merge branch 'main' into fix/6757-vllm-storage-preflight
cjagwani Jul 13, 2026
db8655a
Merge branch 'main' into fix/6757-vllm-storage-preflight
cv Jul 13, 2026
File filter

Filter by extension

Filter by extension

Conversations
Failed to load comments.
Loading
Jump to
Jump to file
Failed to load files.
Loading
Diff view
Diff view
2 changes: 1 addition & 1 deletion ci/platform-matrix.json
Original file line number Diff line number Diff line change
Expand Up @@ -147,7 +147,7 @@
"name": "Local vLLM (managed install/start)",
"status": "caveated",
"endpoint_type": "Local OpenAI-compatible",
"notes": "Appears by default on DGX Spark and DGX Station. Generic Linux NVIDIA GPU hosts require `NEMOCLAW_EXPERIMENTAL=1` or `NEMOCLAW_PROVIDER=install-vllm`. Host must have the NVIDIA Container Toolkit installed and a CDI spec present (`onboard` asserts CDI presence). NemoClaw pulls or starts the stable NGC vLLM container for each host profile. See `src/lib/inference/vllm.ts:55,177` for the pins. DGX Spark and DGX Station use `nvcr.io/nvidia/vllm:26.05.post1-py3`; generic Linux NVIDIA GPU hosts use `nvcr.io/nvidia/vllm:26.03.post1-py3`. Validated defaults are listed in `src/lib/inference/vllm-models.ts`: DGX Spark uses `nvidia/Qwen3.6-35B-A3B-NVFP4`, DGX Station uses `deepseek-ai/DeepSeek-V4-Flash`, and Linux NVIDIA GPU uses `nvidia/NVIDIA-Nemotron-3-Nano-4B-FP8`. Image pulls require NGC registry login (`docker login nvcr.io`); onboard prompts for the NGC API key when authentication is missing."
"notes": "Appears by default on DGX Spark and DGX Station. DGX Station remains deferred until its managed-vLLM onboarding path is validated end-to-end on physical hardware. Generic Linux NVIDIA GPU hosts require `NEMOCLAW_EXPERIMENTAL=1` or `NEMOCLAW_PROVIDER=install-vllm`. Host must have the NVIDIA Container Toolkit installed and a CDI spec present (`onboard` asserts CDI presence). NemoClaw pins each host profile to an immutable NGC platform digest. DGX Spark and DGX Station use the `linux/arm64` digest published under `nvcr.io/nvidia/vllm:26.05.post1-py3`; generic Linux NVIDIA GPU hosts use the matching `linux/arm64` or `linux/amd64` digest published under `nvcr.io/nvidia/vllm:26.03.post1-py3`. Validated defaults are listed in `src/lib/inference/vllm-models.ts`: DGX Spark uses `nvidia/Qwen3.6-35B-A3B-NVFP4`, DGX Station uses `deepseek-ai/DeepSeek-V4-Flash`, and Linux NVIDIA GPU uses `nvidia/NVIDIA-Nemotron-3-Nano-4B-FP8`. Image pulls require NGC registry login (`docker login nvcr.io`); onboard prompts for the NGC API key when authentication is missing."
}
],

Expand Down
2 changes: 1 addition & 1 deletion docs/inference/choose-inference-provider.mdx
Original file line number Diff line number Diff line change
Expand Up @@ -32,7 +32,7 @@ Use this status table to distinguish validated provider integrations from adapte
| Local Ollama | Tested with limitations | Local Ollama API | Available when Ollama is installed or running on the host. Validated default models: `qwen3.6:35b` (high VRAM), `nemotron-3-nano:30b` (medium VRAM), `qwen3.5:9b` (low VRAM fallback). |
| Local NVIDIA NIM | Experimental | Local OpenAI-compatible | Requires `NEMOCLAW_EXPERIMENTAL=1` and a NIM-capable NVIDIA GPU. Host must have the NVIDIA Container Toolkit installed and a CDI spec present (`onboard` asserts CDI presence with `assertCdiNvidiaGpuSpecPresent`, `src/lib/onboard/fatal-runtime-preflight.ts`). NIM images pull from `nvcr.io` and require NGC registry login. NemoClaw gates this path behind the experimental flag because it does not auto-select a NIM image for the host today. You must explicitly pick from the validated image list. On Linux arm64 DGX Spark and DGX Station hosts, onboarding warns that some NIM images may not publish a `linux/arm64` manifest; the warning is advisory, and the selected image pull can still fail when the registry has no matching platform manifest. Managed vLLM has host-specific default models and is not gated on the same boxes. Validated images referenced in `src/lib/inference/config.ts` and `nemoclaw/src/index.ts`: `nvidia/nemotron-3-super-120b-a12b` (default cloud model), `nvidia/nemotron-3-nano-30b-a3b`, `nvidia/llama-3.3-nemotron-super-49b-v1.5`. |
| Local vLLM (already running) | Tested with limitations | Local OpenAI-compatible | Appears in the onboarding menu when NemoClaw detects a server already on `localhost:8000`. No flag required. Model is whatever the existing server serves. |
| Local vLLM (managed install/start) | Tested with limitations | Local OpenAI-compatible | Appears by default on DGX Spark and DGX Station. Generic Linux NVIDIA GPU hosts require `NEMOCLAW_EXPERIMENTAL=1` or `NEMOCLAW_PROVIDER=install-vllm`. Host must have the NVIDIA Container Toolkit installed and a CDI spec present (`onboard` asserts CDI presence). NemoClaw pulls or starts the stable NGC vLLM container for each host profile. See `src/lib/inference/vllm.ts:55,177` for the pins. DGX Spark and DGX Station use `nvcr.io/nvidia/vllm:26.05.post1-py3`; generic Linux NVIDIA GPU hosts use `nvcr.io/nvidia/vllm:26.03.post1-py3`. Validated defaults are listed in `src/lib/inference/vllm-models.ts`: DGX Spark uses `nvidia/Qwen3.6-35B-A3B-NVFP4`, DGX Station uses `deepseek-ai/DeepSeek-V4-Flash`, and Linux NVIDIA GPU uses `nvidia/NVIDIA-Nemotron-3-Nano-4B-FP8`. Image pulls require NGC registry login (`docker login nvcr.io`); onboard prompts for the NGC API key when authentication is missing. |
| Local vLLM (managed install/start) | Tested with limitations | Local OpenAI-compatible | Appears by default on DGX Spark and DGX Station. DGX Station remains deferred until its managed-vLLM onboarding path is validated end-to-end on physical hardware. Generic Linux NVIDIA GPU hosts require `NEMOCLAW_EXPERIMENTAL=1` or `NEMOCLAW_PROVIDER=install-vllm`. Host must have the NVIDIA Container Toolkit installed and a CDI spec present (`onboard` asserts CDI presence). NemoClaw pins each host profile to an immutable NGC platform digest. DGX Spark and DGX Station use the `linux/arm64` digest published under `nvcr.io/nvidia/vllm:26.05.post1-py3`; generic Linux NVIDIA GPU hosts use the matching `linux/arm64` or `linux/amd64` digest published under `nvcr.io/nvidia/vllm:26.03.post1-py3`. Validated defaults are listed in `src/lib/inference/vllm-models.ts`: DGX Spark uses `nvidia/Qwen3.6-35B-A3B-NVFP4`, DGX Station uses `deepseek-ai/DeepSeek-V4-Flash`, and Linux NVIDIA GPU uses `nvidia/NVIDIA-Nemotron-3-Nano-4B-FP8`. Image pulls require NGC registry login (`docker login nvcr.io`); onboard prompts for the NGC API key when authentication is missing. |
{/* provider-status:end */}

## Hosted Providers
Expand Down
42 changes: 41 additions & 1 deletion docs/inference/set-up-vllm.mdx
Original file line number Diff line number Diff line change
Expand Up @@ -68,13 +68,53 @@ In an interactive run, the managed path lists the supported model registry for y
Press **Enter** to use the profile default or select another validated model with its registered `vllm serve` arguments.

NemoClaw pulls the vLLM image, downloads model weights into `~/.cache/huggingface`, starts the `nemoclaw-vllm` container on `localhost:8000`, streams Hugging Face download progress, and polls `/v1/models` until the model is ready.
Managed DGX Spark and DGX Station profiles use `nvcr.io/nvidia/vllm:26.05.post1-py3`.
Managed profiles use immutable NGC platform digests:

- DGX Spark and DGX Station use the `linux/arm64` digest `sha256:9204569b17ee4c0eff75194b8e6e458479c8aee18953b5ab9cf359fcdac659e2` with a compressed layer size of `9.60 GB` under `nvcr.io/nvidia/vllm:26.05.post1-py3`.
DGX Station remains deferred until this path is validated end-to-end on physical hardware.
- Generic Linux `arm64` hosts use `sha256:447995cbb57e6c7cf792cab95e9852e5f62b5fb6d2f39e030fa4eda9a54eadb4` with a compressed layer size of `9.28 GB` under `nvcr.io/nvidia/vllm:26.03.post1-py3`.
- Generic Linux `amd64` hosts use `sha256:7be6c2f676c36059a494fe17254e69ae5c677535ba6191044e5fc8e42a91c773` with a compressed layer size of `8.93 GB` under `nvcr.io/nvidia/vllm:26.03.post1-py3`.

Image pulls from `nvcr.io` require NGC registry login.

The first run can take 10 to 30 minutes.
Later runs reuse the cached image and model weights.
The managed container uses Docker's `--restart unless-stopped` policy, so Docker restarts it unless an operator explicitly stops it.

### Check Storage Before Downloads

For an uncached image, NemoClaw checks Docker image storage before the watched image pull starts.
It checks the Hugging Face model cache before that pull and checks it again afterward before `hf download` starts.
The image estimate includes compressed content, unpacked layers, and pull-staging headroom.
The model estimate includes the model's full published size plus temporary-write headroom because a local cache cannot prove which files a mutable remote revision will reuse.

For Docker's classic image store, NemoClaw checks the absolute `DockerRootDir` reported by `docker info`.
For Docker's containerd image store, it checks the top-level `root` from `/etc/containerd/config.toml`, or `/var/lib/containerd` when that configuration file is absent, and also checks `DockerRootDir` for pull staging.
The check uses the location with the least available capacity.
The model-cache check uses `~/.cache/huggingface`, or its nearest existing parent filesystem when the cache directory does not exist yet.

NemoClaw recognizes an image cache hit only when Docker contains the exact platform digest.
A cache hit skips the cold-pull capacity check, but the explicit watched `docker pull` still runs against that immutable digest.
Before trusting the host model-cache capacity, NemoClaw also verifies that the selected Docker daemon maps bind mounts to the local host; a remote or unverifiable Docker host makes the model-cache check inconclusive.
An explicit Unix socket is trusted only when it is Docker's default `/var/run/docker.sock`; any other socket can forward to a daemon with a different filesystem and is treated as unverifiable.
Known container markers are rejected immediately.
Before trusting client-side image-store capacity, NemoClaw requires the selected default Docker socket peer to have a visible PID and share the client's mount namespace; a nested PID namespace or unverifiable peer fails closed before the image pull.
Before trusting model-cache capacity, NemoClaw also asks the pinned cached vLLM image to read a random mode-`0600` sentinel through a bounded, read-only Docker bind mount.
The pull is idempotent for the cached digest and cannot select different layers from a moved release tag.
The model-download helper and the long-lived vLLM container use `--pull=never`, so an implicit pull cannot bypass the storage gate.
The second model-cache check detects capacity consumed by the image pull before the model download command can start.

When capacity is insufficient or NemoClaw cannot verify it, the warning identifies the image or model, available capacity, approximate requirement, checked path and source, remediation, and diagnostic commands.
Interactive setup continues only when you enter `y` or `yes` explicitly.
Pressing **Enter**, entering `n` or `no`, or entering any other response stops before the guarded pull or model download starts.

<Warning>
`--yes` and `NEMOCLAW_YES=1` do not accept managed-vLLM storage warnings.
Non-interactive setup stops unless you set `NEMOCLAW_IGNORE_VLLM_DISK_SPACE=1` exactly.
This override accepts both insufficient-capacity and inconclusive checks for Docker image storage and the model cache.
Use it only after you independently verify that both storage locations can complete the downloads.
</Warning>

If image pull output stops making progress, a watchdog stops the stalled pull without imposing a fixed wall-clock limit on slow but active downloads.
If vLLM does not become ready, NemoClaw prints a short tail of the container logs before exiting.

Expand Down
3 changes: 2 additions & 1 deletion docs/reference/commands.mdx
Original file line number Diff line number Diff line change
Expand Up @@ -3250,6 +3250,7 @@ Set them before running `$$nemoclaw onboard`.
| `NEMOCLAW_INSTALL_TAG` | release tag | For internal installer commands: the release tag to install. Defaults to the admin-promoted `lkg` tag when unset. Overridden by the `--install-tag` flag. |
| `NEMOCLAW_VLLM_MODEL` | registry slug or Hugging Face model id | Selects the model the managed-vLLM install path serves. Recognised slugs: `qwen3.6-27b`, `qwen3.6-35b-a3b-nvfp4`, `nemotron-3-nano-4b`, `deepseek-v4-flash`, `deepseek-r1-distill-70b`. Unset uses the per-platform profile default. Gated models (e.g. `deepseek-r1-distill-70b`) require `HF_TOKEN` or `HUGGING_FACE_HUB_TOKEN`. |
| `NEMOCLAW_VLLM_EXTRA_ARGS_JSON` | JSON array of non-blank strings | Appends advanced operator-owned tokens to the managed `vllm serve` command after NemoClaw's registry defaults. Example: `["--max-num-seqs","2"]`. Malformed JSON, non-string tokens, or blank tokens fail before Docker work starts. |
| `NEMOCLAW_IGNORE_VLLM_DISK_SPACE` | exactly `1` | Allows managed vLLM to continue when Docker image-storage or model-cache capacity is insufficient or cannot be verified; other values do not bypass the checks. |
<AgentOnly variant="openclaw">
| `NEMOCLAW_MINIMAL_BOOTSTRAP` | `1` to enable | Skips default OpenClaw workspace-template seeding for new pristine workspaces. Existing files are not deleted; refer to [Runtime Controls](../manage-sandboxes/runtime-controls). |
</AgentOnly>
Expand Down Expand Up @@ -3399,7 +3400,7 @@ Set them before running `$$nemoclaw onboard`.

| Variable | Format | Effect |
|----------|--------|--------|
| `NEMOCLAW_YES` | `1` to enable | Auto-accepts confirmation prompts (`--yes` equivalent) including in helpers like the Ollama proxy auth setup. |
| `NEMOCLAW_YES` | `1` to enable | Auto-accepts confirmation prompts (`--yes` equivalent) including in helpers like the Ollama proxy auth setup, but does not accept managed-vLLM storage warnings. |
| `NEMOCLAW_OLLAMA_NO_AUTOSTART` | `1` to enable | Skips the wizard's eager Ollama auto-start during inference-provider selection (equivalent to passing `--no-ollama-autostart`). When set and Ollama is not running on `localhost:11434`, the `$$nemoclaw onboard` Local Ollama path prints a warning and selects the default fallback model instead of spawning `ollama serve`. The flag covers only the provider-selection step; later setup steps (auth proxy, validation, model warm) still expect a reachable Ollama. On Linux hosts with a systemd Ollama unit, the loopback-override path may still restart the daemon before this gate runs. |
| `NEMOCLAW_NON_INTERACTIVE_SUDO_MODE` | `prompt` or empty/unset | When set to `prompt`, allows non-interactive onboarding to use prompt-capable `sudo` for host setup steps that require elevation, which can ask for a password. Empty/unset is the default and uses `sudo -n`, which fails instead of asking for a password. Any other value is rejected. |
| `NEMOCLAW_NO_EXPRESS` | `1` to enable | Installer-only. Skips the DGX Spark, DGX Station, and Windows WSL express install prompt and continues with the normal interactive onboarding flow. |
Expand Down
2 changes: 1 addition & 1 deletion docs/reference/platform-support.mdx
Original file line number Diff line number Diff line change
Expand Up @@ -104,7 +104,7 @@ NemoClaw routes inference through the OpenShell gateway. Each row below is a pro
| Local Ollama | Tested with limitations | Local Ollama API | Available when Ollama is installed or running on the host. Validated default models: `qwen3.6:35b` (high VRAM), `nemotron-3-nano:30b` (medium VRAM), `qwen3.5:9b` (low VRAM fallback). |
| Local NVIDIA NIM | Experimental | Local OpenAI-compatible | Requires `NEMOCLAW_EXPERIMENTAL=1` and a NIM-capable NVIDIA GPU. Host must have the NVIDIA Container Toolkit installed and a CDI spec present (`onboard` asserts CDI presence with `assertCdiNvidiaGpuSpecPresent`, `src/lib/onboard/fatal-runtime-preflight.ts`). NIM images pull from `nvcr.io` and require NGC registry login. NemoClaw gates this path behind the experimental flag because it does not auto-select a NIM image for the host today. You must explicitly pick from the validated image list. On Linux arm64 DGX Spark and DGX Station hosts, onboarding warns that some NIM images may not publish a `linux/arm64` manifest; the warning is advisory, and the selected image pull can still fail when the registry has no matching platform manifest. Managed vLLM has host-specific default models and is not gated on the same boxes. Validated images referenced in `src/lib/inference/config.ts` and `nemoclaw/src/index.ts`: `nvidia/nemotron-3-super-120b-a12b` (default cloud model), `nvidia/nemotron-3-nano-30b-a3b`, `nvidia/llama-3.3-nemotron-super-49b-v1.5`. |
| Local vLLM (already running) | Tested with limitations | Local OpenAI-compatible | Appears in the onboarding menu when NemoClaw detects a server already on `localhost:8000`. No flag required. Model is whatever the existing server serves. |
| Local vLLM (managed install/start) | Tested with limitations | Local OpenAI-compatible | Appears by default on DGX Spark and DGX Station. Generic Linux NVIDIA GPU hosts require `NEMOCLAW_EXPERIMENTAL=1` or `NEMOCLAW_PROVIDER=install-vllm`. Host must have the NVIDIA Container Toolkit installed and a CDI spec present (`onboard` asserts CDI presence). NemoClaw pulls or starts the stable NGC vLLM container for each host profile. See `src/lib/inference/vllm.ts:55,177` for the pins. DGX Spark and DGX Station use `nvcr.io/nvidia/vllm:26.05.post1-py3`; generic Linux NVIDIA GPU hosts use `nvcr.io/nvidia/vllm:26.03.post1-py3`. Validated defaults are listed in `src/lib/inference/vllm-models.ts`: DGX Spark uses `nvidia/Qwen3.6-35B-A3B-NVFP4`, DGX Station uses `deepseek-ai/DeepSeek-V4-Flash`, and Linux NVIDIA GPU uses `nvidia/NVIDIA-Nemotron-3-Nano-4B-FP8`. Image pulls require NGC registry login (`docker login nvcr.io`); onboard prompts for the NGC API key when authentication is missing. |
| Local vLLM (managed install/start) | Tested with limitations | Local OpenAI-compatible | Appears by default on DGX Spark and DGX Station. DGX Station remains deferred until its managed-vLLM onboarding path is validated end-to-end on physical hardware. Generic Linux NVIDIA GPU hosts require `NEMOCLAW_EXPERIMENTAL=1` or `NEMOCLAW_PROVIDER=install-vllm`. Host must have the NVIDIA Container Toolkit installed and a CDI spec present (`onboard` asserts CDI presence). NemoClaw pins each host profile to an immutable NGC platform digest. DGX Spark and DGX Station use the `linux/arm64` digest published under `nvcr.io/nvidia/vllm:26.05.post1-py3`; generic Linux NVIDIA GPU hosts use the matching `linux/arm64` or `linux/amd64` digest published under `nvcr.io/nvidia/vllm:26.03.post1-py3`. Validated defaults are listed in `src/lib/inference/vllm-models.ts`: DGX Spark uses `nvidia/Qwen3.6-35B-A3B-NVFP4`, DGX Station uses `deepseek-ai/DeepSeek-V4-Flash`, and Linux NVIDIA GPU uses `nvidia/NVIDIA-Nemotron-3-Nano-4B-FP8`. Image pulls require NGC registry login (`docker login nvcr.io`); onboard prompts for the NGC API key when authentication is missing. |
{/* provider-status-full:end */}

## Messaging Integrations
Expand Down
41 changes: 41 additions & 0 deletions src/lib/inference/vllm-docker-env.test.ts
Original file line number Diff line number Diff line change
@@ -0,0 +1,41 @@
// SPDX-FileCopyrightText: Copyright (c) 2026 NVIDIA CORPORATION & AFFILIATES. All rights reserved.
// SPDX-License-Identifier: Apache-2.0

import { afterEach, describe, expect, it, vi } from "vitest";
import { buildVllmDockerEnv } from "./vllm-docker-env";

afterEach(() => {
vi.unstubAllEnvs();
});

describe("managed vLLM Docker client environment", () => {
it("forwards one Docker context while retaining subprocess secret filtering (#6757)", () => {
vi.stubEnv("DOCKER_CONFIG", "/tmp/nemoclaw-docker-config");
vi.stubEnv("DOCKER_CONTEXT", "remote-builder");
vi.stubEnv("DOCKER_HOST", "ssh://fallback.example.test");
vi.stubEnv("DOCKER_TLS", "1");
vi.stubEnv("UNRELATED_SECRET", "do-not-forward");

const env = buildVllmDockerEnv({ HF_TOKEN: "hf_test" });

expect(env).toEqual(
expect.objectContaining({
DOCKER_CONFIG: "/tmp/nemoclaw-docker-config",
DOCKER_CONTEXT: "remote-builder",
DOCKER_HOST: "ssh://fallback.example.test",
DOCKER_TLS: "1",
HF_TOKEN: "hf_test",
}),
);
expect(env.UNRELATED_SECRET).toBeUndefined();
});

it("does not inherit Docker selectors omitted from an explicit source (#6757)", () => {
vi.stubEnv("DOCKER_HOST", "ssh://ambient.example.test");

const env = buildVllmDockerEnv({}, { DOCKER_CONTEXT: "requested-context" });

expect(env.DOCKER_CONTEXT).toBe("requested-context");
expect(env.DOCKER_HOST).toBeUndefined();
});
});
Loading
Loading