Skip to content
Merged
Show file tree
Hide file tree
Changes from all commits
Commits
File filter

Filter by extension

Filter by extension

Conversations
Failed to load comments.
Loading
Jump to
Jump to file
Failed to load files.
Loading
Diff view
Diff view
4 changes: 2 additions & 2 deletions ci/platform-matrix.json
Original file line number Diff line number Diff line change
Expand Up @@ -64,7 +64,7 @@
"status": "deferred",
"prd_priority": "P1",
"ci_tested": false,
"notes": "The PRD marks this platform as P1. Workstation form-factor with NVIDIA GPUs and the same Docker + NVIDIA Container Toolkit + CDI requirements as DGX Spark. Onboard path not yet validated end-to-end on the hardware; vLLM has a placeholder default model defined for this host class (`Qwen/Qwen3.6-27B-FP8`) that will move out of `deferred` once the hardware run is signed off."
"notes": "The PRD marks this platform as P1. Workstation form-factor with NVIDIA GPUs and the same Docker + NVIDIA Container Toolkit + CDI requirements as DGX Spark. Onboard path not yet validated end-to-end on the hardware; vLLM defaults to `deepseek-ai/DeepSeek-V4-Flash` for this host class and will move out of `deferred` once the hardware run is signed off."
},
{
"name": "NVIDIA RTX (consumer and Pro workstation GPUs)",
Expand Down Expand Up @@ -141,7 +141,7 @@
"name": "Local vLLM (managed install/start)",
"status": "caveated",
"endpoint_type": "Local OpenAI-compatible",
"notes": "Appears by default on DGX Spark and DGX Station. Generic Linux NVIDIA GPU hosts require `NEMOCLAW_EXPERIMENTAL=1` or `NEMOCLAW_PROVIDER=install-vllm`. Host must have the NVIDIA Container Toolkit installed and a CDI spec present (`onboard` asserts CDI presence). NemoClaw pulls or starts the stable NGC vLLM container for each host profile. See `src/lib/inference/vllm.ts:55,177` for the pins. DGX Spark and DGX Station use `nvcr.io/nvidia/vllm:26.05.post1-py3`; generic Linux NVIDIA GPU hosts use `nvcr.io/nvidia/vllm:26.03.post1-py3`. Validated defaults are listed in `src/lib/inference/vllm-models.ts`: DGX Spark uses `nvidia/Qwen3.6-35B-A3B-NVFP4`, DGX Station uses `Qwen/Qwen3.6-27B-FP8`, and Linux NVIDIA GPU uses `nvidia/NVIDIA-Nemotron-3-Nano-4B-FP8`. Image pulls require NGC registry login (`docker login nvcr.io`); onboard prompts for the NGC API key when authentication is missing."
"notes": "Appears by default on DGX Spark and DGX Station. Generic Linux NVIDIA GPU hosts require `NEMOCLAW_EXPERIMENTAL=1` or `NEMOCLAW_PROVIDER=install-vllm`. Host must have the NVIDIA Container Toolkit installed and a CDI spec present (`onboard` asserts CDI presence). NemoClaw pulls or starts the stable NGC vLLM container for each host profile. See `src/lib/inference/vllm.ts:55,177` for the pins. DGX Spark and DGX Station use `nvcr.io/nvidia/vllm:26.05.post1-py3`; generic Linux NVIDIA GPU hosts use `nvcr.io/nvidia/vllm:26.03.post1-py3`. Validated defaults are listed in `src/lib/inference/vllm-models.ts`: DGX Spark uses `nvidia/Qwen3.6-35B-A3B-NVFP4`, DGX Station uses `deepseek-ai/DeepSeek-V4-Flash`, and Linux NVIDIA GPU uses `nvidia/NVIDIA-Nemotron-3-Nano-4B-FP8`. Image pulls require NGC registry login (`docker login nvcr.io`); onboard prompts for the NGC API key when authentication is missing."
}
],

Expand Down
2 changes: 1 addition & 1 deletion docs/get-started/quickstart.mdx
Original file line number Diff line number Diff line change
Expand Up @@ -78,7 +78,7 @@ curl -fsSL https://www.nvidia.com/nemoclaw.sh | bash

On DGX Spark, DGX Station, and Windows WSL, an interactive installer offers express install after you accept the third-party software notice.
Express install switches onboarding to non-interactive mode, allows `sudo` password prompts for required host changes, and selects the managed local inference path for that platform.
On DGX Spark and DGX Station, express install selects managed vLLM; DGX Spark uses `qwen3.6-35b-a3b-nvfp4` by default.
On DGX Spark and DGX Station, express install selects managed vLLM; DGX Spark uses `qwen3.6-35b-a3b-nvfp4` by default, and DGX Station uses `deepseek-v4-flash` by default.
Unless `NEMOCLAW_POLICY_TIER` is set, it applies sandbox policy in `suggested` mode with the `balanced` tier by default.
That tier uses the base sandbox policy plus supported package, model, web-search, local-inference, and read-only weather presets.
On DGX Spark, express install uses `my-spark-assistant` as the sandbox name unless `NEMOCLAW_SANDBOX_NAME` is already set.
Expand Down
8 changes: 4 additions & 4 deletions docs/inference/inference-options.mdx
Original file line number Diff line number Diff line change
Expand Up @@ -51,7 +51,7 @@ NemoClaw uses provider-specific local tokens for those routes, and rebuilds of l
| Local Ollama | Tested with limitations | Local Ollama API | Available when Ollama is installed or running on the host. Validated default models: `qwen3.6:35b` (high VRAM), `nemotron-3-nano:30b` (medium VRAM), `qwen3.5:9b` (low VRAM fallback). |
| Local NVIDIA NIM | Experimental | Local OpenAI-compatible | Requires `NEMOCLAW_EXPERIMENTAL=1` and a NIM-capable NVIDIA GPU. Host must have the NVIDIA Container Toolkit installed and a CDI spec present (`onboard` asserts CDI presence with `assertCdiNvidiaGpuSpecPresent`, `src/lib/onboard.ts:1586`). NIM images pull from `nvcr.io` and require NGC registry login. NemoClaw gates this path behind the experimental flag because it does not auto-select a NIM image for the host today. You must explicitly pick from the validated image list. Managed vLLM has host-specific default models and is not gated on the same boxes. Validated images referenced in `src/lib/inference/config.ts` and `nemoclaw/src/index.ts`: `nvidia/nemotron-3-super-120b-a12b` (default cloud model), `nvidia/nemotron-3-nano-30b-a3b`, `nvidia/llama-3.3-nemotron-super-49b-v1.5`. |
| Local vLLM (already running) | Tested with limitations | Local OpenAI-compatible | Appears in the onboarding menu when NemoClaw detects a server already on `localhost:8000`. No flag required. Model is whatever the existing server serves. |
| Local vLLM (managed install/start) | Tested with limitations | Local OpenAI-compatible | Appears by default on DGX Spark and DGX Station. Generic Linux NVIDIA GPU hosts require `NEMOCLAW_EXPERIMENTAL=1` or `NEMOCLAW_PROVIDER=install-vllm`. Host must have the NVIDIA Container Toolkit installed and a CDI spec present (`onboard` asserts CDI presence). NemoClaw pulls or starts the stable NGC vLLM container for each host profile. See `src/lib/inference/vllm.ts:55,177` for the pins. DGX Spark and DGX Station use `nvcr.io/nvidia/vllm:26.05.post1-py3`; generic Linux NVIDIA GPU hosts use `nvcr.io/nvidia/vllm:26.03.post1-py3`. Validated defaults are listed in `src/lib/inference/vllm-models.ts`: DGX Spark uses `nvidia/Qwen3.6-35B-A3B-NVFP4`, DGX Station uses `Qwen/Qwen3.6-27B-FP8`, and Linux NVIDIA GPU uses `nvidia/NVIDIA-Nemotron-3-Nano-4B-FP8`. Image pulls require NGC registry login (`docker login nvcr.io`); onboard prompts for the NGC API key when authentication is missing. |
| Local vLLM (managed install/start) | Tested with limitations | Local OpenAI-compatible | Appears by default on DGX Spark and DGX Station. Generic Linux NVIDIA GPU hosts require `NEMOCLAW_EXPERIMENTAL=1` or `NEMOCLAW_PROVIDER=install-vllm`. Host must have the NVIDIA Container Toolkit installed and a CDI spec present (`onboard` asserts CDI presence). NemoClaw pulls or starts the stable NGC vLLM container for each host profile. See `src/lib/inference/vllm.ts:55,177` for the pins. DGX Spark and DGX Station use `nvcr.io/nvidia/vllm:26.05.post1-py3`; generic Linux NVIDIA GPU hosts use `nvcr.io/nvidia/vllm:26.03.post1-py3`. Validated defaults are listed in `src/lib/inference/vllm-models.ts`: DGX Spark uses `nvidia/Qwen3.6-35B-A3B-NVFP4`, DGX Station uses `deepseek-ai/DeepSeek-V4-Flash`, and Linux NVIDIA GPU uses `nvidia/NVIDIA-Nemotron-3-Nano-4B-FP8`. Image pulls require NGC registry login (`docker login nvcr.io`); onboard prompts for the NGC API key when authentication is missing. |
{/* provider-status:end */}

## Provider Options
Expand Down Expand Up @@ -318,7 +318,7 @@ Managed vLLM uses these profiles:
| Host profile | Default model |
|---|---|
| DGX Spark | `nvidia/Qwen3.6-35B-A3B-NVFP4` |
| DGX Station | `Qwen/Qwen3.6-27B-FP8` |
| DGX Station | `deepseek-ai/DeepSeek-V4-Flash` |
| Linux with an NVIDIA GPU | `nvidia/NVIDIA-Nemotron-3-Nano-4B-FP8` |

<Note>
Expand Down Expand Up @@ -356,10 +356,10 @@ Recognized slugs are:

| Slug | Hugging Face model | Notes |
|---|---|---|
| `qwen3.6-27b` | `Qwen/Qwen3.6-27B-FP8` | Default on the DGX Station profile |
| `qwen3.6-27b` | `Qwen/Qwen3.6-27B-FP8` | Supported override |
| `qwen3.6-35b-a3b-nvfp4` | `nvidia/Qwen3.6-35B-A3B-NVFP4` | Default on the DGX Spark profile |
| `nemotron-3-nano-4b` | `nvidia/NVIDIA-Nemotron-3-Nano-4B-FP8` | Default on the generic Linux + NVIDIA GPU profile |
| `deepseek-v4-flash` | `deepseek-ai/DeepSeek-V4-Flash` | Supported override |
| `deepseek-v4-flash` | `deepseek-ai/DeepSeek-V4-Flash` | Default on the DGX Station profile |
| `deepseek-r1-distill-70b` | `deepseek-ai/DeepSeek-R1-Distill-Llama-70B` | Gated. Requires Hugging Face license acceptance |

The slug is case-insensitive; the full Hugging Face id is also accepted.
Expand Down
4 changes: 2 additions & 2 deletions docs/reference/platform-support.mdx
Original file line number Diff line number Diff line change
Expand Up @@ -82,7 +82,7 @@ For the onboarding-time supported set without deferred rows, refer to [Prerequis
| macOS (Apple Silicon) | Colima, Docker Desktop | Tested with limitations | P0 | Yes | Start the container runtime (Colima or Docker Desktop) before running the installer. Xcode Command Line Tools (`xcode-select --install`) are typically required for Node native modules during install. NemoClaw recommends them but does not enforce them during preflight. |
| DGX Spark | Docker | Tested | P1 | Yes | Use the standard installer and `$$nemoclaw onboard`. For an end-to-end walkthrough with local inference, see the [NVIDIA Spark playbook](https://build.nvidia.com/spark/nemoclaw). |
| Windows WSL2 | Docker Desktop (WSL backend) | Tested with limitations | P1 | No | Requires WSL2 with Docker Desktop backend. |
| DGX Station | Docker | Deferred | P1 | No | The PRD marks this platform as P1. Workstation form-factor with NVIDIA GPUs and the same Docker + NVIDIA Container Toolkit + CDI requirements as DGX Spark. Onboard path not yet validated end-to-end on the hardware; vLLM has a placeholder default model defined for this host class (`Qwen/Qwen3.6-27B-FP8`) that will move out of `deferred` once the hardware run is signed off. |
| DGX Station | Docker | Deferred | P1 | No | The PRD marks this platform as P1. Workstation form-factor with NVIDIA GPUs and the same Docker + NVIDIA Container Toolkit + CDI requirements as DGX Spark. Onboard path not yet validated end-to-end on the hardware; vLLM defaults to `deepseek-ai/DeepSeek-V4-Flash` for this host class and will move out of `deferred` once the hardware run is signed off. |
| NVIDIA RTX (consumer and Pro workstation GPUs) | Docker | Deferred | P1 | No | The PRD marks this platform as P1. Covers RTX consumer cards and RTX Pro workstation cards on Linux hosts that meet the generic-Linux-GPU requirements (NVIDIA Container Toolkit + CDI present). The provider menu emits managed vLLM behind `NEMOCLAW_EXPERIMENTAL=1` or `NEMOCLAW_PROVIDER=install-vllm` for this host class today; the end-to-end onboard path on this hardware is not yet validated in CI. |
{/* platform-matrix-full:end */}

Expand All @@ -103,7 +103,7 @@ NemoClaw routes inference through the OpenShell gateway. Each row below is a pro
| Local Ollama | Tested with limitations | Local Ollama API | Available when Ollama is installed or running on the host. Validated default models: `qwen3.6:35b` (high VRAM), `nemotron-3-nano:30b` (medium VRAM), `qwen3.5:9b` (low VRAM fallback). |
| Local NVIDIA NIM | Experimental | Local OpenAI-compatible | Requires `NEMOCLAW_EXPERIMENTAL=1` and a NIM-capable NVIDIA GPU. Host must have the NVIDIA Container Toolkit installed and a CDI spec present (`onboard` asserts CDI presence with `assertCdiNvidiaGpuSpecPresent`, `src/lib/onboard.ts:1586`). NIM images pull from `nvcr.io` and require NGC registry login. NemoClaw gates this path behind the experimental flag because it does not auto-select a NIM image for the host today. You must explicitly pick from the validated image list. Managed vLLM has host-specific default models and is not gated on the same boxes. Validated images referenced in `src/lib/inference/config.ts` and `nemoclaw/src/index.ts`: `nvidia/nemotron-3-super-120b-a12b` (default cloud model), `nvidia/nemotron-3-nano-30b-a3b`, `nvidia/llama-3.3-nemotron-super-49b-v1.5`. |
| Local vLLM (already running) | Tested with limitations | Local OpenAI-compatible | Appears in the onboarding menu when NemoClaw detects a server already on `localhost:8000`. No flag required. Model is whatever the existing server serves. |
| Local vLLM (managed install/start) | Tested with limitations | Local OpenAI-compatible | Appears by default on DGX Spark and DGX Station. Generic Linux NVIDIA GPU hosts require `NEMOCLAW_EXPERIMENTAL=1` or `NEMOCLAW_PROVIDER=install-vllm`. Host must have the NVIDIA Container Toolkit installed and a CDI spec present (`onboard` asserts CDI presence). NemoClaw pulls or starts the stable NGC vLLM container for each host profile. See `src/lib/inference/vllm.ts:55,177` for the pins. DGX Spark and DGX Station use `nvcr.io/nvidia/vllm:26.05.post1-py3`; generic Linux NVIDIA GPU hosts use `nvcr.io/nvidia/vllm:26.03.post1-py3`. Validated defaults are listed in `src/lib/inference/vllm-models.ts`: DGX Spark uses `nvidia/Qwen3.6-35B-A3B-NVFP4`, DGX Station uses `Qwen/Qwen3.6-27B-FP8`, and Linux NVIDIA GPU uses `nvidia/NVIDIA-Nemotron-3-Nano-4B-FP8`. Image pulls require NGC registry login (`docker login nvcr.io`); onboard prompts for the NGC API key when authentication is missing. |
| Local vLLM (managed install/start) | Tested with limitations | Local OpenAI-compatible | Appears by default on DGX Spark and DGX Station. Generic Linux NVIDIA GPU hosts require `NEMOCLAW_EXPERIMENTAL=1` or `NEMOCLAW_PROVIDER=install-vllm`. Host must have the NVIDIA Container Toolkit installed and a CDI spec present (`onboard` asserts CDI presence). NemoClaw pulls or starts the stable NGC vLLM container for each host profile. See `src/lib/inference/vllm.ts:55,177` for the pins. DGX Spark and DGX Station use `nvcr.io/nvidia/vllm:26.05.post1-py3`; generic Linux NVIDIA GPU hosts use `nvcr.io/nvidia/vllm:26.03.post1-py3`. Validated defaults are listed in `src/lib/inference/vllm-models.ts`: DGX Spark uses `nvidia/Qwen3.6-35B-A3B-NVFP4`, DGX Station uses `deepseek-ai/DeepSeek-V4-Flash`, and Linux NVIDIA GPU uses `nvidia/NVIDIA-Nemotron-3-Nano-4B-FP8`. Image pulls require NGC registry login (`docker login nvcr.io`); onboard prompts for the NGC API key when authentication is missing. |
{/* provider-status-full:end */}

## Messaging Integrations
Expand Down
2 changes: 1 addition & 1 deletion src/lib/inference/model-prompts.test.ts
Original file line number Diff line number Diff line change
Expand Up @@ -235,7 +235,7 @@ describe("promptVllmModel", () => {
const sparkDefault = VLLM_MODELS.find((m) => m.envValue === "qwen3.6-35b-a3b-nvfp4")!;
const gatedModel = VLLM_MODELS.find((m) => m.envValue === "deepseek-r1-distill-70b")!;
const stationModels = modelsForPlatform("station");
const stationDefault = VLLM_MODELS.find((m) => m.envValue === "qwen3.6-27b")!;
const stationDefault = VLLM_MODELS.find((m) => m.envValue === "deepseek-v4-flash")!;

it("returns the profile default when the user presses Enter", async () => {
const promptFn = promptSequence([""]);
Expand Down
4 changes: 1 addition & 3 deletions src/lib/inference/vllm-models.test.ts
Original file line number Diff line number Diff line change
Expand Up @@ -170,9 +170,7 @@ describe("vllm model registry", () => {
expect(cmd).toContain("--no-disable-hybrid-kv-cache-manager");
expect(cmd).toContain("--disable-uvicorn-access-log");
expect(cmd).toContain("--max-cudagraph-capture-size 128");
expect(cmd).toContain(
`--speculative-config '{"method":"mtp","num_speculative_tokens":3,"rejection_sample_method":"synthetic","synthetic_acceptance_length":3}'`,
);
expect(cmd).toContain(`--speculative-config '{"method":"mtp","num_speculative_tokens":3}'`);
expect(cmd).toContain("--max-model-len 1048576");
expect(cmd).toContain("--max-num-batched-tokens 8192");
expect(cmd).toContain("--max-num-seqs 16");
Expand Down
6 changes: 3 additions & 3 deletions src/lib/inference/vllm-models.ts
Original file line number Diff line number Diff line change
Expand Up @@ -141,7 +141,7 @@ export const VLLM_MODELS: readonly VllmModelDef[] = [
"--max-cudagraph-capture-size",
"128",
"--speculative-config",
`'{"method":"mtp","num_speculative_tokens":3,"rejection_sample_method":"synthetic","synthetic_acceptance_length":3}'`,
`'{"method":"mtp","num_speculative_tokens":3}'`,
"--max-num-batched-tokens",
"8192",
"--max-num-seqs",
Expand Down Expand Up @@ -214,8 +214,8 @@ export const VLLM_EXTRA_ARGS_ENV = "NEMOCLAW_VLLM_EXTRA_ARGS_JSON";
/**
* Look up the requested express-vLLM model from `NEMOCLAW_VLLM_MODEL`.
* Returns `null` when the env var is empty so the caller can fall back to
* the per-platform profile default (Station prefers Qwen3.6-27B, Spark the
* Qwen3.6-35B-A3B NVFP4 checkpoint, and the generic Linux profile prefers
* the per-platform profile default (Station prefers DeepSeek V4 Flash, Spark
* the Qwen3.6-35B-A3B NVFP4 checkpoint, and the generic Linux profile prefers
* Nemotron-Nano-4B for VRAM headroom).
*
* Match is case-insensitive against either the `envValue` slug or the full
Expand Down
6 changes: 3 additions & 3 deletions src/lib/inference/vllm.test.ts
Original file line number Diff line number Diff line change
Expand Up @@ -34,14 +34,14 @@ describe("vLLM profile detection", () => {
vi.clearAllMocks();
});

it("uses Qwen3.6 27B and the 26.05.post1 NGC image on DGX Station", () => {
it("uses DeepSeek V4 Flash and the 26.05.post1 NGC image on DGX Station", () => {
const profile = detectVllmProfile({ platform: "station", type: "nvidia" });

expect(profile).not.toBeNull();
expect(profile!.name).toBe("DGX Station");
expect(profile!.image).toBe("nvcr.io/nvidia/vllm:26.05.post1-py3");
expect(profile!.defaultModel.id).toBe("Qwen/Qwen3.6-27B-FP8");
expect(profile!.defaultModel.envValue).toBe("qwen3.6-27b");
expect(profile!.defaultModel.id).toBe("deepseek-ai/DeepSeek-V4-Flash");
expect(profile!.defaultModel.envValue).toBe("deepseek-v4-flash");
});

it("keeps DGX Spark on the Qwen3.6 35B NVFP4 default", () => {
Expand Down
8 changes: 4 additions & 4 deletions src/lib/inference/vllm.ts
Original file line number Diff line number Diff line change
Expand Up @@ -62,9 +62,9 @@ function nemotronNanoModel(): VllmModelDef {
return match;
}

function qwen27bFP8Model(): VllmModelDef {
const match = VLLM_MODELS.find((m) => m.envValue === "qwen3.6-27b");
if (!match) throw new Error("vllm-models registry is missing the qwen3.6-27b entry");
function deepseekV4FlashModel(): VllmModelDef {
const match = VLLM_MODELS.find((m) => m.envValue === "deepseek-v4-flash");
if (!match) throw new Error("vllm-models registry is missing the deepseek-v4-flash entry");
return match;
}

Expand Down Expand Up @@ -144,7 +144,7 @@ const STATION_PROFILE: VllmProfile = {
name: "DGX Station",
platform: "station",
image: VLLM_IMAGES.ngc2605Post1,
defaultModel: qwen27bFP8Model(),
defaultModel: deepseekV4FlashModel(),
containerName: "nemoclaw-vllm",
dockerRunFlags: SPARK_PROFILE.dockerRunFlags,
buildDockerRunFlags: () => {
Expand Down
4 changes: 2 additions & 2 deletions test/detect-vllm-profile.test.ts
Original file line number Diff line number Diff line change
Expand Up @@ -29,8 +29,8 @@ describe("detectVllmProfile", () => {
expect(profile).not.toBeNull();
expect(profile!.name).toBe("DGX Station");
expect(profile!.image).toBe("nvcr.io/nvidia/vllm:26.05.post1-py3");
expect(profile!.defaultModel.id).toBe("Qwen/Qwen3.6-27B-FP8");
expect(profile!.defaultModel.envValue).toBe("qwen3.6-27b");
expect(profile!.defaultModel.id).toBe("deepseek-ai/DeepSeek-V4-Flash");
expect(profile!.defaultModel.envValue).toBe("deepseek-v4-flash");
});

it("returns the generic Linux profile for non-Spark/Station NVIDIA hosts", () => {
Expand Down
Loading