Skip to content
Merged
Show file tree
Hide file tree
Changes from all commits
Commits
File filter

Filter by extension

Filter by extension

Conversations
Failed to load comments.
Loading
Jump to
Jump to file
Failed to load files.
Loading
Diff view
Diff view
6 changes: 3 additions & 3 deletions docs/inference/inference-options.mdx
Original file line number Diff line number Diff line change
Expand Up @@ -319,7 +319,7 @@ Managed vLLM uses these profiles:
| Host profile | Default model |
|---|---|
| DGX Spark | `nvidia/Qwen3.6-35B-A3B-NVFP4` |
| DGX Station | `deepseek-ai/DeepSeek-V4-Flash` |
| DGX Station | `Qwen/Qwen3.6-27B-FP8` |
| Linux with an NVIDIA GPU | `nvidia/NVIDIA-Nemotron-3-Nano-4B-FP8` |

<Note>
Expand Down Expand Up @@ -356,10 +356,10 @@ Recognized slugs are:

| Slug | Hugging Face model | Notes |
|---|---|---|
| `deepseek-v4-flash` | `deepseek-ai/DeepSeek-V4-Flash` | Default on the DGX Station profile |
| `qwen3.6-27b` | `Qwen/Qwen3.6-27B-FP8` | Supported override |
| `qwen3.6-27b` | `Qwen/Qwen3.6-27B-FP8` | Default on the DGX Station profile |
| `qwen3.6-35b-a3b-nvfp4` | `nvidia/Qwen3.6-35B-A3B-NVFP4` | Default on the DGX Spark profile |
| `nemotron-3-nano-4b` | `nvidia/NVIDIA-Nemotron-3-Nano-4B-FP8` | Default on the generic Linux + NVIDIA GPU profile |
| `deepseek-v4-flash` | `deepseek-ai/DeepSeek-V4-Flash` | Supported override |
| `deepseek-r1-distill-70b` | `deepseek-ai/DeepSeek-R1-Distill-Llama-70B` | Gated. Requires Hugging Face license acceptance |

The slug is case-insensitive; the full Hugging Face id is also accepted.
Expand Down
2 changes: 1 addition & 1 deletion docs/reference/commands-nemohermes.mdx
Original file line number Diff line number Diff line change
Expand Up @@ -1574,7 +1574,7 @@ Set them before running `nemohermes onboard`.
| `SANDBOX_NAME` | sandbox name | Compatibility spelling used after `NEMOCLAW_SANDBOX_NAME` and `NEMOCLAW_SANDBOX`. |
| `NEMOCLAW_INSTALL_REF` | git ref | For internal installer commands: the git ref to install from. Overridden by the `--install-ref` flag. |
| `NEMOCLAW_INSTALL_TAG` | release tag | For internal installer commands: the release tag to install. Defaults to the admin-promoted `lkg` tag when unset. Overridden by the `--install-tag` flag. |
| `NEMOCLAW_VLLM_MODEL` | registry slug or Hugging Face model id | Selects the model the managed-vLLM install path serves. Recognised slugs: `deepseek-v4-flash`, `qwen3.6-27b`, `qwen3.6-35b-a3b-nvfp4`, `nemotron-3-nano-4b`, `deepseek-r1-distill-70b`. Unset uses the per-platform profile default. Gated models (e.g. `deepseek-r1-distill-70b`) require `HF_TOKEN` or `HUGGING_FACE_HUB_TOKEN`. |
| `NEMOCLAW_VLLM_MODEL` | registry slug or Hugging Face model id | Selects the model the managed-vLLM install path serves. Recognised slugs: `qwen3.6-27b`, `qwen3.6-35b-a3b-nvfp4`, `nemotron-3-nano-4b`, `deepseek-v4-flash`, `deepseek-r1-distill-70b`. Unset uses the per-platform profile default. Gated models (e.g. `deepseek-r1-distill-70b`) require `HF_TOKEN` or `HUGGING_FACE_HUB_TOKEN`. |
| `NEMOCLAW_MINIMAL_BOOTSTRAP` | `1` to enable | Skips default OpenClaw workspace-template seeding for new pristine workspaces. Existing files are not deleted; see [Runtime Controls](../manage-sandboxes/runtime-controls). |
| `NEMOCLAW_MODEL_ROUTER_PYTHON` | absolute path | Pins the host Python interpreter used to create the Model Router virtual environment. Strict. NemoClaw probes only that interpreter and aborts with the failure reason if it does not qualify, rather than silently falling back to another python. Relative command names such as `python3.12` are rejected. When unset, NemoClaw probes `python3.13`, `python3.12`, `python3.11`, `python3.10`, and bare `python3`, retains every interpreter whose version is in `[3.10, 3.14)` and whose `ensurepip`, `pyexpat`, `ssl`, and `venv` stdlib modules import cleanly, and tries `python -m venv` on each in priority order until one succeeds. Set the pin when the auto-discovered interpreter is broken (for example, Homebrew `python@3.14` with a `pyexpat` dlopen mismatch on macOS). |

Expand Down
2 changes: 1 addition & 1 deletion docs/reference/commands.mdx
Original file line number Diff line number Diff line change
Expand Up @@ -1815,7 +1815,7 @@ Set them before running `$$nemoclaw onboard`.
| `SANDBOX_NAME` | sandbox name | Compatibility spelling used after `NEMOCLAW_SANDBOX_NAME` and `NEMOCLAW_SANDBOX`. |
| `NEMOCLAW_INSTALL_REF` | git ref | For internal installer commands: the git ref to install from. Overridden by the `--install-ref` flag. |
| `NEMOCLAW_INSTALL_TAG` | release tag | For internal installer commands: the release tag to install. Defaults to the admin-promoted `lkg` tag when unset. Overridden by the `--install-tag` flag. |
| `NEMOCLAW_VLLM_MODEL` | registry slug or Hugging Face model id | Selects the model the managed-vLLM install path serves. Recognised slugs: `deepseek-v4-flash`, `qwen3.6-27b`, `qwen3.6-35b-a3b-nvfp4`, `nemotron-3-nano-4b`, `deepseek-r1-distill-70b`. Unset uses the per-platform profile default. Gated models (e.g. `deepseek-r1-distill-70b`) require `HF_TOKEN` or `HUGGING_FACE_HUB_TOKEN`. |
| `NEMOCLAW_VLLM_MODEL` | registry slug or Hugging Face model id | Selects the model the managed-vLLM install path serves. Recognised slugs: `qwen3.6-27b`, `qwen3.6-35b-a3b-nvfp4`, `nemotron-3-nano-4b`, `deepseek-v4-flash`, `deepseek-r1-distill-70b`. Unset uses the per-platform profile default. Gated models (e.g. `deepseek-r1-distill-70b`) require `HF_TOKEN` or `HUGGING_FACE_HUB_TOKEN`. |
| `NEMOCLAW_MINIMAL_BOOTSTRAP` | `1` to enable | Skips default OpenClaw workspace-template seeding for new pristine workspaces. Existing files are not deleted; see [Runtime Controls](../manage-sandboxes/runtime-controls). |
| `NEMOCLAW_MODEL_ROUTER_PYTHON` | absolute path | Pins the host Python interpreter used to create the Model Router virtual environment. Strict. NemoClaw probes only that interpreter and aborts with the failure reason if it does not qualify, rather than silently falling back to another python. Relative command names such as `python3.12` are rejected. When unset, NemoClaw probes `python3.13`, `python3.12`, `python3.11`, `python3.10`, and bare `python3`, retains every interpreter whose version is in `[3.10, 3.14)` and whose `ensurepip`, `pyexpat`, `ssl`, and `venv` stdlib modules import cleanly, and tries `python -m venv` on each in priority order until one succeeds. Set the pin when the auto-discovered interpreter is broken (for example, Homebrew `python@3.14` with a `pyexpat` dlopen mismatch on macOS). |

Expand Down
4 changes: 2 additions & 2 deletions src/lib/inference/vllm-models.test.ts
Original file line number Diff line number Diff line change
Expand Up @@ -40,7 +40,7 @@ describe("vllm model registry", () => {
).toEqual(deepseek);
});

it("registers DeepSeek V4 Flash for DGX Station", () => {
it("registers DeepSeek V4 Flash as a managed-vLLM override", () => {
const deepseek = VLLM_MODELS.find((m) => m.envValue === "deepseek-v4-flash");
expect(deepseek).toBeDefined();
expect(deepseek!.id).toBe("deepseek-ai/DeepSeek-V4-Flash");
Expand Down Expand Up @@ -111,7 +111,7 @@ describe("vllm model registry", () => {
expect(cmd).not.toContain("--reasoning-parser qwen3");
});

it("builds the DeepSeek V4 Flash DGX Station serve command with inherited one-GPU defaults", () => {
it("builds the DeepSeek V4 Flash serve command with inherited one-GPU defaults", () => {
const deepseek = VLLM_MODELS.find((m) => m.envValue === "deepseek-v4-flash");
const cmd = buildVllmServeCommand(deepseek!);
expect(cmd).toContain("vllm serve deepseek-ai/DeepSeek-V4-Flash");
Expand Down
4 changes: 2 additions & 2 deletions src/lib/inference/vllm-models.ts
Original file line number Diff line number Diff line change
Expand Up @@ -192,8 +192,8 @@ const HF_TOKEN_ENV_KEYS = ["HF_TOKEN", "HUGGING_FACE_HUB_TOKEN"] as const;
/**
* Look up the requested express-vLLM model from `NEMOCLAW_VLLM_MODEL`.
* Returns `null` when the env var is empty so the caller can fall back to
* the per-platform profile default (Station prefers DeepSeek V4 Flash, Spark
* the Qwen3.6-35B-A3B NVFP4 checkpoint, and the generic Linux profile prefers
* the per-platform profile default (Station prefers Qwen3.6-27B, Spark the
* Qwen3.6-35B-A3B NVFP4 checkpoint, and the generic Linux profile prefers
* Nemotron-Nano-4B for VRAM headroom).
*
* Match is case-insensitive against either the `envValue` slug or the full
Expand Down
6 changes: 3 additions & 3 deletions src/lib/inference/vllm.test.ts
Original file line number Diff line number Diff line change
Expand Up @@ -34,14 +34,14 @@ describe("vLLM profile detection", () => {
vi.clearAllMocks();
});

it("uses DeepSeek V4 Flash and the 26.05.post1 NGC image on DGX Station", () => {
it("uses Qwen3.6 27B and the 26.05.post1 NGC image on DGX Station", () => {
const profile = detectVllmProfile({ platform: "station", type: "nvidia" });

expect(profile).not.toBeNull();
expect(profile!.name).toBe("DGX Station");
expect(profile!.image).toBe("nvcr.io/nvidia/vllm:26.05.post1-py3");
expect(profile!.defaultModel.id).toBe("deepseek-ai/DeepSeek-V4-Flash");
expect(profile!.defaultModel.envValue).toBe("deepseek-v4-flash");
expect(profile!.defaultModel.id).toBe("Qwen/Qwen3.6-27B-FP8");
expect(profile!.defaultModel.envValue).toBe("qwen3.6-27b");
});

it("keeps DGX Spark on the Qwen3.6 35B NVFP4 default", () => {
Expand Down
18 changes: 9 additions & 9 deletions src/lib/inference/vllm.ts
Original file line number Diff line number Diff line change
Expand Up @@ -59,15 +59,15 @@ function nemotronNanoModel(): VllmModelDef {
return match;
}

function qwen35bNvfp4Model(): VllmModelDef {
const match = VLLM_MODELS.find((m) => m.envValue === "qwen3.6-35b-a3b-nvfp4");
if (!match) throw new Error("vllm-models registry is missing the qwen3.6-35b-a3b-nvfp4 entry");
function qwen27bFP8Model(): VllmModelDef {
const match = VLLM_MODELS.find((m) => m.envValue === "qwen3.6-27b");
if (!match) throw new Error("vllm-models registry is missing the qwen3.6-27b entry");
return match;
}

function deepseekV4FlashModel(): VllmModelDef {
const match = VLLM_MODELS.find((m) => m.envValue === "deepseek-v4-flash");
if (!match) throw new Error("vllm-models registry is missing the deepseek-v4-flash entry");
function qwen35bNvfp4Model(): VllmModelDef {
const match = VLLM_MODELS.find((m) => m.envValue === "qwen3.6-35b-a3b-nvfp4");
if (!match) throw new Error("vllm-models registry is missing the qwen3.6-35b-a3b-nvfp4 entry");
return match;
}

Expand Down Expand Up @@ -139,7 +139,7 @@ const SPARK_PROFILE: VllmProfile = {
const STATION_PROFILE: VllmProfile = {
name: "DGX Station",
image: VLLM_IMAGES.ngc2605Post1,
defaultModel: deepseekV4FlashModel(),
defaultModel: qwen27bFP8Model(),
containerName: "nemoclaw-vllm",
dockerRunFlags: SPARK_PROFILE.dockerRunFlags,
buildDockerRunFlags: () => {
Expand Down Expand Up @@ -472,8 +472,8 @@ export async function installVllm(
): Promise<{ ok: boolean }> {
// Resolve the model to serve: `NEMOCLAW_VLLM_MODEL` override if set, else
// the per-platform profile default. The generic-Linux profile defaults to
// Nemotron-Nano-4B for VRAM headroom; Station to DeepSeek V4 Flash; Spark
// to the Qwen3.6-35B-A3B NVFP4 checkpoint.
// Nemotron-Nano-4B for VRAM headroom; Station to Qwen3.6-27B; Spark to the
// Qwen3.6-35B-A3B NVFP4 checkpoint.
// Validate gated-model access (HF_TOKEN required for models like
// DeepSeek-R1 Distill 70B) before touching docker so the user does not
// burn a multi-minute pull on a 401.
Expand Down
4 changes: 2 additions & 2 deletions test/detect-vllm-profile.test.ts
Original file line number Diff line number Diff line change
Expand Up @@ -29,8 +29,8 @@ describe("detectVllmProfile", () => {
expect(profile).not.toBeNull();
expect(profile!.name).toBe("DGX Station");
expect(profile!.image).toBe("nvcr.io/nvidia/vllm:26.05.post1-py3");
expect(profile!.defaultModel.id).toBe("deepseek-ai/DeepSeek-V4-Flash");
expect(profile!.defaultModel.envValue).toBe("deepseek-v4-flash");
expect(profile!.defaultModel.id).toBe("Qwen/Qwen3.6-27B-FP8");
expect(profile!.defaultModel.envValue).toBe("qwen3.6-27b");
});

it("returns the generic Linux profile for non-Spark/Station NVIDIA hosts", () => {
Expand Down
Loading