Skip to content
Merged
Show file tree
Hide file tree
Changes from all commits
Commits
File filter

Filter by extension

Filter by extension

Conversations
Failed to load comments.
Loading
Jump to
Jump to file
Failed to load files.
Loading
Diff view
Diff view
6 changes: 3 additions & 3 deletions ci/source-architecture-budget.json
Original file line number Diff line number Diff line change
Expand Up @@ -27,7 +27,7 @@
"src/lib/runner.ts": 88,
"src/lib/security/redact.ts": 51,
"src/lib/state/onboard-session.ts": 36,
"src/lib/state/registry.ts": 99,
"src/lib/state/registry.ts": 98,
"src/lib/state/state-root.ts": 22,
"src/lib/subprocess-env.ts": 24,
"src/lib/validation.ts": 25
Expand All @@ -47,15 +47,15 @@
"src/lib/actions/uninstall/run-plan.ts": 26,
"src/lib/inference/onboard-probes.ts": 21,
"src/lib/inference/vllm.ts": 21,
"src/lib/onboard.ts": 222,
"src/lib/onboard.ts": 219,
"src/lib/onboard/machine/handlers/sandbox.ts": 21,
"src/lib/sandbox/config.ts": 22,
"src/lib/shields/index.ts": 23
}
},
"allowedCycles": [],
"maxRootFiles": {
"src/lib/onboard": 308,
"src/lib/onboard": 307,
"src/lib/actions": 19,
"src/lib/actions/sandbox": 184,
"src/lib/state": 37,
Expand Down
5 changes: 5 additions & 0 deletions docs/changelog/2026-08-04.mdx
Original file line number Diff line number Diff line change
Expand Up @@ -7,7 +7,12 @@

NemoClaw v0.0.102 adds authenticated attachment of operator-managed llama.cpp servers and an Experimental managed vLLM profile for two DGX Spark systems.
It also improves DGX Station and Windows installation, gateway and sandbox recovery, Shields transactions, inference reliability, multi-port uninstall, and Hermes and LangChain Deep Agents Code workflows.
It reduces the managed single-node DGX Spark Qwen profile's default memory demand during long-context agent workflows.

- The `nvidia/Qwen3.6-35B-A3B-NVFP4` profile no longer enables multi-token prediction (MTP) speculative decoding by default.
It retains async scheduling and all other registered serve arguments.
Operators can apply lower context, concurrency, and batch limits for bounded long-context investigation, but the override does not guarantee protection against a host freeze.
For more information, refer to [Set Up vLLM](/user-guide/openclaw/inference/local-inference/set-up-vllm).
- Onboarding can now attach an authenticated, operator-managed llama.cpp server on loopback port `8081` as the `llama-cpp-local` provider.
NemoClaw requires consistent native fingerprint evidence, bounds probe responses, and retains the generic compatible-endpoint path when the server cannot be identified as llama.cpp.
For more information, refer to the [NemoClaw CLI Commands Reference](/user-guide/openclaw/reference/commands).
Expand Down
18 changes: 17 additions & 1 deletion docs/inference/set-up-vllm.mdx
Original file line number Diff line number Diff line change
Expand Up @@ -195,6 +195,9 @@ When you start managed vLLM outside the installer express flow, NemoClaw uses th
| DGX Station | `deepseek-ai/DeepSeek-V4-Flash` |
| Linux with an NVIDIA GPU | `nvidia/NVIDIA-Nemotron-3-Nano-4B-FP8` |

For the managed single-node DGX Spark `nvidia/Qwen3.6-35B-A3B-NVFP4` profile, NemoClaw enables async scheduling and does not enable multi-token prediction (MTP) speculative decoding.
The profile retains `--gpu-memory-utilization 0.4`, a `262144`-token context window, `4` concurrent sequences, an `8192`-token batch limit, chunked prefill, prefix caching, and its registered parsers and acceleration backends.

On DGX Station, accepting the installer express prompt selects `NEMOCLAW_VLLM_MODEL=nemotron-3-ultra-550b-a55b`.
Express uses the single-Station Ultra recipe unless a trusted reciprocal pair qualifies for distributed serving.
Refer to [Set Up vLLM on Two DGX Stations](set-up-vllm-on-two-dgx-stations) for pair qualification, lifecycle receipts, runtime topology, and network controls.
Expand Down Expand Up @@ -321,7 +324,7 @@ The host checks the token before `docker pull`, so a missing or empty token stop

## Add Managed Serve Arguments

Set `NEMOCLAW_VLLM_EXTRA_ARGS_JSON` to a JSON array of individual non-blank `vllm serve` tokens for operator-owned options that are not in the registry.
Set `NEMOCLAW_VLLM_EXTRA_ARGS_JSON` to a JSON array of individual non-blank `vllm serve` tokens for operator-owned options and overrides.
NemoClaw trims and validates the array before downloads, shell-quotes each token, and appends the tokens after the registry defaults.

```bash
Expand All @@ -332,6 +335,19 @@ NEMOCLAW_PROVIDER=install-vllm \

If the selected image does not support an argument, the managed container exits and NemoClaw prints the log tail.

To bound resource use while investigating long-context workflows on a single DGX Spark, select the Qwen profile.
The following override disables async scheduling and lowers the context window, concurrent-sequence limit, and batch limit.

```bash
NEMOCLAW_PROVIDER=install-vllm \
NEMOCLAW_VLLM_MODEL=qwen3.6-35b-a3b-nvfp4 \
NEMOCLAW_VLLM_EXTRA_ARGS_JSON='["--no-async-scheduling","--max-model-len","32768","--max-num-seqs","1","--max-num-batched-tokens","4096"]' \
$$nemoclaw onboard --non-interactive
```

This override reduces request concurrency and context capacity, so it can reduce throughput.
It does not guarantee protection against a DGX Spark host freeze.

## Related Topics

- [Choose a Local Inference Server](choose-local-inference-server) to compare vLLM with Ollama and NVIDIA NIM.
Expand Down
13 changes: 9 additions & 4 deletions src/lib/inference/vllm-models.test.ts
Original file line number Diff line number Diff line change
Expand Up @@ -360,7 +360,7 @@ describe("vllm model registry", () => {
expect(qwen35b!.gated).toBe(false);
});

it("builds the NVFP4 serve command from the DGX Spark model-card recipe (#6457)", () => {
it("builds the MTP-free NVFP4 serve command for DGX Spark (#7127)", () => {
const qwen35b = VLLM_MODELS.find((m) => m.envValue === "qwen3.6-35b-a3b-nvfp4");
const cmd = buildVllmServeCommand(qwen35b!);
// The current NVIDIA model card no longer needs Spark-specific env exports.
Expand Down Expand Up @@ -390,9 +390,14 @@ describe("vllm model registry", () => {
expect(cmd.match(/--tool-call-parser/g)).toHaveLength(1);
expect(cmd).toContain("--reasoning-parser qwen3");
expect(cmd).toContain("--max-model-len 262144");
expect(cmd).toContain(
`--speculative-config '{"method":"mtp","num_speculative_tokens":3,"moe_backend":"triton"}'`,
);
expect(cmd).toContain("--dtype auto");
expect(cmd).toContain("--max-num-seqs 4");
expect(cmd).toContain("--max-num-batched-tokens 8192");
expect(cmd).toContain("--enable-chunked-prefill");
expect(cmd).toContain("--async-scheduling");
expect(cmd).toContain("--enable-prefix-caching");
expect(cmd).not.toContain("--speculative-config");
expect(cmd).not.toContain('"method":"mtp"');
// Single-node parallel flags stay shared; 0.4 utilization follows the
// current DGX Spark model-card recipe.
expect(cmd).toContain("--gpu-memory-utilization 0.4");
Expand Down
5 changes: 3 additions & 2 deletions src/lib/inference/vllm-models.ts
Original file line number Diff line number Diff line change
Expand Up @@ -331,8 +331,9 @@ export const VLLM_MODELS: readonly VllmModelDef[] = [
"qwen3_coder",
"--reasoning-parser",
"qwen3",
"--speculative-config",
`'{"method":"mtp","num_speculative_tokens":3,"moe_backend":"triton"}'`,
// Keep MTP speculative decoding opt-in on DGX Spark. It increases the
// managed profile's cold-start memory pressure and long-context risk
// without being required for correct model or tool-call behavior (#7127).
"--load-format",
"fastsafetensors",
],
Expand Down
Loading