Skip to content

fix: support DGX Spark vLLM routing - #3051

Closed
dmbyte wants to merge 3 commits into
NVIDIA:mainfrom
dmbyte:codex/vllm-route-and-spark-recipe
Closed

fix: support DGX Spark vLLM routing#3051
dmbyte wants to merge 3 commits into
NVIDIA:mainfrom
dmbyte:codex/vllm-route-and-spark-recipe

Conversation

@dmbyte

@dmbyte dmbyte commented May 5, 2026

Copy link
Copy Markdown

Summary

  • Keep local host aliases out of the sandbox proxy path so local vLLM/Ollama endpoints on host.openshell.internal remain reachable.
  • Use the local vLLM default model when building the OpenClaw primary model for vllm-local.
  • Document a DGX Spark vLLM setup using Gemma 4 NVFP4.

Why

This reproduces the working DGX Spark setup where NemoClaw routes OpenClaw to a host vLLM server while preserving the OpenShell-managed inference.local route for normal provider traffic.

AI assistance

This PR was prepared with Codex assistance. The human operator remains the contributor of record, reviewed the scope, and verified the DGX Spark setup before submission.

Validation

  • bash -n scripts/nemoclaw-start.sh
  • git diff --check

Focused Vitest was not run in the fresh clone because dependencies were not installed.

Summary by CodeRabbit

  • Documentation

    • Added DGX Spark-specific guidance for running local vLLM inference servers, including Docker configuration and network endpoint details.
  • New Features

    • Introduced support for custom no-proxy host configuration in local inference environments via environment variable.
  • Improvements

    • Enhanced default model selection for local vLLM provider with configurable defaults.

@copy-pr-bot

copy-pr-bot Bot commented May 5, 2026

Copy link
Copy Markdown

This pull request requires additional validation before any workflows can run on NVIDIA's runners.

Pull request vetters can view their responsibilities here.

Contributors can view more details about this message here.

@coderabbitai

coderabbitai Bot commented May 5, 2026

Copy link
Copy Markdown
Contributor

Caution

Review failed

Failed to post review comments

📝 Walkthrough

Walkthrough

This PR extends local vLLM inference support by introducing a default model constant, adding proxy configuration for local host aliases, updating test coverage for proxy behavior, and documenting a DGX Spark Host deployment example with environment setup and networking guidance.

Changes

Local vLLM Inference Configuration & Proxy Support

Layer / File(s) Summary
Core Configuration
src/lib/inference-config.ts
Adds exported constant DEFAULT_VLLM_MODEL ("vllm-local") and refactors getOpenClawPrimaryModel to use provider-based default model selection for vllm-local, ollama-local, and cloud providers.
Environment Setup
scripts/nemoclaw-start.sh
Introduces NEMOCLAW_NO_PROXY_EXTRA environment variable handling to allow addition of extra hosts to the NO_PROXY bypass list, with default inclusion of local runtime host aliases.
Test Updates
src/lib/inference-config.test.ts, test/service-env.test.ts
Updates inference config tests to validate DEFAULT_VLLM_MODEL defaulting behavior; replaces existing NO_PROXY test with two new tests that verify loopback plus local runtime hosts (host.openshell.internal, host.docker.internal) in NO_PROXY, and validate NEMOCLAW_NO_PROXY_EXTRA override capability.
Documentation
docs/inference/use-local-inference.md
Adds DGX Spark Host Example subsection with Docker run command, non-interactive onboarding example, and notes on sandbox endpoint reachability via host.openshell.internal and proxy configuration including NEMOCLAW_NO_PROXY_EXTRA.

Estimated code review effort

🎯 2 (Simple) | ⏱️ ~12 minutes


🐰 A vLLM hops on the local stage,
With proxy rules to light the way,
host.openshell.internal's cheer,
And DGX defaults drawing near—
Configuration magic, neat and sage!

🚥 Pre-merge checks | ✅ 5
✅ Passed checks (5 passed)
Check name Status Explanation
Description Check ✅ Passed Check skipped - CodeRabbit’s high-level summary is enabled.
Title check ✅ Passed The title 'fix: support DGX Spark vLLM routing' accurately describes the main objective: enabling vLLM routing support for DGX Spark environments by updating proxy configuration, documentation, and default model handling.
Docstring Coverage ✅ Passed No functions found in the changed files to evaluate docstring coverage. Skipping docstring coverage check.
Linked Issues check ✅ Passed Check skipped because no linked issues were found for this pull request.
Out of Scope Changes check ✅ Passed Check skipped because no linked issues were found for this pull request.

✏️ Tip: You can configure your own custom pre-merge checks in the settings.

✨ Finishing Touches
🧪 Generate unit tests (beta)
  • Create PR with unit tests

Comment @coderabbitai help to get the list of available commands and usage tips.

@dmbyte dmbyte changed the title [codex] support DGX Spark vLLM routing fix: support DGX Spark vLLM routing May 5, 2026
@dmbyte
dmbyte marked this pull request as ready for review May 6, 2026 15:17
@wscurran wscurran added documentation dependencies Pull requests that update a dependency file labels May 6, 2026
@wscurran

wscurran commented May 6, 2026

Copy link
Copy Markdown
Contributor

✨ Thanks for submitting this PR that fixes support for DGX Spark vLLM routing. This change involves updating the sandbox proxy path and documenting a DGX Spark vLLM setup.

@dmbyte

dmbyte commented May 7, 2026

Copy link
Copy Markdown
Author

Closing this as superseded.

The runtime/model fix from this PR has been narrowed and kept alive in #3013. The broader DGX/vLLM onboarding/documentation pieces are now covered by upstream latest/v0.0.36, and the remaining manual DGX Spark Gemma/NVFP4 Docker example is host-specific enough that it belongs in the external recipe rather than the core docs.

Net: #3013 is the smaller still-needed patch; this PR is no longer needed.

@dmbyte dmbyte closed this May 7, 2026
@wscurran wscurran added chore Build, CI, dependency, or tooling maintenance platform: dgx-spark Affects DGX Spark hardware or workflows area: docs Documentation, examples, guides, or docs build and removed Platform: DGX Spark labels Jun 3, 2026
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

area: docs Documentation, examples, guides, or docs build chore Build, CI, dependency, or tooling maintenance dependencies Pull requests that update a dependency file platform: dgx-spark Affects DGX Spark hardware or workflows

Projects

None yet

Development

Successfully merging this pull request may close these issues.

2 participants