Skip to content

fix: remove descoped vLLM from setup-spark and redirect to onboard #223

New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

Sign up for GitHub

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Closed
brianwtaylor wants to merge 1 commit into from
Closed

fix: remove descoped vLLM from setup-spark and redirect to onboard #223

Changes from all commits
Commits
Show all changes
1 commit
Select commit
File filter

Filter by extension

Filter by extension
Conversations
Failed to load comments.
Loading
Jump to
Jump to file
Failed to load files.
Loading
Diff view
Diff view
67 changes: 12 additions & 55 deletions scripts/setup-spark.sh
View file
Open in desktop
Original file line number Diff line number Diff line change
Expand Up @@ -7,18 +7,17 @@
# Spark ships Ubuntu 24.04 (cgroup v2) + Docker 28.x but no k3s.
# OpenShell's gateway starts k3s inside a Docker container, which
# needs cgroup host namespace access. This script configures Docker
# for that, then hands off to the normal setup.sh.
# for that.
#
# Usage:
# sudo nemoclaw setup-spark
# # or directly:
# sudo bash scripts/setup-spark.sh
#
# What it does (beyond setup.sh):
# What it does:
# 1. Adds current user to docker group (avoids sudo for everything else)
# 2. Configures Docker daemon for cgroupns=host (k3s-in-Docker on cgroup v2)
# 3. Restarts Docker
# 4. Runs the normal setup.sh

set -euo pipefail

Expand Down Expand Up @@ -133,57 +132,15 @@ if [ "$NEEDS_RESTART" = true ]; then
info "Docker restarted with cgroupns=host"
fi

# ── 4. Install and start vLLM (local inference on Spark GPU) ──────
# ── 4. Done ──────────────────────────────────────────────────────

if ! python3 -c "import vllm" 2>/dev/null; then
info "Installing vLLM..."
pip3 install --break-system-packages vllm 2>&1 | tail -1
info "vLLM installed"
else
info "vLLM already installed"
fi

# Start vLLM if not already running
VLLM_MODEL="nvidia/nemotron-3-nano-30b-a3b"
if curl -s http://localhost:8000/v1/models > /dev/null 2>&1; then
info "vLLM already running on :8000"
else
if python3 -c "import vllm" 2>/dev/null && command -v nvidia-smi > /dev/null 2>&1; then
info "Starting vLLM with $VLLM_MODEL..."
nohup python3 -m vllm.entrypoints.openai.api_server \
--model "$VLLM_MODEL" \
--port 8000 \
--host 0.0.0.0 \
> /tmp/vllm-server.log 2>&1 &
VLLM_PID=$!
# Wait for vLLM to be ready (model loading can take a while)
info "Waiting for vLLM to load model (this can take a few minutes)..."
for i in $(seq 1 120); do
if curl -s http://localhost:8000/v1/models > /dev/null 2>&1; then
info "vLLM ready (PID $VLLM_PID)"
break
fi
if ! kill -0 "$VLLM_PID" 2>/dev/null; then
warn "vLLM exited. Check /tmp/vllm-server.log"
break
fi
sleep 2
done
fi
fi

# ── 5. Run normal setup ──────────────────────────────────────────

info "Running NemoClaw setup..."
echo ""

# Drop back to the real user for setup.sh (uses docker group, not root)
if [ -n "$REAL_USER" ]; then
# Pass through env vars that setup.sh needs
sudo -u "$REAL_USER" -E \
NVIDIA_API_KEY="${NVIDIA_API_KEY:-}" \
DOCKER_HOST="${DOCKER_HOST:-}" \
bash "$SCRIPT_DIR/setup.sh"
else
bash "$SCRIPT_DIR/setup.sh"
fi
info "DGX Spark configuration complete!"
echo ""
info "What was configured:"
info " - Docker group membership for '${REAL_USER:-root}'"
info " - Docker daemon cgroupns=host (required for k3s-in-Docker)"
echo ""
info "Next step:"
info " nemoclaw onboard"
echo ""