From 20394659032689d17c7a7da5c95b2717d018f122 Mon Sep 17 00:00:00 2001 From: hendrikh Date: Tue, 17 Mar 2026 23:45:36 +0100 Subject: [PATCH 1/5] feat: add Jetson device support (Orin Nano, Orin NX, AGX Orin) MIME-Version: 1.0 Content-Type: text/plain; charset=UTF-8 Content-Transfer-Encoding: 8bit NVIDIA Jetson devices running L4T (Linux for Tegra) cannot run the standard NemoClaw setup because OpenShell's k3s-in-Docker gateway fails on L4T's kernel configuration. This commit adds scripts/setup-jetson.sh, which resolves four distinct L4T incompatibilities before delegating to the normal setup.sh for the full OpenShell/k3s path with security intact. Inference is routed through NVIDIA cloud (nvidia-nim) by default, same as every other platform. Ollama is installed as an available local fallback but no model is pulled automatically. To switch to local inference after setup: ollama pull nemotron-3-nano:4b openshell inference set --provider vllm-local --model nemotron-3-nano:4b ## Problems and fixes ### 1. Missing kernel modules (br_netfilter, xt_comment, ipset) L4T ships the required netfilter and bridge modules as .ko files but does not load them at boot. Without them: - br_netfilter: pod-to-pod and ClusterIP routing fails silently because bridge traffic never enters iptables. - xt_comment: kube-router panics on startup when it cannot insert iptables rules with the --comment match. - xt_conntrack, xt_mark, xt_nat, xt_MASQUERADE: kube-proxy cannot set up service routing rules. - ip_set_hash_net: kube-router's network policy controller cannot create ipset sets for policy enforcement. Fix: modprobe all required modules and persist them via /etc/modules-load.d/k3s-netfilter.conf so they survive reboots. ### 2. iptables backend mismatch (nf_tables vs legacy) The OpenShell gateway image (Ubuntu 24.04) defaults to iptables-nft (nf_tables backend), but L4T's kernel uses iptables-legacy. The nf_tables compatibility layer in the Tegra kernel is incomplete — it lacks translation support for several match extensions including xt_addrtype, which Docker itself requires for bridge networking. Switching the host to nf_tables breaks Docker; keeping the container on nf_tables breaks k3s. Both backends exist in the gateway image. Fix: build a thin wrapper layer over the upstream gateway image that runs update-alternatives to switch to iptables-legacy. The wrapper is tagged with the same image name so openshell gateway start uses it transparently. The upstream image layers remain intact and can be restored with docker pull at any time. ### 3. Incomplete ipset kernel support L4T's kernel only ships the ip_set_hash_net module. kube-router's network policy controller requires additional ipset types (hash:ip, hash:ipport, hash:ipportnet, hash:ipportip, bitmap:port) that are not built into the Tegra kernel. This causes the controller to fail on every sync cycle, and in some configurations it drops all pod traffic by default while in a failed state. Fix: the gateway image wrapper injects --disable-network-policy into the k3s server arguments via an entrypoint shim. This disables kube-router's network policy controller entirely. **Impact on security:** this removes the Kubernetes NetworkPolicy enforcement layer, which acts as a defense-in-depth IP/port-level firewall for sandbox pods. However, OpenShell's application-level policy enforcement remains fully active. This higher-level policy provides: - Deny-by-default outbound network access with an explicit allowlist of permitted hosts (NVIDIA inference APIs, GitHub, npm, etc.) - Per-binary restrictions (e.g. only git can reach github.com) - HTTP method and path filtering (e.g. docs endpoints allow GET only) - Operator approval workflow for requests to unlisted hosts via the OpenShell TUI - Filesystem read-only/read-write policy enforcement - Non-root sandbox user execution The Kubernetes NetworkPolicy layer would catch traffic that bypasses the OpenShell proxy (e.g. raw sockets). On Jetson this gap could be closed in the future by either rebuilding the L4T kernel with full ipset support, or by replacing kube-router with Calico which uses iptables directly without ipset dependencies. ### 4. CoreDNS forwarding to Docker's internal DNS Same issue as Colima environments: k3s CoreDNS forwards to /etc/resolv.conf inside the gateway container, which contains 127.0.0.11 (Docker's embedded DNS resolver). This address is not reachable from k3s pods because they run in separate network namespaces. The gateway entrypoint already sets up a DNS proxy on the container's routable eth0 IP, but CoreDNS is not configured to use it. setup.sh only patches CoreDNS for Colima; Jetson has the same issue. Fix: after setup.sh completes, patch the CoreDNS ConfigMap to forward to the container's DNS proxy IP and restart the CoreDNS deployment. Bounce the sandbox pod if needed so it does not remain in CrashLoopBackOff waiting for the exponential backoff timer. ### 5. cgroup v2 namespace (same as DGX Spark) L4T R36+ runs Ubuntu with cgroup v2. k3s-in-Docker requires --cgroupns=host to manage cgroup hierarchies. Same fix as setup-spark.sh: set default-cgroupns-mode=host in Docker's daemon.json. ## Design The script follows the same pattern as setup-spark.sh: fix platform-specific blockers, then delegate to setup.sh for the standard OpenShell/k3s path. No existing files are modified. - Autodetects Jetson via /etc/nv_tegra_release or tegra kernel string - All fixes are gated behind Jetson detection (FORCE_JETSON=1 to override) - Gateway image patch is non-destructive (docker pull restores the original) - Kernel modules are persisted across reboots via modules-load.d Tested on Jetson Orin (R36.5, L4T 5.15.185-tegra, Docker 29.3, CUDA 12.6, OpenShell 0.0.7). --- scripts/setup-jetson.sh | 304 ++++++++++++++++++++++++++++++++++++++++ 1 file changed, 304 insertions(+) create mode 100755 scripts/setup-jetson.sh diff --git a/scripts/setup-jetson.sh b/scripts/setup-jetson.sh new file mode 100755 index 00000000000..f2ef203a0d4 --- /dev/null +++ b/scripts/setup-jetson.sh @@ -0,0 +1,304 @@ +#!/usr/bin/env bash +# SPDX-FileCopyrightText: Copyright (c) 2026 NVIDIA CORPORATION & AFFILIATES. All rights reserved. +# SPDX-License-Identifier: Apache-2.0 +# +# NemoClaw setup for Jetson devices (Orin Nano, Orin NX, AGX Orin, etc.) +# +# Jetson's L4T kernel ships the iptables/netfilter modules that k3s needs +# (xt_comment, xt_conntrack, etc.) but doesn't load them by default. +# Without them, OpenShell's k3s gateway panics on network policy setup. +# +# This script loads the missing modules, configures Docker for cgroup v2 +# (same as setup-spark.sh), sets up Ollama for local inference, then +# hands off to the normal setup.sh for the full OpenShell/k3s path. +# +# Usage: +# sudo bash scripts/setup-jetson.sh +# +# What it does (beyond setup.sh): +# 1. Autodetects Jetson (Tegra/L4T) +# 2. Loads required kernel modules for k3s (iptables + ipset) +# 3. Configures Docker daemon for cgroupns=host (if cgroup v2) +# 4. Patches OpenShell gateway image: iptables-nft → iptables-legacy +# 5. Ensures Ollama is running with a suitable model +# 6. Runs the normal setup.sh (full OpenShell/k3s path) +# 7. Fixes CoreDNS forwarding (same issue as Colima) + +set -euo pipefail + +RED='\033[0;31m' +GREEN='\033[0;32m' +YELLOW='\033[1;33m' +NC='\033[0m' + +info() { echo -e "${GREEN}>>>${NC} $1"; } +warn() { echo -e "${YELLOW}>>>${NC} $1"; } +fail() { echo -e "${RED}>>>${NC} $1"; exit 1; } + +SCRIPT_DIR="$(cd "$(dirname "$0")" && pwd)" + +# ── Pre-flight checks ──────────────────────────────────────────── + +if [ "$(uname -s)" != "Linux" ]; then + fail "This script is for Jetson (Linux). Use 'nemoclaw setup' for macOS." +fi + +if [ "$(id -u)" -ne 0 ]; then + fail "Must run as root: sudo bash scripts/setup-jetson.sh" +fi + +# Autodetect Jetson (L4T / Tegra) +if [ -f /etc/nv_tegra_release ]; then + TEGRA_INFO=$(head -1 /etc/nv_tegra_release) + info "Detected Jetson: $TEGRA_INFO" +elif uname -r 2>/dev/null | grep -qi tegra; then + info "Detected Jetson kernel: $(uname -r)" +else + warn "No Jetson/Tegra detected. This script is designed for Jetson devices." + warn "If you're sure this is a Jetson, set FORCE_JETSON=1 to continue." + [ "${FORCE_JETSON:-}" = "1" ] || fail "Not a Jetson device. Use 'scripts/setup.sh' for standard Linux." +fi + +command -v docker > /dev/null || fail "Docker not found." + +# Detect the real user (not root) for docker group / handoff +REAL_USER="${SUDO_USER:-$(logname 2>/dev/null || echo "")}" +if [ -z "$REAL_USER" ]; then + warn "Could not detect non-root user. Docker group will not be configured." +fi + +# ── 1. Docker group ────────────────────────────────────────────── + +if [ -n "$REAL_USER" ]; then + if id -nG "$REAL_USER" | grep -qw docker; then + info "User '$REAL_USER' already in docker group" + else + info "Adding '$REAL_USER' to docker group..." + usermod -aG docker "$REAL_USER" + info "Added. Group will take effect on next login (or use 'newgrp docker')." + fi +fi + +# ── 2. Kernel modules for k3s iptables ────────────────────────── +# +# L4T builds xt_comment, xt_conntrack, nf_conntrack etc. as modules +# but doesn't load them at boot. k3s's kube-router panics without +# them because it can't insert iptables rules. + +MODULES=( + # Bridge netfilter — required for pod-to-pod and ClusterIP routing + br_netfilter + # iptables matches for kube-router / kube-proxy + xt_comment xt_conntrack nf_conntrack xt_mark xt_nat xt_MASQUERADE + # ipset types for kube-router network policy (load what's available) + ip_set_hash_net ip_set_hash_ip ip_set_hash_ipport + ip_set_hash_ipportnet ip_set_hash_ipportip ip_set_bitmap_port +) +LOADED_ANY=false + +for mod in "${MODULES[@]}"; do + if ! lsmod | grep -qw "$mod"; then + if modprobe "$mod" 2>/dev/null; then + info "Loaded kernel module: $mod" + LOADED_ANY=true + else + warn "Could not load kernel module: $mod (may not be needed)" + fi + fi +done + +if [ "$LOADED_ANY" = true ]; then + # Persist across reboots + for mod in "${MODULES[@]}"; do + if ! grep -qx "$mod" /etc/modules-load.d/k3s-netfilter.conf 2>/dev/null; then + echo "$mod" >> /etc/modules-load.d/k3s-netfilter.conf + fi + done + info "Modules persisted to /etc/modules-load.d/k3s-netfilter.conf" +fi +info "Kernel modules OK" + +# ── 3. Docker cgroup namespace (same as setup-spark.sh) ────────── +# +# If cgroup v2, k3s-in-Docker needs cgroupns=host. + +if [ "$(stat -fc %T /sys/fs/cgroup/ 2>/dev/null)" = "cgroup2fs" ]; then + DAEMON_JSON="/etc/docker/daemon.json" + NEEDS_RESTART=false + + if [ -f "$DAEMON_JSON" ]; then + CURRENT_MODE=$(python3 -c "import json; print(json.load(open('$DAEMON_JSON')).get('default-cgroupns-mode',''))" 2>/dev/null || echo "") + if [ "$CURRENT_MODE" = "host" ]; then + info "Docker daemon already configured for cgroupns=host" + else + info "Setting Docker daemon cgroupns=host..." + python3 -c " +import json +with open('$DAEMON_JSON') as f: + d = json.load(f) +d['default-cgroupns-mode'] = 'host' +with open('$DAEMON_JSON', 'w') as f: + json.dump(d, f, indent=2) +" + NEEDS_RESTART=true + fi + else + info "Creating Docker daemon config with cgroupns=host..." + mkdir -p "$(dirname "$DAEMON_JSON")" + echo '{ "default-cgroupns-mode": "host" }' > "$DAEMON_JSON" + NEEDS_RESTART=true + fi + + if [ "$NEEDS_RESTART" = true ]; then + info "Restarting Docker daemon..." + systemctl restart docker + for i in 1 2 3 4 5 6 7 8 9 10; do + if docker info > /dev/null 2>&1; then + break + fi + [ "$i" -eq 10 ] && fail "Docker didn't come back after restart. Check 'systemctl status docker'." + sleep 2 + done + info "Docker restarted with cgroupns=host" + fi +else + info "cgroup v1 — no Docker changes needed" +fi + +# ── 4. Patch gateway image: iptables-nft → iptables-legacy ─────── +# +# The OpenShell gateway image ships iptables v1.8.10 defaulting to +# the nf_tables backend. L4T's kernel uses iptables-legacy on the +# host and the nf_tables compat layer is incomplete (xt_addrtype +# etc.). k3s's kube-router panics when nft RULE_INSERT fails. +# +# Fix: build a one-layer wrapper that switches the alternative to +# iptables-legacy, then tag it over the upstream image name so +# `openshell gateway start` uses it. The upstream layers are +# preserved — `docker pull` restores the original at any time. + +GATEWAY_IMAGE="ghcr.io/nvidia/openshell/cluster:0.0.8" + +# Pull upstream image if not present +if ! docker image inspect "$GATEWAY_IMAGE" > /dev/null 2>&1; then + info "Pulling OpenShell gateway image..." + docker pull "$GATEWAY_IMAGE" +fi + +# Check if already patched (iptables-legacy inside the image) +CURRENT_IPT=$(docker run --rm --entrypoint iptables "$GATEWAY_IMAGE" --version 2>&1 || true) +if echo "$CURRENT_IPT" | grep -q "legacy"; then + info "Gateway image already using iptables-legacy" +else + info "Patching gateway image to use iptables-legacy..." + # Save upstream image ID so we can verify we're wrapping the right thing + UPSTREAM_ID=$(docker image inspect --format='{{.Id}}' "$GATEWAY_IMAGE") + + PATCH_CTX="$(mktemp -d)" + cat > "$PATCH_CTX/Dockerfile" <<'DOCKERFILE' +ARG BASE_IMAGE +FROM ${BASE_IMAGE} + +# L4T's iptables uses the legacy backend; the nf_tables compat layer +# is incomplete, so switch k3s to iptables-legacy. +RUN update-alternatives --set iptables /usr/sbin/iptables-legacy \ + && update-alternatives --set ip6tables /usr/sbin/ip6tables-legacy + +# L4T kernel only ships ip_set_hash_net — kube-router's network policy +# controller needs additional ipset types (hash:ip, hash:ipport, etc.) +# that don't exist. Disable the controller so it doesn't block pod traffic. +# Wrap the original entrypoint to inject --disable-network-policy. +RUN mv /usr/local/bin/cluster-entrypoint.sh /usr/local/bin/cluster-entrypoint-orig.sh +COPY entrypoint-jetson.sh /usr/local/bin/cluster-entrypoint.sh +RUN chmod +x /usr/local/bin/cluster-entrypoint.sh +DOCKERFILE + + cat > "$PATCH_CTX/entrypoint-jetson.sh" <<'WRAPPER' +#!/bin/sh +# Jetson wrapper: inject --disable-network-policy then call original entrypoint. +# The original entrypoint ends with: exec /bin/k3s "$@" ... +# We append our flag to the args passed through. +exec /usr/local/bin/cluster-entrypoint-orig.sh "$@" --disable-network-policy +WRAPPER + + docker build -t "$GATEWAY_IMAGE" \ + --build-arg "BASE_IMAGE=$GATEWAY_IMAGE" \ + "$PATCH_CTX" 2>&1 | tail -3 + rm -rf "$PATCH_CTX" + + # Verify + PATCHED_IPT=$(docker run --rm --entrypoint iptables "$GATEWAY_IMAGE" --version 2>&1 || true) + if echo "$PATCHED_IPT" | grep -q "legacy"; then + info "Gateway patched: $PATCHED_IPT" + else + warn "Patch may have failed: $PATCHED_IPT" + fi +fi + +# ── 5. Ollama (optional local inference) ────────────────────────── +# +# Ollama is installed if not present but no model is pulled by default. +# To use local inference after setup, pull a model and switch: +# ollama pull nemotron-3-nano:4b +# openshell inference set --provider vllm-local --model nemotron-3-nano:4b + +if ! command -v ollama > /dev/null 2>&1; then + info "Installing Ollama..." + curl -fsSL https://ollama.com/install.sh | sh +fi + +if command -v ollama > /dev/null 2>&1; then + info "Ollama available (no model pulled — use 'ollama pull ' for local inference)" +fi + +# ── 6. Run normal setup.sh ─────────────────────────────────────── + +info "Running NemoClaw setup..." +echo "" + +if [ -n "$REAL_USER" ]; then + sudo -u "$REAL_USER" -E \ + NVIDIA_API_KEY="${NVIDIA_API_KEY:-}" \ + DOCKER_HOST="${DOCKER_HOST:-}" \ + bash "$SCRIPT_DIR/setup.sh" +else + bash "$SCRIPT_DIR/setup.sh" +fi + +# ── 7. Fix CoreDNS (Jetson-specific) ──────────────────────────── +# +# Same problem as Colima: k3s CoreDNS forwards to /etc/resolv.conf +# which contains 127.0.0.11 (Docker's internal DNS), unreachable +# from k3s pods. The entrypoint sets up a DNS proxy on the +# container's eth0 IP — point CoreDNS there instead. +# +# setup.sh only runs fix-coredns.sh for Colima. On Jetson the +# Docker engine also uses 127.0.0.11 in /etc/resolv.conf, so +# the same fix is needed. + +CLUSTER=$(docker ps --filter "name=openshell-cluster" --format '{{.Names}}' | head -1) +if [ -n "$CLUSTER" ]; then + DNS_IP=$(docker exec "$CLUSTER" cat /etc/rancher/k3s/resolv.conf 2>/dev/null \ + | grep nameserver | awk '{print $2}') + + if [ -n "$DNS_IP" ] && [[ "$DNS_IP" != 127.* ]]; then + # Check if CoreDNS is already forwarding to the right IP + CURRENT_FWD=$(docker exec "$CLUSTER" kubectl get configmap coredns -n kube-system \ + -o jsonpath='{.data.Corefile}' 2>/dev/null | grep -oP 'forward \. \K\S+' || true) + + if [ "$CURRENT_FWD" != "$DNS_IP" ]; then + info "Patching CoreDNS to forward to $DNS_IP..." + docker exec "$CLUSTER" kubectl patch configmap coredns -n kube-system --type merge \ + -p "{\"data\":{\"Corefile\":\".:53 {\\n errors\\n health\\n ready\\n kubernetes cluster.local in-addr.arpa ip6.arpa {\\n pods insecure\\n fallthrough in-addr.arpa ip6.arpa\\n }\\n hosts /etc/coredns/NodeHosts {\\n ttl 60\\n reload 15s\\n fallthrough\\n }\\n prometheus :9153\\n cache 30\\n loop\\n reload\\n loadbalance\\n forward . $DNS_IP\\n}\\n\"}}" > /dev/null + docker exec "$CLUSTER" kubectl rollout restart deploy/coredns -n kube-system > /dev/null + docker exec "$CLUSTER" kubectl rollout status deploy/coredns -n kube-system --timeout=30s > /dev/null 2>&1 + info "CoreDNS patched" + + # Bounce the sandbox pod so it picks up working DNS immediately + # instead of waiting for CrashLoopBackOff to expire + docker exec "$CLUSTER" kubectl delete pod -n openshell -l app=nemoclaw --ignore-not-found > /dev/null 2>&1 || true + else + info "CoreDNS already forwarding to $DNS_IP" + fi + fi +fi From 7264e7ab6ece0c2dd7bc7f00baae8697c505794e Mon Sep 17 00:00:00 2001 From: hendrikh Date: Tue, 17 Mar 2026 23:45:45 +0100 Subject: [PATCH 2/5] feat: add ollama inference profile to blueprint MIME-Version: 1.0 Content-Type: text/plain; charset=UTF-8 Content-Transfer-Encoding: 8bit Add an ollama profile for local inference via Ollama's OpenAI-compatible API. This enables Jetson and other edge devices to use locally-running models (e.g. nemotron-3-nano:4b) without requiring NVIDIA cloud credentials or a vLLM installation. The profile uses http://host.docker.internal:11434/v1 as the endpoint, which resolves to the host's Ollama instance from inside the sandbox container. No API key is required (Ollama accepts any value). This profile is not selected by default — inference routes through nvidia-nim (NVIDIA cloud) unless explicitly changed. --- nemoclaw-blueprint/blueprint.yaml | 9 +++++++++ 1 file changed, 9 insertions(+) diff --git a/nemoclaw-blueprint/blueprint.yaml b/nemoclaw-blueprint/blueprint.yaml index f55f9f651d2..e14c5e5e94c 100644 --- a/nemoclaw-blueprint/blueprint.yaml +++ b/nemoclaw-blueprint/blueprint.yaml @@ -11,6 +11,7 @@ profiles: - ncp - nim-local - vllm + - ollama description: | NemoClaw blueprint: orchestrates OpenClaw sandbox creation, migration, @@ -54,6 +55,14 @@ components: credential_env: "OPENAI_API_KEY" credential_default: "dummy" + ollama: + provider_type: "openai" + provider_name: "ollama-local" + endpoint: "http://host.docker.internal:11434/v1" + model: "nemotron-3-nano:4b" + credential_env: "OPENAI_API_KEY" + credential_default: "ollama" + policy: base: "sandboxes/openclaw/policy.yaml" additions: From 674f8b3a06f1d46a615ee353f59b91970ed61ff8 Mon Sep 17 00:00:00 2001 From: hendrikh Date: Wed, 18 Mar 2026 02:39:03 +0100 Subject: [PATCH 3/5] fix: rework Jetson setup for proper network policy and egress support MIME-Version: 1.0 Content-Type: text/plain; charset=UTF-8 Content-Transfer-Encoding: 8bit - Run setup-jetson.sh as normal user instead of sudo; only kernel module loading and Docker daemon config use sudo internally. This ensures gateway metadata and mTLS certs are stored in the user's home dir, which is required for `openshell term` to connect properly. - Inline gateway/sandbox creation instead of delegating to setup.sh. setup.sh destroys and restarts the gateway, which re-pulls the upstream image and clobbers the iptables-legacy patch. The new flow: start gateway (pulls image) → patch image → restart gateway → create sandbox. - Pass --policy to sandbox create so the baseline network policy (Telegram, NVIDIA API, GitHub, npm, etc.) is loaded at creation time. Without this, the sandbox blocks all egress with 403. - Add Jetson setup documentation (docs/deployment/jetson-setup.md) covering prerequisites, setup steps, network policy activation, local inference with Ollama, and troubleshooting. Tested on Jetson Orin Nano (R36.5, L4T 5.15.185-tegra). --- docs/deployment/jetson-setup.md | 164 ++++++++++++++++++ docs/index.md | 1 + scripts/setup-jetson.sh | 295 +++++++++++++++++++++----------- 3 files changed, 357 insertions(+), 103 deletions(-) create mode 100644 docs/deployment/jetson-setup.md diff --git a/docs/deployment/jetson-setup.md b/docs/deployment/jetson-setup.md new file mode 100644 index 00000000000..101303c610c --- /dev/null +++ b/docs/deployment/jetson-setup.md @@ -0,0 +1,164 @@ +--- +title: + page: "Set Up NemoClaw on Jetson" + nav: "Jetson Setup" +description: "Run NemoClaw on NVIDIA Jetson devices (Orin Nano, Orin NX, AGX Orin)." +keywords: ["nemoclaw jetson", "orin nano", "orin nx", "agx orin", "l4t"] +topics: ["generative_ai", "ai_agents"] +tags: ["openclaw", "openshell", "jetson", "l4t", "edge"] +content: + type: how_to + difficulty: technical_intermediate + audience: ["developer", "engineer"] +status: published +--- + + + +# Set Up NemoClaw on Jetson + +NemoClaw runs on NVIDIA Jetson devices (Orin Nano, Orin NX, AGX Orin) with +L4T (Linux for Tegra). A dedicated setup script handles L4T kernel +incompatibilities and configures the OpenShell gateway for Jetson's +iptables backend. + +## Prerequisites + +- Jetson device running L4T (JetPack 6.x / R36) +- Docker installed and running +- [OpenShell CLI](https://github.com/NVIDIA/OpenShell/releases) installed +- NVIDIA API key from [build.nvidia.com](https://build.nvidia.com) +- Your user in the `docker` group (`sudo usermod -aG docker $USER`, then re-login) + +## Install + +```bash +git clone https://github.com/NVIDIA/NemoClaw.git +cd NemoClaw +cd nemoclaw && npm install && npm run build && cd .. +``` + +## Run the Setup + +The script runs as your **normal user** (not sudo). It uses sudo internally +only for kernel module loading and Docker daemon configuration. + +```bash +export NVIDIA_API_KEY=nvapi-... +bash scripts/setup-jetson.sh +``` + +The script: + +1. Detects the Jetson platform (via `/etc/nv_tegra_release` or kernel string) +2. Loads kernel modules required by k3s (`br_netfilter`, `xt_conntrack`, etc.) +3. Configures Docker for `cgroupns=host` on cgroup v2 +4. Starts the OpenShell gateway and patches the image to use `iptables-legacy` +5. Disables kube-router's network policy controller (L4T lacks required ipset types) +6. Sets up the NVIDIA inference provider +7. Creates the sandbox with a network policy for egress control +8. Patches CoreDNS for Docker DNS forwarding +9. Installs Ollama for optional local inference + +## Activate the Network Policy + +After the setup completes, you must activate the network policy once: + +1. Open the OpenShell TUI: + + ```bash + openshell term + ``` + +2. Approve the pending network policy rules in the TUI. + +3. Once approved, the sandbox can reach all pre-configured endpoints + (Telegram, NVIDIA API, GitHub, npm) without further interaction. + +This is a **one-time step** per gateway session. The policy remains active +until the gateway is destroyed. + +## Connect and Test + +```bash +openshell sandbox connect nemoclaw +``` + +Inside the sandbox, test inference: + +```bash +openclaw agent --agent main --local -m 'hello' --session-id test1 +``` + +## Local Inference with Ollama + +By default, inference routes through NVIDIA cloud. To use local inference +on the Jetson GPU: + +```bash +ollama pull nemotron-3-nano:4b +openshell inference set --provider ollama-local --model nemotron-3-nano:4b +``` + +## What's Different on Jetson + +### iptables-legacy + +L4T's kernel uses the `iptables-legacy` backend. The OpenShell gateway image +defaults to `iptables-nft`, which panics on L4T because the nf_tables +compatibility layer is incomplete. The setup script patches the gateway image +to use `iptables-legacy`. + +### Disabled kube-router Network Policy + +L4T's kernel only ships the `ip_set_hash_net` ipset module. kube-router +requires additional types (`hash:ip`, `hash:ipport`, etc.) that are not +compiled into the Tegra kernel. The setup script disables kube-router's +network policy controller to avoid ipset panics. + +**What remains active:** OpenShell's application-level egress proxy provides: + +- Deny-by-default outbound access with an explicit host allowlist +- HTTP method and path filtering +- Operator approval workflow for unlisted hosts via `openshell term` +- Filesystem and process isolation + +**What is missing:** the Kubernetes NetworkPolicy layer that catches traffic +bypassing the HTTP proxy (e.g., raw sockets). This gap could be closed by +rebuilding the L4T kernel with full ipset support. + +### CoreDNS Fix + +Docker's internal DNS (`127.0.0.11`) is unreachable from k3s pods. The +setup script patches CoreDNS to forward to the gateway container's DNS +proxy, the same fix applied for Colima environments. + +## Tested Devices + +| Device | JetPack | L4T Kernel | Status | +|--------|---------|------------|--------| +| Orin Nano | 6.x | 5.15.185-tegra | Tested | +| Orin NX | 6.x | 5.15.x-tegra | Expected to work | +| AGX Orin | 6.x | 5.15.x-tegra | Expected to work | + +## Troubleshooting + +**Gateway fails with iptables errors:** +The setup script automatically patches the gateway image. If you see +`nf_tables` errors, the patch may not have applied. Run the setup again — +it will detect and re-patch. + +**Sandbox shows `Pending` phase:** +Wait a minute for the image to be pushed into the gateway. If it stays +pending, check `openshell sandbox get nemoclaw` for details. + +**Network requests return 403:** +Open `openshell term` and approve the pending network policy rules. +This is required once after each gateway restart. + +**`openshell term` shows nothing:** +Make sure you ran `setup-jetson.sh` as your normal user (not sudo). +The gateway metadata must be in your user's config directory. diff --git a/docs/index.md b/docs/index.md index 6ef9c075624..8d09ce3c5c6 100644 --- a/docs/index.md +++ b/docs/index.md @@ -220,6 +220,7 @@ Customize the Network Policy :hidden: Deploy to a Remote GPU Instance +Set Up on Jetson Set Up the Telegram Bridge ``` diff --git a/scripts/setup-jetson.sh b/scripts/setup-jetson.sh index f2ef203a0d4..90f4f6d16e0 100755 --- a/scripts/setup-jetson.sh +++ b/scripts/setup-jetson.sh @@ -4,25 +4,25 @@ # # NemoClaw setup for Jetson devices (Orin Nano, Orin NX, AGX Orin, etc.) # -# Jetson's L4T kernel ships the iptables/netfilter modules that k3s needs -# (xt_comment, xt_conntrack, etc.) but doesn't load them by default. -# Without them, OpenShell's k3s gateway panics on network policy setup. -# -# This script loads the missing modules, configures Docker for cgroup v2 -# (same as setup-spark.sh), sets up Ollama for local inference, then -# hands off to the normal setup.sh for the full OpenShell/k3s path. +# Unlike setup.sh, this script is run as a NORMAL USER (not sudo). +# It uses sudo internally only for kernel module loading and Docker +# daemon configuration. All openshell commands run as the user so +# gateway metadata and mTLS certs land in the user's home dir. # # Usage: -# sudo bash scripts/setup-jetson.sh +# export NVIDIA_API_KEY=nvapi-... +# bash scripts/setup-jetson.sh # -# What it does (beyond setup.sh): +# What it does: # 1. Autodetects Jetson (Tegra/L4T) -# 2. Loads required kernel modules for k3s (iptables + ipset) -# 3. Configures Docker daemon for cgroupns=host (if cgroup v2) -# 4. Patches OpenShell gateway image: iptables-nft → iptables-legacy -# 5. Ensures Ollama is running with a suitable model -# 6. Runs the normal setup.sh (full OpenShell/k3s path) -# 7. Fixes CoreDNS forwarding (same issue as Colima) +# 2. Loads required kernel modules for k3s (sudo) +# 3. Configures Docker daemon for cgroupns=host (sudo) +# 4. Starts the OpenShell gateway +# 5. Patches gateway image: iptables-nft → iptables-legacy, restarts +# 6. Sets up inference providers +# 7. Creates NemoClaw sandbox (with network policy) +# 8. Ensures Ollama is available for local inference +# 9. Fixes CoreDNS forwarding (same issue as Colima) set -euo pipefail @@ -36,6 +36,7 @@ warn() { echo -e "${YELLOW}>>>${NC} $1"; } fail() { echo -e "${RED}>>>${NC} $1"; exit 1; } SCRIPT_DIR="$(cd "$(dirname "$0")" && pwd)" +REPO_DIR="$(cd "$SCRIPT_DIR/.." && pwd)" # ── Pre-flight checks ──────────────────────────────────────────── @@ -43,8 +44,8 @@ if [ "$(uname -s)" != "Linux" ]; then fail "This script is for Jetson (Linux). Use 'nemoclaw setup' for macOS." fi -if [ "$(id -u)" -ne 0 ]; then - fail "Must run as root: sudo bash scripts/setup-jetson.sh" +if [ "$(id -u)" -eq 0 ]; then + fail "Do not run as root. Run as your normal user: bash scripts/setup-jetson.sh" fi # Autodetect Jetson (L4T / Tegra) @@ -60,37 +61,25 @@ else fi command -v docker > /dev/null || fail "Docker not found." - -# Detect the real user (not root) for docker group / handoff -REAL_USER="${SUDO_USER:-$(logname 2>/dev/null || echo "")}" -if [ -z "$REAL_USER" ]; then - warn "Could not detect non-root user. Docker group will not be configured." -fi - -# ── 1. Docker group ────────────────────────────────────────────── - -if [ -n "$REAL_USER" ]; then - if id -nG "$REAL_USER" | grep -qw docker; then - info "User '$REAL_USER' already in docker group" - else - info "Adding '$REAL_USER' to docker group..." - usermod -aG docker "$REAL_USER" - info "Added. Group will take effect on next login (or use 'newgrp docker')." - fi +command -v openshell > /dev/null || fail "openshell CLI not found. Install from https://github.com/NVIDIA/OpenShell/releases" +[ -n "${NVIDIA_API_KEY:-}" ] || fail "NVIDIA_API_KEY not set. Get one from build.nvidia.com" + +# Check docker group membership +if ! id -nG | grep -qw docker; then + info "Adding you to the docker group (requires sudo)..." + sudo usermod -aG docker "$USER" + fail "Added to docker group. Please log out and back in (or 'newgrp docker'), then re-run this script." fi -# ── 2. Kernel modules for k3s iptables ────────────────────────── +# ── 1. Kernel modules (requires sudo) ──────────────────────────── # # L4T builds xt_comment, xt_conntrack, nf_conntrack etc. as modules # but doesn't load them at boot. k3s's kube-router panics without # them because it can't insert iptables rules. MODULES=( - # Bridge netfilter — required for pod-to-pod and ClusterIP routing br_netfilter - # iptables matches for kube-router / kube-proxy xt_comment xt_conntrack nf_conntrack xt_mark xt_nat xt_MASQUERADE - # ipset types for kube-router network policy (load what's available) ip_set_hash_net ip_set_hash_ip ip_set_hash_ipport ip_set_hash_ipportnet ip_set_hash_ipportip ip_set_bitmap_port ) @@ -98,7 +87,7 @@ LOADED_ANY=false for mod in "${MODULES[@]}"; do if ! lsmod | grep -qw "$mod"; then - if modprobe "$mod" 2>/dev/null; then + if sudo modprobe "$mod" 2>/dev/null; then info "Loaded kernel module: $mod" LOADED_ANY=true else @@ -108,19 +97,16 @@ for mod in "${MODULES[@]}"; do done if [ "$LOADED_ANY" = true ]; then - # Persist across reboots for mod in "${MODULES[@]}"; do if ! grep -qx "$mod" /etc/modules-load.d/k3s-netfilter.conf 2>/dev/null; then - echo "$mod" >> /etc/modules-load.d/k3s-netfilter.conf + echo "$mod" | sudo tee -a /etc/modules-load.d/k3s-netfilter.conf > /dev/null fi done info "Modules persisted to /etc/modules-load.d/k3s-netfilter.conf" fi info "Kernel modules OK" -# ── 3. Docker cgroup namespace (same as setup-spark.sh) ────────── -# -# If cgroup v2, k3s-in-Docker needs cgroupns=host. +# ── 2. Docker cgroup namespace (requires sudo) ─────────────────── if [ "$(stat -fc %T /sys/fs/cgroup/ 2>/dev/null)" = "cgroup2fs" ]; then DAEMON_JSON="/etc/docker/daemon.json" @@ -131,8 +117,8 @@ if [ "$(stat -fc %T /sys/fs/cgroup/ 2>/dev/null)" = "cgroup2fs" ]; then if [ "$CURRENT_MODE" = "host" ]; then info "Docker daemon already configured for cgroupns=host" else - info "Setting Docker daemon cgroupns=host..." - python3 -c " + info "Setting Docker daemon cgroupns=host (requires sudo)..." + sudo python3 -c " import json with open('$DAEMON_JSON') as f: d = json.load(f) @@ -143,15 +129,15 @@ with open('$DAEMON_JSON', 'w') as f: NEEDS_RESTART=true fi else - info "Creating Docker daemon config with cgroupns=host..." - mkdir -p "$(dirname "$DAEMON_JSON")" - echo '{ "default-cgroupns-mode": "host" }' > "$DAEMON_JSON" + info "Creating Docker daemon config with cgroupns=host (requires sudo)..." + sudo mkdir -p "$(dirname "$DAEMON_JSON")" + echo '{ "default-cgroupns-mode": "host" }' | sudo tee "$DAEMON_JSON" > /dev/null NEEDS_RESTART=true fi if [ "$NEEDS_RESTART" = true ]; then info "Restarting Docker daemon..." - systemctl restart docker + sudo systemctl restart docker for i in 1 2 3 4 5 6 7 8 9 10; do if docker info > /dev/null 2>&1; then break @@ -165,34 +151,38 @@ else info "cgroup v1 — no Docker changes needed" fi -# ── 4. Patch gateway image: iptables-nft → iptables-legacy ─────── +# ── No more sudo needed from here on ───────────────────────────── + +# ── 3. Start gateway ───────────────────────────────────────────── + +info "Starting OpenShell gateway..." +openshell gateway destroy -g nemoclaw > /dev/null 2>&1 || true +GATEWAY_ARGS=(--name nemoclaw) +command -v nvidia-smi > /dev/null 2>&1 && GATEWAY_ARGS+=(--gpu) +openshell gateway start "${GATEWAY_ARGS[@]}" 2>&1 | grep -E "Gateway|✓|Error|error" || true + +# The gateway container will crash because of iptables-nft. +# That's expected — we patch and restart it in the next step. +sleep 3 + +# ── 4. Patch gateway image: iptables-nft → iptables-legacy ────── # # The OpenShell gateway image ships iptables v1.8.10 defaulting to # the nf_tables backend. L4T's kernel uses iptables-legacy on the # host and the nf_tables compat layer is incomplete (xt_addrtype # etc.). k3s's kube-router panics when nft RULE_INSERT fails. # -# Fix: build a one-layer wrapper that switches the alternative to -# iptables-legacy, then tag it over the upstream image name so -# `openshell gateway start` uses it. The upstream layers are -# preserved — `docker pull` restores the original at any time. +# We patch AFTER `openshell gateway start` because that command +# pulls the upstream image from the registry, which would overwrite +# any earlier patch. GATEWAY_IMAGE="ghcr.io/nvidia/openshell/cluster:0.0.8" -# Pull upstream image if not present -if ! docker image inspect "$GATEWAY_IMAGE" > /dev/null 2>&1; then - info "Pulling OpenShell gateway image..." - docker pull "$GATEWAY_IMAGE" -fi - -# Check if already patched (iptables-legacy inside the image) CURRENT_IPT=$(docker run --rm --entrypoint iptables "$GATEWAY_IMAGE" --version 2>&1 || true) if echo "$CURRENT_IPT" | grep -q "legacy"; then info "Gateway image already using iptables-legacy" else info "Patching gateway image to use iptables-legacy..." - # Save upstream image ID so we can verify we're wrapping the right thing - UPSTREAM_ID=$(docker image inspect --format='{{.Id}}' "$GATEWAY_IMAGE") PATCH_CTX="$(mktemp -d)" cat > "$PATCH_CTX/Dockerfile" <<'DOCKERFILE' @@ -206,8 +196,9 @@ RUN update-alternatives --set iptables /usr/sbin/iptables-legacy \ # L4T kernel only ships ip_set_hash_net — kube-router's network policy # controller needs additional ipset types (hash:ip, hash:ipport, etc.) -# that don't exist. Disable the controller so it doesn't block pod traffic. -# Wrap the original entrypoint to inject --disable-network-policy. +# that don't exist. Disable the controller to avoid ipset panics. +# Egress policy enforcement still works via OpenShell's HTTP proxy +# (HTTP_PROXY/HTTPS_PROXY injected into the sandbox). RUN mv /usr/local/bin/cluster-entrypoint.sh /usr/local/bin/cluster-entrypoint-orig.sh COPY entrypoint-jetson.sh /usr/local/bin/cluster-entrypoint.sh RUN chmod +x /usr/local/bin/cluster-entrypoint.sh @@ -215,9 +206,9 @@ DOCKERFILE cat > "$PATCH_CTX/entrypoint-jetson.sh" <<'WRAPPER' #!/bin/sh -# Jetson wrapper: inject --disable-network-policy then call original entrypoint. -# The original entrypoint ends with: exec /bin/k3s "$@" ... -# We append our flag to the args passed through. +# Jetson wrapper: disable kube-router network policy controller to avoid +# ipset panics (L4T lacks hash:ip, hash:ipport kernel modules). +# Egress is still enforced by OpenShell's HTTP proxy layer. exec /usr/local/bin/cluster-entrypoint-orig.sh "$@" --disable-network-policy WRAPPER @@ -235,46 +226,133 @@ WRAPPER fi fi -# ── 5. Ollama (optional local inference) ────────────────────────── -# -# Ollama is installed if not present but no model is pulled by default. -# To use local inference after setup, pull a model and switch: -# ollama pull nemotron-3-nano:4b -# openshell inference set --provider vllm-local --model nemotron-3-nano:4b +# Restart the gateway container with the patched image +CLUSTER_CONTAINER="openshell-cluster-nemoclaw" +info "Restarting gateway with patched image..." +docker rm -f "$CLUSTER_CONTAINER" > /dev/null 2>&1 || true +openshell gateway destroy -g nemoclaw > /dev/null 2>&1 || true +openshell gateway start "${GATEWAY_ARGS[@]}" 2>&1 | grep -E "Gateway|✓|Error|error" || true + +# Verify gateway is healthy +for i in 1 2 3 4 5 6 7 8 9 10; do + if openshell status 2>&1 | grep -q "Connected"; then + break + fi + [ "$i" -eq 10 ] && fail "Gateway failed to start. Check 'docker logs $CLUSTER_CONTAINER'." + sleep 3 +done +info "Gateway is healthy" + +# ── 5. Inference providers ─────────────────────────────────────── + +upsert_provider() { + local name="$1" + local type="$2" + local credential="$3" + local config="$4" + + if openshell provider create --name "$name" --type "$type" \ + --credential "$credential" \ + --config "$config" 2>&1 | grep -q "AlreadyExists"; then + openshell provider update "$name" \ + --credential "$credential" \ + --config "$config" > /dev/null + info "Updated $name provider" + else + info "Created $name provider" + fi +} -if ! command -v ollama > /dev/null 2>&1; then - info "Installing Ollama..." - curl -fsSL https://ollama.com/install.sh | sh -fi +info "Setting up inference providers..." + +upsert_provider \ + "nvidia-nim" \ + "openai" \ + "NVIDIA_API_KEY=$NVIDIA_API_KEY" \ + "OPENAI_BASE_URL=https://integrate.api.nvidia.com/v1" +# Ollama provider (Jetson local inference) if command -v ollama > /dev/null 2>&1; then - info "Ollama available (no model pulled — use 'ollama pull ' for local inference)" + upsert_provider \ + "ollama-local" \ + "openai" \ + "OPENAI_API_KEY=ollama" \ + "OPENAI_BASE_URL=http://host.openshell.internal:11434/v1" fi -# ── 6. Run normal setup.sh ─────────────────────────────────────── +# vllm-local (if vLLM is running) +if curl -s http://localhost:8000/v1/models > /dev/null 2>&1; then + upsert_provider \ + "vllm-local" \ + "openai" \ + "OPENAI_API_KEY=dummy" \ + "OPENAI_BASE_URL=http://host.openshell.internal:8000/v1" +fi -info "Running NemoClaw setup..." -echo "" +info "Setting inference route to nvidia-nim / Nemotron 3 Super..." +openshell inference set --no-verify --provider nvidia-nim --model nvidia/nemotron-3-super-120b-a12b > /dev/null 2>&1 -if [ -n "$REAL_USER" ]; then - sudo -u "$REAL_USER" -E \ - NVIDIA_API_KEY="${NVIDIA_API_KEY:-}" \ - DOCKER_HOST="${DOCKER_HOST:-}" \ - bash "$SCRIPT_DIR/setup.sh" -else - bash "$SCRIPT_DIR/setup.sh" +# ── 6. Create sandbox (with network policy) ───────────────────── + +info "Deleting old nemoclaw sandbox (if any)..." +openshell sandbox delete nemoclaw > /dev/null 2>&1 || true + +info "Building and creating NemoClaw sandbox (this takes a few minutes on first run)..." + +# Stage a clean build context (openshell doesn't honor .dockerignore) +BUILD_CTX="$(mktemp -d)" +cp "$REPO_DIR/Dockerfile" "$BUILD_CTX/" +cp -r "$REPO_DIR/nemoclaw" "$BUILD_CTX/nemoclaw" +cp -r "$REPO_DIR/nemoclaw-blueprint" "$BUILD_CTX/nemoclaw-blueprint" +cp -r "$REPO_DIR/scripts" "$BUILD_CTX/scripts" +rm -rf "$BUILD_CTX/nemoclaw/node_modules" "$BUILD_CTX/nemoclaw/src" + +# Verify nemoclaw/dist/ exists (TypeScript must be pre-built) +if [ ! -d "$BUILD_CTX/nemoclaw/dist" ] || [ -z "$(ls -A "$BUILD_CTX/nemoclaw/dist" 2>/dev/null)" ]; then + rm -rf "$BUILD_CTX" + fail "nemoclaw/dist/ is missing or empty. Run 'cd nemoclaw && npm install && npm run build' first." fi -# ── 7. Fix CoreDNS (Jetson-specific) ──────────────────────────── -# -# Same problem as Colima: k3s CoreDNS forwards to /etc/resolv.conf -# which contains 127.0.0.11 (Docker's internal DNS), unreachable -# from k3s pods. The entrypoint sets up a DNS proxy on the -# container's eth0 IP — point CoreDNS there instead. -# -# setup.sh only runs fix-coredns.sh for Colima. On Jetson the -# Docker engine also uses 127.0.0.11 in /etc/resolv.conf, so -# the same fix is needed. +CREATE_LOG=$(mktemp /tmp/nemoclaw-create-XXXXXX.log) +set +e +openshell sandbox create --from "$BUILD_CTX/Dockerfile" --name nemoclaw \ + --provider nvidia-nim \ + --policy "$REPO_DIR/nemoclaw-blueprint/policies/openclaw-sandbox.yaml" \ + -- env NVIDIA_API_KEY="$NVIDIA_API_KEY" > "$CREATE_LOG" 2>&1 +CREATE_RC=$? +set -e +rm -rf "$BUILD_CTX" + +grep -E "^ (Step |Building |Built |Pushing |\[progress\]|Successfully |Created sandbox|Image )|✓" "$CREATE_LOG" || true + +if [ "$CREATE_RC" != "0" ]; then + echo "" + warn "Last 20 lines of build output:" + tail -20 "$CREATE_LOG" | grep -v "NVIDIA_API_KEY" + echo "" + fail "Sandbox creation failed (exit $CREATE_RC). Full log: $CREATE_LOG" +fi +rm -f "$CREATE_LOG" + +# Verify sandbox is Ready +SANDBOX_LINE=$(openshell sandbox list 2>&1 | sed 's/\x1b\[[0-9;]*m//g' | grep "nemoclaw") +if ! echo "$SANDBOX_LINE" | grep -q "Ready"; then + SANDBOX_PHASE=$(echo "$SANDBOX_LINE" | awk '{print $NF}') + fail "Sandbox created but not Ready (phase: ${SANDBOX_PHASE:-unknown}). Check 'openshell sandbox get nemoclaw'." +fi + +# ── 7. Ollama (optional local inference) ────────────────────────── + +if ! command -v ollama > /dev/null 2>&1; then + info "Installing Ollama (requires sudo)..." + curl -fsSL https://ollama.com/install.sh | sudo sh +fi + +if command -v ollama > /dev/null 2>&1; then + info "Ollama available (no model pulled — use 'ollama pull ' for local inference)" +fi + +# ── 8. Fix CoreDNS (Jetson-specific) ──────────────────────────── CLUSTER=$(docker ps --filter "name=openshell-cluster" --format '{{.Names}}' | head -1) if [ -n "$CLUSTER" ]; then @@ -282,7 +360,6 @@ if [ -n "$CLUSTER" ]; then | grep nameserver | awk '{print $2}') if [ -n "$DNS_IP" ] && [[ "$DNS_IP" != 127.* ]]; then - # Check if CoreDNS is already forwarding to the right IP CURRENT_FWD=$(docker exec "$CLUSTER" kubectl get configmap coredns -n kube-system \ -o jsonpath='{.data.Corefile}' 2>/dev/null | grep -oP 'forward \. \K\S+' || true) @@ -294,11 +371,23 @@ if [ -n "$CLUSTER" ]; then docker exec "$CLUSTER" kubectl rollout status deploy/coredns -n kube-system --timeout=30s > /dev/null 2>&1 info "CoreDNS patched" - # Bounce the sandbox pod so it picks up working DNS immediately - # instead of waiting for CrashLoopBackOff to expire docker exec "$CLUSTER" kubectl delete pod -n openshell -l app=nemoclaw --ignore-not-found > /dev/null 2>&1 || true else info "CoreDNS already forwarding to $DNS_IP" fi fi fi + +# ── Done ───────────────────────────────────────────────────────── + +echo "" +info "Jetson setup complete!" +echo "" +echo " Next steps:" +echo " 1. Run 'openshell term' and approve the pending network policy rules." +echo " (The sandbox blocks all egress until you approve.)" +echo " 2. Connect to the sandbox:" +echo " openshell sandbox connect nemoclaw" +echo " 3. Test the agent:" +echo " openclaw agent --agent main --local -m 'hello' --session-id s1" +echo "" From 3bf7566faa6c75d70567df8603f8e4380c8434f5 Mon Sep 17 00:00:00 2001 From: hendrikh Date: Sun, 22 Mar 2026 12:36:51 +0100 Subject: [PATCH 4/5] fix: align Jetson setup with rebased main - Derive gateway image tag from `openshell --version` instead of hardcoding 0.0.8. The CLI pulls cluster: at runtime, so the iptables-legacy patch must target the matching tag. - Keep nemoclaw/src/ in the Docker build context. Main switched to a multi-stage Dockerfile that builds TypeScript from source (3aaef13), so removing src/ broke the sandbox image build. - Pre-populate network policy draft rules by triggering outbound connections from the sandbox to every endpoint in the policy. This ensures all rules are visible in `openshell term` for approval immediately after setup completes. Tested on Jetson Orin Nano (R36.5, L4T 5.15.185-tegra). --- scripts/setup-jetson.sh | 46 ++++++++++++++++++++++++++++++++++++----- 1 file changed, 41 insertions(+), 5 deletions(-) diff --git a/scripts/setup-jetson.sh b/scripts/setup-jetson.sh index 90f4f6d16e0..01739af867c 100755 --- a/scripts/setup-jetson.sh +++ b/scripts/setup-jetson.sh @@ -176,7 +176,10 @@ sleep 3 # pulls the upstream image from the registry, which would overwrite # any earlier patch. -GATEWAY_IMAGE="ghcr.io/nvidia/openshell/cluster:0.0.8" +# Derive the cluster image tag from the installed openshell CLI version +# so the patch targets the image that `openshell gateway start` actually pulls. +OPENSHELL_VERSION=$(openshell --version 2>&1 | grep -oP '\d+\.\d+\.\d+' | head -1) +GATEWAY_IMAGE="ghcr.io/nvidia/openshell/cluster:${OPENSHELL_VERSION:-0.0.8}" CURRENT_IPT=$(docker run --rm --entrypoint iptables "$GATEWAY_IMAGE" --version 2>&1 || true) if echo "$CURRENT_IPT" | grep -q "legacy"; then @@ -305,12 +308,12 @@ cp "$REPO_DIR/Dockerfile" "$BUILD_CTX/" cp -r "$REPO_DIR/nemoclaw" "$BUILD_CTX/nemoclaw" cp -r "$REPO_DIR/nemoclaw-blueprint" "$BUILD_CTX/nemoclaw-blueprint" cp -r "$REPO_DIR/scripts" "$BUILD_CTX/scripts" -rm -rf "$BUILD_CTX/nemoclaw/node_modules" "$BUILD_CTX/nemoclaw/src" +rm -rf "$BUILD_CTX/nemoclaw/node_modules" -# Verify nemoclaw/dist/ exists (TypeScript must be pre-built) -if [ ! -d "$BUILD_CTX/nemoclaw/dist" ] || [ -z "$(ls -A "$BUILD_CTX/nemoclaw/dist" 2>/dev/null)" ]; then +# Verify nemoclaw/src/ exists (Dockerfile builds from source in a multi-stage build) +if [ ! -d "$BUILD_CTX/nemoclaw/src" ] || [ -z "$(ls -A "$BUILD_CTX/nemoclaw/src" 2>/dev/null)" ]; then rm -rf "$BUILD_CTX" - fail "nemoclaw/dist/ is missing or empty. Run 'cd nemoclaw && npm install && npm run build' first." + fail "nemoclaw/src/ is missing or empty. Are you running from a valid NemoClaw checkout?" fi CREATE_LOG=$(mktemp /tmp/nemoclaw-create-XXXXXX.log) @@ -341,6 +344,39 @@ if ! echo "$SANDBOX_LINE" | grep -q "Ready"; then fail "Sandbox created but not Ready (phase: ${SANDBOX_PHASE:-unknown}). Check 'openshell sandbox get nemoclaw'." fi +# ── 6b. Pre-populate network policy draft rules ────────────────── +# +# On Jetson, kube-router's network policy controller is disabled (ipset +# panics on L4T), so egress goes through OpenShell's HTTP proxy. The +# proxy generates draft rules only when traffic actually hits it. +# Trigger connections to every endpoint in the policy so the rules are +# ready and waiting when the user opens `openshell term` to approve. + +info "Triggering network policy rule generation..." +POLICY_HOSTS=( + api.anthropic.com statsig.anthropic.com sentry.io + integrate.api.nvidia.com inference-api.nvidia.com + github.com api.github.com + clawhub.com openclaw.ai docs.openclaw.ai + registry.npmjs.org + api.telegram.org + discord.com gateway.discord.gg cdn.discordapp.com +) + +# Build a one-liner that curls every host from inside the sandbox +CURL_CMDS="" +for host in "${POLICY_HOSTS[@]}"; do + CURL_CMDS="${CURL_CMDS}curl -sf -o /dev/null --connect-timeout 3 https://${host} 2>/dev/null || true; " +done + +# SSH into the sandbox and fire off all the requests (they'll all 403, +# but that's the point — the proxy records each as a draft rule) +ssh -o StrictHostKeyChecking=no -o UserKnownHostsFile=/dev/null -o LogLevel=ERROR \ + -o "ProxyCommand=openshell ssh-proxy --gateway-name nemoclaw --name nemoclaw" \ + sandbox@openshell-nemoclaw "$CURL_CMDS" > /dev/null 2>&1 || true + +info "Draft rules generated — approve them in 'openshell term'" + # ── 7. Ollama (optional local inference) ────────────────────────── if ! command -v ollama > /dev/null 2>&1; then From 2648f3e82440dfbf7a9ad8d2b6667e80d967e843 Mon Sep 17 00:00:00 2001 From: hendrikh Date: Sun, 22 Mar 2026 12:56:50 +0100 Subject: [PATCH 5/5] fix: address code review feedback on Jetson setup - Use console fences with $ prompts in docs (docs formatter contract) - Fail fast in upsert_provider when create fails for non-AlreadyExists - Move ollama-local provider creation after Ollama install step; make Ollama install non-fatal so setup continues with cloud inference - Guard draft-rule warm-up message on SSH success - Use exact CLUSTER_CONTAINER name for CoreDNS patch instead of fuzzy docker ps match --- docs/deployment/jetson-setup.md | 32 +++++++++--------- scripts/setup-jetson.sh | 58 +++++++++++++++++++-------------- 2 files changed, 49 insertions(+), 41 deletions(-) diff --git a/docs/deployment/jetson-setup.md b/docs/deployment/jetson-setup.md index 101303c610c..0fe371a78f1 100644 --- a/docs/deployment/jetson-setup.md +++ b/docs/deployment/jetson-setup.md @@ -35,10 +35,10 @@ iptables backend. ## Install -```bash -git clone https://github.com/NVIDIA/NemoClaw.git -cd NemoClaw -cd nemoclaw && npm install && npm run build && cd .. +```console +$ git clone https://github.com/NVIDIA/NemoClaw.git +$ cd NemoClaw +$ cd nemoclaw && npm install && npm run build && cd .. ``` ## Run the Setup @@ -46,9 +46,9 @@ cd nemoclaw && npm install && npm run build && cd .. The script runs as your **normal user** (not sudo). It uses sudo internally only for kernel module loading and Docker daemon configuration. -```bash -export NVIDIA_API_KEY=nvapi-... -bash scripts/setup-jetson.sh +```console +$ export NVIDIA_API_KEY=nvapi-... +$ bash scripts/setup-jetson.sh ``` The script: @@ -69,8 +69,8 @@ After the setup completes, you must activate the network policy once: 1. Open the OpenShell TUI: - ```bash - openshell term + ```console + $ openshell term ``` 2. Approve the pending network policy rules in the TUI. @@ -83,14 +83,14 @@ until the gateway is destroyed. ## Connect and Test -```bash -openshell sandbox connect nemoclaw +```console +$ openshell sandbox connect nemoclaw ``` Inside the sandbox, test inference: -```bash -openclaw agent --agent main --local -m 'hello' --session-id test1 +```console +$ openclaw agent --agent main --local -m 'hello' --session-id test1 ``` ## Local Inference with Ollama @@ -98,9 +98,9 @@ openclaw agent --agent main --local -m 'hello' --session-id test1 By default, inference routes through NVIDIA cloud. To use local inference on the Jetson GPU: -```bash -ollama pull nemotron-3-nano:4b -openshell inference set --provider ollama-local --model nemotron-3-nano:4b +```console +$ ollama pull nemotron-3-nano:4b +$ openshell inference set --provider ollama-local --model nemotron-3-nano:4b ``` ## What's Different on Jetson diff --git a/scripts/setup-jetson.sh b/scripts/setup-jetson.sh index 01739af867c..c2dadd70274 100755 --- a/scripts/setup-jetson.sh +++ b/scripts/setup-jetson.sh @@ -254,15 +254,24 @@ upsert_provider() { local credential="$3" local config="$4" - if openshell provider create --name "$name" --type "$type" \ + local output rc + set +e + output=$(openshell provider create --name "$name" --type "$type" \ --credential "$credential" \ - --config "$config" 2>&1 | grep -q "AlreadyExists"; then + --config "$config" 2>&1) + rc=$? + set -e + + if [ "$rc" -eq 0 ]; then + info "Created $name provider" + elif echo "$output" | grep -q "AlreadyExists"; then openshell provider update "$name" \ --credential "$credential" \ --config "$config" > /dev/null info "Updated $name provider" else - info "Created $name provider" + echo "$output" >&2 + fail "Failed to create provider $name (exit $rc)" fi } @@ -274,15 +283,6 @@ upsert_provider \ "NVIDIA_API_KEY=$NVIDIA_API_KEY" \ "OPENAI_BASE_URL=https://integrate.api.nvidia.com/v1" -# Ollama provider (Jetson local inference) -if command -v ollama > /dev/null 2>&1; then - upsert_provider \ - "ollama-local" \ - "openai" \ - "OPENAI_API_KEY=ollama" \ - "OPENAI_BASE_URL=http://host.openshell.internal:11434/v1" -fi - # vllm-local (if vLLM is running) if curl -s http://localhost:8000/v1/models > /dev/null 2>&1; then upsert_provider \ @@ -371,43 +371,51 @@ done # SSH into the sandbox and fire off all the requests (they'll all 403, # but that's the point — the proxy records each as a draft rule) -ssh -o StrictHostKeyChecking=no -o UserKnownHostsFile=/dev/null -o LogLevel=ERROR \ +if ssh -o StrictHostKeyChecking=no -o UserKnownHostsFile=/dev/null -o LogLevel=ERROR \ -o "ProxyCommand=openshell ssh-proxy --gateway-name nemoclaw --name nemoclaw" \ - sandbox@openshell-nemoclaw "$CURL_CMDS" > /dev/null 2>&1 || true - -info "Draft rules generated — approve them in 'openshell term'" + sandbox@openshell-nemoclaw "$CURL_CMDS" > /dev/null 2>&1; then + info "Draft rules generated — approve them in 'openshell term'" +else + warn "SSH into sandbox failed — draft rules may not have been generated. You can trigger them manually after connecting." +fi # ── 7. Ollama (optional local inference) ────────────────────────── if ! command -v ollama > /dev/null 2>&1; then info "Installing Ollama (requires sudo)..." - curl -fsSL https://ollama.com/install.sh | sudo sh + if ! curl -fsSL https://ollama.com/install.sh | sudo sh; then + warn "Ollama installation failed — skipping local inference setup" + fi fi if command -v ollama > /dev/null 2>&1; then info "Ollama available (no model pulled — use 'ollama pull ' for local inference)" + upsert_provider \ + "ollama-local" \ + "openai" \ + "OPENAI_API_KEY=ollama" \ + "OPENAI_BASE_URL=http://host.openshell.internal:11434/v1" fi # ── 8. Fix CoreDNS (Jetson-specific) ──────────────────────────── -CLUSTER=$(docker ps --filter "name=openshell-cluster" --format '{{.Names}}' | head -1) -if [ -n "$CLUSTER" ]; then - DNS_IP=$(docker exec "$CLUSTER" cat /etc/rancher/k3s/resolv.conf 2>/dev/null \ +if docker ps --format '{{.Names}}' | grep -qx "$CLUSTER_CONTAINER"; then + DNS_IP=$(docker exec "$CLUSTER_CONTAINER" cat /etc/rancher/k3s/resolv.conf 2>/dev/null \ | grep nameserver | awk '{print $2}') if [ -n "$DNS_IP" ] && [[ "$DNS_IP" != 127.* ]]; then - CURRENT_FWD=$(docker exec "$CLUSTER" kubectl get configmap coredns -n kube-system \ + CURRENT_FWD=$(docker exec "$CLUSTER_CONTAINER" kubectl get configmap coredns -n kube-system \ -o jsonpath='{.data.Corefile}' 2>/dev/null | grep -oP 'forward \. \K\S+' || true) if [ "$CURRENT_FWD" != "$DNS_IP" ]; then info "Patching CoreDNS to forward to $DNS_IP..." - docker exec "$CLUSTER" kubectl patch configmap coredns -n kube-system --type merge \ + docker exec "$CLUSTER_CONTAINER" kubectl patch configmap coredns -n kube-system --type merge \ -p "{\"data\":{\"Corefile\":\".:53 {\\n errors\\n health\\n ready\\n kubernetes cluster.local in-addr.arpa ip6.arpa {\\n pods insecure\\n fallthrough in-addr.arpa ip6.arpa\\n }\\n hosts /etc/coredns/NodeHosts {\\n ttl 60\\n reload 15s\\n fallthrough\\n }\\n prometheus :9153\\n cache 30\\n loop\\n reload\\n loadbalance\\n forward . $DNS_IP\\n}\\n\"}}" > /dev/null - docker exec "$CLUSTER" kubectl rollout restart deploy/coredns -n kube-system > /dev/null - docker exec "$CLUSTER" kubectl rollout status deploy/coredns -n kube-system --timeout=30s > /dev/null 2>&1 + docker exec "$CLUSTER_CONTAINER" kubectl rollout restart deploy/coredns -n kube-system > /dev/null + docker exec "$CLUSTER_CONTAINER" kubectl rollout status deploy/coredns -n kube-system --timeout=30s > /dev/null 2>&1 info "CoreDNS patched" - docker exec "$CLUSTER" kubectl delete pod -n openshell -l app=nemoclaw --ignore-not-found > /dev/null 2>&1 || true + docker exec "$CLUSTER_CONTAINER" kubectl delete pod -n openshell -l app=nemoclaw --ignore-not-found > /dev/null 2>&1 || true else info "CoreDNS already forwarding to $DNS_IP" fi