fix: propagate --sandbox flag through start/stop to Telegram bridge - #235
fix: propagate --sandbox flag through start/stop to Telegram bridge#235SARPAT wants to merge 1 commit into
Conversation
The Telegram bridge failed with "sandbox not found" when using `nemoclaw start --sandbox <name>` because SANDBOX_NAME never reached the child process. Two gaps fixed: 1. Export SANDBOX_NAME in start-services.sh so nohup children inherit it 2. Parse --sandbox in bin/nemoclaw.js start()/stop() and forward it via both process.env.NEMOCLAW_SANDBOX and the shell script flag Also updates help text and adds tests for the new flag. Signed-off-by: $(git config user.name) <$(git config user.email)> Signed-off-by: Saransh Patel <saranshappy@gmail.com>
Three bugs caused onboarding to silently fail on DGX when GPU was detected: 1. Pipe masked sandbox creation exit code: `openshell sandbox create ... | awk` exits with awk's status (0), not openshell's, so GPU errors were ignored and NemoClaw reported success while no sandbox existed — causing "sandbox not found" in Step 7. Fixed with `set -o pipefail`. 2. GPU device plugin race on DGX: the k3s GPU device plugin needs extra time to register GPUs with the Kubernetes scheduler after the gateway HTTP endpoint becomes healthy. The 5-second DNS sleep was not enough. Added a polling loop (up to 2 minutes) for GPU allocatability before sandbox creation. 3. Stale port 18789 forward from a previous onboard blocked the new sandbox's dashboard port. Added explicit `openshell forward stop 18789` cleanup before the new forward. Co-Authored-By: Claude Sonnet 4.6 <noreply@anthropic.com>
* fix: resolve GPU sandbox creation failures on DGX machines (#235) Three bugs caused onboarding to silently fail on DGX when GPU was detected: 1. Pipe masked sandbox creation exit code: `openshell sandbox create ... | awk` exits with awk's status (0), not openshell's, so GPU errors were ignored and NemoClaw reported success while no sandbox existed — causing "sandbox not found" in Step 7. Fixed with `set -o pipefail`. 2. GPU device plugin race on DGX: the k3s GPU device plugin needs extra time to register GPUs with the Kubernetes scheduler after the gateway HTTP endpoint becomes healthy. The 5-second DNS sleep was not enough. Added a polling loop (up to 2 minutes) for GPU allocatability before sandbox creation. 3. Stale port 18789 forward from a previous onboard blocked the new sandbox's dashboard port. Added explicit `openshell forward stop 18789` cleanup before the new forward. Co-Authored-By: Claude Sonnet 4.6 <noreply@anthropic.com> * fix: stop passing --gpu to gateway and sandbox on DGX Spark The official NVIDIA DGX Spark playbooks do not use --gpu on openshell gateway start or sandbox create. On Spark, inference is routed through a host-side provider (Ollama, vLLM, or cloud API) via openshell's inference routing layer — the sandbox does not need direct GPU access. Passing --gpu causes FailedPrecondition errors because the gateway's k3s GPU device plugin cannot allocate GPUs on Spark, which cascades into sandbox creation failures and broken policy application. Removes the GPU allocatability poll loop (no longer needed) while preserving the pipefail and stale port forward fixes from the parent commit. Refs: #239 See: https://build.nvidia.com/spark/nemoclaw/instructions --------- Co-authored-by: NemoClaw Dev <dev@nemoclaw.local> Co-authored-by: Claude Sonnet 4.6 <noreply@anthropic.com> Co-authored-by: Aaron Erickson <aerickson@nvidia.com>
|
Hey @SARPAT — just flagging that #222 (filed ~3 hours earlier) already covers the #198 fix here. The #222 also addresses #199 (openshell path resolution in the bridge) which is part of the same propagation chain — the bridge fails on both missing Happy to coordinate if maintainers prefer a different breakdown, but wanted to make sure the overlap is visible. -BT |
Thanks for flagging the overlap, @brianwtaylor. #222 covers the same root cause fix and also addresses #199, so it makes more sense to keep that as the single PR for this. Closing this one. |
* fix: resolve GPU sandbox creation failures on DGX machines (NVIDIA#235) Three bugs caused onboarding to silently fail on DGX when GPU was detected: 1. Pipe masked sandbox creation exit code: `openshell sandbox create ... | awk` exits with awk's status (0), not openshell's, so GPU errors were ignored and NemoClaw reported success while no sandbox existed — causing "sandbox not found" in Step 7. Fixed with `set -o pipefail`. 2. GPU device plugin race on DGX: the k3s GPU device plugin needs extra time to register GPUs with the Kubernetes scheduler after the gateway HTTP endpoint becomes healthy. The 5-second DNS sleep was not enough. Added a polling loop (up to 2 minutes) for GPU allocatability before sandbox creation. 3. Stale port 18789 forward from a previous onboard blocked the new sandbox's dashboard port. Added explicit `openshell forward stop 18789` cleanup before the new forward. Co-Authored-By: Claude Sonnet 4.6 <noreply@anthropic.com> * fix: stop passing --gpu to gateway and sandbox on DGX Spark The official NVIDIA DGX Spark playbooks do not use --gpu on openshell gateway start or sandbox create. On Spark, inference is routed through a host-side provider (Ollama, vLLM, or cloud API) via openshell's inference routing layer — the sandbox does not need direct GPU access. Passing --gpu causes FailedPrecondition errors because the gateway's k3s GPU device plugin cannot allocate GPUs on Spark, which cascades into sandbox creation failures and broken policy application. Removes the GPU allocatability poll loop (no longer needed) while preserving the pipefail and stale port forward fixes from the parent commit. Refs: NVIDIA#239 See: https://build.nvidia.com/spark/nemoclaw/instructions --------- Co-authored-by: NemoClaw Dev <dev@nemoclaw.local> Co-authored-by: Claude Sonnet 4.6 <noreply@anthropic.com> Co-authored-by: Aaron Erickson <aerickson@nvidia.com>
* fix: resolve GPU sandbox creation failures on DGX machines (NVIDIA#235) Three bugs caused onboarding to silently fail on DGX when GPU was detected: 1. Pipe masked sandbox creation exit code: `openshell sandbox create ... | awk` exits with awk's status (0), not openshell's, so GPU errors were ignored and NemoClaw reported success while no sandbox existed — causing "sandbox not found" in Step 7. Fixed with `set -o pipefail`. 2. GPU device plugin race on DGX: the k3s GPU device plugin needs extra time to register GPUs with the Kubernetes scheduler after the gateway HTTP endpoint becomes healthy. The 5-second DNS sleep was not enough. Added a polling loop (up to 2 minutes) for GPU allocatability before sandbox creation. 3. Stale port 18789 forward from a previous onboard blocked the new sandbox's dashboard port. Added explicit `openshell forward stop 18789` cleanup before the new forward. Co-Authored-By: Claude Sonnet 4.6 <noreply@anthropic.com> * fix: stop passing --gpu to gateway and sandbox on DGX Spark The official NVIDIA DGX Spark playbooks do not use --gpu on openshell gateway start or sandbox create. On Spark, inference is routed through a host-side provider (Ollama, vLLM, or cloud API) via openshell's inference routing layer — the sandbox does not need direct GPU access. Passing --gpu causes FailedPrecondition errors because the gateway's k3s GPU device plugin cannot allocate GPUs on Spark, which cascades into sandbox creation failures and broken policy application. Removes the GPU allocatability poll loop (no longer needed) while preserving the pipefail and stale port forward fixes from the parent commit. Refs: NVIDIA#239 See: https://build.nvidia.com/spark/nemoclaw/instructions --------- Co-authored-by: NemoClaw Dev <dev@nemoclaw.local> Co-authored-by: Claude Sonnet 4.6 <noreply@anthropic.com> Co-authored-by: Aaron Erickson <aerickson@nvidia.com>
Summary
SANDBOX_NAMEinstart-services.shso child processes (telegram-bridge) inherit it--sandbox <name>inbin/nemoclaw.jsstart()/stop()and forward to the shell script via both env var and CLI flag--sandboxflag is providedFixes #198
Root cause
Two gaps in the env propagation chain:
bin/nemoclaw.jsstart()/stop()ignored CLI arguments —--sandboxwas never parsed or forwardedstart-services.shsetSANDBOX_NAMEas a local variable (not exported), sonohup node telegram-bridge.jsnever inherited itHow this differs from existing PRs
--sandboxstart-services.shbut bundles unrelated SSH retry + vLLM venv changesSANDBOX_NAMEexport but bundles unrelated openshell path resolution (Telegram bridge fails with "openshell: not found" because ~/.local/bin isn't on PATH for child processes #199)This PR covers both gaps with minimal scope and includes tests.
Test plan
npm test— 59 tests pass (includes newtest/start-sandbox-flag.test.js)nemoclaw start --sandbox <name>→ verify/tmp/nemoclaw-services-<name>/PID dir + correct sandbox in bridge logsnemoclaw startwith no flag → uses registry default