fix(gateway): respawn openclaw gateway after a managed restart (#6868)

[atulya-singh]

Summary

nemoclaw <sandbox> config set --key … --value … --restart wrote the config to disk and then hung until the user killed it (or, if they waited, failed with GATEWAY_HEALTH_TIMEOUT after ~150s), leaving the config written but never applied to the running agent. The managed restart path terminated the openclaw gateway with SIGTERM and then only waited for a replacement — but openclaw exits 0 on SIGTERM, and nemoclaw-start reads a clean exit as an intentional shutdown and deliberately does not respawn. The restart therefore killed the gateway with the one signal that guaranteed it would not come back. After this change the root controller leases the exit it is about to cause, the entrypoint relaunches the gateway, and config set --restart completes normally.

Related Issue

Fixes #6868

Root cause

The managed path (gateway-control.sh routes to managed-gateway-control.py when PID 1 is /opt/openshell/bin/openshell-sandbox) has no explicit restart branch: it falls through to _terminate_gateway() + _wait_for_healthy_gateway(), and never launches a replacement. Its comment states the assumption — "PID 1 continuously supervises the managed gateway" — but the process that would respawn it is nemoclaw-start, whose respawn loop treats RC == 0 as an intentional stop and exit 0s.

The mechanism was already solved for Hermes by the expected-exit lease (_publish_expected_exit_lease), which authorizes one exact gateway exit while the root controller is live. It was gated behind spec.name == "hermes", so openclaw never got one.

Evidence gathered against ghcr.io/nvidia/nemoclaw/sandbox-base:latest (OpenClaw 2026.6.10 / aa69b12, aarch64 — the exact build in the report):

• A real gateway (openclaw gateway run --port … --allow-unconfigured, healthy on HTTP 200) sent the same SIGTERM _terminate_gateway sends exits with status 0.
• openshell-sandbox v0.0.72 as PID 1 reaps children but never respawns them (its CLI has no restart policy; killing its child propagates the exit).

Changes

• scripts/managed-gateway-control.py — publish the expected-exit lease for every managed agent, not only Hermes, so the entrypoint can distinguish a host-requested restart from a self-requested shutdown. The lease is published before the pidfd signal and cleared after the health wait, so it is live exactly while the controller waits.
• scripts/lib/gateway-supervisor.sh — add gateway_control_exit_was_host_authorized (plus the controller argv/liveness proofs) to the shared PID 1 control primitives both entrypoints already source. It validates the root-owned 0:0 711 directory and 0:0 444 marker, requires the lease to name the exact pid + start identity, and requires the named controller to still be the live root helper, so an orphaned or mismatched marker fails closed as an ordinary crash.
• scripts/nemoclaw-start.sh — both openclaw respawn loops relaunch a leased exit instead of exit 0. An unleased clean exit still stops, so genuine shutdowns are unchanged.
• src/lib/sandbox/config.ts — when the restart does not complete, say plainly that the config was written to disk but not applied to the running agent, and give the idempotent retry (nemoclaw <sandbox> gateway restart) instead of a generic failure line.

No new abstraction, configuration, fallback, or compatibility path: this wires openclaw into the existing lease mechanism and its existing consumer (nemoclaw-start), protected by the tests below.

Type of Change

• Code change (feature, bug fix, or refactor)
• Code change with doc updates
• Doc only (prose changes, no code sample modifications)
• Doc only (includes code sample changes)

Quality Gates

• Tests added or updated for changed behavior
• Existing tests cover changed behavior — justification:
• Tests not applicable — justification:
• Docs updated for user-facing behavior changes
• Docs not applicable — justification: restores the documented config set --restart behavior; no documented interface, flag, or output contract changes.
• Sensitive paths changed (security, policy, credentials, preflight, onboarding, inference, runner, sandbox, or messaging)
• Sensitive-path review completed or maintainer-approved waiver recorded — reviewer/approval link/justification: requesting maintainer review of the root-controller trust boundary — the lease check is the new authorization surface and is intended to fail closed on a mismatched pid/start identity, an orphaned marker, a non-root marker, or a controller whose argv is not python3 -I <helper> restart|recover <64-hex nonce>.
• Non-success, skipped, or missing CI check accepted by maintainer — check name, approval link, and follow-up issue:

Verification

• PR description includes the DCO sign-off declaration and every commit appears as Verified in GitHub
• Normal pre-commit, commit-msg, and pre-push hooks passed, or npm run check:diff passed when hooks were skipped or unavailable
• Targeted behavior tests pass for the current change set, or tests are marked not applicable above — command/result: npx vitest run test/openclaw-managed-restart-respawn.test.ts test/config-set.test.ts src/lib/actions/sandbox/gateway-restart.test.ts → 92 passed. npm run typecheck:cli and npm --prefix nemoclaw run typecheck → 0 errors.
• Applicable broad gate passed — npm test for broad runtime/test-harness changes; npm run check for repo-wide validation/coverage changes — command/result:
• Quality Gates section completed with required justifications or waivers
• No secrets, API keys, or credentials committed
• npm run docs builds without warnings (doc changes only)
• Doc pages follow the style guide (doc changes only)
• New doc pages include SPDX header and frontmatter (new pages only)

Regression coverage

test/openclaw-managed-restart-respawn.test.ts is the test that should have caught this. Each case was confirmed to fail against the unfixed code and pass after:

• Reverting the lease to Hermes-only fails "leases the gateway exit before terminating" (expected false to be true — no lease published).
• Reverting the guard in either respawn loop fails "respawns a clean gateway exit that the root controller leased".
• The lease authorization is also exercised directly on a real Linux procfs with the shipped library: a valid lease naming a live controller authorizes the respawn, while a mismatched pid, an orphaned marker (controller dead), and no marker at all each fail closed.

Not reproduced

The report notes the container stays Up (healthy). By this mechanism nemoclaw-start exiting should take the container down with it; NemoClaw sets no Docker restart policy (OpenShell creates the container), so this is most likely OpenShell restarting it. That leg was not verified and is not relied on by this fix.

---

Signed-off-by: Atulya Singh atulyarajsingh@gmail.com

Summary by CodeRabbit

• New Features

  • Managed gateway restarts now reliably distinguish authorized recoveries from unexpected exits, improving service continuity.
  • Restart behavior is supported consistently across gateway types and supervisor modes.

• Bug Fixes

  • Prevented authorized gateway shutdowns from being mistaken for failures requiring termination.
  • Configuration restart failures now clearly indicate that settings were saved but not applied, with a retry command.

• Tests

  • Added coverage for managed restart, respawn, authorization, and gateway identity validation scenarios.

[copy-pr-bot]

This pull request requires additional validation before any workflows can run on NVIDIA's runners.

Pull request vetters can view their responsibilities here.

Contributors can view more details about this message here.

[coderabbitai]

📝 Walkthrough

Walkthrough

The change adds managed expected-exit leases, validates them during gateway respawn, publishes leases for all managed agents, and improves config restart failure guidance with corresponding tests.

Changes

Managed gateway restart flow

Layer / File(s)	Summary
Expected-exit lease validation scripts/lib/gateway-supervisor.sh	Adds strict managed-controller argv and liveness validation, plus ownership, permission, identity, and marker checks for expected-exit leases.
Restart authorization and respawn wiring scripts/managed-gateway-control.py, scripts/nemoclaw-start.sh	Publishes expected-exit authorization for any agent with an existing identity and prevents authorized clean exits from terminating either respawn loop.
Config restart messaging and behavioral coverage src/lib/sandbox/config.ts, test/config-set.test.ts, test/openclaw-managed-restart-respawn.test.ts	Adds agent-specific retry guidance for failed restarts and tests lease publication, respawn behavior, lease identity matching, and error messaging.

Estimated code review effort: 4 (Complex) | ~45 minutes

Sequence Diagram(s)

sequenceDiagram
  participant ManagedController
  participant GatewaySupervisor
  participant Gateway
  participant RespawnLoop
  ManagedController->>GatewaySupervisor: publish expected-exit lease
  ManagedController->>Gateway: signal old gateway
  Gateway-->>RespawnLoop: report clean exit
  RespawnLoop->>GatewaySupervisor: validate pid and start identity
  GatewaySupervisor-->>RespawnLoop: return authorization result
  RespawnLoop->>Gateway: respawn authorized gateway

Loading

Suggested labels: integration: openclaw, area: sandbox, bug-fix

Suggested reviewers: ericksoa, cv, laitingsheng

🚥 Pre-merge checks | ✅ 5

✅ Passed checks (5 passed)

Check name	Status	Explanation
Description Check	✅ Passed	Check skipped - CodeRabbit’s high-level summary is enabled.
Title check	✅ Passed	The title matches the main change: managed OpenClaw gateway respawn after restart.
Linked Issues check	✅ Passed	The changes address #6868 by making managed restarts complete, apply config to the running agent, and improve restart handling.
Out of Scope Changes check	✅ Passed	The added validation, respawn logic, message update, and tests all support the restart fix; no clear unrelated changes stand out.
Docstring Coverage	✅ Passed	No functions found in the changed files to evaluate docstring coverage. Skipping docstring coverage check.

✨ Finishing Touches

🧪 Generate unit tests (beta)

• Create PR with unit tests

---

_{Comment @coderabbitai help to get the list of available commands.}

[github-actions]

PR Review Advisor — Informational

Advisor assessment: Informational / high confidence
Next action: No advisor follow-up needed.
Findings: 0 blockers · 0 warnings · 0 suggestions
Status: No actionable findings remain in the canonical review ledger.

Model lanes

• GPT-5.6 Terra (primary): Completed · high confidence · 0 blockers · 0 warnings · 0 suggestions
• Nemotron 3 Ultra (second opinion): Completed · high confidence · 0 blockers · 0 warnings · 0 suggestions
• Model comparison: normalized findings match; normalized E2E selections differ; severity counts match.

Nemotron output stays in workflow artifacts and does not change the assessment above.

E2E guidance

Advisory only. E2E / PR Gate selects and runs jobs independently.

Recommended E2E: None

2 optional E2E recommendations

• gateway-guard-recovery
• sandbox-operations

Workflow run details

This automated review informs maintainers. Warnings and suggestions do not require a response. A maintainer decides whether to merge.

[coderabbitai]

🧹 Nitpick comments (1)

test/openclaw-managed-restart-respawn.test.ts (1)

177-203: 📐 Maintainability & Code Quality | 🔵 Trivial | 🏗️ Heavy lift

Exercise the actual entrypoint instead of extracted shell fragments.

These tests can pass even if sourcing or surrounding lifecycle wiring prevents the shipped guards from executing. Use a sandboxed nemoclaw-start.sh harness and assert the gateway process is respawned or stopped through that public boundary.

As per path instructions, prefer observable outcomes through the public boundary over source-text or private-shape assertions.

Also applies to: 239-312

🤖 Prompt for AI Agents

Verify each finding against current code. Fix only still-valid issues, skip the
rest with a brief reason, keep changes minimal, and validate.

In `@test/openclaw-managed-restart-respawn.test.ts` around lines 177 - 203,
Replace the extractRespawnGuards source-text testing in the respawn tests with a
sandboxed nemoclaw-start.sh harness that executes the real entrypoint and
observes gateway lifecycle outcomes. Cover both guard paths, asserting leased
exits respawn the gateway and non-leased or terminal exits stop it through the
public boundary; remove source-marker and extracted-fragment assertions.

Source: Path instructions

🤖 Prompt for all review comments with AI agents

Verify each finding against current code. Fix only still-valid issues, skip the
rest with a brief reason, keep changes minimal, and validate.

Nitpick comments:
In `@test/openclaw-managed-restart-respawn.test.ts`:
- Around line 177-203: Replace the extractRespawnGuards source-text testing in
the respawn tests with a sandboxed nemoclaw-start.sh harness that executes the
real entrypoint and observes gateway lifecycle outcomes. Cover both guard paths,
asserting leased exits respawn the gateway and non-leased or terminal exits stop
it through the public boundary; remove source-marker and extracted-fragment
assertions.

---

ℹ️ Review info

⚙️ Run configuration

Configuration used: Path: .coderabbit.yaml

Review profile: CHILL

Plan: Enterprise

Run ID: 9093f075-7976-4c6f-be70-6c4c4470f668

📥 Commits

Reviewing files that changed from the base of the PR and between 3cccebf and 1252959.

📒 Files selected for processing (6)

• scripts/lib/gateway-supervisor.sh
• scripts/managed-gateway-control.py
• scripts/nemoclaw-start.sh
• src/lib/sandbox/config.ts
• test/config-set.test.ts
• test/openclaw-managed-restart-respawn.test.ts

[wscurran]

✨ Thanks for the fix. The managed restart now leases the exit and respawns the gateway so config set --restart completes normally. Ready for maintainer review.

---

Related open issues:

• #6868 [All Platforms][CLI&UX] config set --restart hangs on the gateway restart — config written but never applied to the running agent

---

Related open issues:

• #6868 [All Platforms][CLI&UX] config set --restart hangs on the gateway restart — config written but never applied to the running agent

[apurvvkumaria]

Resolved the current main merge conflict append-only in signed commit 8b755d7 while preserving the original author as co-author. The resolution keeps current managed-controller diagnostics and extends the existing exact root-authorized exit lease to OpenClaw restarts. Validation: 95 focused tests passed; repository hooks, CLI typecheck, and plugin typecheck passed; independent security review passed with no blocker; existing docs already cover the behavior. Current-head CI and automated review are now being monitored.

[apurvvkumaria]

Approved at exact head acda1f5 after resolving the main merge conflict and the source-shape gate. Independent security review passed; the expected-exit lease remains root-owned, exact-process-bound, single-use, and fail-closed. The follow-up test refactor preserves all positive and negative authorization cases while reducing TypeScript IfStatements from 8 to 0. Final exact-head result: 37 checks passed, 0 failed, 0 pending; advisor ledger is clear and review threads are resolved.