fix(sandbox): restore startup before readiness

docs/deployment/deploy-to-headless-server.mdx

@@ -303,8 +303,9 @@ Model traffic uses the OpenShell-managed `inference.local` route.
 
 <AgentOnly variant="openclaw">
 
-OpenClaw generates a new gateway token each time the sandbox container starts.
-Retrieve the dashboard URL or token again after the container restarts or a replacement sandbox is created.
+OpenClaw generates a new gateway token when the sandbox container starts with mutable configuration.
+If Shields are up, a non-root start preserves the sealed token because the sandbox user cannot replace the protected configuration.
+Retrieve the dashboard URL or token again after the container starts or a replacement sandbox is created.
 
 </AgentOnly>
 <AgentOnly variant="hermes">
@@ -389,7 +390,7 @@ Use `$$nemoclaw headless-agent rebuild` when you need the current agent image wh
 | Manually installed system or global packages | Usually remain in the same writable layer | Not preserved | Not preserved |
 | Direct edits to generated profile, config, or environment files | May remain until regeneration | Agent-specific and usually excluded or filtered | Regenerated or filtered by the current manifest |
 <AgentOnly variant="openclaw">
-| OpenClaw gateway token | Rotated when the container starts | Not captured; a replacement sandbox generates a new token | Rotated for the replacement sandbox |
+| OpenClaw gateway token | Rotated when the container starts with mutable configuration; preserved for a non-root start while Shields are up | Not captured; a replacement sandbox generates a new token | Rotated for the replacement sandbox |
 </AgentOnly>
 <AgentOnly variant="hermes">
 | Hermes `API_SERVER_KEY` | Preserved | Not captured; a replacement sandbox generates a new token | Rotated for the replacement sandbox |

docs/manage-sandboxes/run-sandboxes.mdx

@@ -132,10 +132,11 @@ $$nemoclaw <sandbox-name> start
 ```
 
 <AgentOnly variant="openclaw,hermes">
-NemoClaw restarts the container and repairs the in-sandbox gateway and host forwards.
+After Docker reports the existing container as running, NemoClaw checks the selected agent's managed startup state and recovers missing processes when needed.
+NemoClaw completes this recovery before its final OpenShell readiness and host-forward checks.
 </AgentOnly>
 <AgentOnly variant="deepagents">
-NemoClaw restarts the container so the managed terminal runtime can run again.
+After Docker reports the existing container as running, NemoClaw verifies the managed terminal runtime before the command reports success.
 </AgentOnly>
 Refer to [`$$nemoclaw <name> stop`](../../reference/commands#$$nemoclaw-name-stop) and [`$$nemoclaw <name> start`](../../reference/commands#$$nemoclaw-name-start) for details.
 Use [`$$nemoclaw <name> destroy`](../../reference/commands#$$nemoclaw-name-destroy) when you want to delete the sandbox instead.

docs/security/best-practices.mdx

@@ -297,8 +297,10 @@ After the top-level config binding is frozen, lockdown makes containment monoton
 It removes unsafe symlinks, special entries, and protected-root names that are not directories through descriptor-relative operations without following their targets.
 For protected regular files, lockdown publishes a fresh inode, severing hardlinks while preserving file content, read/execute mode, timestamps, and supported extended attributes; this also revokes write authority held through a descriptor opened before `shields up`.
 The OpenClaw gateway (a member of the `sandbox` group) keeps read access to plugin and agent code; the sandbox user can no longer write them.
-The same workflow also locks the secret-bearing directories (`credentials`, `identity`, `pairing`) to `root:root 700` with `chmod -R go-rwX`.
-Neither the sandbox user nor the gateway can read those secrets while the lock is active.
+The same workflow locks `identity`, `pairing`, and a non-empty `credentials` directory to `root:root 0700`.
+An empty `credentials` directory uses `root:sandbox 0710` so OpenClaw can confirm that optional credential files are absent during startup.
+The sandbox group cannot list, create, or remove entries, and it cannot read credential files.
+Neither the sandbox user nor the gateway can read stored secrets while the lock is active.
 Restoring the mutable-default posture returns those directories to `sandbox:sandbox 2770`.
 The list is the union of state directories declared by every shipped agent manifest.
 The lock helper silently skips dirs that are not present in a given agent's config tree.

docs/security/credential-storage.mdx

@@ -16,7 +16,15 @@ When you provide a provider credential, either interactively during `$$nemoclaw
 The gateway stores the credential and the OpenShell L7 proxy substitutes it into outbound requests at egress, so sandboxed agents see placeholders instead of the raw secret.
 
 <AgentOnly variant="openclaw">
-The sandbox-side OpenClaw gateway token is generated at container startup and is not rotated through provider credential commands.
+The sandbox-side OpenClaw gateway token is generated when the container starts with mutable configuration.
+Provider credential commands do not rotate this token.
+A non-root start while Shields are up preserves the sealed token because the sandbox user cannot replace the protected configuration.
+If the sealed gateway token or required auth profile is unavailable, the start fails instead of weakening the sealed configuration.
+The error tells you to lower Shields before you restart.
+When the sealed `credentials` directory is empty, NemoClaw grants the sandbox group search-only access with mode `0710`.
+This access lets OpenClaw confirm that optional credential files are absent during start.
+The sandbox group cannot list, create, or remove entries, and it cannot read credential files.
+A non-empty `credentials` directory remains root-only.
 </AgentOnly>
 <AgentOnly variant="hermes">
 NemoClaw manages Hermes API credentials and provider credentials through the same OpenShell provider boundary.

scripts/nemoclaw-start.sh

@@ -2409,6 +2409,21 @@ needs_gateway_token_for_current_command() {
 
 prepare_gateway_token_for_current_command() {
   if [ ${#NEMOCLAW_CMD[@]} -eq 0 ]; then
+    # OpenShell launches the persisted workload as the sandbox user. When
+    # Shields are up, the root-owned config seal deliberately prevents that
+    # identity from replacing openclaw.json. Preserve the sealed startup token
+    # rather than weakening the lock; mutable and root-owned startup paths keep
+    # rotating it. A sealed config without a token cannot be repaired safely by
+    # this identity, so fail before attempting a write.
+    if [ "$(id -u)" -ne 0 ] \
+      && [ "$(openclaw_config_dir_owner /sandbox/.openclaw)" = "root" ]; then
+      if [ -n "$(_read_gateway_token)" ]; then
+        printf '[token] Shields are up; preserving the sealed gateway token for startup\n' >&2
+        return 0
+      fi
+      printf '[SECURITY] Shields are up but the sealed OpenClaw config has no gateway token; lower Shields before restarting\n' >&2
+      return 1
+    fi
     ensure_gateway_token
     return $?
   fi
@@ -2436,6 +2451,17 @@ write_auth_profile() {
   # fallback in v0.0.90.
   # See: https://github.com/NVIDIA/NemoClaw/issues/1332
   local provider_key="${NEMOCLAW_INFERENCE_PROVIDER_ID:-${NEMOCLAW_PROVIDER_KEY:-inference}}"
+  local auth_profile_path="${HOME}/.openclaw/agents/main/agent/auth-profiles.json"
+
+  if [ "$(id -u)" -ne 0 ] \
+    && [ "$(openclaw_config_dir_owner "${HOME}/.openclaw")" = "root" ]; then
+    if [ -L "$auth_profile_path" ] || [ ! -f "$auth_profile_path" ]; then
+      printf '[SECURITY] Shields are up but the sealed OpenClaw auth profile is unavailable; lower Shields before restarting\n' >&2
+      return 1
+    fi
+    printf '[auth] Shields are up; preserving the sealed OpenClaw auth profile\n' >&2
+    return 0
+  fi
 
   python3 - "$provider_key" <<'PYAUTH'
 import json
@@ -4625,6 +4651,7 @@ seed_default_workspace_templates_as_sandbox() {
 setup_auth_profile_as_sandbox() {
   run_step_down_as_sandbox \
     "export HOME=/sandbox; write_auth_profile; harden_auth_profiles" \
+    openclaw_config_dir_owner \
     write_auth_profile \
     harden_auth_profiles
 }

scripts/state-dir-guard.py

@@ -89,7 +89,7 @@
 FS_IOC_GETFLAGS = 0x80086601
 FS_IOC_SETFLAGS = 0x40086602
 
-Action = Literal["preflight", "lock", "unlock"]
+Action = Literal["preflight", "lock", "unlock", "startup"]
 Policy = Literal["high-risk", "confidentiality"]
 
 
@@ -921,6 +921,32 @@ def _set_dir_metadata(
     os.fchmod(dir_fd, _expected_dir_mode(policy, action))
 
 
+def _set_empty_credentials_startup_metadata(
+    dir_fd: int,
+    identity: Identity,
+) -> None:
+    """Allow the sandbox group to probe names in an empty sealed credentials dir."""
+
+    os.fchown(dir_fd, identity.root_uid, identity.sandbox_gid)
+    os.fchmod(dir_fd, 0o710)
+
+
+def _is_empty_credentials_root(
+    relative_dir: str,
+    policy: Policy,
+    action: Action,
+    is_root: bool,
+    names: list[str],
+) -> bool:
+    return (
+        action == "lock"
+        and policy == "confidentiality"
+        and is_root
+        and relative_dir == "credentials"
+        and not names
+    )
+
+
 def _copy_extent(
     source_fd: int,
     temp_fd: int,
@@ -1337,6 +1363,14 @@ def _mutate_dir(
         names = _bounded_directory_names(
             dir_fd, context.display(relative_dir), context.budget
         )
+        if _is_empty_credentials_root(
+            relative_dir, policy, action, is_root, names
+        ):
+            # OpenClaw probes optional credential paths while starting.  An
+            # empty directory contains no secret metadata to expose, so allow
+            # the sandbox group to traverse it without granting list, read, or
+            # write access.  Non-empty confidentiality roots remain root-only.
+            _set_empty_credentials_startup_metadata(dir_fd, identity)
     except OSError as exc:
         raise GuardOperationError(
             _os_issue(
@@ -1510,8 +1544,12 @@ def _verify_metadata(
     policy: Policy,
     action: Action,
     identity: Identity,
+    empty_credentials_root: bool = False,
 ) -> Issue | None:
-    expected_uid, expected_gid = _expected_ids(policy, action, identity)
+    if empty_credentials_root:
+        expected_uid, expected_gid = identity.root_uid, identity.sandbox_gid
+    else:
+        expected_uid, expected_gid = _expected_ids(policy, action, identity)
     if st.st_uid != expected_uid or st.st_gid != expected_gid:
         return Issue(
             "verification-owner-mismatch",
@@ -1522,7 +1560,7 @@ def _verify_metadata(
         return None
     mode = stat.S_IMODE(st.st_mode)
     if entry_type == "directory":
-        expected_mode = _expected_dir_mode(policy, action)
+        expected_mode = 0o710 if empty_credentials_root else _expected_dir_mode(policy, action)
         if mode != expected_mode:
             return Issue(
                 "verification-mode-mismatch",
@@ -1579,16 +1617,6 @@ def _verify_dir(
             )
         )
         return
-    dir_issue = _verify_metadata(
-        context.display(relative_dir),
-        os.fstat(dir_fd),
-        "directory",
-        policy,
-        action,
-        identity,
-    )
-    if dir_issue is not None:
-        issues.append(dir_issue)
     try:
         names = _bounded_directory_names(
             dir_fd, context.display(relative_dir), context.budget
@@ -1600,6 +1628,19 @@ def _verify_dir(
             )
         )
         return
+    dir_issue = _verify_metadata(
+        context.display(relative_dir),
+        os.fstat(dir_fd),
+        "directory",
+        policy,
+        action,
+        identity,
+        _is_empty_credentials_root(
+            relative_dir, policy, action, relative_dir == "credentials", names
+        ),
+    )
+    if dir_issue is not None:
+        issues.append(dir_issue)
     for name in names:
         relative_path = posixpath.join(relative_dir, name)
         path = context.display(relative_path)
@@ -1697,6 +1738,118 @@ def _verify_dir(
             )
 
 
+def _restore_empty_credentials_startup_access(
+    config_dir: str,
+    identity: Identity,
+    deadline: float,
+) -> GuardResult:
+    """Restore only the empty credentials traversal needed during startup."""
+
+    result = GuardResult(action="startup")
+    config_fd = -1
+    credentials_fd = -1
+    path = _display_path(config_dir, "credentials")
+    try:
+        config_fd = _open_absolute_dir_nofollow(config_dir)
+        config_st = os.fstat(config_fd)
+        config_mode = stat.S_IMODE(config_st.st_mode)
+        if config_st.st_uid != identity.root_uid or config_mode & 0o022:
+            result.issues.append(
+                Issue(
+                    "startup-posture-mismatch",
+                    config_dir,
+                    "sealed config directory must be root-owned and not group/world writable",
+                )
+            )
+            return result
+        try:
+            credentials_st = os.stat(
+                "credentials", dir_fd=config_fd, follow_symlinks=False
+            )
+        except FileNotFoundError:
+            return result
+        if (
+            not stat.S_ISDIR(credentials_st.st_mode)
+            or credentials_st.st_dev != config_st.st_dev
+        ):
+            result.issues.append(
+                Issue(
+                    "unsafe-startup-credentials-root",
+                    path,
+                    "credentials root must be a directory on the config filesystem",
+                )
+            )
+            return result
+        credentials_fd = _open_child_dir(config_fd, "credentials", credentials_st)
+        names = _bounded_directory_names(
+            credentials_fd, path, WorkBudget(deadline)
+        )
+        current = os.fstat(credentials_fd)
+        mode = stat.S_IMODE(current.st_mode)
+        root_only = (
+            current.st_uid == identity.root_uid
+            and current.st_gid == identity.root_gid
+            and mode == 0o700
+        )
+        startup_traversable = (
+            current.st_uid == identity.root_uid
+            and current.st_gid == identity.sandbox_gid
+            and mode == 0o710
+        )
+        if not root_only and not startup_traversable:
+            result.issues.append(
+                Issue(
+                    "startup-posture-mismatch",
+                    path,
+                    f"credentials root has unexpected owner or mode {mode:04o}",
+                )
+            )
+            return result
+        if names:
+            if startup_traversable:
+                _set_dir_metadata(
+                    credentials_fd, "confidentiality", "lock", identity
+                )
+                os.fsync(credentials_fd)
+            return result
+        _set_empty_credentials_startup_metadata(credentials_fd, identity)
+        after = os.fstat(credentials_fd)
+        if (
+            after.st_uid != identity.root_uid
+            or after.st_gid != identity.sandbox_gid
+            or stat.S_IMODE(after.st_mode) != 0o710
+        ):
+            result.issues.append(
+                Issue(
+                    "startup-verification-failed",
+                    path,
+                    "empty credentials root did not reach root:sandbox 0710",
+                )
+            )
+            return result
+        os.fsync(credentials_fd)
+        result.roots = 1
+        result.directories = 1
+        return result
+    except (OSError, GuardOperationError) as exc:
+        result.issues.append(
+            exc.issue
+            if isinstance(exc, GuardOperationError)
+            else _os_issue(
+                "startup-restore-failed",
+                path,
+                "restore empty credentials startup access",
+                exc,
+            )
+        )
+        return result
+    finally:
+        if credentials_fd >= 0:
+            os.close(credentials_fd)
+        if config_fd >= 0:
+            os.close(config_fd)
+
+
 def _run_guard_unserialized(
     action: Action,
     config_dir: str,
@@ -1707,6 +1860,10 @@ def _run_guard_unserialized(
     result = GuardResult(action=action)
     deadline = time.monotonic() + MAX_GUARD_SECONDS
     normalized_config = posixpath.normpath(config_dir)
+    if action == "startup":
+        return _restore_empty_credentials_startup_access(
+            normalized_config, identity, deadline
+        )
     fail_closed_config_root = action == "lock" and (
         normalized_config in PRODUCTION_FAIL_CLOSED_CONFIG_DIRS
         or os.environ.get("NEMOCLAW_TEST_OPENCLAW_FAIL_CLOSED") == "1"
@@ -2003,9 +2160,9 @@ def _production_identity() -> Identity:
 
 def _parse_args(argv: list[str]) -> argparse.Namespace:
     parser = argparse.ArgumentParser(
-        description="Safely preflight, lock, or unlock recursive agent state directories"
+        description="Safely manage recursive agent state directories"
     )
-    parser.add_argument("action", choices=("preflight", "lock", "unlock"))
+    parser.add_argument("action", choices=("preflight", "lock", "unlock", "startup"))
     parser.add_argument("--config-dir", required=True)
     return parser.parse_args(argv)
 

src/lib/actions/sandbox/connect.ts

@@ -891,6 +891,10 @@ function maybeEnsureHermesToolGatewayBroker(sb: SandboxEntry | null): void {
   }
 }
 
+export function restoreSandboxStartupState(sandboxName: string): void {
+  checkAndRecoverSandboxProcesses(sandboxName, { quiet: true });
+}
+
 function restoreInteractiveTerminal(): void {
   if (!process.stdin.isTTY) return;