Skip to content
Merged
Show file tree
Hide file tree
Changes from 7 commits
Commits
File filter

Filter by extension

Filter by extension

Conversations
Failed to load comments.
Loading
Jump to
Jump to file
Failed to load files.
Loading
Diff view
Diff view
8 changes: 8 additions & 0 deletions ci/env-var-doc-allowlist.json
Original file line number Diff line number Diff line change
Expand Up @@ -59,6 +59,14 @@
"name": "NEMOCLAW_TEST_NO_SLEEP",
"reason": "Test sentinel that bypasses real-time sleep() calls in onboard inference probes. Set to '1' only by Vitest tests; never user-set."
},
{
"name": "NEMOCLAW_TEST_BASE_HOME",
"reason": "Internal Vitest-only baseline used with NEMOCLAW_TEST_STATE_DIR so tests that explicitly replace HOME retain their fixture paths. Never user-set in production."
},
{
"name": "NEMOCLAW_TEST_STATE_DIR",
"reason": "Internal Vitest-only state root that keeps lifecycle locks and Shields artifacts out of the caller's real NemoClaw state. The production resolver honors it only while Vitest is active."
},
{
"name": "NEMOCLAW_TELEGRAM_STARTUP_GRACE_MS",
"reason": "Internal Vitest-only override that shortens the Telegram diagnostics startup-grace timer. Production uses the built-in default."
Expand Down
2 changes: 1 addition & 1 deletion ci/source-architecture-budget.json
Original file line number Diff line number Diff line change
Expand Up @@ -24,7 +24,7 @@
"src/lib/inference/web-search.ts": 21,
"src/lib/messaging/channels/index.ts": 26,
"src/lib/onboard/gateway-binding.ts": 48,
"src/lib/runner.ts": 89,
"src/lib/runner.ts": 88,
"src/lib/security/redact.ts": 51,
"src/lib/state/onboard-session.ts": 36,
"src/lib/state/registry.ts": 101,
Expand Down
5 changes: 5 additions & 0 deletions ci/source-shape-test-budget.json
Original file line number Diff line number Diff line change
Expand Up @@ -721,6 +721,11 @@
"test": "wires cleanup into root and standalone plugin test runs",
"category": "compatibility"
},
{
"file": "test/vitest-temp-root.test.ts",
"test": "isolates stateful non-live projects without redirecting live E2E state",
"category": "security"
},
{
"file": "test/vitest-watch-triggers.test.ts",
"test": "registers the focused mappings at the root configuration boundary (#6692)",
Expand Down
20 changes: 14 additions & 6 deletions docs/manage-sandboxes/backup-restore.mdx
Original file line number Diff line number Diff line change
Expand Up @@ -91,8 +91,14 @@ NemoClaw computes versions (`v1`, `v2`, through `vN`) from timestamp order, so `
`snapshot create` requires shields to be down.
Snapshot creation and restore share the per-sandbox transition lock with the shields auto-restore timer.

If a timed shields-down window expires during snapshot work, auto-restore can interrupt the operation and restore lockdown instead of allowing state or policy changes to continue past the deadline.
Retry the snapshot in a new shields-down window if the deadline interrupts it.
If a timed shields-down window expires during snapshot work, auto-restore closes the per-sandbox lifecycle deadline gate.
The gate blocks new mutations and waits for the recorded live owner to release its exact lock generation.
NemoClaw does not signal the snapshot process.
If ownership becomes ambiguous, NemoClaw attempts to record durable containment and reports exact-generation recovery guidance.
If the containment commit fails, NemoClaw retains any exact lifecycle and deadline gates it already owns.
A state-directory failure that prevented gate publication also prevents normal mutation-lock acquisition.
Correct the reported state-directory write failure, then retry a NemoClaw command for the sandbox to obtain exact-generation recovery guidance.
Stop all NemoClaw processes for the sandbox, then follow that guidance before you retry the snapshot.

Tag a snapshot with a human-readable label:

Expand Down Expand Up @@ -207,11 +213,13 @@ If a registered docker-driver sandbox's container is stopped, `backup-all` start
If the container cannot be returned to the stopped state, the backup run fails and reports that the container was left running.
If a sandbox is not running and its container cannot be started this way, start the sandbox or its container and rerun `$$nemoclaw backup-all`.

When Shields are UP for an eligible sandbox, `backup-all` opens a 30-minute shields-down window before it creates that sandbox's snapshot.
When an eligible sandbox starts with Shields up, `backup-all` acquires the lifecycle lock and opens a 30-minute shields-down window.
A sandbox that starts with Shields down remains down.
An unlock failure marks that sandbox as failed, and `backup-all` continues with the next sandbox.
Because the timer does not defer to the backup process, it can restore lockdown when the 30-minute deadline expires.
NemoClaw always attempts to restore Shields lockdown before it processes the next sandbox, including when the backup fails.
`backup-all` reacquires the lock under that exact timer generation while it copies sandbox state.
After the copy, it uses a third lock interval with a 30-second acquisition budget to restore the previous Shields posture.
If the timer expires during the copy, the deadline gate blocks new mutations and waits for the exact backup owner to finish without signaling it.
An initial lock or unlock failure marks that sandbox as failed, and `backup-all` continues with the next sandbox.
NemoClaw attempts to restore the previous Shields posture before it processes the next sandbox, including when the backup fails.
If lockdown cannot be restored, `backup-all` stops and does not process the remaining sandboxes.
Correct the reported issue, run the printed `$$nemoclaw <name> shields up` command, and rerun `$$nemoclaw backup-all`.

Expand Down
28 changes: 26 additions & 2 deletions docs/manage-sandboxes/runtime-controls.mdx
Original file line number Diff line number Diff line change
Expand Up @@ -108,10 +108,34 @@ Run `$$nemoclaw <name> shields down` before the change, then restore lockdown wi
NemoClaw serializes host-side config and inference writes, snapshot mutation, sandbox destruction, and shields transitions for each sandbox.
When `shields down --timeout` is active, each mutation binds to that exact timer generation so a replaced or expired timer cannot race a later command or a new sandbox that reuses the same name.

If the timeout expires while a mutation is still changing sandbox state, auto-restore can stop that exact process tree, reclaim the transition, and restore the restrictive policy and config posture.
If the timeout expires while a mutation is changing sandbox state, auto-restore closes the per-sandbox lifecycle deadline gate.
The gate blocks new mutations and waits for the recorded live owner to release its exact lock generation.
NemoClaw does not signal that process because portable process inspection cannot prove that all descendants stopped.
After the owner releases the lock, auto-restore restores the restrictive policy and config posture.
The ownership check includes both the process ID and process start identity so PID reuse does not grant control over an unrelated process.

Retry a command that the auto-restore deadline interrupts after you open a new shields-down window.
Before a manual Shields transition replaces a policy, NemoClaw requires exact Model Context Protocol (MCP) agreement among the sandbox registry, generated-policy record, and live gateway policy.
`shields down` carries the proven managed MCP policy entries into the relaxed policy.
Restoration removes snapshot-time managed MCP entries before it overlays current exact entries.

Copy link
Copy Markdown
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

📐 Maintainability & Code Quality | 🟡 Minor | ⚡ Quick win

Hyphenate the compound modifier.

Change snapshot-time managed MCP entries to snapshot-time-managed MCP entries.

🧰 Tools
🪛 LanguageTool

[grammar] ~119-~119: Use a hyphen to join words.
Context: ...olicy. Restoration removes snapshot-time managed MCP entries before it overlays c...

(QB_NEW_EN_HYPHEN)

🤖 Prompt for AI Agents
Verify each finding against current code. Fix only still-valid issues, skip the
rest with a brief reason, keep changes minimal, and validate.

In `@docs/manage-sandboxes/runtime-controls.mdx` at line 119, Update the sentence
in the runtime controls documentation to hyphenate the compound modifier,
changing “snapshot-time managed MCP entries” to “snapshot-time-managed MCP
entries.”

Source: Linters/SAST tools

If exact agreement is absent, a manual Shields transition refuses the replacement policy.
At an expired deadline, auto-restore omits unproven managed MCP policy entries, restores lockdown, and records the omission count in its audit entry.
An MCP server removed during the shields-down window stays removed.
A surviving server keeps its recorded endpoint and address pins while its policy ownership remains exact.

An interactive command can take over an expired timer.
The interactive takeover makes up to 7 restoration attempts.
The detached auto-restore timer makes up to 7 total recovery attempts across deadline setup and restoration.
Both paths wait 5 seconds between failed attempts, for up to 30 seconds of retry delay.
The deadline gate remains closed during those attempts.
If restoration cannot commit, NemoClaw attempts to record durable containment.
If that containment commit also fails, NemoClaw retains any exact lifecycle and deadline gates it already owns.
A state-directory failure that prevented gate publication also prevents normal mutation-lock acquisition.
Correct the reported state-directory write failure, then retry a NemoClaw command for the sandbox to obtain exact-generation recovery guidance.
The interactive command returns an error, or the detached timer exits with a failure status.
NemoClaw also records durable containment when an owner exits before it can prove that the owner's descendants stopped, or when ownership becomes ambiguous.
Durable containment, retained exact gates, or the fail-closed state-directory error keeps new mutations blocked until you complete exact-generation operator recovery.
Stop all NemoClaw processes for the sandbox, then follow the paths, identities, tokens, and removal order in the reported error.
Verify each recorded generation is unchanged, remove only the exact stale generations first, and remove the exact containment generation last.

## Related Topics

Expand Down
48 changes: 44 additions & 4 deletions docs/reference/commands.mdx
Original file line number Diff line number Diff line change
Expand Up @@ -1173,8 +1173,31 @@ Follow [Deep Agents Config Lock Failure Recovery](troubleshooting#deep-agents-co
</AgentOnly>

Host-side config and inference writes, snapshot mutation, sandbox destruction, and shields transitions serialize per sandbox.
When a timed shields-down window reaches its deadline, auto-restore can interrupt the exact process tree holding that transition and restore lockdown.
Retry an interrupted command in a new shields-down window.
When a timed shields-down window reaches its deadline, auto-restore closes the per-sandbox lifecycle deadline gate.
The gate blocks new mutations and waits for the recorded live owner to release its exact lock generation before auto-restore restores lockdown.
NemoClaw does not signal that process because portable process inspection cannot prove that all descendants stopped.
An interactive command can take over an expired timer.
The interactive takeover makes up to 7 restoration attempts.
The detached auto-restore timer makes up to 7 total recovery attempts across deadline setup and restoration.
Both paths wait 5 seconds between failed attempts, for up to 30 seconds of retry delay.
The deadline gate remains closed during those attempts.
If restoration cannot commit, NemoClaw attempts to record durable containment.
If that containment commit also fails, NemoClaw retains any exact lifecycle and deadline gates it already owns.
A state-directory failure that prevented gate publication also prevents normal mutation-lock acquisition.
Correct the reported state-directory write failure, then retry a NemoClaw command for the sandbox to obtain exact-generation recovery guidance.
The interactive command returns an error, or the detached timer exits with a failure status.
NemoClaw also records durable containment when an owner exits before it can prove that the owner's descendants stopped, or when ownership becomes ambiguous.
Durable containment, retained exact gates, or the fail-closed state-directory error blocks new mutations until you complete exact-generation operator recovery.
Stop all NemoClaw processes for the sandbox, then follow the paths, identities, tokens, and removal order in the reported error.
Verify each recorded generation is unchanged, remove only the exact stale generations first, and remove the exact containment generation last.

Before a manual Shields transition replaces a policy, NemoClaw requires exact agreement among the sandbox registry, generated-policy record, and live gateway policy.
`shields down` carries the proven managed MCP policy entries into the relaxed policy.
Restoration removes snapshot-time managed MCP entries before it overlays current exact entries.
If exact agreement is absent, a manual Shields transition refuses the replacement policy.
At an expired deadline, auto-restore omits unproven managed MCP policy entries, restores lockdown, and records the omission count in its audit entry.
An MCP server removed during the shields-down window stays removed.
A surviving server keeps its recorded endpoint and address pins while its policy ownership remains exact.

<AgentOnly variant="openclaw,hermes">

Expand Down Expand Up @@ -2919,6 +2942,13 @@ A registered docker-driver sandbox whose container is stopped is started for the
If the container cannot be returned to the stopped state, the command fails and reports that the container was left running.
Sandboxes that are not running and cannot be started this way are skipped with remediation guidance.

For each eligible sandbox, `backup-all` acquires the lifecycle lock to open a 30-minute shields-down window when needed.
It reacquires the lock under that exact timer generation while it copies sandbox state.
After the copy, it uses a third lock interval with a 30-second acquisition budget to restore the previous Shields posture.
If the timer expires during the copy, the deadline gate blocks new mutations and waits for the exact backup owner to finish without signaling it.
An initial lock or unlock failure marks that sandbox as failed, and `backup-all` continues with the next sandbox.
A failure to restore the previous Shields posture stops `backup-all` before it processes another sandbox.

```bash
$$nemoclaw backup-all
```
Expand All @@ -2938,7 +2968,12 @@ A skipped sandbox's uncommitted state is not included in its last successful bac
Create a timestamped snapshot of sandbox state.
Snapshots are stored in `~/.nemoclaw/rebuild-backups/<name>/`.
The command requires shields to be down and keeps the shields check and backup under one per-sandbox transition.
An expired auto-restore timer can interrupt a long-running backup and restore lockdown.
If the timer expires during a long-running backup, the deadline gate blocks new mutations and waits for the exact backup owner to finish.
Auto-restore does not signal the backup process.
If ownership becomes ambiguous, NemoClaw attempts to record durable containment and reports exact-generation recovery guidance.
If the containment commit fails, NemoClaw retains any exact lifecycle and deadline gates it already owns.
A state-directory failure that prevented gate publication also prevents normal mutation-lock acquisition.
Correct the reported state-directory write failure, then retry a NemoClaw command for the sandbox to obtain exact-generation recovery guidance.
When the sandbox has active baseline exclusions, successful output lists their keys and repeats that excluded egress leaves dependent agent features unsupported for that sandbox.

```bash
Expand Down Expand Up @@ -2972,7 +3007,12 @@ If no selector is provided, the latest snapshot is used.
Restore removes files added after the snapshot only from state directories selected for cleanup.
It preserves directories that exist only in the target manifest or whose backup failed.
The state replacement, mutable-config permission repair, and policy reconciliation run under the same per-sandbox transition.
An expired auto-restore timer can interrupt that work and restore lockdown.
If the timer expires during that work, the deadline gate blocks new mutations and waits for the exact restore owner to finish.
Auto-restore does not signal the restore process.
If ownership becomes ambiguous, NemoClaw attempts to record durable containment and reports exact-generation recovery guidance.
If the containment commit fails, NemoClaw retains any exact lifecycle and deadline gates it already owns.
A state-directory failure that prevented gate publication also prevents normal mutation-lock acquisition.
Correct the reported state-directory write failure, then retry a NemoClaw command for the sandbox to obtain exact-generation recovery guidance.

The selector accepts any of:

Expand Down
2 changes: 1 addition & 1 deletion scripts/checks/openshell-policy-mutation-read.mts
Original file line number Diff line number Diff line change
Expand Up @@ -58,7 +58,7 @@ export const MUTATION_READS: readonly AuditedMutationRead[] = [
},
{
relativePath: "src/lib/shields/index.ts",
expectedReadCalls: 1,
expectedReadCalls: 3,
Comment thread
coderabbitai[bot] marked this conversation as resolved.
Resolved
baseCommand: "runCapture(buildPolicyGetCommand(sandboxName))",
unsafeBaseCommand: "runCapture(buildPolicyGetCommand(sandboxName), {",
fullCommand: "runCapture(buildPolicyGetFullCommand(sandboxName))",
Expand Down
3 changes: 1 addition & 2 deletions src/commands/sandbox/shields/status.ts
Original file line number Diff line number Diff line change
Expand Up @@ -4,7 +4,6 @@
import { NemoClawCommand } from "../../../lib/cli/nemoclaw-oclif-command";
import { sandboxNameArg } from "../../../lib/sandbox/command-support";
import * as shields from "../../../lib/shields/index";
import { withSandboxMutationLock } from "../../../lib/state/mcp-lifecycle-lock";

export default class ShieldsStatusCommand extends NemoClawCommand {
static id = "sandbox:shields:status";
Expand All @@ -18,6 +17,6 @@ export default class ShieldsStatusCommand extends NemoClawCommand {

public async run(): Promise<void> {
const { args } = await this.parse(ShieldsStatusCommand);
await withSandboxMutationLock(args.sandboxName, () => shields.shieldsStatus(args.sandboxName));
shields.shieldsStatus(args.sandboxName);
}
}
5 changes: 1 addition & 4 deletions src/commands/sandbox/shields/up.ts
Original file line number Diff line number Diff line change
Expand Up @@ -4,7 +4,6 @@
import { NemoClawCommand } from "../../../lib/cli/nemoclaw-oclif-command";
import { sandboxNameArg } from "../../../lib/sandbox/command-support";
import * as shields from "../../../lib/shields/index";
import { withSandboxMutationLock } from "../../../lib/state/mcp-lifecycle-lock";

export default class ShieldsUpCommand extends NemoClawCommand {
static id = "sandbox:shields:up";
Expand All @@ -18,8 +17,6 @@ export default class ShieldsUpCommand extends NemoClawCommand {

public async run(): Promise<void> {
const { args } = await this.parse(ShieldsUpCommand);
await withSandboxMutationLock(args.sandboxName, () =>
shields.shieldsUp(args.sandboxName, { throwOnError: true }),
);
shields.shieldsUp(args.sandboxName, { throwOnError: true });
}
}
Loading
Loading