fix(backup): explain stopped sandbox remediation

[deepujain]

Summary

• keep strict pre-upgrade backup behavior intact for stopped registered sandboxes
• make the skipped stopped-sandbox message tell users to start the sandbox/container and rerun backup/install
• document the remediation path in backup and command references

Addresses #6500

Testing

• npm run build:cli
• npm run typecheck:cli
• npm run source-shape:check
• npm run lint
• npm run --silent test -- src/lib/actions/maintenance.test.ts

Summary by CodeRabbit

• New Features

  • Enhanced backup-all output to clearly explain when a sandbox is skipped because it isn’t running, including step-by-step remediation to start the sandbox/container and rerun the backup or installer.

• Documentation

  • Updated backup-all docs with explicit guidance for stopped sandboxes, including what to do after strict pre-upgrade checks report skipped, non-running instances.

• Tests

  • Expanded backup-all test coverage to confirm the new skip messaging and the additional strict failure summary for skipped non-running sandboxes.

Signed-off-by: Deepak Jain deepujain@gmail.com

[copy-pr-bot]

This pull request requires additional validation before any workflows can run on NVIDIA's runners.

Pull request vetters can view their responsibilities here.

Contributors can view more details about this message here.

[coderabbitai]

📝 Walkthrough

Walkthrough

backupAll() now identifies stopped sandboxes separately, provides restart and rerun instructions, reports strict-mode details, validates the messages in tests, and documents the remediation steps.

Changes

Backup guidance

Layer / File(s)	Summary
Stopped sandbox reporting src/lib/actions/maintenance.ts, src/lib/actions/maintenance.test.ts	backupAll() generates actionable messages for non-running sandboxes, tracks their count, reports strict-mode remediation, and tests the added output.
Backup remediation documentation docs/manage-sandboxes/backup-restore.mdx, docs/reference/commands.mdx	Documentation instructs operators to start stopped sandboxes and rerun the installer or nemoclaw backup-all.

Estimated code review effort: 2 (Simple) | ~10 minutes

Possibly related PRs

• NVIDIA/NemoClaw#3423: Both PRs modify backupAll() and its tests to update backup state handling and emitted guidance.
• NVIDIA/NemoClaw#4164: Both PRs update backup-and-restore documentation with guidance for using nemoclaw backup-all.

Suggested labels: bug-fix, area: cli, area: sandbox, area: docs

Suggested reviewers: miyoungc, cv

🚥 Pre-merge checks | ✅ 4 | ❌ 1

❌ Failed checks (1 warning)

Check name	Status	Explanation	Resolution
Linked Issues check	⚠️ Warning	The PR adds guidance, but it does not fix the strict gate or provide the explicit escape hatch required for stopped sandboxes.	Either stop counting not-running sandboxes as fatal or add a documented override plus exact remediation text for skipped sandboxes.

✅ Passed checks (4 passed)

Check name	Status	Explanation
Out of Scope Changes check	✅ Passed	All code, docs, and tests stay focused on stopped-sandbox backup messaging and strict-backup behavior.
Docstring Coverage	✅ Passed	No functions found in the changed files to evaluate docstring coverage. Skipping docstring coverage check.
Description Check	✅ Passed	Check skipped - CodeRabbit’s high-level summary is enabled.
Title check	✅ Passed	The title matches the core change: clearer remediation guidance for stopped sandbox backups.

✨ Finishing Touches

🧪 Generate unit tests (beta)

• Create PR with unit tests

---

_{Comment @coderabbitai help to get the list of available commands.}

[github-actions]

E2E Advisor Recommendation

Required E2E: state-backup-restore, upgrade-stale-sandbox
Optional E2E: None

Dispatch hint: state-backup-restore,upgrade-stale-sandbox

Workflow run

Full advisor summary

E2E Recommendation Advisor

Base: target/main
Head: HEAD
Confidence: high

Required E2E

• state-backup-restore: Required live sandbox lifecycle coverage for the backup/restore boundary affected by backup-all maintenance behavior.
• upgrade-stale-sandbox: Required real upgrade/rebuild coverage because strict backup-all is used as a pre-upgrade safety gate and its stopped-sandbox handling affects maintenance workflows.

Optional E2E

• None.

New E2E recommendations

• sandbox-lifecycle (high): Existing state-backup-restore covers backup and restore through the legacy workspace script, but does not execute nemoclaw backup-all with a registered stopped sandbox or verify installer-strict failure and remediation output.
  • Suggested test: Add a live backup-all E2E that registers one Ready and one stopped sandbox, verifies standalone backup-all reports the stopped sandbox with start/rerun guidance, then verifies NEMOCLAW_REQUIRE_ALL_SANDBOX_BACKUPS=1 fails before upgrade continuation with the strict remediation message.

Dispatch hint

• Workflow: E2E
• jobs input: state-backup-restore,upgrade-stale-sandbox

[github-actions]

E2E Target Recommendation

Required E2E targets: None
Optional E2E targets: None

Workflow run

Full E2E target advisor summary

E2E Target Advisor

Base: target/main
Head: HEAD
Confidence: high

Required E2E targets

• None. No E2E target dispatch is required. The PR changes documentation plus backup-all remediation output covered by non-E2E unit tests; the canonical live E2E targets/jobs do not exercise the changed stopped-sandbox backup-all messaging path.

Optional E2E targets

• None.

Relevant changed files

• None.

[coderabbitai]

🧹 Nitpick comments (1)

src/lib/actions/maintenance.test.ts (1)

169-170: 🎯 Functional Correctness | 🔵 Trivial | ⚡ Quick win

Assert the strict-mode rerun instruction too.

This test checks the skipped count and restart guidance, but not the new rerun the installer or ... backup-all portion emitted by maintenance.ts. Add an assertion so the complete remediation contract is covered.

As per path instructions, review tests for behavioral confidence through observable public outcomes.

Proposed assertion

 expect(errorOutput).toContain("Start each sandbox/container");
+expect(errorOutput).toContain("rerun the installer or");

🤖 Prompt for AI Agents

Verify each finding against current code. Fix only still-valid issues, skip the
rest with a brief reason, keep changes minimal, and validate.

In `@src/lib/actions/maintenance.test.ts` around lines 169 - 170, Add an assertion
in the maintenance error-output test covering the strict-mode rerun guidance
emitted by maintenance.ts, specifically the “rerun the installer or …
backup-all” remediation text. Keep the assertion based on the observable
errorOutput so the complete remediation contract is verified alongside the
skipped-count and restart-guidance assertions.

Source: Path instructions

🤖 Prompt for all review comments with AI agents

Verify each finding against current code. Fix only still-valid issues, skip the
rest with a brief reason, keep changes minimal, and validate.

Nitpick comments:
In `@src/lib/actions/maintenance.test.ts`:
- Around line 169-170: Add an assertion in the maintenance error-output test
covering the strict-mode rerun guidance emitted by maintenance.ts, specifically
the “rerun the installer or … backup-all” remediation text. Keep the assertion
based on the observable errorOutput so the complete remediation contract is
verified alongside the skipped-count and restart-guidance assertions.

---

ℹ️ Review info

⚙️ Run configuration

Configuration used: Path: .coderabbit.yaml

Review profile: CHILL

Plan: Enterprise

Run ID: 272415aa-34e1-437d-aa73-ff5cdf67c406

📥 Commits

Reviewing files that changed from the base of the PR and between e4d2e91 and 85cc0d2.

📒 Files selected for processing (4)

• docs/manage-sandboxes/backup-restore.mdx
• docs/reference/commands.mdx
• src/lib/actions/maintenance.test.ts
• src/lib/actions/maintenance.ts

[deepujain]

Added the missing strict-mode assertion for the rerun guidance and pushed it. Focused maintenance tests, source-shape, and lint/checks pass. Should be ready for another look once the refreshed checks finish.

[deepujain]

Rebased onto current main and resolved the backup test conflict by keeping the mixed-outcome accounting checks plus the stopped-sandbox remediation assertions. Focused maintenance test, source-shape, and lint/checks pass.

[prekshivyas]

PR Review Advisor — Informational

Merge posture: Informational / low confidence
Primary next action: Add or justify PRA-T1 and any related test follow-ups.
Open items: 0 required · 0 warnings · 0 suggestions · 1 test follow-up

Action checklist

• PRA-T1 Add or justify test follow-up: Runtime validation

Test follow-ups to resolve or justify

If these cover changed behavior, prefer adding them in this PR; otherwise state why existing coverage is enough or link the follow-up.

• PRA-T1 Runtime validation — Add or identify targeted runtime/integration validation for the changed behavior; do not report external E2E job pass/fail here.. Runtime/sandbox/infrastructure paths need behavioral runtime validation: docs/manage-sandboxes/backup-restore.mdx. The changed behavior is output-only handling for an existing readiness classification. Updated focused unit tests assert normal and strict stopped-sandbox remediation, while nearby tests cover gateway-preflight aborts, strict skipped-sandbox behavior, orphan-manifest boundaries, and unreachable-backup handling.

Workflow run details

This is an automated, non-binding review; it still expects maintainers and agents to respond to each required or warning item. Treat suggestions as current-PR improvements when they touch changed code; defer only with maintainer rationale or a linked follow-up. A human maintainer must make the final merge decision.

[prekshivyas]

PR Review Advisor (Nemotron Ultra) — No blocking findings

Merge posture: No blocking advisor findings
Primary next action: Add or justify PRA-T1 and any related test follow-ups.
Open items: 0 required · 0 warnings · 0 suggestions · 2 test follow-ups
Since last review: 0 prior items resolved · 0 still apply · 0 new items found

Action checklist

• PRA-T1 Add or justify test follow-up: Runtime validation
• PRA-T2 Add or justify test follow-up: Acceptance clause

Test follow-ups to resolve or justify

If these cover changed behavior, prefer adding them in this PR; otherwise state why existing coverage is enough or link the follow-up.

• PRA-T1 Runtime validation — Add or identify targeted runtime/integration validation for the changed behavior; do not report external E2E job pass/fail here.. Runtime/sandbox/infrastructure paths need behavioral runtime validation: docs/manage-sandboxes/backup-restore.mdx. Unit/mock tests cover all changed decision logic (strict gate, orphan skip, unreachable skip, mixed outcomes, gateway preflight). Runtime validation recommended only for documentation files (backup-restore.mdx, commands.mdx) which is expected for doc-only changes. No behavior-specific test gap for implementation.
• PRA-T2 Acceptance clause — Issue [NemoClaw][onboard][All Platforms] nemoclaw installer strict pre-upgrade backup fails when a registered sandbox is not running #6500 Design question (1): whether not-running skips count against strict gate — deferred by maintainer — add test evidence or identify existing coverage. jason-ma-nv (COLLABORATOR) explicitly deferred design decision; PR preserves [All Platforms][Upgrade] v0.0.55 → v0.0.76 leaves pre-existing sandboxes stuck in Provisioning/Error — user data inaccessible until manual rebuild #6114 invariant (requireAll && skipped > 0 → exit 1) unchanged; strict-gate test still expects exit 1

Workflow run details

This is an automated, non-binding review; it still expects maintainers and agents to respond to each required or warning item. Treat suggestions as current-PR improvements when they touch changed code; defer only with maintainer rationale or a linked follow-up. A human maintainer must make the final merge decision.

[github-actions]

E2E Target Results — ✅ All requested tests passed

Run: 29183055032
Workflow ref: tmp/e2e/pr-6673-6ae60850
Requested targets: (default — all supported)
Requested test IDs: upgrade-stale-sandbox,snapshot-commands
Summary: 2 passed, 0 failed, 0 cancelled, 0 skipped, 0 unknown

Test	Result
snapshot-commands	✅ success
upgrade-stale-sandbox	✅ success

[cv]

Maintainer refresh update: current main is merged into this branch and the merge commit is GitHub Verified. Refreshed CI passed build/typecheck, all eight CLI shards, installer integration, static checks, e2e-support, macOS, docs, DCO, commit lint, and security checks. The sole failure is plugin coverage inherited from current main: 98.66% functions against the new 99% floor. #6706 contains the separate schema regression test that reproduces locally at 99.10% with all 557 plugin tests passing. Keeping this PR blocked until #6706 lands and this branch receives the next main refresh.