test(rebuild): prove live recovery boundaries - #6574
Conversation
Signed-off-by: Julie Yaunches <jyaunches@nvidia.com>
|
Auto-sync is disabled for draft pull requests in this repository. Workflows must be run manually. Contributors can view more details about this message here. |
|
Important Review skippedDraft detected. Please check the settings in the CodeRabbit UI or the ⚙️ Run configurationConfiguration used: Path: .coderabbit.yaml Review profile: CHILL Plan: Enterprise Run ID: You can disable this status message by setting the Use the checkbox below for a quick retry:
✨ Finishing Touches🧪 Generate unit tests (beta)
Comment |
PR Review Advisor — No blocking findingsMerge posture: No blocking advisor findings Action checklist
Test follow-ups to resolve or justifyIf these cover changed behavior, prefer adding them in this PR; otherwise state why existing coverage is enough or link the follow-up.
This is an automated, non-binding review; it still expects maintainers and agents to respond to each required or warning item. Treat suggestions as current-PR improvements when they touch changed code; defer only with maintainer rationale or a linked follow-up. A human maintainer must make the final merge decision. |
E2E Advisor RecommendationRequired E2E: Dispatch hint: Full advisor summaryE2E Recommendation AdvisorBase: Required E2E
Optional E2E
New E2E recommendations
Dispatch hint
|
E2E Target RecommendationRequired E2E targets: Dispatch required E2E targets:
Full E2E target advisor summaryE2E Target AdvisorBase: Required E2E targets
Optional E2E targets
Relevant changed files
|
PR Review Advisor (Nemotron Ultra) — InformationalMerge posture: Informational / low confidence Action checklist
Findings index
Review findings by urgency: 0 required fixes, 1 item to resolve/justify, 0 in-scope improvements
|
Code Coverage OverviewLanguages: TypeScript TypeScript / code-coverage/pluginThe overall coverage remains at 96%, unchanged from the TypeScript / code-coverage/cliThe overall coverage in the Show a code coverage summary of the most impacted files.
Updated |
E2E Target Results — ❌ Some jobs failedRun: 29019873920
|
Signed-off-by: Julie Yaunches <jyaunches@nvidia.com>
Signed-off-by: Julie Yaunches <jyaunches@nvidia.com>
E2E Target Results —
|
| Job | Result |
|---|---|
| agent-turn-latency | |
| bedrock-runtime-compatible-anthropic | |
| brave-search | |
| channels-add-remove | |
| channels-stop-start | |
| cloud-inference | |
| cloud-onboard | |
| common-egress-agent | |
| concurrent-gateway-ports | |
| credential-migration | |
| credential-sanitization | |
| cron-preflight-inference-local | |
| device-auth-health | |
| diagnostics | |
| docs-validation | |
| double-onboard | |
| full-e2e | |
| gateway-drift-preflight | |
| gateway-guard-recovery | |
| gateway-health-honest | |
| generate-matrix | |
| gpu-double-onboard | |
| gpu-e2e | |
| hermes-dashboard | |
| hermes-discord | |
| hermes-e2e | |
| hermes-gpu-startup | |
| hermes-inference-switch | |
| hermes-shields-config | |
| hermes-slack | |
| inference-routing | |
| issue-2478-crash-loop-recovery | |
| issue-4434-tui-unreachable-inference | |
| issue-4462-scope-upgrade-approval | |
| jetson-nvmap-gpu | |
| kimi-inference-compat | |
| launchable-smoke | |
| live | |
| mcp-bridge | |
| mcp-bridge-dev | |
| messaging-compatible-endpoint | |
| messaging-providers | |
| model-router-provider-routed-inference | |
| network-policy | |
| ollama-auth-proxy | |
| onboard-negative-paths | |
| onboard-repair | |
| onboard-resume | |
| openclaw-discord-pairing | |
| openclaw-inference-switch | |
| openclaw-skill-cli | |
| openclaw-slack-pairing | |
| openclaw-tui-chat-correlation | |
| openshell-gateway-auth-contract | |
| openshell-gateway-upgrade | |
| openshell-version-pin | |
| overlayfs-autofix | |
| rebuild-hermes | |
| rebuild-hermes-stale-base | |
| rebuild-openclaw | |
| sandbox-operations | |
| sandbox-rebuild | |
| sandbox-rlimits-connect | |
| sandbox-survival | |
| security-posture | |
| sessions-agents-cli | |
| shields-config | |
| skill-agent | |
| snapshot-commands | |
| spark-install | |
| state-backup-restore | |
| telegram-injection | |
| token-rotation | |
| tunnel-lifecycle | |
| upgrade-stale-sandbox |
Explicit-only jobs skipped: .
Signed-off-by: Julie Yaunches <jyaunches@nvidia.com>
E2E Target Results — ❌ Some jobs failedRun: 29022741069
|
Signed-off-by: Julie Yaunches <jyaunches@nvidia.com>
Signed-off-by: Julie Yaunches <jyaunches@nvidia.com>
E2E Target Results —
|
| Job | Result |
|---|---|
| agent-turn-latency | |
| bedrock-runtime-compatible-anthropic | |
| brave-search | |
| channels-add-remove | |
| channels-stop-start | |
| cloud-inference | |
| cloud-onboard | |
| common-egress-agent | |
| concurrent-gateway-ports | |
| credential-migration | |
| credential-sanitization | |
| cron-preflight-inference-local | |
| device-auth-health | |
| diagnostics | |
| docs-validation | |
| double-onboard | |
| full-e2e | |
| gateway-drift-preflight | |
| gateway-guard-recovery | |
| gateway-health-honest | |
| generate-matrix | ✅ success |
| gpu-double-onboard | |
| gpu-e2e | |
| hermes-dashboard | |
| hermes-discord | |
| hermes-e2e | |
| hermes-gpu-startup | ⏭️ skipped |
| hermes-inference-switch | |
| hermes-shields-config | |
| hermes-slack | |
| inference-routing | |
| issue-2478-crash-loop-recovery | |
| issue-4434-tui-unreachable-inference | |
| issue-4462-scope-upgrade-approval | |
| jetson-nvmap-gpu | ⏭️ skipped |
| kimi-inference-compat | |
| launchable-smoke | |
| live | |
| mcp-bridge | |
| mcp-bridge-dev | ⏭️ skipped |
| messaging-compatible-endpoint | |
| messaging-providers | |
| model-router-provider-routed-inference | |
| network-policy | |
| ollama-auth-proxy | |
| onboard-negative-paths | |
| onboard-repair | |
| onboard-resume | |
| openclaw-discord-pairing | |
| openclaw-inference-switch | |
| openclaw-skill-cli | |
| openclaw-slack-pairing | |
| openclaw-tui-chat-correlation | |
| openshell-gateway-auth-contract | ⏭️ skipped |
| openshell-gateway-upgrade | |
| openshell-version-pin | |
| overlayfs-autofix | |
| rebuild-hermes | |
| rebuild-hermes-stale-base | |
| rebuild-openclaw | |
| sandbox-operations | |
| sandbox-rebuild | |
| sandbox-rlimits-connect | ⏭️ skipped |
| sandbox-survival | |
| security-posture | |
| sessions-agents-cli | |
| shields-config | |
| skill-agent | |
| snapshot-commands | |
| spark-install | |
| state-backup-restore | |
| telegram-injection | |
| token-rotation | |
| tunnel-lifecycle | |
| upgrade-stale-sandbox |
Explicit-only jobs skipped:
openshell-gateway-auth-contract(default dispatch excludes the resource-heavy OpenShell auth-contract probe unless selected; validate withjobs=openshell-gateway-auth-contractortargets=openshell-gateway-auth-contract),mcp-bridge-dev(default dispatch excludes moving OpenShell dev artifacts unless explicitly selected; validate withjobs=mcp-bridge-devortargets=mcp-bridge-dev),hermes-gpu-startup(default dispatch excludes this explicit-only job unless selected; validate withjobs=hermes-gpu-startuportargets=hermes-gpu-startup),sandbox-rlimits-connect(default dispatch excludes the destructive rlimit fork/connect probe unless selected; validate withjobs=sandbox-rlimits-connectortargets=sandbox-rlimits-connect),jetson-nvmap-gpu(default dispatch excludes Jetson until a stable Jetson runner is available; validate withjobs=jetson-nvmap-gpuortargets=jetson-nvmap-gpu).
Signed-off-by: Julie Yaunches <jyaunches@nvidia.com>
Signed-off-by: Julie Yaunches <jyaunches@nvidia.com>
E2E Target Results — ❌ Some jobs failedRun: 29025419580
|
Signed-off-by: Julie Yaunches <jyaunches@nvidia.com>
E2E Target Results — ❌ Some jobs failedRun: 29028975951
|
E2E Target Results — ❌ Some jobs failedRun: 29028975951
|
|
Closing this draft stack as requested while the broader human-and-agent lifecycle architecture is reconsidered. |
Summary
This stacked draft turns the existing
sandbox-rebuildlive target into the rollout proof for both destructive recovery forks: recreation after durable deletion and adoption after unjournaled replacement creation. It uses the public rebuild command for both resumes and adds no workflow job or recovery engine. After review-driven fixture consolidation and the production defects found by the live test, the full PR is+1104/-193(net+911), almost entirely test and fixture coverage; production code is+15/-9(net+6) versus #6522.Related Issue
Closes #6437.
Part of #6433.
Changes
old_deletedtransition while preserving the existing hermetic interruption points.SIGSTOPped, captures redacted artifacts, and verifies the subsequentSIGKILL.sandbox-rebuildtarget to recover two distinct generations on one provisioned sandbox, prove workspace restoration, both observed completions with latest-only durable tombstone semantics, host/runtime version agreement, exact registry/default ownership, and replacement adoption without another create/delete.upgrade-sandboxesrecovery coverage to prove an existingold_deletedtransaction reaches completion through the installer runner and that a real unsupported transaction file blocks only its sandbox while an unrelated transaction completes; extract its shared harness so the prior monolith shrinks from 566 to 427 lines.SIGKILLverification.snapshotVersionmetadata from durable backup receipt fingerprints; the live test proved that otherwise a process restart rejects its own validold_deletedtransaction before recovery.old_deletedsandbox when the preserved registry row matches the transaction's normalized target; this remains a create path, not adoption, and still refuses uncorrelated live replacements.+15/-9(net+6), tests+1085/-184(net+901), and test-only config+4/-0; no workflow files change.+2055, tests net+3396, and total net+5459; Epic: make sandbox rebuild resumable across host-process interruption #6433 therefore still requires a concrete consolidation child before completion.Type of Change
Quality Gates
Verification
Verifiedin GitHubpre-commit,commit-msg, andpre-pushhooks passed, ornpm run check:diffpassed when hooks were skipped or unavailablenpm run typecheck:cli; 78 focused CLI tests; 43 E2E-support tests; 6 process-death integration tests;npm run test:projects:check;npm run test:titles:check.npm testfor broad runtime/test-harness changes;npm run checkfor repo-wide validation/coverage changes — not claimed:npm run test:fastdeterministically loses the generated oclif manifest afterclean:clion both this branch and exact base refactor(rebuild): resume transaction-owned replacements #6522; a build-first local pass reached 8510/8533 with the remaining failures attributable to local macOS/PyYAML prerequisites and parallel five-second timeouts. Required CI remains authoritative.npm run docsbuilds without warnings (doc changes only)Signed-off-by: Julie Yaunches jyaunches@nvidia.com