Skip to content

fix(hermes): retire stale OpenClaw base guard - #5949

Merged
cv merged 5 commits into
mainfrom
codex/retire-hermes-stale-openclaw-guard
Jun 29, 2026
Merged

fix(hermes): retire stale OpenClaw base guard#5949
cv merged 5 commits into
mainfrom
codex/retire-hermes-stale-openclaw-guard

Conversation

@cv

@cv cv commented Jun 28, 2026

Copy link
Copy Markdown
Collaborator

Summary

Advance the default Hermes final-image base to the newly published post-stale-layout digest and retire the temporary digest-coupled repair that intentionally rejected newer published bases. This restores Hermes onboarding and live E2E builds after the latest base moved, while keeping current-state fail-closed layout checks and the older .hermes-data compatibility migration.

Related Issue

Unblocks #5947. Follow-up to #5882.

Changes

  • Pin the default Hermes base to sha256:8dad3b989a9ed1e601743310b97be21be5f59f89f7913a47d04f3ec3c40b8ce6, whose published ARM64 image has neither /sandbox/.openclaw nor /sandbox/.hermes-data.
  • Remove the fired NEMOCLAW_STALE_OPENCLAW_BASE_DIGEST guard and temporary /sandbox/.openclaw repair; replace it with a small fail-closed invariant that rejects retired OpenClaw state rather than maintaining it.
  • Keep published-image provenance in the resolver: export immutable official GHCR digests and reject candidates containing .openclaw or .hermes-data state, while preserving local rebuild and caller-selected base refs in the final Dockerfile.
  • Delete the dedicated stale-layout verifier job, script, helpers, and tests now that their documented removal trigger has fired.
  • Add a focused final-image layout suite covering OpenClaw-state refusal, the retained .hermes-data migration, and symlink refusal; keep immutable published-ref coverage at the resolver boundary and extend live secret-boundary inspection to both retired paths.
  • Reject nested symlinks before copying retained .hermes-data compatibility state so migration cannot preserve links outside the legacy tree.

Type of Change

  • Code change (feature, bug fix, or refactor)
  • Code change with doc updates
  • Doc only (prose changes, no code sample modifications)
  • Doc only (includes code sample changes)

Quality Gates

  • Tests added or updated for changed behavior
  • Existing tests cover changed behavior — justification:
  • Tests not applicable — justification:
  • Docs updated for user-facing behavior changes
  • Docs not applicable — justification: internal published-image lifecycle cleanup; CLI behavior, configuration, supported workflows, and the documented Hermes state location are unchanged.
  • Sensitive paths changed (security, policy, credentials, preflight, onboarding, inference, runner, sandbox, or messaging)
  • Sensitive-path review completed or maintainer-approved waiver recorded — reviewer/approval link/justification: pending independent review of the Hermes sandbox image boundary.
  • Non-success, skipped, or missing CI check accepted by maintainer — check name, approval link, and follow-up issue:

Verification

  • PR description includes the DCO sign-off declaration and every commit appears as Verified in GitHub
  • Git hooks passed during commit and push, or npx prek run --from-ref main --to-ref HEAD passes
  • Targeted tests pass for changed behavior
  • Full npm test passes (broad runtime changes only)
  • Quality Gates section completed with required justifications or waivers
  • No secrets, API keys, or credentials committed
  • npm run docs builds without warnings (doc changes only)
  • Doc pages follow the style guide (doc changes only)
  • New doc pages include SPDX header and frontmatter (new pages only)

Verification evidence:

  • Pulled and inspected the published ARM64 8dad… image; /sandbox/.openclaw and /sandbox/.hermes-data are absent.
  • Built agents/hermes/Dockerfile against the exact 8dad… digest and verified sandbox-user readability, runtime permissions, runtime symlinks, and absence of both stale paths.
  • Focused integration run passed 61 tests across hermes-final-image-layout, hermes-doctor-config-hash, pr-workflow-contract, and sandbox-provisioning.
  • npm run build:cli, npm run typecheck:cli, npm run checks, and npm run test:projects:check passed.
  • Normal commit and push hooks passed for all commits, including the full CLI test lane, shellcheck, hadolint, repository checks, source-shape and test-size budgets, gitleaks, commitlint, and CLI TypeScript checks.
  • Vitest E2E run 28339846140 passed Hermes install/inference, root entrypoint, and secret-boundary jobs, then exposed that Dockerfile-level registry validation rejected the forced local :latest base-cache rebuild. Commit ce85e762a moves provenance back to the resolver boundary; exact-head rerun 28340627350 passed all four jobs, including stale-base rebuild.
  • Downloaded exact-head artifacts have no credential-shaped matches; every command result in the rebuild artifact is zero, post-rebuild inference returned PONG, backup scanning found no leaks, and all scenario cleanups report no failures.
  • Required documentation writer assessment, rerun after the fail-closed review fixes, reported no user-facing docs changes needed.

Signed-off-by: Carlos Villela cvillela@nvidia.com

@cv cv self-assigned this Jun 28, 2026
@coderabbitai

coderabbitai Bot commented Jun 28, 2026

Copy link
Copy Markdown
Contributor

Review Change Stack

Note

Reviews paused

It looks like this branch is under active development. To avoid overwhelming you with review comments due to an influx of new commits, CodeRabbit has automatically paused this review. You can configure this behavior by changing the reviews.auto_review.auto_pause_after_reviewed_commits setting.

Use the following commands to manage reviews:

  • @coderabbitai resume to resume automatic reviews.
  • @coderabbitai review to trigger a single review.

Use the checkboxes below for quick actions:

  • ▶️ Resume reviews
  • 🔍 Trigger review
📝 Walkthrough

Walkthrough

The PR removes the stale OpenClaw verifier path, updates Hermes base-image retired-layout checks, and adds final-image layout tests for mutable base rejection, retired state rejection, and legacy-data migration.

Changes

Hermes base image layout validation

Layer / File(s) Summary
Base image layout validation
agents/hermes/Dockerfile, .github/actions/resolve-hermes-base-image/action.yaml, test/e2e/test-hermes-sandbox-secret-boundary.sh, test/pr-workflow-contract.test.ts
BASE_IMAGE now defaults to a pinned digest directly, the Dockerfile repair block now targets legacy .hermes-data bridging, and the resolver/action checks /sandbox/.openclaw and /sandbox/.hermes-data before accepting candidate images. The boundary inspection test now rejects those retired paths, and the contract test checks the new layout_ok output.
Test and workflow cleanup
.github/workflows/pr-self-hosted.yaml, scripts/verify-hermes-stale-openclaw-image.sh, test/helpers/hermes-dockerfile-run.ts, test/sandbox-provisioning.test.ts, test/hermes-stale-openclaw-guard.test.ts, test/pr-workflow-contract.test.ts
Removes the stale OpenClaw workflow job, deletes the verifier script and guard suite, inlines set -euo pipefail in the Docker shell helper, removes stale OpenClaw provisioning support and assertions, and drops the deleted workflow job contract test.
Final image layout regressions
test/hermes-final-image-layout.test.ts
Adds a new layout test helper and cases covering mutable base-image rejection, retired OpenClaw rejection, legacy-data migration, and legacy symlink cleanup refusal.

Estimated code review effort

🎯 4 (Complex) | ⏱️ ~45 minutes

Possibly related PRs

  • NVIDIA/NemoClaw#5882: Introduced the stale OpenClaw verifier workflow job, Dockerfile logic, and guard tests that this PR removes or replaces.

Suggested labels

integration: openclaw, integration: hermes, area: packaging, area: sandbox, platform: container, bug-fix

Suggested reviewers

  • ericksoa
🚥 Pre-merge checks | ✅ 4 | ❌ 1

❌ Failed checks (1 warning)

Check name Status Explanation Resolution
Docstring Coverage ⚠️ Warning Docstring coverage is 0.00% which is insufficient. The required threshold is 80.00%. Write docstrings for the functions missing them to satisfy the coverage threshold.
✅ Passed checks (4 passed)
Check name Status Explanation
Title check ✅ Passed The title accurately reflects the main change: retiring the stale OpenClaw base guard in Hermes.
Linked Issues check ✅ Passed Check skipped because no linked issues were found for this pull request.
Out of Scope Changes check ✅ Passed Check skipped because no linked issues were found for this pull request.
Description Check ✅ Passed Check skipped - CodeRabbit’s high-level summary is enabled.
✨ Finishing Touches
📝 Generate docstrings
  • Create stacked PR
  • Commit on current branch
🧪 Generate unit tests (beta)
  • Create PR with unit tests
  • Commit unit tests in branch codex/retire-hermes-stale-openclaw-guard

Comment @coderabbitai help to get the list of available commands.

@github-code-quality

github-code-quality Bot commented Jun 28, 2026

Copy link
Copy Markdown
Contributor

Code Coverage Overview

Languages: TypeScript

TypeScript / code-coverage/plugin

The overall coverage in the codex/retire-hermes-... branch is 96%. Coverage data for the main branch is not yet available.

Show a code coverage summary of the most covered files.
File main codex/retire-hermes-... ce85e76 +/-
nemoclaw/src/se...cret-scanner.ts 100%
nemoclaw/src/commands/slash.ts 100%
nemoclaw/src/li...bprocess-env.ts 100%
nemoclaw/src/bl...eprint/state.ts 98%
nemoclaw/src/onboard/config.ts 98%
nemoclaw/src/bl...int/snapshot.ts 97%
nemoclaw/src/bl...print/runner.ts 95%
nemoclaw/src/co...ration-state.ts 94%
nemoclaw/src/bl...ate-networks.ts 94%
nemoclaw/src/index.ts 94%

TypeScript / code-coverage/cli

The overall coverage in the codex/retire-hermes-... branch is 68%. Coverage data for the main branch is not yet available.

Show a code coverage summary of the most covered files.
File main codex/retire-hermes-... ce85e76 +/-
src/lib/actions...all/run-plan.ts 80%
src/lib/state/o...oard-session.ts 79%
src/lib/actions...dbox/rebuild.ts 74%
src/lib/state/sandbox.ts 72%
src/lib/shields/index.ts 70%
src/lib/onboard/preflight.ts 69%
src/lib/actions...licy-channel.ts 59%
src/lib/onboard...er-gpu-patch.ts 59%
src/lib/policy/index.ts 52%
src/lib/onboard.ts 20%

Updated June 29, 2026 00:11 UTC
Code Coverage is in Public Preview. Learn more and provide us with your feedback.

@github-actions

github-actions Bot commented Jun 28, 2026

Copy link
Copy Markdown
Contributor

E2E Advisor Recommendation

Required E2E: hermes-secret-boundary-e2e, hermes-root-entrypoint-smoke-e2e, rebuild-hermes-stale-base-e2e, hermes-e2e
Optional E2E: rebuild-hermes-e2e, hermes-onboard-security-posture-e2e

Dispatch hint: hermes-secret-boundary-e2e,hermes-root-entrypoint-smoke-e2e,rebuild-hermes-stale-base-e2e,hermes-e2e

Auto-dispatched E2E: hermes-secret-boundary-e2e, hermes-root-entrypoint-smoke-e2e via nightly-e2e.yaml at ce85e762aebf57f2358d75fd36108520f33070c8nightly run

Workflow run

Full advisor summary

E2E Recommendation Advisor

Base: origin/main
Head: HEAD
Confidence: high

Required E2E

  • hermes-secret-boundary-e2e (medium): Directly exercises the modified test/e2e/test-hermes-sandbox-secret-boundary.sh and validates that Hermes images do not bake raw secret-shaped values or retired /sandbox/.openclaw and /sandbox/.hermes-data state into the sandbox boundary.
  • hermes-root-entrypoint-smoke-e2e (medium): Builds the real Hermes image and verifies root-entrypoint startup, gateway-user execution, and Hermes layout repair, which are directly affected by the Dockerfile layout cleanup and base-image change.
  • rebuild-hermes-stale-base-e2e (high): Covers the Hermes rebuild path when the cached Hermes base tag is stale, which is the closest existing E2E coverage for the removed stale OpenClaw verifier and the changed base-image/layout provenance behavior.
  • hermes-e2e (high): Runs the real install → onboard --agent hermes → health probe → live inference user flow, providing end-to-end confidence that the changed Hermes final image and default base digest still produce a usable assistant sandbox.

Optional E2E

  • rebuild-hermes-e2e (high): Useful adjacent confidence for the normal Hermes rebuild/upgrade path without the explicit stale-base mode.
  • hermes-onboard-security-posture-e2e (high): Optional broader security posture check for full Hermes onboarding on a non-root host user after image layout and sandbox-state boundary changes.

New E2E recommendations

  • Hermes base resolver stale-layout rejection (high): The dedicated self-hosted stale OpenClaw verifier was removed, and the new resolver layout_ok check is only contract/unit covered. Add an E2E or self-hosted validation that builds synthetic Hermes bases containing /sandbox/.openclaw and /sandbox/.hermes-data, runs the resolver/final-image path, and proves stale retired state is rejected before dispatching a base image.
    • Suggested test: Add a Hermes base resolver stale-layout rejection E2E or restore equivalent self-hosted image validation coverage.

Dispatch hint

  • Workflow: nightly-e2e.yaml
  • jobs input: hermes-secret-boundary-e2e,hermes-root-entrypoint-smoke-e2e,rebuild-hermes-stale-base-e2e,hermes-e2e

@github-actions

github-actions Bot commented Jun 28, 2026

Copy link
Copy Markdown
Contributor

Vitest E2E Scenario Recommendation

Required Vitest E2E scenarios: hermes-sandbox-secret-boundary-vitest
Optional Vitest E2E scenarios: None

Dispatch required Vitest E2E scenarios:

  • gh workflow run e2e-vitest-scenarios.yaml --ref <pr-head-ref> --field jobs=hermes-sandbox-secret-boundary-vitest

Workflow run

Full Vitest E2E advisor summary

Vitest E2E Scenario Advisor

Base: origin/main
Head: HEAD
Confidence: high

Required Vitest E2E scenarios

  • hermes-sandbox-secret-boundary-vitest: The PR changes the Hermes final image Dockerfile layout and retired sandbox-state handling. The wired free-standing Hermes sandbox secret-boundary Vitest job builds the Hermes images and probes the final image/startup boundary, making it the smallest Vitest scenario dispatch that exercises this changed Hermes image surface.
    • Dispatch: gh workflow run e2e-vitest-scenarios.yaml --ref <pr-head-ref> --field jobs=hermes-sandbox-secret-boundary-vitest

Optional Vitest E2E scenarios

  • None.

Relevant changed files

  • agents/hermes/Dockerfile

@github-actions

github-actions Bot commented Jun 28, 2026

Copy link
Copy Markdown
Contributor

PR Review Advisor (Nemotron Ultra) — Blocked

Merge posture: Do not merge until addressed
Primary next action: Fix PRA-4: Dockerfile BASE_IMAGE validation completely removed — no build-time guardrail; then add or justify PRA-T1.
Open items: 2 required · 9 warnings · 3 suggestions · 8 test follow-ups
Since last review: 1 prior item resolved · 6 still apply · 5 new items found

Action checklist

  • PRA-4 Fix: Dockerfile BASE_IMAGE validation completely removed — no build-time guardrail in agents/hermes/Dockerfile:173
  • PRA-5 Fix: Holistic Security Posture degraded — build-time trust boundary for sandbox base image removed in agents/hermes/Dockerfile:173
  • PRA-1 Resolve or justify: Source-of-truth review needed: agents/hermes/Dockerfile:173-270 (cleanup block for .openclaw and .hermes-data)
  • PRA-2 Resolve or justify: Source-of-truth review needed: agents/hermes/start.sh:1090 (migrate_legacy_layout runtime migration)
  • PRA-3 Resolve or justify: Source-of-truth review needed: .github/actions/resolve-hermes-base-image/action.yaml:34-44 (layout_ok CI enforcement)
  • PRA-6 Resolve or justify: Resolver action layout_ok() executes candidate image before digest verification in .github/actions/resolve-hermes-base-image/action.yaml:34
  • PRA-7 Resolve or justify: New test file missing BASE_IMAGE validation tests in test/hermes-final-image-layout.test.ts:1
  • PRA-8 Resolve or justify: Source-of-truth review needed for Dockerfile cleanup block in agents/hermes/Dockerfile:173
  • PRA-9 Resolve or justify: Source-of-truth review needed for start.sh migrate_legacy_layout() runtime migration in agents/hermes/start.sh:1090
  • PRA-10 Resolve or justify: Source-of-truth review needed for resolver action layout_ok() in .github/actions/resolve-hermes-base-image/action.yaml:34
  • PRA-11 Resolve or justify: CI contract test doesn't verify resolver stale-image rejection path in test/pr-workflow-contract.test.ts:1
  • PRA-T1 Add or justify test follow-up: Runtime validation
  • PRA-T2 Add or justify test follow-up: Runtime validation
  • PRA-T3 Add or justify test follow-up: Runtime validation
  • PRA-T4 Add or justify test follow-up: Runtime validation
  • PRA-T5 Add or justify test follow-up: Runtime validation
  • PRA-T6 Add or justify test follow-up: New test file missing BASE_IMAGE validation tests
  • PRA-T7 Add or justify test follow-up: CI contract test doesn't verify resolver stale-image rejection path
  • PRA-T8 Add or justify test follow-up: Removed test coverage for symlinked stale OpenClaw state — verify replacement coverage
  • PRA-12 In-scope improvement: Removed test coverage for symlinked stale OpenClaw state — verify replacement coverage in test/sandbox-provisioning.test.ts:1
  • PRA-13 In-scope improvement: Dockerfile comment claims caller-selected bases supported but lacks validation in agents/hermes/Dockerfile:173
  • PRA-14 In-scope improvement: Missing TODO with removal condition for .hermes-data migration block in agents/hermes/Dockerfile:271

Findings index

ID Severity Category Location Required action
PRA-1 Resolve/justify architecture Identify the invalid state, source boundary, source-fix constraint, regression test, and removal condition before merging the localized behavior.
PRA-2 Resolve/justify architecture Identify the invalid state, source boundary, source-fix constraint, regression test, and removal condition before merging the localized behavior.
PRA-3 Resolve/justify architecture Identify the invalid state, source boundary, source-fix constraint, regression test, and removal condition before merging the localized behavior.
PRA-4 Required security agents/hermes/Dockerfile:173 Restore lightweight BASE_IMAGE validation in Dockerfile as defense-in-depth: require @sha256: digest for ghcr.io/nvidia/nemoclaw/hermes-sandbox-base refs; reject mutable tags for official images; reject unsupported registries; explicitly allow local test tags (nemoclaw-hermes-base-local, nemoclaw-hermes-stale-openclaw-dir-base:*, nemoclaw-hermes-stale-openclaw-link-base:*). Keep resolver action as primary CI enforcement.
PRA-5 Required security agents/hermes/Dockerfile:173 Same as SEC-1 — restore Dockerfile BASE_IMAGE validation as defense-in-depth.
PRA-6 Resolve/justify security .github/actions/resolve-hermes-base-image/action.yaml:34 Consider adding a digest pinning check before running layout_ok() to avoid pulling and running untrusted images. Since candidates are GHCR tags, verify the tag resolves to an expected digest pattern before docker run. Alternatively, use docker image inspect on the pulled image without running it, or run layout_ok in a more restricted context.
PRA-7 Resolve/justify tests test/hermes-final-image-layout.test.ts:1 Add test cases in test/hermes-final-image-layout.test.ts for: BASE_IMAGE mutable tag rejection, unsupported registry rejection, local test tag acceptance, migration resolution loop with nested symlinks, final verification loop detecting new legacy symlinks after cleanup.
PRA-8 Resolve/justify architecture agents/hermes/Dockerfile:173 Document the source-of-truth analysis in a comment near the cleanup block. The comment should cover: invalid state handled, source boundary, source-fix constraint, regression test, and removal condition. Add a TODO with removal condition.
PRA-9 Resolve/justify architecture agents/hermes/start.sh:1090 Document the source-of-truth analysis in a comment near migrate_legacy_layout() in start.sh, or in a linked design doc. The Dockerfile comment at line 174 partially addresses this but start.sh lacks it.
PRA-10 Resolve/justify architecture .github/actions/resolve-hermes-base-image/action.yaml:34 Document the source-of-truth analysis in a comment in the resolver action. Add a unit test for layout_ok() that can run without Docker daemon (mock or fixture-based).
PRA-11 Resolve/justify tests test/pr-workflow-contract.test.ts:1 Add a test that verifies the resolver action's layout_ok is exercised in CI for all candidate images and rejects stale images. Ensure the pr-workflow-contract.test.ts validates the new CI flow.
PRA-12 Improvement tests test/sandbox-provisioning.test.ts:1 Verify test/hermes-final-image-layout.test.ts fully replaces the removed test. The new test covers openclaw directory and symlink rejection, plus legacy data migration. Confirm no regression in coverage.
PRA-13 Improvement correctness agents/hermes/Dockerfile:173 Add BASE_IMAGE validation (see SEC-1) to make the 'caller-selected bases remain supported' claim accurate — currently caller-selected bases are accepted without ANY validation.
PRA-14 Improvement scope agents/hermes/Dockerfile:271 Add TODO comment at the migration block with removal condition: 'TODO: Remove after all supported GHCR Hermes base tags verified to lack /sandbox/.hermes-data (see SEC-1).'

🚨 Required before merge

Address these before merging unless a maintainer explicitly overrides the advisor with rationale.

PRA-4 Required — Dockerfile BASE_IMAGE validation completely removed — no build-time guardrail

  • Location: agents/hermes/Dockerfile:173
  • Category: security
  • Problem: Previous version (lines 183-192) had a case statement requiring @sha256: digest for GHCR refs, rejecting mutable tags, rejecting unsupported registries, and explicitly allowing local test tags. Current version has NO validation — accepts any BASE_IMAGE value. This removes defense-in-depth for the sandbox root-of-trust. The resolver action's layout_ok() only runs in CI; local builds, fork CI, and alternate CI paths have no build-time guardrail.
  • Impact: The entire Hermes sandbox boundary (SSRF protection via OpenShell L7 proxy, network policy enforcement, credential redaction at egress, blueprint integrity) depends on BASE_IMAGE integrity. A malicious or mistaken BASE_IMAGE would only be caught if the CI resolver action runs. Local development builds, fork PRs, dependent repository builds, and any CI path that doesn't invoke the resolver action have zero protection.
  • Required action: Restore lightweight BASE_IMAGE validation in Dockerfile as defense-in-depth: require @sha256: digest for ghcr.io/nvidia/nemoclaw/hermes-sandbox-base refs; reject mutable tags for official images; reject unsupported registries; explicitly allow local test tags (nemoclaw-hermes-base-local, nemoclaw-hermes-stale-openclaw-dir-base:*, nemoclaw-hermes-stale-openclaw-link-base:*). Keep resolver action as primary CI enforcement.
  • Expected follow-up: Fix before merge or get explicit maintainer override.
  • Verification: Attempt docker build with BASE_IMAGE=ghcr.io/nvidia/nemoclaw/hermes-sandbox-base:latest — should fail with 'must use an immutable sha256 digest'. Attempt BASE_IMAGE=localhost:5000/evil:tag — should fail with 'unsupported base image registry'. Test that nemoclaw-hermes-base-local still works.
  • Missing regression test: Dockerfile unit/integration test that passes mutable tag BASE_IMAGE (ghcr.io/nvidia/nemoclaw/hermes-sandbox-base:latest) and expects build failure with 'must be an immutable digest' error. Test that unsupported registry BASE_IMAGE (localhost:5000/evil:tag) fails with 'unsupported base image registry'. Test that local test tags (nemoclaw-hermes-base-local, nemoclaw-hermes-stale-openclaw-dir-base:test, nemoclaw-hermes-stale-openclaw-link-base:test) are accepted.
  • Done when: The required change is committed and verification passes: Attempt docker build with BASE_IMAGE=ghcr.io/nvidia/nemoclaw/hermes-sandbox-base:latest — should fail with 'must use an immutable sha256 digest'. Attempt BASE_IMAGE=localhost:5000/evil:tag — should fail with 'unsupported base image registry'. Test that nemoclaw-hermes-base-local still works.
  • Evidence: agents/hermes/Dockerfile lines 173-270: cleanup block validates layout but no BASE_IMAGE validation. Old Dockerfile (git history) lines 176-240 had case statement with digest equality check, mutable tag rejection, unsupported registry rejection, explicit local tag allowlist.

PRA-5 Required — Holistic Security Posture degraded — build-time trust boundary for sandbox base image removed

  • Location: agents/hermes/Dockerfile:173
  • Category: security
  • Problem: The Dockerfile was the last line of defense for BASE_IMAGE integrity outside of the CI resolver action. With its removal, a malicious or mistaken BASE_IMAGE would only be caught if the CI resolver action runs. This affects local development builds, fork PRs, dependent repository builds, and any CI path that doesn't invoke the resolver action.
  • Impact: The entire Hermes sandbox boundary (SSRF protection via OpenShell L7 proxy, network policy enforcement, credential redaction at egress, blueprint integrity) is only as strong as its base image. This is a defense-in-depth regression affecting the root of trust for all non-CI build paths.
  • Required action: Same as SEC-1 — restore Dockerfile BASE_IMAGE validation as defense-in-depth.
  • Expected follow-up: Fix before merge or get explicit maintainer override.
  • Verification: Attempt docker build with BASE_IMAGE=ghcr.io/nvidia/nemoclaw/hermes-sandbox-base:latest — should fail with 'must use an immutable sha256 digest'. Attempt BASE_IMAGE=localhost:5000/evil:tag — should fail with 'unsupported base image registry'. Test that nemoclaw-hermes-base-local still works.
  • Missing regression test: Same as SEC-1 — Dockerfile validation tests for mutable tag rejection, unsupported registry rejection, local test tag acceptance.
  • Done when: The required change is committed and verification passes: Attempt docker build with BASE_IMAGE=ghcr.io/nvidia/nemoclaw/hermes-sandbox-base:latest — should fail with 'must use an immutable sha256 digest'. Attempt BASE_IMAGE=localhost:5000/evil:tag — should fail with 'unsupported base image registry'. Test that nemoclaw-hermes-base-local still works.
  • Evidence: agents/hermes/Dockerfile: BASE_IMAGE ARG at line 4, FROM ${BASE_IMAGE} at line 7, no validation between line 7 and cleanup block at line 173.
Review findings by urgency: 2 required fixes, 9 items to resolve/justify, 3 in-scope improvements

⚠️ Resolve or justify before merge

Investigate these in the current review; either fix them, explain why they are not applicable, or document the accepted risk.

PRA-1 Resolve/justify — Source-of-truth review needed: agents/hermes/Dockerfile:173-270 (cleanup block for .openclaw and .hermes-data)

  • Location: not file-specific
  • Category: architecture
  • Problem: The advisor marked localized patch analysis as needs_followup.
  • Impact: A localized workaround can preserve or hide an invalid state when the source boundary is unclear.
  • Recommended action: Identify the invalid state, source boundary, source-fix constraint, regression test, and removal condition before merging the localized behavior.
  • Expected follow-up: Resolve in this PR or explain why the risk is acceptable.
  • Verification: Inspect the localized patch and source-of-truth review fields for a concrete invalid state, source boundary, source-fix constraint, regression test, and removal condition.
  • Missing regression test: test/hermes-final-image-layout.test.ts covers rejection + migration; resolver layout_ok() validates new bases
  • Done when: The risk is fixed or explicitly justified in the PR. Verification: Inspect the localized patch and source-of-truth review fields for a concrete invalid state, source boundary, source-fix constraint, regression test, and removal condition.
  • Evidence: Dockerfile lines 173-270: cleanup block comment mentions resolver ownership but lacks explicit removal condition or full source-of-truth analysis

PRA-2 Resolve/justify — Source-of-truth review needed: agents/hermes/start.sh:1090 (migrate_legacy_layout runtime migration)

  • Location: not file-specific
  • Category: architecture
  • Problem: The advisor marked localized patch analysis as needs_followup.
  • Impact: A localized workaround can preserve or hide an invalid state when the source boundary is unclear.
  • Recommended action: Identify the invalid state, source boundary, source-fix constraint, regression test, and removal condition before merging the localized behavior.
  • Expected follow-up: Resolve in this PR or explain why the risk is acceptable.
  • Verification: Inspect the localized patch and source-of-truth review fields for a concrete invalid state, source boundary, source-fix constraint, regression test, and removal condition.
  • Missing regression test: test/nemoclaw-start.test.ts covers migrate_legacy_layout (sentinel trust, symlink refusal, root check)
  • Done when: The risk is fixed or explicitly justified in the PR. Verification: Inspect the localized patch and source-of-truth review fields for a concrete invalid state, source boundary, source-fix constraint, regression test, and removal condition.
  • Evidence: agents/hermes/start.sh line 1090: migrate_legacy_layout() function with no source-of-truth comment

PRA-3 Resolve/justify — Source-of-truth review needed: .github/actions/resolve-hermes-base-image/action.yaml:34-44 (layout_ok CI enforcement)

  • Location: not file-specific
  • Category: architecture
  • Problem: The advisor marked localized patch analysis as needs_followup.
  • Impact: A localized workaround can preserve or hide an invalid state when the source boundary is unclear.
  • Recommended action: Identify the invalid state, source boundary, source-fix constraint, regression test, and removal condition before merging the localized behavior.
  • Expected follow-up: Resolve in this PR or explain why the risk is acceptable.
  • Verification: Inspect the localized patch and source-of-truth review fields for a concrete invalid state, source boundary, source-fix constraint, regression test, and removal condition.
  • Missing regression test: Runs in CI via pr-self-hosted.yaml / sandbox-images-and-e2e.yaml; no standalone unit test
  • Done when: The risk is fixed or explicitly justified in the PR. Verification: Inspect the localized patch and source-of-truth review fields for a concrete invalid state, source boundary, source-fix constraint, regression test, and removal condition.
  • Evidence: action.yaml layout_ok() function with no source-of-truth comment. No test file exercises layout_ok in isolation.

PRA-6 Resolve/justify — Resolver action layout_ok() executes candidate image before digest verification

  • Location: .github/actions/resolve-hermes-base-image/action.yaml:34
  • Category: security
  • Problem: The layout_ok() function executes 'docker run --rm --entrypoint sh "$ref"' to check for retired sandbox state. This means an untrusted candidate image is executed (albeit with a limited shell command) before its digest is verified. While the candidate list is constrained to GHCR, a compromised GHCR tag could execute code during the layout check.
  • Impact: Supply-chain risk if GHCR is compromised or a malicious tag is pushed. The layout check runs the image code as part of validation.
  • Recommended action: Consider adding a digest pinning check before running layout_ok() to avoid pulling and running untrusted images. Since candidates are GHCR tags, verify the tag resolves to an expected digest pattern before docker run. Alternatively, use docker image inspect on the pulled image without running it, or run layout_ok in a more restricted context.
  • Expected follow-up: Resolve in this PR or explain why the risk is acceptable.
  • Verification: Review action.yaml lines 34-44: layout_ok() calls docker run on the candidate ref. Check if any digest verification happens before this.
  • Missing regression test: Test that layout_ok is not called on images that fail digest verification. Add a test with a malicious candidate that would execute code if run.
  • Done when: The risk is fixed or explicitly justified in the PR. Verification: Review action.yaml lines 34-44: layout_ok() calls docker run on the candidate ref. Check if any digest verification happens before this.
  • Evidence: .github/actions/resolve-hermes-base-image/action.yaml lines 34-44: layout_ok() function does docker run --rm --entrypoint sh "$ref" before try_image() extracts digest_ref via docker image inspect.

PRA-7 Resolve/justify — New test file missing BASE_IMAGE validation tests

  • Location: test/hermes-final-image-layout.test.ts:1
  • Category: tests
  • Problem: New test file covers layout migration logic but has NO tests for BASE_IMAGE validation. Previous review (PRA-9, PRA-11) required tests for: (1) migration resolution loop and final verification loop; (2) local test tag acceptance; (3) unsupported registry rejection; (4) mutable tag rejection. None of these are covered.
  • Impact: The removed validation logic has no test coverage. The deleted test/hermes-stale-openclaw-guard.test.ts HAD tests for mutable tag rejection, unsupported registry rejection, local test tag acceptance, and Dockerfile BASE_IMAGE validation contract. These are now completely untested.
  • Recommended action: Add test cases in test/hermes-final-image-layout.test.ts for: BASE_IMAGE mutable tag rejection, unsupported registry rejection, local test tag acceptance, migration resolution loop with nested symlinks, final verification loop detecting new legacy symlinks after cleanup.
  • Expected follow-up: Resolve in this PR or explain why the risk is acceptable.
  • Verification: Run vitest on test/hermes-final-image-layout.test.ts and verify no test cases exist for BASE_IMAGE validation.
  • Missing regression test: Test cases for BASE_IMAGE validation: mutable tag -> build failure; unsupported registry -> build failure; local test tags -> build success. Migration resolution loop: create .hermes-data with nested symlinks, run layout, verify symlinks replaced. Final verification: create new legacy symlink after cleanup, verify build fails.
  • Done when: The risk is fixed or explicitly justified in the PR. Verification: Run vitest on test/hermes-final-image-layout.test.ts and verify no test cases exist for BASE_IMAGE validation.
  • Evidence: test/hermes-final-image-layout.test.ts: 4 test blocks covering OpenClaw rejection and legacy migration only. No BASE_IMAGE validation tests.

PRA-8 Resolve/justify — Source-of-truth review needed for Dockerfile cleanup block

  • Location: agents/hermes/Dockerfile:173
  • Category: architecture
  • Problem: Cleanup block (lines 173-270) handles invalid state (stale .openclaw, .hermes-data with symlinks) from published base images. Questions: What invalid state? (Retired OpenClaw state and legacy .hermes-data symlink bridge). Where created? (Stale published GHCR base images). Why can't source be fixed? (Published images immutable; new bases from Dockerfile.base clean but old tags exist). What regression test? (Resolver layout_ok validates new bases; test/hermes-final-image-layout.test.ts validates cleanup). When removable? (When all supported GHCR tags resolve to post-stale-layout digests — no timeline documented).
  • Impact: Without documented source-of-truth analysis, the workaround's scope, necessity, and removal criteria are unclear. Future maintainers cannot evaluate if it's still needed.
  • Recommended action: Document the source-of-truth analysis in a comment near the cleanup block. The comment should cover: invalid state handled, source boundary, source-fix constraint, regression test, and removal condition. Add a TODO with removal condition.
  • Expected follow-up: Resolve in this PR or explain why the risk is acceptable.
  • Verification: Read Dockerfile lines 173-270 and verify comment addresses the five source-of-truth questions.
  • Missing regression test: Test that validates the cleanup block correctly handles all legacy fixture types (already partially covered in test/hermes-final-image-layout.test.ts). Need test for migration resolution loop and final verification loop (PRA-9).
  • Done when: The risk is fixed or explicitly justified in the PR. Verification: Read Dockerfile lines 173-270 and verify comment addresses the five source-of-truth questions.
  • Evidence: agents/hermes/Dockerfile lines 173-270: cleanup block comment mentions 'base resolver owns published-image provenance' but lacks explicit removal condition or full source-of-truth analysis.

PRA-9 Resolve/justify — Source-of-truth review needed for start.sh migrate_legacy_layout() runtime migration

  • Location: agents/hermes/start.sh:1090
  • Category: architecture
  • Problem: Runtime migration duplicates Dockerfile cleanup logic at sandbox startup (non-root). Questions: What invalid state? (Same legacy layouts). Where created? (Stale bases that escaped Dockerfile cleanup, or runtime mutation). Why can't source be fixed? (Runtime cannot rely on root; OpenShell starts sandbox as sandbox user). What regression test? (test/nemoclaw-start.test.ts covers migrate_legacy_layout). When removable? (When all supported bases are clean AND runtime mutation vectors closed).
  • Impact: Drift risk between build-time and runtime cleanup logic. Without documented analysis, cannot verify both paths stay in sync or when either can be removed.
  • Recommended action: Document the source-of-truth analysis in a comment near migrate_legacy_layout() in start.sh, or in a linked design doc. The Dockerfile comment at line 174 partially addresses this but start.sh lacks it.
  • Expected follow-up: Resolve in this PR or explain why the risk is acceptable.
  • Verification: Read agents/hermes/start.sh lines 1090-1200 and verify comment addresses the five source-of-truth questions.
  • Missing regression test: test/nemoclaw-start.test.ts should cover migrate_legacy_layout with all legacy fixture types including nested symlinks and sentinel trust logic.
  • Done when: The risk is fixed or explicitly justified in the PR. Verification: Read agents/hermes/start.sh lines 1090-1200 and verify comment addresses the five source-of-truth questions.
  • Evidence: agents/hermes/start.sh line 1090: migrate_legacy_layout() function with no source-of-truth comment. Dockerfile line 174 comment references resolver ownership but start.sh has no equivalent.

PRA-10 Resolve/justify — Source-of-truth review needed for resolver action layout_ok()

  • Location: .github/actions/resolve-hermes-base-image/action.yaml:34
  • Category: architecture
  • Problem: CI enforcement layer that validates base images before use. Questions: What invalid state? (Retired sandbox state in base images). Where created? (Stale published GHCR tags). Why can't source be fixed? (Published tags immutable; Dockerfile.base produces clean images but old tags persist). What regression test? (Resolver runs in CI; no standalone unit test for layout_ok). When removable? (When all supported tags clean).
  • Impact: Without documented analysis, the resolver's role as CI-only enforcement vs build-time boundary is unclear. No unit test for layout_ok logic.
  • Recommended action: Document the source-of-truth analysis in a comment in the resolver action. Add a unit test for layout_ok() that can run without Docker daemon (mock or fixture-based).
  • Expected follow-up: Resolve in this PR or explain why the risk is acceptable.
  • Verification: Review action.yaml layout_ok() function and verify it has adequate documentation and test coverage.
  • Missing regression test: Unit test for layout_ok() logic using mocked docker or test fixtures, verifying it detects /sandbox/.openclaw and /sandbox/.hermes-data in various forms (dir, symlink, nested).
  • Done when: The risk is fixed or explicitly justified in the PR. Verification: Review action.yaml layout_ok() function and verify it has adequate documentation and test coverage.
  • Evidence: .github/actions/resolve-hermes-base-image/action.yaml lines 34-44: layout_ok() function with no source-of-truth comment. No test file exercises layout_ok in isolation.

PRA-11 Resolve/justify — CI contract test doesn't verify resolver stale-image rejection path

  • Location: test/pr-workflow-contract.test.ts:1
  • Category: tests
  • Problem: Test for Hermes stale OpenClaw image validation in self-hosted PR CI was removed (previously verified job 'build-hermes-stale-openclaw-image' existed). The verifier script is deleted and the job is gone. Need to verify the new resolver action + layout_ok provides equivalent or better coverage in CI.
  • Impact: CI may not actually reject candidate images with retired sandbox state. The test only checks that 'layout_ok' string exists in the action, not that the rejection behavior works.
  • Recommended action: Add a test that verifies the resolver action's layout_ok is exercised in CI for all candidate images and rejects stale images. Ensure the pr-workflow-contract.test.ts validates the new CI flow.
  • Expected follow-up: Resolve in this PR or explain why the risk is acceptable.
  • Verification: Check test/pr-workflow-contract.test.ts for test 'exports immutable GHCR digests from the Hermes base resolver' — it now checks for layout_ok but doesn't verify the stale-image rejection path.
  • Missing regression test: Test that CI rejects a candidate image containing retired sandbox state (integration test or mock-based).
  • Done when: The risk is fixed or explicitly justified in the PR. Verification: Check test/pr-workflow-contract.test.ts for test 'exports immutable GHCR digests from the Hermes base resolver' — it now checks for layout_ok but doesn't verify the stale-image rejection path.
  • Evidence: test/pr-workflow-contract.test.ts: test 'exports immutable GHCR digests from the Hermes base resolver' checks runs contain 'layout_ok' but not rejection behavior.

💡 In-scope improvements

These are lower-risk, not throwaway. Prefer fixing them in this PR when they are local to changed code; defer only with rationale or a linked follow-up.

PRA-12 Improvement — Removed test coverage for symlinked stale OpenClaw state — verify replacement coverage

  • Location: test/sandbox-provisioning.test.ts:1
  • Category: tests
  • Problem: Test 'refuses symlinked stale OpenClaw state during Hermes final image cleanup' was removed. The new test/hermes-final-image-layout.test.ts covers similar ground but in a different test file. Ensure coverage is not duplicated or gapped.
  • Impact: Potential coverage gap if new test doesn't fully replace old test scenarios.
  • Suggested action: Verify test/hermes-final-image-layout.test.ts fully replaces the removed test. The new test covers openclaw directory and symlink rejection, plus legacy data migration. Confirm no regression in coverage.
  • Expected follow-up: Prefer a current-PR fix when local to changed code; defer only with rationale or linked follow-up.
  • Verification: Compare test coverage between old test/sandbox-provisioning.test.ts 'refuses symlinked stale OpenClaw state' and new test/hermes-final-image-layout.test.ts 'rejects retired OpenClaw state represented as a directory/symlink'.
  • Missing regression test: None if coverage is equivalent; otherwise add missing cases.
  • Done when: The local improvement is applied, or the PR notes why it should be deferred.
  • Evidence: test/sandbox-provisioning.test.ts: removed test 'refuses symlinked stale OpenClaw state during Hermes final image cleanup'. test/hermes-final-image-layout.test.ts: tests 'rejects retired OpenClaw state represented as a directory' and 'rejects retired OpenClaw state represented as a symlink without following it'.

PRA-13 Improvement — Dockerfile comment claims caller-selected bases supported but lacks validation

  • Location: agents/hermes/Dockerfile:173
  • Category: correctness
  • Problem: Cleanup block comment says 'The base resolver owns published-image provenance and exports immutable digests; this final-image boundary owns layout validation so local, rebuilt, and caller-selected bases remain supported.' This is a good architectural clarification but the Dockerfile lacks the BASE_IMAGE validation that would enforce this boundary for caller-selected bases.
  • Impact: Claim about supporting caller-selected bases is inaccurate — currently caller-selected bases are accepted without ANY validation.
  • Suggested action: Add BASE_IMAGE validation (see SEC-1) to make the 'caller-selected bases remain supported' claim accurate — currently caller-selected bases are accepted without ANY validation.
  • Expected follow-up: Prefer a current-PR fix when local to changed code; defer only with rationale or linked follow-up.
  • Verification: Attempt docker build with BASE_IMAGE=docker.io/library/ubuntu:latest — currently succeeds (no validation), should fail.
  • Missing regression test: Test that arbitrary BASE_IMAGE values are rejected.
  • Done when: The local improvement is applied, or the PR notes why it should be deferred.
  • Evidence: agents/hermes/Dockerfile line 174-176 comment vs actual validation (none).

PRA-14 Improvement — Missing TODO with removal condition for .hermes-data migration block

  • Location: agents/hermes/Dockerfile:271
  • Category: scope
  • Problem: Previous review (PRA-13) recommended adding TODO comment with removal condition for .hermes-data migration block. Not yet added.
  • Impact: No documented criteria for when this workaround can be removed. Future maintainers cannot track cleanup progress.
  • Suggested action: Add TODO comment at the migration block with removal condition: 'TODO: Remove after all supported GHCR Hermes base tags verified to lack /sandbox/.hermes-data (see SEC-1).'
  • Expected follow-up: Prefer a current-PR fix when local to changed code; defer only with rationale or linked follow-up.
  • Verification: Check Dockerfile line 271 area for TODO comment.
  • Missing regression test: None — documentation improvement.
  • Done when: The local improvement is applied, or the PR notes why it should be deferred.
  • Evidence: agents/hermes/Dockerfile: cleanup block ends around line 270-271 with no TODO comment.
Test follow-ups to resolve or justify

If these cover changed behavior, prefer adding them in this PR; otherwise state why existing coverage is enough or link the follow-up.

  • PRA-T1 Runtime validation — BASE_IMAGE:mutable_tag_rejected — ghcr.io/nvidia/nemoclaw/hermes-sandbox-base:latest -> build fails with 'must use immutable sha256 digest'. Runtime/sandbox/infrastructure paths need behavioral runtime validation: resolver action layout_ok() executes images, Dockerfile BASE_IMAGE validation removed (was build-time), migration resolution/final verification loops have no test coverage. Deleted test file had BASE_IMAGE validation tests now completely untested.
  • PRA-T2 Runtime validation — BASE_IMAGE:unsupported_registry_rejected — localhost:5000/evil:tag -> build fails with 'unsupported base image registry'. Runtime/sandbox/infrastructure paths need behavioral runtime validation: resolver action layout_ok() executes images, Dockerfile BASE_IMAGE validation removed (was build-time), migration resolution/final verification loops have no test coverage. Deleted test file had BASE_IMAGE validation tests now completely untested.
  • PRA-T3 Runtime validation — BASE_IMAGE:local_test_tags_accepted — nemoclaw-hermes-base-local, nemoclaw-hermes-stale-openclaw-dir-base:test, nemoclaw-hermes-stale-openclaw-link-base:test -> build succeeds. Runtime/sandbox/infrastructure paths need behavioral runtime validation: resolver action layout_ok() executes images, Dockerfile BASE_IMAGE validation removed (was build-time), migration resolution/final verification loops have no test coverage. Deleted test file had BASE_IMAGE validation tests now completely untested.
  • PRA-T4 Runtime validation — migration:resolution_loop_nested_symlinks — create .hermes-data with entries, create nested symlinks in .hermes pointing into .hermes-data, run layout, verify symlinks replaced with actual dirs/files. Runtime/sandbox/infrastructure paths need behavioral runtime validation: resolver action layout_ok() executes images, Dockerfile BASE_IMAGE validation removed (was build-time), migration resolution/final verification loops have no test coverage. Deleted test file had BASE_IMAGE validation tests now completely untested.
  • PRA-T5 Runtime validation — migration:final_verification_new_legacy_symlink_fails — after successful migration, create new legacy symlink in .hermes pointing into old .hermes-data location, verify build fails with 'legacy symlink remains after cleanup'. Runtime/sandbox/infrastructure paths need behavioral runtime validation: resolver action layout_ok() executes images, Dockerfile BASE_IMAGE validation removed (was build-time), migration resolution/final verification loops have no test coverage. Deleted test file had BASE_IMAGE validation tests now completely untested.
  • PRA-T6 New test file missing BASE_IMAGE validation tests — Add test cases in test/hermes-final-image-layout.test.ts for: BASE_IMAGE mutable tag rejection, unsupported registry rejection, local test tag acceptance, migration resolution loop with nested symlinks, final verification loop detecting new legacy symlinks after cleanup.
  • PRA-T7 CI contract test doesn't verify resolver stale-image rejection path — Add a test that verifies the resolver action's layout_ok is exercised in CI for all candidate images and rejects stale images. Ensure the pr-workflow-contract.test.ts validates the new CI flow.
  • PRA-T8 Removed test coverage for symlinked stale OpenClaw state — verify replacement coverage — Verify test/hermes-final-image-layout.test.ts fully replaces the removed test. The new test covers openclaw directory and symlink rejection, plus legacy data migration. Confirm no regression in coverage.
Since last review details

Current findings, using the urgency labels above:

PRA-1 Resolve/justify — Source-of-truth review needed: agents/hermes/Dockerfile:173-270 (cleanup block for .openclaw and .hermes-data)

  • Location: not file-specific
  • Category: architecture
  • Problem: The advisor marked localized patch analysis as needs_followup.
  • Impact: A localized workaround can preserve or hide an invalid state when the source boundary is unclear.
  • Recommended action: Identify the invalid state, source boundary, source-fix constraint, regression test, and removal condition before merging the localized behavior.
  • Expected follow-up: Resolve in this PR or explain why the risk is acceptable.
  • Verification: Inspect the localized patch and source-of-truth review fields for a concrete invalid state, source boundary, source-fix constraint, regression test, and removal condition.
  • Missing regression test: test/hermes-final-image-layout.test.ts covers rejection + migration; resolver layout_ok() validates new bases
  • Done when: The risk is fixed or explicitly justified in the PR. Verification: Inspect the localized patch and source-of-truth review fields for a concrete invalid state, source boundary, source-fix constraint, regression test, and removal condition.
  • Evidence: Dockerfile lines 173-270: cleanup block comment mentions resolver ownership but lacks explicit removal condition or full source-of-truth analysis

PRA-2 Resolve/justify — Source-of-truth review needed: agents/hermes/start.sh:1090 (migrate_legacy_layout runtime migration)

  • Location: not file-specific
  • Category: architecture
  • Problem: The advisor marked localized patch analysis as needs_followup.
  • Impact: A localized workaround can preserve or hide an invalid state when the source boundary is unclear.
  • Recommended action: Identify the invalid state, source boundary, source-fix constraint, regression test, and removal condition before merging the localized behavior.
  • Expected follow-up: Resolve in this PR or explain why the risk is acceptable.
  • Verification: Inspect the localized patch and source-of-truth review fields for a concrete invalid state, source boundary, source-fix constraint, regression test, and removal condition.
  • Missing regression test: test/nemoclaw-start.test.ts covers migrate_legacy_layout (sentinel trust, symlink refusal, root check)
  • Done when: The risk is fixed or explicitly justified in the PR. Verification: Inspect the localized patch and source-of-truth review fields for a concrete invalid state, source boundary, source-fix constraint, regression test, and removal condition.
  • Evidence: agents/hermes/start.sh line 1090: migrate_legacy_layout() function with no source-of-truth comment

PRA-3 Resolve/justify — Source-of-truth review needed: .github/actions/resolve-hermes-base-image/action.yaml:34-44 (layout_ok CI enforcement)

  • Location: not file-specific
  • Category: architecture
  • Problem: The advisor marked localized patch analysis as needs_followup.
  • Impact: A localized workaround can preserve or hide an invalid state when the source boundary is unclear.
  • Recommended action: Identify the invalid state, source boundary, source-fix constraint, regression test, and removal condition before merging the localized behavior.
  • Expected follow-up: Resolve in this PR or explain why the risk is acceptable.
  • Verification: Inspect the localized patch and source-of-truth review fields for a concrete invalid state, source boundary, source-fix constraint, regression test, and removal condition.
  • Missing regression test: Runs in CI via pr-self-hosted.yaml / sandbox-images-and-e2e.yaml; no standalone unit test
  • Done when: The risk is fixed or explicitly justified in the PR. Verification: Inspect the localized patch and source-of-truth review fields for a concrete invalid state, source boundary, source-fix constraint, regression test, and removal condition.
  • Evidence: action.yaml layout_ok() function with no source-of-truth comment. No test file exercises layout_ok in isolation.

PRA-4 Required — Dockerfile BASE_IMAGE validation completely removed — no build-time guardrail

  • Location: agents/hermes/Dockerfile:173
  • Category: security
  • Problem: Previous version (lines 183-192) had a case statement requiring @sha256: digest for GHCR refs, rejecting mutable tags, rejecting unsupported registries, and explicitly allowing local test tags. Current version has NO validation — accepts any BASE_IMAGE value. This removes defense-in-depth for the sandbox root-of-trust. The resolver action's layout_ok() only runs in CI; local builds, fork CI, and alternate CI paths have no build-time guardrail.
  • Impact: The entire Hermes sandbox boundary (SSRF protection via OpenShell L7 proxy, network policy enforcement, credential redaction at egress, blueprint integrity) depends on BASE_IMAGE integrity. A malicious or mistaken BASE_IMAGE would only be caught if the CI resolver action runs. Local development builds, fork PRs, dependent repository builds, and any CI path that doesn't invoke the resolver action have zero protection.
  • Required action: Restore lightweight BASE_IMAGE validation in Dockerfile as defense-in-depth: require @sha256: digest for ghcr.io/nvidia/nemoclaw/hermes-sandbox-base refs; reject mutable tags for official images; reject unsupported registries; explicitly allow local test tags (nemoclaw-hermes-base-local, nemoclaw-hermes-stale-openclaw-dir-base:*, nemoclaw-hermes-stale-openclaw-link-base:*). Keep resolver action as primary CI enforcement.
  • Expected follow-up: Fix before merge or get explicit maintainer override.
  • Verification: Attempt docker build with BASE_IMAGE=ghcr.io/nvidia/nemoclaw/hermes-sandbox-base:latest — should fail with 'must use an immutable sha256 digest'. Attempt BASE_IMAGE=localhost:5000/evil:tag — should fail with 'unsupported base image registry'. Test that nemoclaw-hermes-base-local still works.
  • Missing regression test: Dockerfile unit/integration test that passes mutable tag BASE_IMAGE (ghcr.io/nvidia/nemoclaw/hermes-sandbox-base:latest) and expects build failure with 'must be an immutable digest' error. Test that unsupported registry BASE_IMAGE (localhost:5000/evil:tag) fails with 'unsupported base image registry'. Test that local test tags (nemoclaw-hermes-base-local, nemoclaw-hermes-stale-openclaw-dir-base:test, nemoclaw-hermes-stale-openclaw-link-base:test) are accepted.
  • Done when: The required change is committed and verification passes: Attempt docker build with BASE_IMAGE=ghcr.io/nvidia/nemoclaw/hermes-sandbox-base:latest — should fail with 'must use an immutable sha256 digest'. Attempt BASE_IMAGE=localhost:5000/evil:tag — should fail with 'unsupported base image registry'. Test that nemoclaw-hermes-base-local still works.
  • Evidence: agents/hermes/Dockerfile lines 173-270: cleanup block validates layout but no BASE_IMAGE validation. Old Dockerfile (git history) lines 176-240 had case statement with digest equality check, mutable tag rejection, unsupported registry rejection, explicit local tag allowlist.

PRA-5 Required — Holistic Security Posture degraded — build-time trust boundary for sandbox base image removed

  • Location: agents/hermes/Dockerfile:173
  • Category: security
  • Problem: The Dockerfile was the last line of defense for BASE_IMAGE integrity outside of the CI resolver action. With its removal, a malicious or mistaken BASE_IMAGE would only be caught if the CI resolver action runs. This affects local development builds, fork PRs, dependent repository builds, and any CI path that doesn't invoke the resolver action.
  • Impact: The entire Hermes sandbox boundary (SSRF protection via OpenShell L7 proxy, network policy enforcement, credential redaction at egress, blueprint integrity) is only as strong as its base image. This is a defense-in-depth regression affecting the root of trust for all non-CI build paths.
  • Required action: Same as SEC-1 — restore Dockerfile BASE_IMAGE validation as defense-in-depth.
  • Expected follow-up: Fix before merge or get explicit maintainer override.
  • Verification: Attempt docker build with BASE_IMAGE=ghcr.io/nvidia/nemoclaw/hermes-sandbox-base:latest — should fail with 'must use an immutable sha256 digest'. Attempt BASE_IMAGE=localhost:5000/evil:tag — should fail with 'unsupported base image registry'. Test that nemoclaw-hermes-base-local still works.
  • Missing regression test: Same as SEC-1 — Dockerfile validation tests for mutable tag rejection, unsupported registry rejection, local test tag acceptance.
  • Done when: The required change is committed and verification passes: Attempt docker build with BASE_IMAGE=ghcr.io/nvidia/nemoclaw/hermes-sandbox-base:latest — should fail with 'must use an immutable sha256 digest'. Attempt BASE_IMAGE=localhost:5000/evil:tag — should fail with 'unsupported base image registry'. Test that nemoclaw-hermes-base-local still works.
  • Evidence: agents/hermes/Dockerfile: BASE_IMAGE ARG at line 4, FROM ${BASE_IMAGE} at line 7, no validation between line 7 and cleanup block at line 173.

PRA-6 Resolve/justify — Resolver action layout_ok() executes candidate image before digest verification

  • Location: .github/actions/resolve-hermes-base-image/action.yaml:34
  • Category: security
  • Problem: The layout_ok() function executes 'docker run --rm --entrypoint sh "$ref"' to check for retired sandbox state. This means an untrusted candidate image is executed (albeit with a limited shell command) before its digest is verified. While the candidate list is constrained to GHCR, a compromised GHCR tag could execute code during the layout check.
  • Impact: Supply-chain risk if GHCR is compromised or a malicious tag is pushed. The layout check runs the image code as part of validation.
  • Recommended action: Consider adding a digest pinning check before running layout_ok() to avoid pulling and running untrusted images. Since candidates are GHCR tags, verify the tag resolves to an expected digest pattern before docker run. Alternatively, use docker image inspect on the pulled image without running it, or run layout_ok in a more restricted context.
  • Expected follow-up: Resolve in this PR or explain why the risk is acceptable.
  • Verification: Review action.yaml lines 34-44: layout_ok() calls docker run on the candidate ref. Check if any digest verification happens before this.
  • Missing regression test: Test that layout_ok is not called on images that fail digest verification. Add a test with a malicious candidate that would execute code if run.
  • Done when: The risk is fixed or explicitly justified in the PR. Verification: Review action.yaml lines 34-44: layout_ok() calls docker run on the candidate ref. Check if any digest verification happens before this.
  • Evidence: .github/actions/resolve-hermes-base-image/action.yaml lines 34-44: layout_ok() function does docker run --rm --entrypoint sh "$ref" before try_image() extracts digest_ref via docker image inspect.

PRA-7 Resolve/justify — New test file missing BASE_IMAGE validation tests

  • Location: test/hermes-final-image-layout.test.ts:1
  • Category: tests
  • Problem: New test file covers layout migration logic but has NO tests for BASE_IMAGE validation. Previous review (PRA-9, PRA-11) required tests for: (1) migration resolution loop and final verification loop; (2) local test tag acceptance; (3) unsupported registry rejection; (4) mutable tag rejection. None of these are covered.
  • Impact: The removed validation logic has no test coverage. The deleted test/hermes-stale-openclaw-guard.test.ts HAD tests for mutable tag rejection, unsupported registry rejection, local test tag acceptance, and Dockerfile BASE_IMAGE validation contract. These are now completely untested.
  • Recommended action: Add test cases in test/hermes-final-image-layout.test.ts for: BASE_IMAGE mutable tag rejection, unsupported registry rejection, local test tag acceptance, migration resolution loop with nested symlinks, final verification loop detecting new legacy symlinks after cleanup.
  • Expected follow-up: Resolve in this PR or explain why the risk is acceptable.
  • Verification: Run vitest on test/hermes-final-image-layout.test.ts and verify no test cases exist for BASE_IMAGE validation.
  • Missing regression test: Test cases for BASE_IMAGE validation: mutable tag -> build failure; unsupported registry -> build failure; local test tags -> build success. Migration resolution loop: create .hermes-data with nested symlinks, run layout, verify symlinks replaced. Final verification: create new legacy symlink after cleanup, verify build fails.
  • Done when: The risk is fixed or explicitly justified in the PR. Verification: Run vitest on test/hermes-final-image-layout.test.ts and verify no test cases exist for BASE_IMAGE validation.
  • Evidence: test/hermes-final-image-layout.test.ts: 4 test blocks covering OpenClaw rejection and legacy migration only. No BASE_IMAGE validation tests.

PRA-8 Resolve/justify — Source-of-truth review needed for Dockerfile cleanup block

  • Location: agents/hermes/Dockerfile:173
  • Category: architecture
  • Problem: Cleanup block (lines 173-270) handles invalid state (stale .openclaw, .hermes-data with symlinks) from published base images. Questions: What invalid state? (Retired OpenClaw state and legacy .hermes-data symlink bridge). Where created? (Stale published GHCR base images). Why can't source be fixed? (Published images immutable; new bases from Dockerfile.base clean but old tags exist). What regression test? (Resolver layout_ok validates new bases; test/hermes-final-image-layout.test.ts validates cleanup). When removable? (When all supported GHCR tags resolve to post-stale-layout digests — no timeline documented).
  • Impact: Without documented source-of-truth analysis, the workaround's scope, necessity, and removal criteria are unclear. Future maintainers cannot evaluate if it's still needed.
  • Recommended action: Document the source-of-truth analysis in a comment near the cleanup block. The comment should cover: invalid state handled, source boundary, source-fix constraint, regression test, and removal condition. Add a TODO with removal condition.
  • Expected follow-up: Resolve in this PR or explain why the risk is acceptable.
  • Verification: Read Dockerfile lines 173-270 and verify comment addresses the five source-of-truth questions.
  • Missing regression test: Test that validates the cleanup block correctly handles all legacy fixture types (already partially covered in test/hermes-final-image-layout.test.ts). Need test for migration resolution loop and final verification loop (PRA-9).
  • Done when: The risk is fixed or explicitly justified in the PR. Verification: Read Dockerfile lines 173-270 and verify comment addresses the five source-of-truth questions.
  • Evidence: agents/hermes/Dockerfile lines 173-270: cleanup block comment mentions 'base resolver owns published-image provenance' but lacks explicit removal condition or full source-of-truth analysis.

PRA-9 Resolve/justify — Source-of-truth review needed for start.sh migrate_legacy_layout() runtime migration

  • Location: agents/hermes/start.sh:1090
  • Category: architecture
  • Problem: Runtime migration duplicates Dockerfile cleanup logic at sandbox startup (non-root). Questions: What invalid state? (Same legacy layouts). Where created? (Stale bases that escaped Dockerfile cleanup, or runtime mutation). Why can't source be fixed? (Runtime cannot rely on root; OpenShell starts sandbox as sandbox user). What regression test? (test/nemoclaw-start.test.ts covers migrate_legacy_layout). When removable? (When all supported bases are clean AND runtime mutation vectors closed).
  • Impact: Drift risk between build-time and runtime cleanup logic. Without documented analysis, cannot verify both paths stay in sync or when either can be removed.
  • Recommended action: Document the source-of-truth analysis in a comment near migrate_legacy_layout() in start.sh, or in a linked design doc. The Dockerfile comment at line 174 partially addresses this but start.sh lacks it.
  • Expected follow-up: Resolve in this PR or explain why the risk is acceptable.
  • Verification: Read agents/hermes/start.sh lines 1090-1200 and verify comment addresses the five source-of-truth questions.
  • Missing regression test: test/nemoclaw-start.test.ts should cover migrate_legacy_layout with all legacy fixture types including nested symlinks and sentinel trust logic.
  • Done when: The risk is fixed or explicitly justified in the PR. Verification: Read agents/hermes/start.sh lines 1090-1200 and verify comment addresses the five source-of-truth questions.
  • Evidence: agents/hermes/start.sh line 1090: migrate_legacy_layout() function with no source-of-truth comment. Dockerfile line 174 comment references resolver ownership but start.sh has no equivalent.

PRA-10 Resolve/justify — Source-of-truth review needed for resolver action layout_ok()

  • Location: .github/actions/resolve-hermes-base-image/action.yaml:34
  • Category: architecture
  • Problem: CI enforcement layer that validates base images before use. Questions: What invalid state? (Retired sandbox state in base images). Where created? (Stale published GHCR tags). Why can't source be fixed? (Published tags immutable; Dockerfile.base produces clean images but old tags persist). What regression test? (Resolver runs in CI; no standalone unit test for layout_ok). When removable? (When all supported tags clean).
  • Impact: Without documented analysis, the resolver's role as CI-only enforcement vs build-time boundary is unclear. No unit test for layout_ok logic.
  • Recommended action: Document the source-of-truth analysis in a comment in the resolver action. Add a unit test for layout_ok() that can run without Docker daemon (mock or fixture-based).
  • Expected follow-up: Resolve in this PR or explain why the risk is acceptable.
  • Verification: Review action.yaml layout_ok() function and verify it has adequate documentation and test coverage.
  • Missing regression test: Unit test for layout_ok() logic using mocked docker or test fixtures, verifying it detects /sandbox/.openclaw and /sandbox/.hermes-data in various forms (dir, symlink, nested).
  • Done when: The risk is fixed or explicitly justified in the PR. Verification: Review action.yaml layout_ok() function and verify it has adequate documentation and test coverage.
  • Evidence: .github/actions/resolve-hermes-base-image/action.yaml lines 34-44: layout_ok() function with no source-of-truth comment. No test file exercises layout_ok in isolation.

PRA-11 Resolve/justify — CI contract test doesn't verify resolver stale-image rejection path

  • Location: test/pr-workflow-contract.test.ts:1
  • Category: tests
  • Problem: Test for Hermes stale OpenClaw image validation in self-hosted PR CI was removed (previously verified job 'build-hermes-stale-openclaw-image' existed). The verifier script is deleted and the job is gone. Need to verify the new resolver action + layout_ok provides equivalent or better coverage in CI.
  • Impact: CI may not actually reject candidate images with retired sandbox state. The test only checks that 'layout_ok' string exists in the action, not that the rejection behavior works.
  • Recommended action: Add a test that verifies the resolver action's layout_ok is exercised in CI for all candidate images and rejects stale images. Ensure the pr-workflow-contract.test.ts validates the new CI flow.
  • Expected follow-up: Resolve in this PR or explain why the risk is acceptable.
  • Verification: Check test/pr-workflow-contract.test.ts for test 'exports immutable GHCR digests from the Hermes base resolver' — it now checks for layout_ok but doesn't verify the stale-image rejection path.
  • Missing regression test: Test that CI rejects a candidate image containing retired sandbox state (integration test or mock-based).
  • Done when: The risk is fixed or explicitly justified in the PR. Verification: Check test/pr-workflow-contract.test.ts for test 'exports immutable GHCR digests from the Hermes base resolver' — it now checks for layout_ok but doesn't verify the stale-image rejection path.
  • Evidence: test/pr-workflow-contract.test.ts: test 'exports immutable GHCR digests from the Hermes base resolver' checks runs contain 'layout_ok' but not rejection behavior.

PRA-12 Improvement — Removed test coverage for symlinked stale OpenClaw state — verify replacement coverage

  • Location: test/sandbox-provisioning.test.ts:1
  • Category: tests
  • Problem: Test 'refuses symlinked stale OpenClaw state during Hermes final image cleanup' was removed. The new test/hermes-final-image-layout.test.ts covers similar ground but in a different test file. Ensure coverage is not duplicated or gapped.
  • Impact: Potential coverage gap if new test doesn't fully replace old test scenarios.
  • Suggested action: Verify test/hermes-final-image-layout.test.ts fully replaces the removed test. The new test covers openclaw directory and symlink rejection, plus legacy data migration. Confirm no regression in coverage.
  • Expected follow-up: Prefer a current-PR fix when local to changed code; defer only with rationale or linked follow-up.
  • Verification: Compare test coverage between old test/sandbox-provisioning.test.ts 'refuses symlinked stale OpenClaw state' and new test/hermes-final-image-layout.test.ts 'rejects retired OpenClaw state represented as a directory/symlink'.
  • Missing regression test: None if coverage is equivalent; otherwise add missing cases.
  • Done when: The local improvement is applied, or the PR notes why it should be deferred.
  • Evidence: test/sandbox-provisioning.test.ts: removed test 'refuses symlinked stale OpenClaw state during Hermes final image cleanup'. test/hermes-final-image-layout.test.ts: tests 'rejects retired OpenClaw state represented as a directory' and 'rejects retired OpenClaw state represented as a symlink without following it'.

PRA-13 Improvement — Dockerfile comment claims caller-selected bases supported but lacks validation

  • Location: agents/hermes/Dockerfile:173
  • Category: correctness
  • Problem: Cleanup block comment says 'The base resolver owns published-image provenance and exports immutable digests; this final-image boundary owns layout validation so local, rebuilt, and caller-selected bases remain supported.' This is a good architectural clarification but the Dockerfile lacks the BASE_IMAGE validation that would enforce this boundary for caller-selected bases.
  • Impact: Claim about supporting caller-selected bases is inaccurate — currently caller-selected bases are accepted without ANY validation.
  • Suggested action: Add BASE_IMAGE validation (see SEC-1) to make the 'caller-selected bases remain supported' claim accurate — currently caller-selected bases are accepted without ANY validation.
  • Expected follow-up: Prefer a current-PR fix when local to changed code; defer only with rationale or linked follow-up.
  • Verification: Attempt docker build with BASE_IMAGE=docker.io/library/ubuntu:latest — currently succeeds (no validation), should fail.
  • Missing regression test: Test that arbitrary BASE_IMAGE values are rejected.
  • Done when: The local improvement is applied, or the PR notes why it should be deferred.
  • Evidence: agents/hermes/Dockerfile line 174-176 comment vs actual validation (none).

PRA-14 Improvement — Missing TODO with removal condition for .hermes-data migration block

  • Location: agents/hermes/Dockerfile:271
  • Category: scope
  • Problem: Previous review (PRA-13) recommended adding TODO comment with removal condition for .hermes-data migration block. Not yet added.
  • Impact: No documented criteria for when this workaround can be removed. Future maintainers cannot track cleanup progress.
  • Suggested action: Add TODO comment at the migration block with removal condition: 'TODO: Remove after all supported GHCR Hermes base tags verified to lack /sandbox/.hermes-data (see SEC-1).'
  • Expected follow-up: Prefer a current-PR fix when local to changed code; defer only with rationale or linked follow-up.
  • Verification: Check Dockerfile line 271 area for TODO comment.
  • Missing regression test: None — documentation improvement.
  • Done when: The local improvement is applied, or the PR notes why it should be deferred.
  • Evidence: agents/hermes/Dockerfile: cleanup block ends around line 270-271 with no TODO comment.

Workflow run details

This is an automated, non-binding review; it still expects maintainers and agents to respond to each required or warning item. Treat suggestions as current-PR improvements when they touch changed code; defer only with maintainer rationale or a linked follow-up. A human maintainer must make the final merge decision.

@github-actions

github-actions Bot commented Jun 28, 2026

Copy link
Copy Markdown
Contributor

PR Review Advisor — Changes requested

Merge posture: Do not merge yet
Primary next action: Resolve or justify PRA-1: Source-of-truth review needed: Hermes final-image compatibility migration for `/sandbox/.hermes-data`.
Open items: 0 required · 3 warnings · 0 suggestions · 8 test follow-ups
Since last review: 1 prior item resolved · 0 still apply · 2 new items found

Action checklist

  • PRA-1 Resolve or justify: Source-of-truth review needed: Hermes final-image compatibility migration for `/sandbox/.hermes-data`
  • PRA-2 Resolve or justify: Restore real Docker-build coverage for stale Hermes base refusal in scripts/verify-hermes-stale-openclaw-image.sh:1
  • PRA-3 Resolve or justify: Document the retained .hermes-data migration source and removal condition in agents/hermes/Dockerfile:252
  • PRA-T1 Add or justify test follow-up: Runtime validation
  • PRA-T2 Add or justify test follow-up: Runtime validation
  • PRA-T3 Add or justify test follow-up: Runtime validation
  • PRA-T4 Add or justify test follow-up: Runtime validation
  • PRA-T5 Add or justify test follow-up: Restore real Docker-build coverage for stale Hermes base refusal
  • PRA-T6 Add or justify test follow-up: Acceptance clause
  • PRA-T7 Add or justify test follow-up: Acceptance clause
  • PRA-T8 Add or justify test follow-up: Acceptance clause

Findings index

ID Severity Category Location Required action
PRA-1 Resolve/justify architecture Identify the invalid state, source boundary, source-fix constraint, regression test, and removal condition before merging the localized behavior.
PRA-2 Resolve/justify tests scripts/verify-hermes-stale-openclaw-image.sh:1 Add or identify a focused runtime/integration test in this PR that builds a minimal synthetic Hermes base with `/sandbox/.openclaw` as a directory and as a symlink, then asserts `docker build -f agents/hermes/Dockerfile --build-arg BASE_IMAGE=<synthetic>` fails closed without following/deleting the symlink target. If resolver behavior is the intended enforcement point for published candidates, also cover stale candidate rejection with a stubbed `docker` or minimal local-image scenario rather than only string containment.
PRA-3 Resolve/justify correctness agents/hermes/Dockerfile:252 Extend the Dockerfile comment or nearby tests with a concrete removal checklist for `.hermes-data`: identify the old base/source boundary, state why caller-selected/local stale bases still require the migration, assert that `agents/hermes/Dockerfile.base` does not create `/sandbox/.hermes-data`, and define the supported-base cutoff after which the migration can be removed.
Review findings by urgency: 0 required fixes, 3 items to resolve/justify, 0 in-scope improvements

⚠️ Resolve or justify before merge

Investigate these in the current review; either fix them, explain why they are not applicable, or document the accepted risk.

PRA-1 Resolve/justify — Source-of-truth review needed: Hermes final-image compatibility migration for `/sandbox/.hermes-data`

  • Location: not file-specific
  • Category: architecture
  • Problem: The advisor marked localized patch analysis as needs_followup.
  • Impact: A localized workaround can preserve or hide an invalid state when the source boundary is unclear.
  • Recommended action: Identify the invalid state, source boundary, source-fix constraint, regression test, and removal condition before merging the localized behavior.
  • Expected follow-up: Resolve in this PR or explain why the risk is acceptable.
  • Verification: Inspect the localized patch and source-of-truth review fields for a concrete invalid state, source boundary, source-fix constraint, regression test, and removal condition.
  • Missing regression test: `test/hermes-final-image-layout.test.ts` covers content migration and symlink refusal, including nested symlinks.
  • Done when: The risk is fixed or explicitly justified in the PR. Verification: Inspect the localized patch and source-of-truth review fields for a concrete invalid state, source boundary, source-fix constraint, regression test, and removal condition.
  • Evidence: `agents/hermes/Dockerfile` retains `.hermes-data` migration logic; no changed file adds a removal checklist or source-level assertion that `agents/hermes/Dockerfile.base` cannot recreate the legacy path.

PRA-2 Resolve/justify — Restore real Docker-build coverage for stale Hermes base refusal

  • Location: scripts/verify-hermes-stale-openclaw-image.sh:1
  • Category: tests
  • Problem: This PR deletes the dedicated stale OpenClaw verifier script and self-hosted workflow job that built synthetic stale Hermes bases through the real `agents/hermes/Dockerfile`. The replacement `test/hermes-final-image-layout.test.ts` exercises the extracted RUN block in a host shell, and the resolver contract test only checks string fragments. That covers most shell logic, but it no longer proves the actual Docker build fails closed when a selected base image contains `/sandbox/.openclaw` as a directory or symlink, nor that resolver candidate rejection behaves correctly end-to-end.
  • Impact: A regression in Dockerfile build-time behavior, ARG/FROM interaction, shell environment, or resolver fallback could let retired OpenClaw state reach a Hermes final image or break legitimate local fallback builds without being caught by the focused unit-style tests.
  • Recommended action: Add or identify a focused runtime/integration test in this PR that builds a minimal synthetic Hermes base with `/sandbox/.openclaw` as a directory and as a symlink, then asserts `docker build -f agents/hermes/Dockerfile --build-arg BASE_IMAGE=<synthetic>` fails closed without following/deleting the symlink target. If resolver behavior is the intended enforcement point for published candidates, also cover stale candidate rejection with a stubbed `docker` or minimal local-image scenario rather than only string containment.
  • Expected follow-up: Resolve in this PR or explain why the risk is acceptable.
  • Verification: Read the diff for `scripts/verify-hermes-stale-openclaw-image.sh` deletion and `.github/workflows/pr-self-hosted.yaml` job removal, then compare with `test/hermes-final-image-layout.test.ts` and `test/pr-workflow-contract.test.ts` to confirm the remaining coverage is extracted-shell/text-contract rather than a real Docker-build negative test.
  • Missing regression test: A behavior-specific test named like `Hermes Dockerfile fails closed when BASE_IMAGE contains retired /sandbox/.openclaw directory or symlink` that builds synthetic stale base images and asserts the final image build fails with the retired-state error while preserving the symlink target sentinel.
  • Done when: The risk is fixed or explicitly justified in the PR. Verification: Read the diff for `scripts/verify-hermes-stale-openclaw-image.sh` deletion and `.github/workflows/pr-self-hosted.yaml` job removal, then compare with `test/hermes-final-image-layout.test.ts` and `test/pr-workflow-contract.test.ts` to confirm the remaining coverage is extracted-shell/text-contract rather than a real Docker-build negative test.
  • Evidence: Deleted verifier previously built `nemoclaw-hermes-stale-openclaw-dir-base` and `nemoclaw-hermes-stale-openclaw-link-base` through the real Dockerfile; new coverage in `test/hermes-final-image-layout.test.ts` calls `dockerRunCommandBetween(...)` and `runDockerShell(...)` on an extracted RUN snippet.

PRA-3 Resolve/justify — Document the retained .hermes-data migration source and removal condition

  • Location: agents/hermes/Dockerfile:252
  • Category: correctness
  • Problem: The PR intentionally keeps a compatibility migration for legacy `/sandbox/.hermes-data` while rejecting retired `/sandbox/.openclaw`. The new comment explains the invalid state and that local/rebuilt/caller-selected bases remain supported, but it does not state where `.hermes-data` can still be created, why that source cannot be eliminated in this PR, what source-level regression prevents new bases from reintroducing it, or when this migration can be removed.
  • Impact: Without a source-of-truth contract, this root-owned compatibility cleanup can become permanent legacy code. Future base-image changes may accidentally recreate `.hermes-data` or change the migration assumptions without a clear removal trigger or source-level test failure.
  • Recommended action: Extend the Dockerfile comment or nearby tests with a concrete removal checklist for `.hermes-data`: identify the old base/source boundary, state why caller-selected/local stale bases still require the migration, assert that `agents/hermes/Dockerfile.base` does not create `/sandbox/.hermes-data`, and define the supported-base cutoff after which the migration can be removed.
  • Expected follow-up: Resolve in this PR or explain why the risk is acceptable.
  • Verification: Inspect the comment beginning `# Flatten stale published base images that still contain the old .hermes-data` and the tests in `test/hermes-final-image-layout.test.ts`; they cover migration behavior and symlink refusal but not a source-level non-regression or removal condition.
  • Missing regression test: A focused source contract test named like `Hermes base Dockerfile does not create legacy /sandbox/.hermes-data` plus a comment/removal checklist tied to the minimum supported Hermes base version or digest.
  • Done when: The risk is fixed or explicitly justified in the PR. Verification: Inspect the comment beginning `# Flatten stale published base images that still contain the old .hermes-data` and the tests in `test/hermes-final-image-layout.test.ts`; they cover migration behavior and symlink refusal but not a source-level non-regression or removal condition.
  • Evidence: `agents/hermes/Dockerfile` retains migration logic for `/sandbox/.hermes-data` and new tests cover migration/refusal, but no changed file states a removal condition for that compatibility path.

💡 In-scope improvements

These are lower-risk, not throwaway. Prefer fixing them in this PR when they are local to changed code; defer only with rationale or a linked follow-up.

  • None.
Test follow-ups to resolve or justify

If these cover changed behavior, prefer adding them in this PR; otherwise state why existing coverage is enough or link the follow-up.

  • PRA-T1 Runtime validation — Hermes Dockerfile fails closed when BASE_IMAGE contains retired /sandbox/.openclaw directory. The PR changes sandbox image layout enforcement, Dockerfile base selection, and self-hosted workflow validation. Existing tests cover extracted shell behavior and final image inspection, but the deleted verifier previously exercised real Docker builds against synthetic stale bases.
  • PRA-T2 Runtime validation — Hermes Dockerfile fails closed when BASE_IMAGE contains retired /sandbox/.openclaw symlink without following or deleting the symlink target. The PR changes sandbox image layout enforcement, Dockerfile base selection, and self-hosted workflow validation. Existing tests cover extracted shell behavior and final image inspection, but the deleted verifier previously exercised real Docker builds against synthetic stale bases.
  • PRA-T3 Runtime validation — Hermes base resolver rejects a candidate image containing /sandbox/.hermes-data or /sandbox/.openclaw and falls back to the next candidate or local build. The PR changes sandbox image layout enforcement, Dockerfile base selection, and self-hosted workflow validation. Existing tests cover extracted shell behavior and final image inspection, but the deleted verifier previously exercised real Docker builds against synthetic stale bases.
  • PRA-T4 Runtime validation — Hermes base Dockerfile does not create legacy /sandbox/.hermes-data. The PR changes sandbox image layout enforcement, Dockerfile base selection, and self-hosted workflow validation. Existing tests cover extracted shell behavior and final image inspection, but the deleted verifier previously exercised real Docker builds against synthetic stale bases.
  • PRA-T5 Restore real Docker-build coverage for stale Hermes base refusal — Add or identify a focused runtime/integration test in this PR that builds a minimal synthetic Hermes base with `/sandbox/.openclaw` as a directory and as a symlink, then asserts `docker build -f agents/hermes/Dockerfile --build-arg BASE_IMAGE=<synthetic>` fails closed without following/deleting the symlink target. If resolver behavior is the intended enforcement point for published candidates, also cover stale candidate rejection with a stubbed `docker` or minimal local-image scenario rather than only string containment.
  • PRA-T6 Acceptance clause — No deterministic linked issue clauses were available in the validation context. — add test evidence or identify existing coverage. The synthetic validation context reported `linkedIssues: []`; PR body references to issues were treated as untrusted descriptive evidence rather than authoritative issue acceptance clauses.
  • PRA-T7 Acceptance clause — Keep published-image provenance in the resolver: export immutable official GHCR digests and reject candidates containing `.openclaw` or `.hermes-data` state, while preserving local rebuild and caller-selected base refs in the final Dockerfile. — add test evidence or identify existing coverage. The resolver now runs `layout_ok`, extracts `${image}@sha256:` repo digests, and exports either `HERMES_BASE_IMAGE=${digest_ref}` or `nemoclaw-hermes-base-local`; `test/pr-workflow-contract.test.ts` checks these strings. Behavioral resolver coverage for stale candidate rejection/fallback is not present.
  • PRA-T8 Acceptance clause — Add a focused final-image layout suite covering immutable official refs, OpenClaw-state refusal, the retained `.hermes-data` migration, and symlink refusal; extend live secret-boundary inspection to both retired paths. — add test evidence or identify existing coverage. `test/hermes-final-image-layout.test.ts` covers OpenClaw directory/symlink refusal, `.hermes-data` content migration, and `.hermes-data` symlink refusal including nested symlinks. `test/e2e/test-hermes-sandbox-secret-boundary.sh` checks both retired paths. Immutable official ref behavior is only covered by resolver string checks, not a focused behavioral test.
Since last review details

Current findings, using the urgency labels above:

PRA-1 Resolve/justify — Source-of-truth review needed: Hermes final-image compatibility migration for `/sandbox/.hermes-data`

  • Location: not file-specific
  • Category: architecture
  • Problem: The advisor marked localized patch analysis as needs_followup.
  • Impact: A localized workaround can preserve or hide an invalid state when the source boundary is unclear.
  • Recommended action: Identify the invalid state, source boundary, source-fix constraint, regression test, and removal condition before merging the localized behavior.
  • Expected follow-up: Resolve in this PR or explain why the risk is acceptable.
  • Verification: Inspect the localized patch and source-of-truth review fields for a concrete invalid state, source boundary, source-fix constraint, regression test, and removal condition.
  • Missing regression test: `test/hermes-final-image-layout.test.ts` covers content migration and symlink refusal, including nested symlinks.
  • Done when: The risk is fixed or explicitly justified in the PR. Verification: Inspect the localized patch and source-of-truth review fields for a concrete invalid state, source boundary, source-fix constraint, regression test, and removal condition.
  • Evidence: `agents/hermes/Dockerfile` retains `.hermes-data` migration logic; no changed file adds a removal checklist or source-level assertion that `agents/hermes/Dockerfile.base` cannot recreate the legacy path.

PRA-2 Resolve/justify — Restore real Docker-build coverage for stale Hermes base refusal

  • Location: scripts/verify-hermes-stale-openclaw-image.sh:1
  • Category: tests
  • Problem: This PR deletes the dedicated stale OpenClaw verifier script and self-hosted workflow job that built synthetic stale Hermes bases through the real `agents/hermes/Dockerfile`. The replacement `test/hermes-final-image-layout.test.ts` exercises the extracted RUN block in a host shell, and the resolver contract test only checks string fragments. That covers most shell logic, but it no longer proves the actual Docker build fails closed when a selected base image contains `/sandbox/.openclaw` as a directory or symlink, nor that resolver candidate rejection behaves correctly end-to-end.
  • Impact: A regression in Dockerfile build-time behavior, ARG/FROM interaction, shell environment, or resolver fallback could let retired OpenClaw state reach a Hermes final image or break legitimate local fallback builds without being caught by the focused unit-style tests.
  • Recommended action: Add or identify a focused runtime/integration test in this PR that builds a minimal synthetic Hermes base with `/sandbox/.openclaw` as a directory and as a symlink, then asserts `docker build -f agents/hermes/Dockerfile --build-arg BASE_IMAGE=<synthetic>` fails closed without following/deleting the symlink target. If resolver behavior is the intended enforcement point for published candidates, also cover stale candidate rejection with a stubbed `docker` or minimal local-image scenario rather than only string containment.
  • Expected follow-up: Resolve in this PR or explain why the risk is acceptable.
  • Verification: Read the diff for `scripts/verify-hermes-stale-openclaw-image.sh` deletion and `.github/workflows/pr-self-hosted.yaml` job removal, then compare with `test/hermes-final-image-layout.test.ts` and `test/pr-workflow-contract.test.ts` to confirm the remaining coverage is extracted-shell/text-contract rather than a real Docker-build negative test.
  • Missing regression test: A behavior-specific test named like `Hermes Dockerfile fails closed when BASE_IMAGE contains retired /sandbox/.openclaw directory or symlink` that builds synthetic stale base images and asserts the final image build fails with the retired-state error while preserving the symlink target sentinel.
  • Done when: The risk is fixed or explicitly justified in the PR. Verification: Read the diff for `scripts/verify-hermes-stale-openclaw-image.sh` deletion and `.github/workflows/pr-self-hosted.yaml` job removal, then compare with `test/hermes-final-image-layout.test.ts` and `test/pr-workflow-contract.test.ts` to confirm the remaining coverage is extracted-shell/text-contract rather than a real Docker-build negative test.
  • Evidence: Deleted verifier previously built `nemoclaw-hermes-stale-openclaw-dir-base` and `nemoclaw-hermes-stale-openclaw-link-base` through the real Dockerfile; new coverage in `test/hermes-final-image-layout.test.ts` calls `dockerRunCommandBetween(...)` and `runDockerShell(...)` on an extracted RUN snippet.

PRA-3 Resolve/justify — Document the retained .hermes-data migration source and removal condition

  • Location: agents/hermes/Dockerfile:252
  • Category: correctness
  • Problem: The PR intentionally keeps a compatibility migration for legacy `/sandbox/.hermes-data` while rejecting retired `/sandbox/.openclaw`. The new comment explains the invalid state and that local/rebuilt/caller-selected bases remain supported, but it does not state where `.hermes-data` can still be created, why that source cannot be eliminated in this PR, what source-level regression prevents new bases from reintroducing it, or when this migration can be removed.
  • Impact: Without a source-of-truth contract, this root-owned compatibility cleanup can become permanent legacy code. Future base-image changes may accidentally recreate `.hermes-data` or change the migration assumptions without a clear removal trigger or source-level test failure.
  • Recommended action: Extend the Dockerfile comment or nearby tests with a concrete removal checklist for `.hermes-data`: identify the old base/source boundary, state why caller-selected/local stale bases still require the migration, assert that `agents/hermes/Dockerfile.base` does not create `/sandbox/.hermes-data`, and define the supported-base cutoff after which the migration can be removed.
  • Expected follow-up: Resolve in this PR or explain why the risk is acceptable.
  • Verification: Inspect the comment beginning `# Flatten stale published base images that still contain the old .hermes-data` and the tests in `test/hermes-final-image-layout.test.ts`; they cover migration behavior and symlink refusal but not a source-level non-regression or removal condition.
  • Missing regression test: A focused source contract test named like `Hermes base Dockerfile does not create legacy /sandbox/.hermes-data` plus a comment/removal checklist tied to the minimum supported Hermes base version or digest.
  • Done when: The risk is fixed or explicitly justified in the PR. Verification: Inspect the comment beginning `# Flatten stale published base images that still contain the old .hermes-data` and the tests in `test/hermes-final-image-layout.test.ts`; they cover migration behavior and symlink refusal but not a source-level non-regression or removal condition.
  • Evidence: `agents/hermes/Dockerfile` retains migration logic for `/sandbox/.hermes-data` and new tests cover migration/refusal, but no changed file states a removal condition for that compatibility path.

Workflow run details

This is an automated, non-binding review; it still expects maintainers and agents to respond to each required or warning item. Treat suggestions as current-PR improvements when they touch changed code; defer only with maintainer rationale or a linked follow-up. A human maintainer must make the final merge decision.

@coderabbitai coderabbitai Bot left a comment

Copy link
Copy Markdown
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Caution

Some comments are outside the diff and can’t be posted inline due to platform limitations.

⚠️ Outside diff range comments (1)
test/sandbox-provisioning.test.ts (1)

1291-1306: 🔒 Security & Privacy | 🟡 Minor | ⚡ Quick win

Add a Hermes negative-path test for legacy .hermes-data cleanup. test/sandbox-provisioning.test.ts only covers the successful migration path today, so the Dockerfile’s symlink-rejection branches can still regress without a failing test. Add one case that starts with a legacy .hermes-data symlink and asserts the build exits non-zero.

🤖 Prompt for AI Agents
Verify each finding against current code. Fix only still-valid issues, skip the
rest with a brief reason, keep changes minimal, and validate.

In `@test/sandbox-provisioning.test.ts` around lines 1291 - 1306, Add a
negative-path test in sandbox-provisioning to cover legacy .hermes-data cleanup
failure: extend the existing docker-based helper around
runDockerShell/dockerRunCommandBetween so one case starts with a preexisting
.hermes-data symlink and verifies the build fails with a non-zero exit. Reuse
the current sandbox setup in test/sandbox-provisioning.test.ts and the migration
setup around precreateConfig, but assert the symlink-rejection path instead of
the successful config migration.

Source: Path instructions

🤖 Prompt for all review comments with AI agents
Verify each finding against current code. Fix only still-valid issues, skip the
rest with a brief reason, keep changes minimal, and validate.

Outside diff comments:
In `@test/sandbox-provisioning.test.ts`:
- Around line 1291-1306: Add a negative-path test in sandbox-provisioning to
cover legacy .hermes-data cleanup failure: extend the existing docker-based
helper around runDockerShell/dockerRunCommandBetween so one case starts with a
preexisting .hermes-data symlink and verifies the build fails with a non-zero
exit. Reuse the current sandbox setup in test/sandbox-provisioning.test.ts and
the migration setup around precreateConfig, but assert the symlink-rejection
path instead of the successful config migration.

ℹ️ Review info
⚙️ Run configuration

Configuration used: Path: .coderabbit.yaml

Review profile: CHILL

Plan: Enterprise

Run ID: 962818d7-70c1-40cc-8d68-32a0a743eaa6

📥 Commits

Reviewing files that changed from the base of the PR and between d2d0e7b and e3dd76b.

📒 Files selected for processing (7)
  • .github/workflows/pr-self-hosted.yaml
  • agents/hermes/Dockerfile
  • scripts/verify-hermes-stale-openclaw-image.sh
  • test/helpers/hermes-dockerfile-run.ts
  • test/hermes-stale-openclaw-guard.test.ts
  • test/pr-workflow-contract.test.ts
  • test/sandbox-provisioning.test.ts
💤 Files with no reviewable changes (4)
  • scripts/verify-hermes-stale-openclaw-image.sh
  • test/hermes-stale-openclaw-guard.test.ts
  • test/pr-workflow-contract.test.ts
  • .github/workflows/pr-self-hosted.yaml

@github-actions

Copy link
Copy Markdown
Contributor

Vitest E2E Scenario Results — ✅ All requested jobs passed

Run: 28339310217
Workflow ref: codex/retire-hermes-stale-openclaw-guard
Requested scenarios: (default — all supported)
Requested jobs: hermes-root-entrypoint-smoke-vitest,hermes-sandbox-secret-boundary-vitest,hermes-e2e-vitest
Summary: 3 passed, 0 failed, 0 cancelled, 0 skipped

Job Result
hermes-e2e-vitest ✅ success
hermes-root-entrypoint-smoke-vitest ✅ success
hermes-sandbox-secret-boundary-vitest ✅ success

@coderabbitai coderabbitai Bot left a comment

Copy link
Copy Markdown
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Actionable comments posted: 3

🤖 Prompt for all review comments with AI agents
Verify each finding against current code. Fix only still-valid issues, skip the
rest with a brief reason, keep changes minimal, and validate.

Inline comments:
In @.github/actions/resolve-hermes-base-image/action.yaml:
- Around line 29-39: The layout_ok check is too strict because it rejects any
/sandbox/.hermes-data entry, but the Hermes Dockerfile still allows a real
directory there and migrates it. Update layout_ok in resolve-hermes-base-image
action to only fail on retired sandbox state that is actually invalid for the
final image contract (for example, symlink forms) while allowing a real
.hermes-data directory so try_image() does not incorrectly skip a valid
published digest.

In `@test/hermes-final-image-layout.test.ts`:
- Around line 42-70: The new hermes final image layout test is tripping the
conditional guardrail because it contains too many inline branches in the
fixture setup and OpenClaw assertions. Refactor the setup logic in the affected
test to use table-driven cases or small helper functions instead of multiple
if/else blocks, and move any OpenClaw-specific assertion into a separate helper
or dedicated test. Keep the unique fixtures grouped by scenario using the
existing hermes fixture builder and related test symbols so the behavior stays
the same while reducing conditional count.
- Line 117: The `.hermes-data` migration assertion in the test is too weak
because `fs.existsSync()` can return false for a dangling symlink, so it may not
prove the old path was actually removed. Update the check in
`hermes-final-image-layout.test.ts` to use a stronger existence/removal
assertion against `run.sandboxRoot` and `.hermes-data`, so the test verifies the
superseded path is truly gone or unreachable rather than merely non-resolvable.
🪄 Autofix (Beta)

Fix all unresolved CodeRabbit comments on this PR:

  • Push a commit to this branch (recommended)
  • Create a new PR with the fixes

ℹ️ Review info
⚙️ Run configuration

Configuration used: Path: .coderabbit.yaml

Review profile: CHILL

Plan: Enterprise

Run ID: 98bd0e74-ce9e-4fcb-aee2-689cbb8df787

📥 Commits

Reviewing files that changed from the base of the PR and between e3dd76b and e87b6b4.

📒 Files selected for processing (5)
  • .github/actions/resolve-hermes-base-image/action.yaml
  • agents/hermes/Dockerfile
  • test/e2e/test-hermes-sandbox-secret-boundary.sh
  • test/hermes-final-image-layout.test.ts
  • test/pr-workflow-contract.test.ts
🚧 Files skipped from review as they are similar to previous changes (2)
  • agents/hermes/Dockerfile
  • test/pr-workflow-contract.test.ts

Comment thread .github/actions/resolve-hermes-base-image/action.yaml
Comment thread test/hermes-final-image-layout.test.ts Outdated
Comment thread test/hermes-final-image-layout.test.ts Outdated
@github-actions

Copy link
Copy Markdown
Contributor

Vitest E2E Scenario Results — ✅ All requested jobs passed

Run: 28339588125
Workflow ref: codex/retire-hermes-stale-openclaw-guard
Requested scenarios: (default — all supported)
Requested jobs: hermes-root-entrypoint-smoke-vitest,hermes-sandbox-secret-boundary-vitest,hermes-e2e-vitest
Summary: 3 passed, 0 failed, 0 cancelled, 0 skipped

Job Result
hermes-e2e-vitest ✅ success
hermes-root-entrypoint-smoke-vitest ✅ success
hermes-sandbox-secret-boundary-vitest ✅ success

@github-actions

Copy link
Copy Markdown
Contributor

Selective E2E Results — ✅ All requested jobs passed

Run: 28339640498
Target ref: e87b6b435631c2887789fa4fe976443433f6ab1d
Workflow ref: main
Requested jobs: hermes-secret-boundary-e2e,hermes-root-entrypoint-smoke-e2e
Summary: 2 passed, 0 failed, 0 cancelled, 0 skipped

Job Result
hermes-root-entrypoint-smoke-e2e ✅ success
hermes-secret-boundary-e2e ✅ success

@github-actions

Copy link
Copy Markdown
Contributor

Vitest E2E Scenario Results — ⚠️ Run cancelled — no signal

Run: 28339757271
Workflow ref: codex/retire-hermes-stale-openclaw-guard
Requested scenarios: (default — all supported)
Requested jobs: hermes-root-entrypoint-smoke-vitest,hermes-sandbox-secret-boundary-vitest,hermes-e2e-vitest
Summary: 0 passed, 0 failed, 3 cancelled, 0 skipped

Job Result
hermes-e2e-vitest ⚠️ cancelled
hermes-root-entrypoint-smoke-vitest ⚠️ cancelled
hermes-sandbox-secret-boundary-vitest ⚠️ cancelled

Signed-off-by: Carlos Villela <cvillela@nvidia.com>
@cv

cv commented Jun 28, 2026

Copy link
Copy Markdown
Collaborator Author

Advisor disposition for final head de8e3ba62:

  • PRA-5 / PRA-6 / PRA-8 / PRA-11 / PRA-17 — do not apply the requested registry allowlist. src/lib/agent/onboard.ts deliberately passes the public NEMOCLAW_HERMES_SANDBOX_BASE_IMAGE_REF override into resolveSandboxBaseImage(), whose override path accepts a caller-selected registry after pull and ABI validation. The Dockerfile therefore cannot reject all non-NVIDIA registries without breaking that supported escape hatch. The removed allowlist was scoped to the temporary stale-OpenClaw repair and its synthetic verifier tags; those tags no longer exist. This PR keeps the permanent boundary it can enforce without changing that contract: every ghcr.io/nvidia/nemoclaw/hermes-sandbox-base reference must be immutable. A caller who explicitly selects another base owns that base's trust decision, and the final-image layout checks still fail closed on retired state.
  • PRA-9 is based on stale test analysis. The content fixture creates top-level file/directory symlinks and a nested .hermes/profiles/legacy-sessions symlink into .hermes-data; the test proves all three are materialized and no longer symlinks. de8e3ba62 additionally rejects symlinks anywhere inside the legacy data tree before copying and preserves the external sentinel in the negative test.
  • PRA-10 is already enforced on every resolver use. layout_ok() checks each SHA/latest GHCR candidate before exporting its immutable digest, and checks the local source build before accepting it. A second scheduled copy of the same rule would add machinery without strengthening the selection boundary. The default pinned digest was also pulled and inspected directly.
  • PRA-16 is not introduced by layout_ok(). Candidate compatibility already requires running the fixed-repository image's /usr/bin/ldd before layout inspection. Changing the resolver to inspect an unexecuted image filesystem would be a broader trust-model redesign, not a localized fix for this lifecycle cleanup.
  • PRA-1 through PRA-4 / PRA-7 / PRA-13: the invalid source was immutable, already-published legacy image state; Dockerfile.base is the clean source of truth, the resolver now prevents published-state regression, and the retained final/runtime migration remains compatibility defense for direct/local builds. I am not adding an unowned TODO or a second repository-local tracker; this PR's removal trigger and evidence are recorded here and in the linked GitHub work.

Runtime evidence for PRA-T1PRA-T8 is exact-head Vitest E2E run 28339846140, covering Hermes install/live inference, root entrypoint, final-image secret/layout boundary, and stale-base rebuild. Local focused coverage is 62 passing tests, including all seven final-image layout cases.

@github-actions

Copy link
Copy Markdown
Contributor

Selective E2E Results — ✅ All requested jobs passed

Run: 28339907806
Target ref: de8e3ba62da8304f073fd18ca85174d2f5e3653c
Workflow ref: main
Requested jobs: hermes-secret-boundary-e2e,hermes-root-entrypoint-smoke-e2e
Summary: 2 passed, 0 failed, 0 cancelled, 0 skipped

Job Result
hermes-root-entrypoint-smoke-e2e ✅ success
hermes-secret-boundary-e2e ✅ success

@github-actions

Copy link
Copy Markdown
Contributor

Vitest E2E Scenario Results — ❌ Some jobs failed

Run: 28339846140
Workflow ref: codex/retire-hermes-stale-openclaw-guard
Requested scenarios: (default — all supported)
Requested jobs: hermes-root-entrypoint-smoke-vitest,hermes-sandbox-secret-boundary-vitest,hermes-e2e-vitest,rebuild-hermes-stale-base-vitest
Summary: 3 passed, 1 failed, 0 cancelled, 0 skipped

Job Result
hermes-e2e-vitest ✅ success
hermes-root-entrypoint-smoke-vitest ✅ success
hermes-sandbox-secret-boundary-vitest ✅ success
rebuild-hermes-stale-base-vitest ❌ failure

Failed jobs: rebuild-hermes-stale-base-vitest. Check run artifacts for logs.

Signed-off-by: Carlos Villela <cvillela@nvidia.com>
@github-actions

Copy link
Copy Markdown
Contributor

Selective E2E Results — ✅ All requested jobs passed

Run: 28340700087
Target ref: ce85e762aebf57f2358d75fd36108520f33070c8
Workflow ref: main
Requested jobs: hermes-secret-boundary-e2e,hermes-root-entrypoint-smoke-e2e
Summary: 2 passed, 0 failed, 0 cancelled, 0 skipped

Job Result
hermes-root-entrypoint-smoke-e2e ✅ success
hermes-secret-boundary-e2e ✅ success

@github-actions

Copy link
Copy Markdown
Contributor

Vitest E2E Scenario Results — ✅ All requested jobs passed

Run: 28340627350
Workflow ref: codex/retire-hermes-stale-openclaw-guard
Requested scenarios: (default — all supported)
Requested jobs: hermes-root-entrypoint-smoke-vitest,hermes-sandbox-secret-boundary-vitest,hermes-e2e-vitest,rebuild-hermes-stale-base-vitest
Summary: 4 passed, 0 failed, 0 cancelled, 0 skipped

Job Result
hermes-e2e-vitest ✅ success
hermes-root-entrypoint-smoke-vitest ✅ success
hermes-sandbox-secret-boundary-vitest ✅ success
rebuild-hermes-stale-base-vitest ✅ success

@cv

cv commented Jun 29, 2026

Copy link
Copy Markdown
Collaborator Author

Final-head evidence for ce85e762a:

  • Vitest E2E run 28340627350 passed all four requested jobs: Hermes install/live inference, root entrypoint, final-image secret/layout boundary, and stale-base rebuild.
  • The first run's failed rebuild was useful negative evidence: Dockerfile-level registry validation rejected NemoClaw's intentional forced local ghcr…:latest cache rebuild after sandbox deletion. Moving provenance back to the resolver fixed that separation-of-concerns error; the exact same stale-cache scenario now completes the rebuild, preserves state/config, upgrades Hermes, and returns PONG after recreation.
  • Downloaded exact-head artifacts contain no credential-shaped matches. The rebuild has no nonzero command results, its backup credential scan reports leaks: [], and every scenario cleanup reports failures: [].
  • All 39 non-skipped PR checks pass; CodeRabbit has no remaining actionable finding. Local hooks and the focused 61-test integration slice pass.

Advisor dispositions on the final head:

  • Base provenance belongs at resolution: the Actions resolver exports immutable official GHCR digests and rejects retired layouts; the CLI resolver pins published candidates when a repo digest exists. The final Dockerfile must accept locally rebuilt and explicit caller-selected bases, then enforce the filesystem/layout boundary. The live stale-cache regression demonstrates why duplicating registry policy in the Dockerfile is incorrect.
  • The retained .hermes-data path is compatibility for immutable old or explicitly selected local bases. Dockerfile.base is exercised by the exact-head root/secret-boundary image builds, and the resulting images prove both retired paths absent. The focused suite covers content migration plus directory, entry, and nested-symlink refusal. Removing compatibility is a future product-support cutoff, not something this lifecycle fix can infer safely.
  • The deleted synthetic verifier existed to support and police a temporary repair that removed stale .openclaw directories. That repair is now gone: selected published bases are rejected by the resolver when dirty, and the final layout fails closed on either a directory or symlink using the exact Dockerfile RUN block. Keeping a dedicated 190-line script and self-hosted job after its documented trigger fired would preserve the migration scaffolding this PR is intended to retire.

The remaining gate is the PR template's independent sensitive-path human review; I have intentionally left that checkbox unchecked.

@wscurran wscurran added area: sandbox OpenShell sandbox lifecycle, runtime, config, or recovery bug-fix PR fixes a bug or regression integration: hermes Hermes integration behavior labels Jun 29, 2026
@cv
cv merged commit 87058e0 into main Jun 29, 2026
129 of 133 checks passed
@cv
cv deleted the codex/retire-hermes-stale-openclaw-guard branch June 29, 2026 01:58
@cv cv added the v0.0.70 label Jun 29, 2026
@miyoungc miyoungc mentioned this pull request Jun 30, 2026
21 tasks
jyaunches pushed a commit that referenced this pull request Jun 30, 2026
## Summary
Refreshes the v0.0.70 release docs from the release announcement and the
`v0.0.69..v0.0.70` commit range.
It also documents the `channels start` policy restoration behavior that
was missing from the shared OpenClaw and Hermes command references, and
bumps the Fern CLI version used for docs validation.

## Changes
- Replaced the stale `v0.0.70` release-notes entry with the actual
release themes, including CLI, onboarding, inference, messaging,
Windows, documentation, and release-validation changes.
- Documented that `channels start` reapplies the matching built-in
network policy preset before rebuild and rolls back to disabled if
policy restoration fails.
- Bumped `fern/fern.config.json` from `5.55.0` to `5.59.0` for the docs
refresh.
- Source summary:
- #5754 -> `docs/about/release-notes.mdx`: Notes Docker Desktop gateway
bridge retry behavior during onboarding.
- #5930 -> `docs/about/release-notes.mdx`: Links `nemoclaw use` default
sandbox selection to the command reference.
- #5948 -> `docs/about/release-notes.mdx`: Links reasoning-compatible
endpoint validation to inference documentation.
- #5950 -> `docs/about/release-notes.mdx`: Links Windows bootstrap WSL
recovery behavior to Windows preparation and troubleshooting docs.
- #5856 -> `docs/about/release-notes.mdx`: Notes rebuilt policy preset
registry repair.
- #5882 and #5949 -> `docs/about/release-notes.mdx`: Notes Hermes stale
base-image state repair.
- #6016 -> `docs/reference/commands.mdx`,
`docs/reference/commands-nemohermes.mdx`, and
`docs/manage-sandboxes/messaging-channels.mdx`: Documents channel policy
restoration and rollback on `channels start`.
- #5859 -> `docs/about/release-notes.mdx`: Links quickstart network
approval guidance.
- #5863 -> `docs/about/release-notes.mdx`: Links Teams allowlist
guidance in the messaging page.
- #5756, #5926, #6010, and #6011 -> `docs/about/release-notes.mdx`:
Summarizes the Vitest E2E validation cutover.

## Type of Change

- [ ] Code change (feature, bug fix, or refactor)
- [ ] Code change with doc updates
- [x] Doc only (prose changes, no code sample modifications)
- [ ] Doc only (includes code sample changes)

## Quality Gates
- [ ] Tests added or updated for changed behavior
- [ ] Existing tests cover changed behavior — justification:
- [x] Tests not applicable — justification: doc-only prose refresh with
no runtime behavior change.
- [x] Docs updated for user-facing behavior changes
- [ ] Docs not applicable — justification:
- [ ] Sensitive paths changed (security, policy, credentials, preflight,
onboarding, inference, runner, sandbox, or messaging)
- [ ] Sensitive-path review completed or maintainer-approved waiver
recorded — reviewer/approval link/justification:
- [ ] Non-success, skipped, or missing CI check accepted by maintainer —
check name, approval link, and follow-up issue:

## Verification
- [x] PR description includes the DCO sign-off declaration and every
commit appears as `Verified` in GitHub
- [x] Git hooks passed during commit and push, or `npx prek run
--from-ref main --to-ref HEAD` passes
- [ ] Targeted tests pass for changed behavior
- [ ] Full `npm test` passes (broad runtime changes only)
- [x] Quality Gates section completed with required justifications or
waivers
- [x] No secrets, API keys, or credentials committed
- [ ] `npm run docs` builds without warnings (doc changes only)
- [x] Doc pages follow the [style
guide](https://github.com/NVIDIA/NemoClaw/blob/main/docs/CONTRIBUTING.md)
(doc changes only)
- [ ] New doc pages include SPDX header and frontmatter (new pages only)

`npm run docs` exited 0 and Fern reported one existing light-mode accent
contrast warning.
`fern check --warnings` confirmed the warning is the site theme contrast
ratio, not content introduced by this PR.

---
Signed-off-by: Miyoung Choi <miyoungc@nvidia.com>
Hadar301 pushed a commit to Hadar301/NemoClaw-OpenShift that referenced this pull request Jul 12, 2026
<!-- markdownlint-disable MD041 -->
## Summary
Advance the default Hermes final-image base to the newly published
post-stale-layout digest and retire the temporary digest-coupled repair
that intentionally rejected newer published bases. This restores Hermes
onboarding and live E2E builds after the `latest` base moved, while
keeping current-state fail-closed layout checks and the older
`.hermes-data` compatibility migration.

## Related Issue

Unblocks NVIDIA#5947. Follow-up to NVIDIA#5882.

## Changes

- Pin the default Hermes base to
`sha256:8dad3b989a9ed1e601743310b97be21be5f59f89f7913a47d04f3ec3c40b8ce6`,
whose published ARM64 image has neither `/sandbox/.openclaw` nor
`/sandbox/.hermes-data`.
- Remove the fired `NEMOCLAW_STALE_OPENCLAW_BASE_DIGEST` guard and
temporary `/sandbox/.openclaw` repair; replace it with a small
fail-closed invariant that rejects retired OpenClaw state rather than
maintaining it.
- Keep published-image provenance in the resolver: export immutable
official GHCR digests and reject candidates containing `.openclaw` or
`.hermes-data` state, while preserving local rebuild and caller-selected
base refs in the final Dockerfile.
- Delete the dedicated stale-layout verifier job, script, helpers, and
tests now that their documented removal trigger has fired.
- Add a focused final-image layout suite covering OpenClaw-state
refusal, the retained `.hermes-data` migration, and symlink refusal;
keep immutable published-ref coverage at the resolver boundary and
extend live secret-boundary inspection to both retired paths.
- Reject nested symlinks before copying retained `.hermes-data`
compatibility state so migration cannot preserve links outside the
legacy tree.

## Type of Change

- [x] Code change (feature, bug fix, or refactor)
- [ ] Code change with doc updates
- [ ] Doc only (prose changes, no code sample modifications)
- [ ] Doc only (includes code sample changes)

## Quality Gates
<!-- Check all that apply. For any "covered by existing tests", "not
applicable", or waiver entry, add a brief justification on the same line
or in the Changes section. -->
- [x] Tests added or updated for changed behavior
- [ ] Existing tests cover changed behavior — justification:
- [ ] Tests not applicable — justification:
- [ ] Docs updated for user-facing behavior changes
- [x] Docs not applicable — justification: internal published-image
lifecycle cleanup; CLI behavior, configuration, supported workflows, and
the documented Hermes state location are unchanged.
- [x] Sensitive paths changed (security, policy, credentials, preflight,
onboarding, inference, runner, sandbox, or messaging)
- [ ] Sensitive-path review completed or maintainer-approved waiver
recorded — reviewer/approval link/justification: pending independent
review of the Hermes sandbox image boundary.
- [ ] Non-success, skipped, or missing CI check accepted by maintainer —
check name, approval link, and follow-up issue:

## Verification
<!-- Check each item you ran and confirmed. Leave unchecked items you
skipped. Doc-only changes do not require npm test unless you ran it. -->
- [x] PR description includes the DCO sign-off declaration and every
commit appears as `Verified` in GitHub
- [x] Git hooks passed during commit and push, or `npx prek run
--from-ref main --to-ref HEAD` passes
- [x] Targeted tests pass for changed behavior
- [ ] Full `npm test` passes (broad runtime changes only)
- [ ] Quality Gates section completed with required justifications or
waivers
- [x] No secrets, API keys, or credentials committed
- [ ] `npm run docs` builds without warnings (doc changes only)
- [ ] Doc pages follow the [style
guide](https://github.com/NVIDIA/NemoClaw/blob/main/docs/CONTRIBUTING.md)
(doc changes only)
- [ ] New doc pages include SPDX header and frontmatter (new pages only)

Verification evidence:

- Pulled and inspected the published ARM64 `8dad…` image;
`/sandbox/.openclaw` and `/sandbox/.hermes-data` are absent.
- Built `agents/hermes/Dockerfile` against the exact `8dad…` digest and
verified sandbox-user readability, runtime permissions, runtime
symlinks, and absence of both stale paths.
- Focused integration run passed 61 tests across
`hermes-final-image-layout`, `hermes-doctor-config-hash`,
`pr-workflow-contract`, and `sandbox-provisioning`.
- `npm run build:cli`, `npm run typecheck:cli`, `npm run checks`, and
`npm run test:projects:check` passed.
- Normal commit and push hooks passed for all commits, including the
full CLI test lane, shellcheck, hadolint, repository checks,
source-shape and test-size budgets, gitleaks, commitlint, and CLI
TypeScript checks.
- [Vitest E2E run
28339846140](https://github.com/NVIDIA/NemoClaw/actions/runs/28339846140)
passed Hermes install/inference, root entrypoint, and secret-boundary
jobs, then exposed that Dockerfile-level registry validation rejected
the forced local `:latest` base-cache rebuild. Commit `ce85e762a` moves
provenance back to the resolver boundary; [exact-head rerun
28340627350](https://github.com/NVIDIA/NemoClaw/actions/runs/28340627350)
passed all four jobs, including stale-base rebuild.
- Downloaded exact-head artifacts have no credential-shaped matches;
every command result in the rebuild artifact is zero, post-rebuild
inference returned `PONG`, backup scanning found no leaks, and all
scenario cleanups report no failures.
- Required documentation writer assessment, rerun after the fail-closed
review fixes, reported no user-facing docs changes needed.

---
<!-- DCO sign-off is required in this PR description, and every commit
must appear as Verified in GitHub. Run: git config user.name && git
config user.email -->
Signed-off-by: Carlos Villela <cvillela@nvidia.com>

---------

Signed-off-by: Carlos Villela <cvillela@nvidia.com>
Hadar301 pushed a commit to Hadar301/NemoClaw-OpenShift that referenced this pull request Jul 12, 2026
## Summary
Refreshes the v0.0.70 release docs from the release announcement and the
`v0.0.69..v0.0.70` commit range.
It also documents the `channels start` policy restoration behavior that
was missing from the shared OpenClaw and Hermes command references, and
bumps the Fern CLI version used for docs validation.

## Changes
- Replaced the stale `v0.0.70` release-notes entry with the actual
release themes, including CLI, onboarding, inference, messaging,
Windows, documentation, and release-validation changes.
- Documented that `channels start` reapplies the matching built-in
network policy preset before rebuild and rolls back to disabled if
policy restoration fails.
- Bumped `fern/fern.config.json` from `5.55.0` to `5.59.0` for the docs
refresh.
- Source summary:
- NVIDIA#5754 -> `docs/about/release-notes.mdx`: Notes Docker Desktop gateway
bridge retry behavior during onboarding.
- NVIDIA#5930 -> `docs/about/release-notes.mdx`: Links `nemoclaw use` default
sandbox selection to the command reference.
- NVIDIA#5948 -> `docs/about/release-notes.mdx`: Links reasoning-compatible
endpoint validation to inference documentation.
- NVIDIA#5950 -> `docs/about/release-notes.mdx`: Links Windows bootstrap WSL
recovery behavior to Windows preparation and troubleshooting docs.
- NVIDIA#5856 -> `docs/about/release-notes.mdx`: Notes rebuilt policy preset
registry repair.
- NVIDIA#5882 and NVIDIA#5949 -> `docs/about/release-notes.mdx`: Notes Hermes stale
base-image state repair.
- NVIDIA#6016 -> `docs/reference/commands.mdx`,
`docs/reference/commands-nemohermes.mdx`, and
`docs/manage-sandboxes/messaging-channels.mdx`: Documents channel policy
restoration and rollback on `channels start`.
- NVIDIA#5859 -> `docs/about/release-notes.mdx`: Links quickstart network
approval guidance.
- NVIDIA#5863 -> `docs/about/release-notes.mdx`: Links Teams allowlist
guidance in the messaging page.
- NVIDIA#5756, NVIDIA#5926, NVIDIA#6010, and NVIDIA#6011 -> `docs/about/release-notes.mdx`:
Summarizes the Vitest E2E validation cutover.

## Type of Change

- [ ] Code change (feature, bug fix, or refactor)
- [ ] Code change with doc updates
- [x] Doc only (prose changes, no code sample modifications)
- [ ] Doc only (includes code sample changes)

## Quality Gates
- [ ] Tests added or updated for changed behavior
- [ ] Existing tests cover changed behavior — justification:
- [x] Tests not applicable — justification: doc-only prose refresh with
no runtime behavior change.
- [x] Docs updated for user-facing behavior changes
- [ ] Docs not applicable — justification:
- [ ] Sensitive paths changed (security, policy, credentials, preflight,
onboarding, inference, runner, sandbox, or messaging)
- [ ] Sensitive-path review completed or maintainer-approved waiver
recorded — reviewer/approval link/justification:
- [ ] Non-success, skipped, or missing CI check accepted by maintainer —
check name, approval link, and follow-up issue:

## Verification
- [x] PR description includes the DCO sign-off declaration and every
commit appears as `Verified` in GitHub
- [x] Git hooks passed during commit and push, or `npx prek run
--from-ref main --to-ref HEAD` passes
- [ ] Targeted tests pass for changed behavior
- [ ] Full `npm test` passes (broad runtime changes only)
- [x] Quality Gates section completed with required justifications or
waivers
- [x] No secrets, API keys, or credentials committed
- [ ] `npm run docs` builds without warnings (doc changes only)
- [x] Doc pages follow the [style
guide](https://github.com/NVIDIA/NemoClaw/blob/main/docs/CONTRIBUTING.md)
(doc changes only)
- [ ] New doc pages include SPDX header and frontmatter (new pages only)

`npm run docs` exited 0 and Fern reported one existing light-mode accent
contrast warning.
`fern check --warnings` confirmed the warning is the site theme contrast
ratio, not content introduced by this PR.

---
Signed-off-by: Miyoung Choi <miyoungc@nvidia.com>
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

area: sandbox OpenShell sandbox lifecycle, runtime, config, or recovery bug-fix PR fixes a bug or regression integration: hermes Hermes integration behavior

Projects

None yet

Development

Successfully merging this pull request may close these issues.

2 participants