Skip to content

fix(openclaw): isolate managed heartbeat sessions - #11487

Merged
prekshivyas merged 10 commits into
mainfrom
codex/fix-10262-isolated-heartbeats
Sep 12, 2026
Merged

fix(openclaw): isolate managed heartbeat sessions#11487
prekshivyas merged 10 commits into
mainfrom
codex/fix-10262-isolated-heartbeats

Conversation

@jyaunches

@jyaunches jyaunches commented Sep 10, 2026

Copy link
Copy Markdown
Contributor

Outcome

An explicitly configured NemoClaw-managed OpenClaw heartbeat runs in a dedicated heartbeat session. Managed-image checks reject a main-session heartbeat, a different cadence, or a stale configuration hash.

Reason

NemoClaw previously set only agents.defaults.heartbeat.every, leaving heartbeat turns in the ordinary main session. Main-session activity could make liveness appear current without a scheduled heartbeat turn.

Related issues

Fixes #10262. Relates to #10244. Advisor dependency #11500 and main test repair #11503 have merged and are incorporated.

Changes

  • Set isolatedSession: true with explicitly configured heartbeat cadences, including 0m, without adding a public setting.
  • Qualify the managed-image fixture with 2m, its generated configuration hash, and the 120000 ms startup interval. The sandbox probe verifies the built image beyond generator-only assertions.
  • Exercise the actual probe with valid, main-session, wrong-cadence, and stale-hash fixtures. Resolve the test hash executable through PATH and quote shell inputs literally, including paths that contain quotes and shell metacharacters.
  • Document dedicated heartbeat sessions and accepted duration syntax. The restore-merger defect in Configured OpenClaw heartbeat interval is lost during managed rebuild #10244 remains separate scope.

Verification

  • Generator and managed-runtime contract suites: 197 tests passed.
  • Managed-startup environment suite: 41 tests passed.
  • Final probe contract suite with shell metacharacters in the temporary path: 48 tests passed.
  • Maintainer focused review rerun: 238 tests passed across the generator, managed-startup mapping, and protected runtime contracts.
  • CLI and plugin builds passed.
  • npm run validate:pr passed against canonical main fa97e069a0b1ae8922f112e7260a9cbc9e6d7c7f, including CLI type-checking.
  • The incorporated main test repair passes all 88 live export tests.
  • Required GitHub checks pass on candidate df172e751692babe739d9b38d936fe0c0903b81b.
  • CodeRabbit completed a full review of the candidate with no actionable findings and minimal merge risk.
  • Repository checks and signed-commit hooks passed. The diff contains no secrets, API keys, or credentials.
  • Documentation validation passed on the candidate.

Review notes

Candidate: df172e751692babe739d9b38d936fe0c0903b81b in NVIDIA/NemoClaw. Sensitive paths are the config generator and managed-image qualification scripts. Full diff review covered correctness, shell quoting, credential-safe diagnostics, test coverage, and deterministic risk selection.

The maintainer review approved this candidate. The bounded cross-issue sweep found no adjacent fix or contradiction. The complete PR Review Advisor report remains unavailable because the Advisor service is failing before specialist execution, so merge remains blocked on that evidence.

Both OpenClaw MCP discovery passes now succeed on the candidate. The second pass succeeded when rerun after its external Cloudflare tunnel process exited before readiness on all three bounded attempts.

The deterministic risk plan requires managed-image-multiarch-startup and managed-image-protected-runtime. These manual qualifications have not been dispatched. Merge remains separate from automated evaluation.


Signed-off-by: Julie Yaunches jyaunches@nvidia.com

Summary by CodeRabbit

  • New Features

    • Agent heartbeats now run in dedicated, isolated sessions.
    • Configured heartbeat intervals automatically enable isolated-session behavior.
    • Managed OpenClaw environments use a validated two-minute heartbeat interval.
  • Documentation

    • Updated heartbeat documentation to explain periodic isolated-session turns.
  • Bug Fixes

    • Improved validation of heartbeat settings, startup behavior, and configuration integrity in managed environments.

Run explicitly configured managed heartbeats in OpenClaw's dedicated heartbeat session.

Validate the isolated setting, configuration hash, and startup interval.

Signed-off-by: Julie Yaunches <jyaunches@nvidia.com>
@jyaunches jyaunches self-assigned this Sep 10, 2026
@copy-pr-bot

copy-pr-bot Bot commented Sep 10, 2026

Copy link
Copy Markdown

Auto-sync is disabled for draft pull requests in this repository. Workflows must be run manually.

Contributors can view more details about this message here.

@coderabbitai

coderabbitai Bot commented Sep 10, 2026

Copy link
Copy Markdown
Contributor

Review Change StackReview Change Stack

No actionable comments were generated in the recent review. 🎉

ℹ️ Recent review info
⚙️ Run configuration

Configuration used: Path: .coderabbit.yaml

Review profile: CHILL

Plan: Enterprise

Run ID: f503aad6-60d4-4ecc-a647-96f873f3cca4

📥 Commits

Reviewing files that changed from the base of the PR and between ffc1116 and df172e7.

📒 Files selected for processing (8)
  • docs/configure-agents/configure-agent-heartbeats.mdx
  • scripts/checks/generate-managed-startup-profile-fixture.mts
  • scripts/checks/run-managed-image-direct-e2e.ts
  • scripts/checks/run-managed-image-openshell-e2e.ts
  • scripts/generate-openclaw-config.mts
  • src/lib/onboard/managed-startup-agent-environment.test.ts
  • test/generation/generate-openclaw-config.test.ts
  • test/inference/managed/managed-image-protected-runtime-contract.test.ts

Included review availability: Your plan provides up to 12 included reviews per hour; 11 remain after this review.


📝 Walkthrough

Walkthrough

Configured OpenClaw heartbeats now use isolated sessions. Managed-image fixtures and direct and OpenShell checks validate cadence, isolation, startup behavior, configuration hashes, and protected error handling.

Changes

OpenClaw heartbeat configuration

Layer / File(s) Summary
Heartbeat configuration contract
docs/configure-agents/configure-agent-heartbeats.mdx, scripts/generate-openclaw-config.mts, src/lib/onboard/managed-startup-agent-environment.test.ts, test/generation/generate-openclaw-config.test.ts
Configured heartbeat objects now include isolatedSession: true. Documentation and generation tests cover configured and disabled 0m heartbeats.
Direct managed-image validation
scripts/checks/generate-managed-startup-profile-fixture.mts, scripts/checks/run-managed-image-direct-e2e.ts
Managed OpenClaw fixtures use a two-minute heartbeat. Direct E2E checks validate the cadence, isolated session setting, and configuration hash.
OpenShell runtime validation
scripts/checks/run-managed-image-openshell-e2e.ts, test/inference/managed/managed-image-protected-runtime-contract.test.ts
OpenShell checks validate startup logs, configuration and hash integrity, accepted and rejected heartbeat settings, stale hashes, and credential-safe errors.

Priority: ➖ Normal

Estimated code review effort: 3 (Moderate) | ~25 minutes

Change: Bug fix · Severity of issue fixed: Medium

Suggested reviewers: cv, ericksoa

Merge Risk: ⚪ Minimal · up to df172

Managed OpenClaw heartbeats now use dedicated sessions and the updated validation paths cover the configured cadence and generated configuration integrity. No merge-blocking risk remains.

🚥 Pre-merge checks | ✅ 4 | ❌ 1

❌ Failed checks (1 warning)

Check name Status Explanation Resolution
Docstring Coverage ⚠️ Warning Docstring coverage is 0.00% which is insufficient. The required threshold is 80.00%. Docstring coverage is scoped to functions touched by this diff. Analyzed 10 functions across 7 files. (1 skipped: 1… Write docstrings for the functions missing them to satisfy the coverage threshold.
✅ Passed checks (4 passed)
Check name Status Explanation
Linked Issues check ✅ Passed The PR satisfies the coding requirements in issue #10262. Config generation adds isolatedSession: true to explicitly configured managed heartbeats, including 0m. The generator tests cover configur…
Out of Scope Changes check ✅ Passed The changed files stay within issue #10262. They update heartbeat generation, lifecycle fixtures and tests, managed-image validation and probes, protected runtime contract tests, and heartbeat documen…
Description Check ✅ Passed Check skipped - CodeRabbit’s high-level summary is enabled.
Title check ✅ Passed The title clearly and concisely describes the primary change: configuring managed OpenClaw heartbeat sessions to run in isolation.
Full details: Docstring Coverage

Explanation

Docstring coverage is 0.00% which is insufficient. The required threshold is 80.00%. Docstring coverage is scoped to functions touched by this diff. Analyzed 10 functions across 7 files. (1 skipped: 1 unsupported.)

  • Fix all pre-merge checks with AI
✨ Finishing Touches 💡 2
📝 Generate docstrings 💡
  • Create stacked PR
  • Commit on current branch
🛠️ Fix failing CI checks 💡
  • Create stacked PR
  • Commit on current branch
🧪 Generate unit tests (beta)
  • Create PR with unit tests
  • Commit unit tests in branch codex/fix-10262-isolated-heartbeats

Comment @coderabbitai help to get the list of available commands.

@github-actions

Copy link
Copy Markdown
Contributor

@github-code-quality

github-code-quality Bot commented Sep 10, 2026

Copy link
Copy Markdown
Contributor

Code Coverage Overview

Languages: TypeScript

TypeScript / code-coverage/plugin

The overall line coverage in commit df172e7 in the codex/fix-10262-isol... branch remains at 96%, unchanged from commit e8071e8 in the main branch.

TypeScript / code-coverage/cli

The overall line coverage in commit df172e7 in the codex/fix-10262-isol... branch remains at 83%, unchanged from commit e8071e8 in the main branch.

Show a line coverage summary of the most impacted files.
File main e8071e8 codex/fix-10262-isol... df172e7 +/-
src/lib/onboard...uter-command.ts 65% 52% -13%
src/lib/onboard...outer-python.ts 96% 84% -12%
src/lib/onboard...eway-process.ts 90% 89% -1%
src/lib/actions...oy-preflight.ts 84% 83% -1%
src/lib/onboard...uild-context.ts 74% 75% +1%
src/lib/actions...-add-restart.ts 30% 31% +1%
src/lib/onboard...load/rebuild.ts 87% 88% +1%
src/lib/domain/...ycle/options.ts 85% 87% +2%
src/lib/actions...dbox/destroy.ts 89% 92% +3%
src/lib/actions...oy-execution.ts 91% 94% +3%

Updated September 11, 2026 20:07 UTC

Run explicitly configured managed heartbeats in OpenClaw's dedicated heartbeat session.

Validate the isolated setting, configuration hash, and startup interval.

Signed-off-by: Julie Yaunches <jyaunches@nvidia.com>
jyaunches added a commit that referenced this pull request Sep 11, 2026
<!-- markdownlint-disable MD041 -->
## Outcome

Local PR review now starts reliably from macOS temporary directories,
finds Homebrew-installed tools, and removes its owned temporary root
after success or failure. Previously, the trusted bootstrap could exit
silently through a symlinked `TMPDIR`, omit `rg` from its constrained
path, or leave immutable Advisor input behind.

## Reason

PR #11487 cannot complete its required local review because the Advisor
bootstrap and cleanup fail on a standard macOS checkout. These failures
also reproduce on canonical `main`.

### Related issues

Relates to #11487.

## Changes

- Resolve the trusted Advisor implementation to its canonical path
before starting Node. This is required because macOS exposes `/var`
through `/private/var`; the bootstrap is the consumer, and string
comparison alone cannot establish direct execution through that alias.
The bootstrap-isolation regression test uses a synthetic symlinked
temporary root.
- Add `/opt/homebrew/bin` to the constrained host tool path so the
Advisor can invoke its required `rg` executable on Apple Silicon
Homebrew installations. Keeping the fixed allowlist preserves the
existing trust boundary.
- Restore write permission only on directories inside the Advisor-owned
unpredictable temporary root before removal. Direct unlinking is
insufficient because specialist input is intentionally read-only; the
cleanup regression test also proves that a symlinked external directory
is not modified.

## Verification

- `npx vitest run --project integration
test/automation/pull-requests/pr-review-advisor-local.test.ts -t
'installs trusted dependencies|removes its temporary root'
--reporter=dot` — 3 passed, 15 skipped.
- `npm run checks:repository` — passed.
- `npm run build:cli && npm --prefix nemoclaw run build` — passed after
the final base integration.
- `NODE_OPTIONS=--max-old-space-size=5120 npm run validate:pr` — passed
against canonical `main` `79c09d969ab4d3c2d5446c8b985fc0e513c9054b`
after the final base integration.
- Manual reproduction on canonical `main` — confirmed the silent
canonical-path exit, missing Homebrew `rg`, and immutable-directory
cleanup failure.
- Diff inspection — no secrets, API keys, or credentials are present.

## Review notes

Sensitive paths: `tools/pr-review-advisor/local-review.mts` and
`tools/pr-review-advisor/local-review-implementation.mts`. Repository
`NVIDIA/NemoClaw`, commit `64b54ea1ee7269f2f488c3eaefcdca5ac681be0f`,
reviewed locally through focused regression tests, repository checks,
canonical builds, full PR validation, and manual diff/security-boundary
inspection; no candidate-owned findings remain. Independent local
Advisor clearance is unavailable because the trusted bootstrap
deliberately loads its implementation from canonical `main`, which is
the defect this PR repairs. The PR remains draft for independent review
and hosted checks.

---
Signed-off-by: Julie Yaunches <jyaunches@nvidia.com>

Signed-off-by: Julie Yaunches <jyaunches@nvidia.com>
Signed-off-by: Julie Yaunches <jyaunches@nvidia.com>
Signed-off-by: Julie Yaunches <jyaunches@nvidia.com>
Signed-off-by: Julie Yaunches <jyaunches@nvidia.com>
@jyaunches

Copy link
Copy Markdown
Contributor Author

@coderabbitai review

@coderabbitai

coderabbitai Bot commented Sep 11, 2026

Copy link
Copy Markdown
Contributor
✅ Action performed

Review finished.

Note: CodeRabbit is an incremental review system and does not re-review already reviewed commits. This command is applicable only when automatic reviews are paused.

Signed-off-by: Julie Yaunches <jyaunches@nvidia.com>
Signed-off-by: Julie Yaunches <jyaunches@nvidia.com>
@github-actions

Copy link
Copy Markdown
Contributor

PR Review Advisor finished for commit 0a56298. Include the Advisor findings in the complete PR feedback collection. Verify and group valid findings before repair.

All previous runs

@github-actions github-actions Bot added v0.0.124 Release target and removed v0.0.123 labels Sep 11, 2026
Signed-off-by: Julie Yaunches <jyaunches@nvidia.com>
@jyaunches
jyaunches marked this pull request as ready for review September 11, 2026 17:22
jyaunches added a commit that referenced this pull request Sep 11, 2026
## Outcome

The launch supervisor timeout test now lets the host timeout terminate
the command before the fixture's internal deadline expires.

## Reason

The fixture's two-second session budget could expire before the
2.5-second host timeout. This caused `fixture.result.timedOut` to be
false.
The same assertion failed on [main
CI](https://github.com/NVIDIA/NemoClaw/actions/runs/34602042635/job/103271974108)
and blocked [PR #11487](#11487).

## Changes

Give `supervisor-timeout` the existing ten-second session budget used by
`restored-canonical-timeout`. Keep the host timeout and all
descendant-cleanup assertions unchanged.
No production code, retry policy, or new mechanism changes.

## Verification

- Linux, Node 22.23.2 container with `--init`: `npx vitest run --project
e2e-support test/e2e/support/launch-agent-turn.test.ts -t "host command
times out" --reporter=verbose` — passed; 1 test passed, 43 deselected.
This ran before rebase; the test file is unchanged by rebase.
- macOS: `npx vitest run --project e2e-support
test/e2e/support/launch-agent-turn.test.ts` — 6 passed, 38 Linux-only
tests skipped.
- `npm --prefix nemoclaw run build` and `npm run build:cli` — passed.
- `NODE_OPTIONS=--max-old-space-size=5120 npm run validate:pr` — passed
for `b1aeedef5b80ae061745a502eb5938b755526f43`, with canonical
validation base `41c5625e8b831ed213cd5c381385973adc58659c`.
- Diff review found no secrets, API keys, or credentials.

## Review notes

Local Advisor sandbox creation is unavailable in this shepherd session.
Alternative review completed for the one-line diff: correctness,
unchanged cleanup assertions, sibling timeout budgets, and security
boundaries were checked. This is self-review, not independent Advisor
clearance. Hosted checks and review remain pending.
An initial Linux container without an init process retained zombie
descendants; the corrected container passed the affected test. The full
Linux suite did not complete, so no full-suite pass is claimed.

---
Signed-off-by: Julie Yaunches <jyaunches@nvidia.com>


<!-- This is an auto-generated comment: release notes by coderabbit.ai
-->

## Summary by CodeRabbit

* **Bug Fixes**
* Updated the supervisor timeout test scenario to use a 10-second launch
session budget, aligning it with the restored canonical timeout
scenario.
  * Other timeout modes and default budgets remain unchanged.

<!-- end of auto-generated comment: release notes by coderabbit.ai -->

Signed-off-by: Julie Yaunches <jyaunches@nvidia.com>
@jyaunches

Copy link
Copy Markdown
Contributor Author

@coderabbitai review

@coderabbitai

coderabbitai Bot commented Sep 11, 2026

Copy link
Copy Markdown
Contributor
⚠️ Action not completed

Already reviewed the last commit. Use @coderabbitai full review to rerun a review of the entire changeset.

Note: CodeRabbit is an incremental review system and does not re-review already reviewed commits. This command is applicable only when automatic reviews are paused.

@jyaunches

Copy link
Copy Markdown
Contributor Author

@coderabbitai full review

@coderabbitai

coderabbitai Bot commented Sep 11, 2026

Copy link
Copy Markdown
Contributor
✅ Action performed

Full review finished.

@rsliter rsliter left a comment

Copy link
Copy Markdown
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Reviewed commit df172e7 against accepted issue #10262. The shared generator emits the isolated heartbeat for configured cadences, including 0m; managed startup and image checks validate cadence, isolation, startup evidence, and config integrity. Shell inputs are quoted, failed log reads omit captured output, focused validation passed, the current CodeRabbit full review has no actionable findings, and the bounded cross-issue sweep found no adjacent conflict.

@jyaunches

Copy link
Copy Markdown
Contributor Author

The earlier MCP-discovery setup failure is cleared: managed-image workflow 34641125743 attempt 2 passed, including both MCP passes and all-agent activation. CI and the full CodeRabbit review also passed for df172e7 (no actionable review findings).

We are now waiting on the selected protected-runtime E2E, including multi-architecture managed-image startup: https://github.com/NVIDIA/NemoClaw/actions/runs/34666235561

The run is in progress, not yet passing evidence. It targets PR head df172e7, base ffc1116, with trusted workflow 37261ef. Selector: managed-image-protected-runtime; no Launchable or Jetson opt-ins. Human review and merge remain separate from this automated qualification.

@jyaunches

Copy link
Copy Markdown
Contributor Author

The protected E2E failure is also present on main: both native architectures fail while building Hermes because the protected builder selects root but Hermes requires sandbox. Main comparison: https://github.com/NVIDIA/NemoClaw/actions/runs/34670833866. Fix PR #11613 changes the default to sandbox and passes 37 focused tests plus publication validation. Its CI, automated review, and selected E2E are next. This PR remains waiting for that dependency; five-minute follow-up is active. No merge has been performed.

@prekshivyas
prekshivyas merged commit 7872cb9 into main Sep 12, 2026
136 of 137 checks passed
@prekshivyas
prekshivyas deleted the codex/fix-10262-isolated-heartbeats branch September 12, 2026 05:02
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

v0.0.124 Release target

Projects

None yet

Development

Successfully merging this pull request may close these issues.

Managed OpenClaw heartbeats are not isolated from the main session

4 participants