Skip to content

fix(onboard): use deadline wait for gateway recovery - #6320

Merged
cv merged 10 commits into
NVIDIA:mainfrom
HOYALIM:codex/issue-3768-gateway-recovery-deadline
Jul 8, 2026
Merged

fix(onboard): use deadline wait for gateway recovery#6320
cv merged 10 commits into
NVIDIA:mainfrom
HOYALIM:codex/issue-3768-gateway-recovery-deadline

Conversation

@HOYALIM

@HOYALIM HOYALIM commented Jul 6, 2026

Copy link
Copy Markdown
Contributor

Summary

  • replace the fixed gateway recovery polling loop with the shared deadline/attempt helper waitUntilAsync
  • report the actual recovery attempt count and interval alongside the upper wait bound in timeout errors
  • cover the configured attempt/interval budget without sleeping after the final probe
  • add caller-level tests for the immediate-success, retry-then-success, and zero-count-edge paths

Scope

This PR converts the gateway recovery polling loop only. #3768's broader acceptance criterion (all remaining fixed readiness polls in onboard become deadline-based) is not fully addressed here: fixed poll loops in src/lib/onboard.ts (around lines 2205 and 2328) using NEMOCLAW_HEALTH_POLL_COUNT / NEMOCLAW_HEALTH_POLL_INTERVAL remain. Follow-up PRs will address those.

Refs #3768

Tests

  • npx vitest run src/lib/onboard/gateway-recovery.test.ts
  • npm run build:cli
  • npm run typecheck:cli
  • npx biome check src/lib/onboard/gateway-recovery.ts src/lib/onboard/gateway-recovery.test.ts

Summary by CodeRabbit

  • Bug Fixes
    • Improved gateway recovery to use deadline-based readiness polling with consistent probe cadence.
    • Enhanced failure messaging to report the configured recovery deadline/budget when readiness isn’t reached.
    • Ensures readiness is detected promptly and the gateway readiness flag is set on success.
  • Tests
    • Expanded gateway recovery coverage with deterministic virtual time and injectable timing controls.
    • Added assertions for timeout budget calculations, exact probe/sleep cadence, success on immediate/after recovery, and fast-fail when polling is disabled.

Signed-off-by: Ho Lim holim@nvidia.com
Signed-off-by: Charan Jagwani cjagwani@nvidia.com

Copilot AI review requested due to automatic review settings July 6, 2026 10:30
@copy-pr-bot

copy-pr-bot Bot commented Jul 6, 2026

Copy link
Copy Markdown

This pull request requires additional validation before any workflows can run on NVIDIA's runners.

Pull request vetters can view their responsibilities here.

Contributors can view more details about this message here.

Copilot AI left a comment

Copy link
Copy Markdown
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Pull request overview

This PR updates the gateway recovery path in the CLI onboarding flow to use the shared waitUntilAsync helper instead of a hand-rolled polling loop, and improves timeout reporting to reflect the configured recovery wait budget.

Changes:

  • Replaces the fixed for-loop polling in startTargetGatewayForRecovery with waitUntilAsync, including a computed deadline budget.
  • Enhances timeout errors to include the configured wait budget and poll attempt/interval configuration.
  • Adds a regression test asserting the configured attempt/interval budget is applied without sleeping after the final probe.

Reviewed changes

Copilot reviewed 2 out of 2 changed files in this pull request and generated 2 comments.

File Description
src/lib/onboard/gateway-recovery.ts Switches gateway recovery polling to waitUntilAsync, adds budget computation/formatting, and improves timeout error text.
src/lib/onboard/gateway-recovery.test.ts Injects a sleepSeconds dep and adds a test validating the attempt/interval budget and sleep behavior.

💡 Add Copilot custom instructions for smarter, more guided reviews. Learn how to get started.

Comment thread src/lib/onboard/gateway-recovery.ts
Comment thread src/lib/onboard/gateway-recovery.ts
@coderabbitai

coderabbitai Bot commented Jul 6, 2026

Copy link
Copy Markdown
Contributor

Review Change Stack

Note

Reviews paused

It looks like this branch is under active development. To avoid overwhelming you with review comments due to an influx of new commits, CodeRabbit has automatically paused this review. You can configure this behavior by changing the reviews.auto_review.auto_pause_after_reviewed_commits setting.

Use the following commands to manage reviews:

  • @coderabbitai resume to resume automatic reviews.
  • @coderabbitai review to trigger a single review.

Use the checkboxes below for quick actions:

  • ▶️ Resume reviews
  • 🔍 Trigger review
📝 Walkthrough

Walkthrough

Gateway recovery now polls until a configured deadline using injectable clock, sleep, and readiness checks. The failure path reports the deadline budget and interval. Tests cover timeout, first-probe success, retry success, and zero-budget failure.

Changes

Gateway recovery timing refactor

Layer / File(s) Summary
Deps type and wait-budget helpers
src/lib/onboard/gateway-recovery.ts
Adds injectable gateway readiness, sleep, and clock dependencies, updates port imports, and adds helpers to compute and format the recovery wait budget.
Deadline-based wait loop and error message
src/lib/onboard/gateway-recovery.ts
Replaces the retry sleep loop with deadline-based polling that checks gateway connectivity, health, and HTTP readiness, then sets the gateway environment variable or throws a detailed timeout error.
Recovery timing tests
src/lib/onboard/gateway-recovery.test.ts
Adds a virtual clock helper, updates failure expectations, and covers deadline exhaustion, first-probe success, retry-then-success, and zero-attempt fail-fast cases.

Estimated code review effort: 3 (Moderate) | ~25 minutes

Sequence Diagram(s)

sequenceDiagram
  participant startGatewayForRecovery
  participant waitUntilAsync
  participant Gateway probe

  startGatewayForRecovery->>waitUntilAsync: poll until deadline
  loop each attempt
    waitUntilAsync->>Gateway probe: check connectivity, health, HTTP readiness
    Gateway probe-->>waitUntilAsync: ready or not ready
    waitUntilAsync->>waitUntilAsync: sleepSeconds(interval) unless final attempt
  end
  alt gateway ready
    waitUntilAsync-->>startGatewayForRecovery: success
    startGatewayForRecovery->>startGatewayForRecovery: set OPENSHELL_GATEWAY
  else deadline exceeded
    waitUntilAsync-->>startGatewayForRecovery: timeout
    startGatewayForRecovery-->>startGatewayForRecovery: throw error with budget and interval
  end
Loading
🚥 Pre-merge checks | ✅ 5
✅ Passed checks (5 passed)
Check name Status Explanation
Docstring Coverage ✅ Passed No functions found in the changed files to evaluate docstring coverage. Skipping docstring coverage check.
Linked Issues check ✅ Passed Check skipped because no linked issues were found for this pull request.
Out of Scope Changes check ✅ Passed Check skipped because no linked issues were found for this pull request.
Description Check ✅ Passed Check skipped - CodeRabbit’s high-level summary is enabled.
Title check ✅ Passed The title clearly summarizes the main change: switching gateway recovery to deadline-based waiting.
✨ Finishing Touches
🧪 Generate unit tests (beta)
  • Create PR with unit tests

Comment @coderabbitai help to get the list of available commands.

Signed-off-by: Ho Lim <subhoya@gmail.com>
@HOYALIM
HOYALIM force-pushed the codex/issue-3768-gateway-recovery-deadline branch from cad6567 to c9f3979 Compare July 6, 2026 10:50
@wscurran wscurran added area: cli Command line interface, flags, terminal UX, or output area: onboarding Onboarding FSM, provider setup, sandbox launch, or first-run flow bug-fix PR fixes a bug or regression labels Jul 7, 2026
@wscurran

wscurran commented Jul 7, 2026

Copy link
Copy Markdown
Contributor

✨ Thanks for the PR. This fixes the gateway recovery path during onboarding by replacing the fixed polling loop with a deadline-based wait helper and improves timeout error messages. Ready for maintainer review.


Related open issues:


Related open issues:

@cv cv added the v0.0.77 label Jul 8, 2026
@cjagwani

cjagwani commented Jul 8, 2026

Copy link
Copy Markdown
Collaborator

Hey, nice small refactor. A few things to work through before we merge:

1. The error message over-promises the wait budget. The message says "configured 6s recovery wait budget (3 attempts, 2s interval)" but the loop can actually exit at the maxAttempts cap before the deadline fires. On fast-failing probes it stops earlier than 6s, and on slow probes the reported budget is optimistic. Two ways to reconcile:

2. Scope vs the linked issue. #3768's acceptance criterion is broader than gateway recovery. There are still fixed poll loops in src/lib/onboard.ts (around line 2205 and 2328) using the same NEMOCLAW_HEALTH_POLL_COUNT / NEMOCLAW_HEALTH_POLL_INTERVAL pattern. Either extend this PR to cover them, or update the description so it's clear this is a gateway-recovery-only slice and doesn't close #3768.

3. Missing success-path test coverage. The new test pins the timeout shape and the no-final-sleep behavior, which is great. But there's no test for the actual happy path: immediate success without sleeping, retry-then-success, OPENSHELL_GATEWAY env being set after readiness, the CoreDNS patch firing. Refactor could silently break any of those.

Small nit: the sleep: (ms) => sleeper(ms / 1000) unit adapter is correct but worth a one-line comment noting waitUntilAsync speaks ms while sleepSeconds takes seconds.

@cjagwani

cjagwani commented Jul 8, 2026

Copy link
Copy Markdown
Collaborator

Please run the ubuntu-repo-docker-post-reboot-recovery typed target before merging. That's the registry target that actually exercises host-side gateway recovery through the shared live runner, so the refactor is validated end-to-end and not just at the unit-mock layer.

Dispatch:

gh workflow run e2e.yaml --ref <branch> --field targets=ubuntu-repo-docker-post-reboot-recovery

Optionally worth running gateway-guard-recovery for adjacent coverage of the sandbox guard chain, but not required.

@cjagwani cjagwani self-assigned this Jul 8, 2026
@github-actions

github-actions Bot commented Jul 8, 2026

Copy link
Copy Markdown
Contributor

E2E Target Results — ✅ All selected jobs passed

Run: 28950873219
Workflow ref: pr-6320-mirror
Requested targets: ubuntu-repo-docker-post-reboot-recovery
Requested jobs: (default — all default-enabled free-standing jobs; explicit-only jobs openshell-gateway-auth-contract, mcp-bridge-dev, hermes-gpu-startup, sandbox-rlimits-connect, and jetson-nvmap-gpu are skipped unless selected)
Summary: 1 passed, 0 failed, 0 cancelled, 0 skipped

Job Result
live ✅ success

cjagwani and others added 2 commits July 8, 2026 07:39
…ss paths

Address the advisor review on PR NVIDIA#6320.

1. Message/behavior reconciliation. The prior message said "configured Xs
   recovery wait budget" while waitUntilAsync can also terminate via
   maxAttempts, so the reported budget was an upper bound rather than
   the actual guarantee. Rewrite as "N recovery attempt(s) at Ns
   interval (max wait Xs)" so an operator scanning the error sees both
   dimensions: the attempt cap that likely fired, and the hard upper
   time bound the loop cannot exceed.

2. Add success-path coverage. Inject isGatewayHealthy and
   isGatewayHttpReady as optional deps (default to the production
   implementations) so caller-level tests can pin the happy path
   without standing up a real gateway. New tests:
     - "succeeds on the first healthy probe without sleeping and sets
       OPENSHELL_GATEWAY"
     - "succeeds after retrying past unhealthy probes and still sets
       OPENSHELL_GATEWAY"
     - "with NEMOCLAW_HEALTH_POLL_COUNT=0 fails fast without invoking
       the probe" (edge case: zero attempts must not silently pass)

3. Document the sleep unit adapter with a one-line comment noting
   waitUntilAsync passes durations in ms while the injected sleeper
   (sleepSeconds) takes seconds; adapt at the boundary only.

11/11 tests pass locally. Biome and the changed-file typecheck are
clean. Pre-existing typecheck error in test/helpers/mcp-lifecycle-
lock-properties.ts is unrelated to this branch's diff.

Co-Authored-By: Ho Lim <holim@nvidia.com>
Signed-off-by: Charan Jagwani <cjagwani@nvidia.com>
@github-actions

github-actions Bot commented Jul 8, 2026

Copy link
Copy Markdown
Contributor

E2E Target Results — ✅ All selected jobs passed

Run: 28952907265
Workflow ref: pr-6320-mirror
Requested targets: ubuntu-repo-docker-post-reboot-recovery
Requested jobs: (default — all default-enabled free-standing jobs; explicit-only jobs openshell-gateway-auth-contract, mcp-bridge-dev, hermes-gpu-startup, sandbox-rlimits-connect, and jetson-nvmap-gpu are skipped unless selected)
Summary: 1 passed, 0 failed, 0 cancelled, 0 skipped

Job Result
live ✅ success

@ericksoa ericksoa added v0.0.78 and removed v0.0.77 labels Jul 8, 2026
cjagwani added a commit that referenced this pull request Jul 8, 2026
…6320 advisor)

Advisor re-review kept blocking on the same concern: with maxAttempts
still passed to waitUntilAsync alongside the deadline, a fast-failing
probe sequence exits after N attempts even though the deadline still
permits more polling. The reported "max wait Xs" language then
under-promises the wait the system was willing to spend, and the loop
regresses to the exact fixed-attempt behavior #3768 is meant to
replace.

Fix:

1. Drop `maxAttempts: recoveryPollCount` from the waitUntilAsync
   options. The loop is now purely deadline-driven; the interval and
   probe cost naturally bound the total attempt count. Under a
   3 * 2s = 6s budget with fast-failing mocked probes, the loop now
   runs 3 probes and 3 sleeps until the top-of-loop deadline check
   terminates it, instead of the earlier 3 probes / 2 sleeps under the
   attempt cap.

2. Rewrite the error message to describe the actual deadline, not a
   hedged "budget + attempt cap" mix: "Gateway 'X' did not become
   ready within the configured Ns recovery deadline (Is poll
   interval)".

3. Inject Date.now via a new `now` deps hook so the deadline test can
   drive a captured virtual clock without vi.useFakeTimers globally
   patching timers (which hangs the async loop). The virtual clock
   advances only when the injected sleepSeconds mock is called, so
   time progresses deterministically at unit-test speed with zero
   real wall-clock waits.

4. Rewrite the timeout test to match the new deadline-only semantics:
   asserts 3 probes and 3 sleeps at 2s each under a 6s budget, and
   that the error message mentions "6s recovery deadline (2s poll
   interval)".

5. Adjust the zero-count edge test's message assertion to match the
   new deadline-only error text.

11/11 tests pass. Biome and changed-file typecheck are clean. The
pre-existing typecheck error in test/helpers/mcp-lifecycle-lock-
properties.ts is unrelated to this branch's diff.

Signed-off-by: Charan Jagwani <cjagwani@nvidia.com>
…VIDIA#6320 advisor)

Advisor re-review kept blocking on the same concern: with maxAttempts
still passed to waitUntilAsync alongside the deadline, a fast-failing
probe sequence exits after N attempts even though the deadline still
permits more polling. The reported "max wait Xs" language then
under-promises the wait the system was willing to spend, and the loop
regresses to the exact fixed-attempt behavior NVIDIA#3768 is meant to
replace.

Fix:

1. Drop `maxAttempts: recoveryPollCount` from the waitUntilAsync
   options. The loop is now purely deadline-driven; the interval and
   probe cost naturally bound the total attempt count. Under a
   3 * 2s = 6s budget with fast-failing mocked probes, the loop now
   runs 3 probes and 3 sleeps until the top-of-loop deadline check
   terminates it, instead of the earlier 3 probes / 2 sleeps under the
   attempt cap.

2. Rewrite the error message to describe the actual deadline, not a
   hedged "budget + attempt cap" mix: "Gateway 'X' did not become
   ready within the configured Ns recovery deadline (Is poll
   interval)".

3. Inject Date.now via a new `now` deps hook so the deadline test can
   drive a captured virtual clock without vi.useFakeTimers globally
   patching timers (which hangs the async loop). The virtual clock
   advances only when the injected sleepSeconds mock is called, so
   time progresses deterministically at unit-test speed with zero
   real wall-clock waits.

4. Rewrite the timeout test to match the new deadline-only semantics:
   asserts 3 probes and 3 sleeps at 2s each under a 6s budget, and
   that the error message mentions "6s recovery deadline (2s poll
   interval)".

5. Adjust the zero-count edge test's message assertion to match the
   new deadline-only error text.

11/11 tests pass. Biome and changed-file typecheck are clean. The
pre-existing typecheck error in test/helpers/mcp-lifecycle-lock-
properties.ts is unrelated to this branch's diff.

Signed-off-by: Charan Jagwani <cjagwani@nvidia.com>
@github-actions

github-actions Bot commented Jul 8, 2026

Copy link
Copy Markdown
Contributor

E2E Target Results — ✅ All selected jobs passed

Run: 28953876797
Workflow ref: pr-6320-mirror
Requested targets: ubuntu-repo-docker-post-reboot-recovery
Requested jobs: (default — all default-enabled free-standing jobs; explicit-only jobs openshell-gateway-auth-contract, mcp-bridge-dev, hermes-gpu-startup, sandbox-rlimits-connect, and jetson-nvmap-gpu are skipped unless selected)
Summary: 1 passed, 0 failed, 0 cancelled, 0 skipped

Job Result
live ✅ success

@coderabbitai coderabbitai Bot left a comment

Copy link
Copy Markdown
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Actionable comments posted: 2

🧹 Nitpick comments (1)
src/lib/onboard/gateway-recovery.test.ts (1)

118-125: 📐 Maintainability & Code Quality | 🔵 Trivial | ⚡ Quick win

Probe count derived from a magic subprocess-call divisor.

probeCount = runCaptureCalls / 3 hard-codes the current number of runCaptureOpenshell calls per probe cycle (status + gateway info by name + gateway info current). If the production probe sequence adds/removes a subprocess call, this test silently miscounts without failing meaningfully, and it couples the test to implementation shape rather than the observable retry cadence.

As per path instructions for **/*.test.{ts,js,mts,mjs,cts,cjs}: "Prefer observable outcomes through the public boundary over source-text, private-shape, or mock-call assertions."

Consider asserting on a more explicit signal — e.g., have the mocked runCaptureOpenshell increment a probe counter only on the first call of each cycle (args[0] === "status"), rather than deriving it via division.

🤖 Prompt for AI Agents
Verify each finding against current code. Fix only still-valid issues, skip the
rest with a brief reason, keep changes minimal, and validate.

In `@src/lib/onboard/gateway-recovery.test.ts` around lines 118 - 125, The test in
gateway-recovery.test.ts is inferring probe count from a hard-coded
subprocess-call divisor, which ties it to implementation details of
runCaptureOpenshell. Update the assertion to count observable probe cycles
directly by using the mocked runCaptureOpenshell and incrementing a probe
counter only when the first call in each cycle occurs (for example, the status
command), instead of dividing total mock calls by 3. Keep the expectation
focused on the retry cadence rather than the internal call shape.

Source: Path instructions

🤖 Prompt for all review comments with AI agents
Verify each finding against current code. Fix only still-valid issues, skip the
rest with a brief reason, keep changes minimal, and validate.

Inline comments:
In `@src/lib/onboard/gateway-recovery.test.ts`:
- Line 105: The new behavior-oriented test titles in gateway-recovery.test.ts
are missing the required local issue-reference suffix. Update the affected test
names in the relevant `it(...)` blocks to end with the same final `(`#3768`)`
suffix, keeping the titles behavior-focused while preserving the existing intent
in the test descriptions.
- Around line 105-129: The timeout test in gateway-recovery should prove it is
stopping on the recovery deadline, not just on a fixed attempt count. Update the
test around startGatewayForRecovery, makeVirtualClock, and the health-check
probe path so each probe advances the injected clock (for example via the mocked
subprocess/health check behavior), which makes deadline-driven polling and a
hidden maxAttempts limit diverge. Then assert on the resulting probe/sleep count
and timeout message in a way that only passes when the loop exits because the
configured deadline is reached.

---

Nitpick comments:
In `@src/lib/onboard/gateway-recovery.test.ts`:
- Around line 118-125: The test in gateway-recovery.test.ts is inferring probe
count from a hard-coded subprocess-call divisor, which ties it to implementation
details of runCaptureOpenshell. Update the assertion to count observable probe
cycles directly by using the mocked runCaptureOpenshell and incrementing a probe
counter only when the first call in each cycle occurs (for example, the status
command), instead of dividing total mock calls by 3. Keep the expectation
focused on the retry cadence rather than the internal call shape.
🪄 Autofix (Beta)

Fix all unresolved CodeRabbit comments on this PR:

  • Push a commit to this branch (recommended)
  • Create a new PR with the fixes

ℹ️ Review info
⚙️ Run configuration

Configuration used: Path: .coderabbit.yaml

Review profile: CHILL

Plan: Enterprise

Run ID: 90bdbcb4-486d-4d5a-8864-f1bc33c26af8

📥 Commits

Reviewing files that changed from the base of the PR and between fcfa968 and 691f863.

📒 Files selected for processing (2)
  • src/lib/onboard/gateway-recovery.test.ts
  • src/lib/onboard/gateway-recovery.ts
🚧 Files skipped from review as they are similar to previous changes (1)
  • src/lib/onboard/gateway-recovery.ts

Comment thread src/lib/onboard/gateway-recovery.test.ts Outdated
Comment thread src/lib/onboard/gateway-recovery.test.ts Outdated
cjagwani added 2 commits July 8, 2026 08:23
…NVIDIA#6320)

Advisor in-scope cleanup: `beforeEach` was left over from the earlier
vi.useFakeTimers approach that the virtual-clock refactor replaced.
Not used anywhere in the file.

Signed-off-by: Charan Jagwani <cjagwani@nvidia.com>
…VIDIA#3768)

Address two CodeRabbit nits on PR NVIDIA#6320.

1. Test titles missing local issue reference (Major, quick win). Add
   the required `(NVIDIA#3768)` suffix to the four new tests so they match
   the repo's test-title-style contract.

2. Deadline-driven proof (Major, heavy lift). The prior timeout test
   used fast-failing probes that only advanced the clock via the
   sleep mock. Under that setup, a hidden `maxAttempts: recoveryPollCount`
   would have produced the same probe/sleep counts as a pure deadline,
   so the test could not tell them apart.

   Redesign the timeout test to make probes ALSO advance the virtual
   clock: NEMOCLAW_HEALTH_POLL_COUNT=10, NEMOCLAW_HEALTH_POLL_INTERVAL=1
   with each probe consuming 1s of virtual time. Under the pure-deadline
   implementation the loop runs ~5 iterations before the 10s budget is
   exhausted (probes and sleeps consume 2s per iteration cumulatively).
   Under a hidden attempt cap of 10 the loop would run exactly 10
   iterations. The new assertion `probeCount < 10` therefore only
   passes when the deadline (not an attempt cap) terminates the loop,
   which is the invariant NVIDIA#3768 asks for.

   Expose `advance(seconds)` on the makeVirtualClock helper so a mocked
   subprocess-probe implementation can move time forward while keeping
   the sleeper as the primary in-loop clock driver.

11/11 tests still pass. Biome and repo checks clean.

Signed-off-by: Charan Jagwani <cjagwani@nvidia.com>

@cjagwani cjagwani left a comment

Copy link
Copy Markdown
Collaborator

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

LGTM. Passes the PR review advisor, E2E (ubuntu-repo-docker-post-reboot-recovery), and functional testing. CodeRabbit threads resolved.

…IA#6320)

The new deadline-driven timeout test added a mocked runCaptureOpenshell
that used `if (probesRun * 3 === ...)` to advance the virtual clock
once per probe. That single new `if` tripped
codebase-growth-guardrails' "no new if statements in changed test
files" gate. Move the branching into a named helper `advanceOnStatusCall`
that uses a ternary, so the test body stays linear and the guardrail
passes.

11/11 tests still pass.

Signed-off-by: Charan Jagwani <cjagwani@nvidia.com>
@github-actions

github-actions Bot commented Jul 8, 2026

Copy link
Copy Markdown
Contributor

E2E Target Results — ✅ All selected jobs passed

Run: 28955158984
Workflow ref: pr-6320-mirror
Requested targets: ubuntu-repo-docker-post-reboot-recovery
Requested jobs: (default — all default-enabled free-standing jobs; explicit-only jobs openshell-gateway-auth-contract, mcp-bridge-dev, hermes-gpu-startup, sandbox-rlimits-connect, and jetson-nvmap-gpu are skipped unless selected)
Summary: 1 passed, 0 failed, 0 cancelled, 0 skipped

Job Result
live ✅ success

cjagwani added 2 commits July 8, 2026 08:48
CI's `Files were modified by following hooks` step tripped on the
prior commit: Biome format collapsed the `advanceOnStatusCall`
ternary from multi-line onto a single line. Apply the same
formatter locally so the committed state matches what CI expects.

Signed-off-by: Charan Jagwani <cjagwani@nvidia.com>
@cv
cv merged commit 4ff5756 into NVIDIA:main Jul 8, 2026
29 checks passed
@cjagwani cjagwani mentioned this pull request Jul 9, 2026
21 tasks
cv pushed a commit that referenced this pull request Jul 9, 2026
<!-- markdownlint-disable MD041 -->
## Summary

Prepares the user documentation for NemoClaw v0.0.78 by replacing the
unreleased section with release highlights and synchronizing the
affected inference, lifecycle, messaging, and CLI reference pages with
merged behavior.

## Changes

- Publish the v0.0.78 release-notes section with links to the most
specific user guides for each shipped behavior.
- Document authoritative Deep Agents route health, Nemotron Ultra
profile behavior, and Hermes compatible-endpoint context metadata.
- Document forced rebuild recovery after total backup failure and the
ownership-safe tunnel/full-stop behavior.
- Keep command examples and shared agent variants aligned with the
current OpenClaw, Hermes, and Deep Agents interfaces.

Source mapping:

- [#3787](#3787) ->
`docs/about/release-notes.mdx`: Record reliable workspace template
seeding during sandbox startup.
- [#4960](#4960) ->
`docs/about/release-notes.mdx`: Record safer detection of rewritten
OpenClaw gateway processes.
- [#5676](#5676) ->
`docs/about/release-notes.mdx`: Record warning-tolerant agent-list JSON
handling.
- [#5857](#5857) ->
`docs/about/release-notes.mdx`: Record synchronization of explicit
OpenClaw main-agent model state.
- [#5929](#5929) ->
`docs/about/release-notes.mdx`: Record copyable SSH port-forward
guidance for remote dashboards.
- [#6068](#6068) ->
`docs/about/release-notes.mdx`: Record custom-image plugin provenance
reconciliation.
- [#6116](#6116) ->
`docs/about/release-notes.mdx`: Record live-loopback dashboard-forward
recovery.
- [#6122](#6122) ->
`docs/about/release-notes.mdx`: Announce validated, round-trippable
policy YAML output.
- [#6211](#6211) ->
`docs/manage-sandboxes/lifecycle.mdx`, `docs/reference/commands.mdx`,
`docs/about/release-notes.mdx`: Explain the explicit no-backup `rebuild
--force` recovery boundary.
- [#6283](#6283) ->
`docs/about/release-notes.mdx`: Record Hermes WebUI port alignment.
- [#6293](#6293) ->
`docs/inference/switch-inference-providers.mdx`,
`docs/about/release-notes.mdx`: Document compatible-endpoint
context-length probing for Hermes.
- [#6320](#6320) ->
`docs/about/release-notes.mdx`: Record bounded gateway-recovery waits.
- [#6377](#6377) ->
`docs/reference/commands.mdx`, `docs/about/release-notes.mdx`: Explain
rebuild diagnostics and prepared MCP-destroy recovery.
- [#6412](#6412) ->
`docs/get-started/quickstart-langchain-deepagents-code.mdx`,
`docs/about/release-notes.mdx`: Document authoritative agent-visible
inference route health.
- [#6421](#6421) ->
`docs/about/release-notes.mdx`: Record the longer quiet-pull window for
managed vLLM images.
- [#6431](#6431) ->
`docs/inference/model-capability-audit.mdx`,
`docs/about/release-notes.mdx`: Document the version-pinned Nemotron
Ultra profile plugin.
- [#6439](#6439) ->
`docs/about/release-notes.mdx`: Summarize the authenticated, pinned
credential-capture helper boundary.
- [#6450](#6450) ->
`docs/manage-sandboxes/messaging-channels.mdx`,
`docs/reference/commands.mdx`, `docs/about/release-notes.mdx`: Document
host-forward cleanup and ownership-safe gateway-port release.
- [#6474](#6474) ->
`docs/manage-sandboxes/messaging-channels.mdx`,
`docs/about/release-notes.mdx`: Record composable OpenClaw messaging
runtime loaders.
- [#6475](#6475) ->
`docs/about/release-notes.mdx`: Record removal of the unavailable Kimi
K2.6 production endpoint option.
- [#6480](#6480) ->
`docs/about/release-notes.mdx`: Record stderr routing for the plugin
registration banner.
- [#6481](#6481) ->
`docs/about/release-notes.mdx`: Record post-pull Ollama model discovery
checks.
- [#6482](#6482) ->
`docs/about/release-notes.mdx`: Record Ollama model warm-up after daemon
restart.
- [#6486](#6486) ->
`docs/about/release-notes.mdx`: Publish the opt-in, thread-scoped Deep
Agents auto-approval boundary.
- [#6490](#6490) ->
`docs/about/release-notes.mdx`: Record diagnostics for custom images
missing the managed runtime.
- [#6494](#6494) ->
`docs/inference/model-capability-audit.mdx`,
`docs/about/release-notes.mdx`: Document nonempty tool-call content
preservation and placeholder rejection.
- [#6497](#6497) ->
`docs/get-started/quickstart-langchain-deepagents-code.mdx`,
`docs/about/release-notes.mdx`: Document isolated Deep Agents
route-probe output.
- [#6506](#6506) ->
`docs/get-started/quickstart-langchain-deepagents-code.mdx`,
`docs/about/release-notes.mdx`: Document observability-preserving
managed route probes.
- [#6508](#6508) ->
`docs/about/release-notes.mdx`: Link the new extension taxonomy and
SDK-readiness reference from the release summary.

Release-source verification: GitHub reports all 29 cited source PRs as
merged with base `main`, and every merge commit is an ancestor of
`origin/main` at `17bf9a6a9688b3b1d69cf4b37d3f23110acb055e`. No
source-mapping mismatches were found.

## Type of Change

- [ ] Code change (feature, bug fix, or refactor)
- [ ] Code change with doc updates
- [ ] Doc only (prose changes, no code sample modifications)
- [x] Doc only (includes code sample changes)

## Quality Gates

<!-- Check exactly one tests line and one docs line. Check other lines
when applicable. Add every requested justification or approval
reference. -->
- [ ] Tests added or updated for changed behavior
- [ ] Existing tests cover changed behavior — justification:
- [x] Tests not applicable — justification: Documentation-only
release-prep changes; `npm run docs` validates variants, routes, and
Fern content.
- [x] Docs updated for user-facing behavior changes
- [ ] Docs not applicable — justification:
- [ ] Sensitive paths changed (security, policy, credentials, preflight,
onboarding, inference, runner, sandbox, or messaging)
- [ ] Sensitive-path review completed or maintainer-approved waiver
recorded — reviewer/approval link/justification:
- [ ] Non-success, skipped, or missing CI check accepted by maintainer —
check name, approval link, and follow-up issue:

## Verification

<!-- Check each applicable item only when supported by the requested
evidence. Run targeted tests once per relevant change set and rerun
after later edits or hook autofixes that can affect the tested behavior.
Do not rerun hook-covered checks. -->
- [x] PR description includes the DCO sign-off declaration and every
commit appears as `Verified` in GitHub
- [x] Normal `pre-commit`, `commit-msg`, and `pre-push` hooks passed, or
`npm run check:diff` passed when hooks were skipped or unavailable
- [x] Targeted behavior tests pass for the current change set, or tests
are marked not applicable above — command/result or justification: Tests
are not applicable to this documentation-only change set.
- [ ] Applicable broad gate passed — `npm test` for broad
runtime/test-harness changes; `npm run check` for repo-wide
validation/coverage changes — command/result:
- [x] Quality Gates section completed with required justifications or
waivers
- [x] No secrets, API keys, or credentials committed
- [ ] `npm run docs` builds without warnings (doc changes only) — exited
0 with zero errors; Fern reported the existing unauthenticated
redirect-check and light-mode contrast warnings.
- [x] Doc pages follow the [style
guide](https://github.com/NVIDIA/NemoClaw/blob/main/docs/CONTRIBUTING.md)
(doc changes only)
- [ ] New doc pages include SPDX header and frontmatter (new pages only)

---
<!-- DCO sign-off is required in this PR description, and every commit
must appear as Verified in GitHub. Run: git config user.name && git
config user.email -->
Signed-off-by: Charan Jagwani <cjagwani@nvidia.com>

---------

Signed-off-by: cjagwani <cjagwani@nvidia.com>
Hadar301 pushed a commit to Hadar301/NemoClaw-OpenShift that referenced this pull request Jul 12, 2026
## Summary
- replace the fixed gateway recovery polling loop with the shared
deadline/attempt helper waitUntilAsync
- report the actual recovery attempt count and interval alongside the
upper wait bound in timeout errors
- cover the configured attempt/interval budget without sleeping after
the final probe
- add caller-level tests for the immediate-success, retry-then-success,
and zero-count-edge paths

## Scope
This PR converts the gateway recovery polling loop only. NVIDIA#3768's broader
acceptance criterion (all remaining fixed readiness polls in onboard
become deadline-based) is not fully addressed here: fixed poll loops in
`src/lib/onboard.ts` (around lines 2205 and 2328) using
`NEMOCLAW_HEALTH_POLL_COUNT` / `NEMOCLAW_HEALTH_POLL_INTERVAL` remain.
Follow-up PRs will address those.

Refs NVIDIA#3768

## Tests
- `npx vitest run src/lib/onboard/gateway-recovery.test.ts`
- `npm run build:cli`
- `npm run typecheck:cli`
- `npx biome check src/lib/onboard/gateway-recovery.ts
src/lib/onboard/gateway-recovery.test.ts`

<!-- This is an auto-generated comment: release notes by coderabbit.ai
-->
## Summary by CodeRabbit

* **Bug Fixes**
* Improved gateway recovery to use deadline-based readiness polling with
consistent probe cadence.
* Enhanced failure messaging to report the configured recovery
deadline/budget when readiness isn’t reached.
* Ensures readiness is detected promptly and the gateway readiness flag
is set on success.
* **Tests**
* Expanded gateway recovery coverage with deterministic virtual time and
injectable timing controls.
* Added assertions for timeout budget calculations, exact probe/sleep
cadence, success on immediate/after recovery, and fast-fail when polling
is disabled.
<!-- end of auto-generated comment: release notes by coderabbit.ai -->

Signed-off-by: Ho Lim <holim@nvidia.com>
Signed-off-by: Charan Jagwani <cjagwani@nvidia.com>

---------

Signed-off-by: Ho Lim <subhoya@gmail.com>
Signed-off-by: Charan Jagwani <cjagwani@nvidia.com>
Co-authored-by: cjagwani <cjagwani@nvidia.com>
Co-authored-by: Ho Lim <holim@nvidia.com>
Hadar301 pushed a commit to Hadar301/NemoClaw-OpenShift that referenced this pull request Jul 12, 2026
<!-- markdownlint-disable MD041 -->
## Summary

Prepares the user documentation for NemoClaw v0.0.78 by replacing the
unreleased section with release highlights and synchronizing the
affected inference, lifecycle, messaging, and CLI reference pages with
merged behavior.

## Changes

- Publish the v0.0.78 release-notes section with links to the most
specific user guides for each shipped behavior.
- Document authoritative Deep Agents route health, Nemotron Ultra
profile behavior, and Hermes compatible-endpoint context metadata.
- Document forced rebuild recovery after total backup failure and the
ownership-safe tunnel/full-stop behavior.
- Keep command examples and shared agent variants aligned with the
current OpenClaw, Hermes, and Deep Agents interfaces.

Source mapping:

- [NVIDIA#3787](NVIDIA#3787) ->
`docs/about/release-notes.mdx`: Record reliable workspace template
seeding during sandbox startup.
- [NVIDIA#4960](NVIDIA#4960) ->
`docs/about/release-notes.mdx`: Record safer detection of rewritten
OpenClaw gateway processes.
- [NVIDIA#5676](NVIDIA#5676) ->
`docs/about/release-notes.mdx`: Record warning-tolerant agent-list JSON
handling.
- [NVIDIA#5857](NVIDIA#5857) ->
`docs/about/release-notes.mdx`: Record synchronization of explicit
OpenClaw main-agent model state.
- [NVIDIA#5929](NVIDIA#5929) ->
`docs/about/release-notes.mdx`: Record copyable SSH port-forward
guidance for remote dashboards.
- [NVIDIA#6068](NVIDIA#6068) ->
`docs/about/release-notes.mdx`: Record custom-image plugin provenance
reconciliation.
- [NVIDIA#6116](NVIDIA#6116) ->
`docs/about/release-notes.mdx`: Record live-loopback dashboard-forward
recovery.
- [NVIDIA#6122](NVIDIA#6122) ->
`docs/about/release-notes.mdx`: Announce validated, round-trippable
policy YAML output.
- [NVIDIA#6211](NVIDIA#6211) ->
`docs/manage-sandboxes/lifecycle.mdx`, `docs/reference/commands.mdx`,
`docs/about/release-notes.mdx`: Explain the explicit no-backup `rebuild
--force` recovery boundary.
- [NVIDIA#6283](NVIDIA#6283) ->
`docs/about/release-notes.mdx`: Record Hermes WebUI port alignment.
- [NVIDIA#6293](NVIDIA#6293) ->
`docs/inference/switch-inference-providers.mdx`,
`docs/about/release-notes.mdx`: Document compatible-endpoint
context-length probing for Hermes.
- [NVIDIA#6320](NVIDIA#6320) ->
`docs/about/release-notes.mdx`: Record bounded gateway-recovery waits.
- [NVIDIA#6377](NVIDIA#6377) ->
`docs/reference/commands.mdx`, `docs/about/release-notes.mdx`: Explain
rebuild diagnostics and prepared MCP-destroy recovery.
- [NVIDIA#6412](NVIDIA#6412) ->
`docs/get-started/quickstart-langchain-deepagents-code.mdx`,
`docs/about/release-notes.mdx`: Document authoritative agent-visible
inference route health.
- [NVIDIA#6421](NVIDIA#6421) ->
`docs/about/release-notes.mdx`: Record the longer quiet-pull window for
managed vLLM images.
- [NVIDIA#6431](NVIDIA#6431) ->
`docs/inference/model-capability-audit.mdx`,
`docs/about/release-notes.mdx`: Document the version-pinned Nemotron
Ultra profile plugin.
- [NVIDIA#6439](NVIDIA#6439) ->
`docs/about/release-notes.mdx`: Summarize the authenticated, pinned
credential-capture helper boundary.
- [NVIDIA#6450](NVIDIA#6450) ->
`docs/manage-sandboxes/messaging-channels.mdx`,
`docs/reference/commands.mdx`, `docs/about/release-notes.mdx`: Document
host-forward cleanup and ownership-safe gateway-port release.
- [NVIDIA#6474](NVIDIA#6474) ->
`docs/manage-sandboxes/messaging-channels.mdx`,
`docs/about/release-notes.mdx`: Record composable OpenClaw messaging
runtime loaders.
- [NVIDIA#6475](NVIDIA#6475) ->
`docs/about/release-notes.mdx`: Record removal of the unavailable Kimi
K2.6 production endpoint option.
- [NVIDIA#6480](NVIDIA#6480) ->
`docs/about/release-notes.mdx`: Record stderr routing for the plugin
registration banner.
- [NVIDIA#6481](NVIDIA#6481) ->
`docs/about/release-notes.mdx`: Record post-pull Ollama model discovery
checks.
- [NVIDIA#6482](NVIDIA#6482) ->
`docs/about/release-notes.mdx`: Record Ollama model warm-up after daemon
restart.
- [NVIDIA#6486](NVIDIA#6486) ->
`docs/about/release-notes.mdx`: Publish the opt-in, thread-scoped Deep
Agents auto-approval boundary.
- [NVIDIA#6490](NVIDIA#6490) ->
`docs/about/release-notes.mdx`: Record diagnostics for custom images
missing the managed runtime.
- [NVIDIA#6494](NVIDIA#6494) ->
`docs/inference/model-capability-audit.mdx`,
`docs/about/release-notes.mdx`: Document nonempty tool-call content
preservation and placeholder rejection.
- [NVIDIA#6497](NVIDIA#6497) ->
`docs/get-started/quickstart-langchain-deepagents-code.mdx`,
`docs/about/release-notes.mdx`: Document isolated Deep Agents
route-probe output.
- [NVIDIA#6506](NVIDIA#6506) ->
`docs/get-started/quickstart-langchain-deepagents-code.mdx`,
`docs/about/release-notes.mdx`: Document observability-preserving
managed route probes.
- [NVIDIA#6508](NVIDIA#6508) ->
`docs/about/release-notes.mdx`: Link the new extension taxonomy and
SDK-readiness reference from the release summary.

Release-source verification: GitHub reports all 29 cited source PRs as
merged with base `main`, and every merge commit is an ancestor of
`origin/main` at `17bf9a6a9688b3b1d69cf4b37d3f23110acb055e`. No
source-mapping mismatches were found.

## Type of Change

- [ ] Code change (feature, bug fix, or refactor)
- [ ] Code change with doc updates
- [ ] Doc only (prose changes, no code sample modifications)
- [x] Doc only (includes code sample changes)

## Quality Gates

<!-- Check exactly one tests line and one docs line. Check other lines
when applicable. Add every requested justification or approval
reference. -->
- [ ] Tests added or updated for changed behavior
- [ ] Existing tests cover changed behavior — justification:
- [x] Tests not applicable — justification: Documentation-only
release-prep changes; `npm run docs` validates variants, routes, and
Fern content.
- [x] Docs updated for user-facing behavior changes
- [ ] Docs not applicable — justification:
- [ ] Sensitive paths changed (security, policy, credentials, preflight,
onboarding, inference, runner, sandbox, or messaging)
- [ ] Sensitive-path review completed or maintainer-approved waiver
recorded — reviewer/approval link/justification:
- [ ] Non-success, skipped, or missing CI check accepted by maintainer —
check name, approval link, and follow-up issue:

## Verification

<!-- Check each applicable item only when supported by the requested
evidence. Run targeted tests once per relevant change set and rerun
after later edits or hook autofixes that can affect the tested behavior.
Do not rerun hook-covered checks. -->
- [x] PR description includes the DCO sign-off declaration and every
commit appears as `Verified` in GitHub
- [x] Normal `pre-commit`, `commit-msg`, and `pre-push` hooks passed, or
`npm run check:diff` passed when hooks were skipped or unavailable
- [x] Targeted behavior tests pass for the current change set, or tests
are marked not applicable above — command/result or justification: Tests
are not applicable to this documentation-only change set.
- [ ] Applicable broad gate passed — `npm test` for broad
runtime/test-harness changes; `npm run check` for repo-wide
validation/coverage changes — command/result:
- [x] Quality Gates section completed with required justifications or
waivers
- [x] No secrets, API keys, or credentials committed
- [ ] `npm run docs` builds without warnings (doc changes only) — exited
0 with zero errors; Fern reported the existing unauthenticated
redirect-check and light-mode contrast warnings.
- [x] Doc pages follow the [style
guide](https://github.com/NVIDIA/NemoClaw/blob/main/docs/CONTRIBUTING.md)
(doc changes only)
- [ ] New doc pages include SPDX header and frontmatter (new pages only)

---
<!-- DCO sign-off is required in this PR description, and every commit
must appear as Verified in GitHub. Run: git config user.name && git
config user.email -->
Signed-off-by: Charan Jagwani <cjagwani@nvidia.com>

---------

Signed-off-by: cjagwani <cjagwani@nvidia.com>
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

area: cli Command line interface, flags, terminal UX, or output area: onboarding Onboarding FSM, provider setup, sandbox launch, or first-run flow bug-fix PR fixes a bug or regression

Projects

None yet

Development

Successfully merging this pull request may close these issues.

6 participants