Skip to content
Merged
Show file tree
Hide file tree
Changes from all commits
Commits
File filter

Filter by extension

Filter by extension

Conversations
Failed to load comments.
Loading
Jump to
Jump to file
Failed to load files.
Loading
Diff view
Diff view
2 changes: 2 additions & 0 deletions docs/get-started/quickstart-hermes.mdx
Original file line number Diff line number Diff line change
Expand Up @@ -267,6 +267,8 @@ Use these details when your first-run path needs more control.
```

The onboard flow starts both port forwards automatically.
If OpenShell reports `sandbox is not ready`, NemoClaw waits 5 seconds and retries the affected forward up to three times.
These retries preserve the existing sandbox and selected host port.
The Hermes dashboard URL does not include an OpenClaw `#token=` fragment.
`nemohermes my-hermes dashboard-url --quiet` returns `http://127.0.0.1:18789/` when the default local forward is active.
Check the API health endpoint from the host.
Expand Down
2 changes: 2 additions & 0 deletions docs/get-started/quickstart.mdx
Original file line number Diff line number Diff line change
Expand Up @@ -398,6 +398,8 @@ Use these details when your first-run path needs more control.
The wizard starts a background dashboard port forward and prints its URL in the ready summary.
The default host port is `18789`.
When that port is occupied, NemoClaw uses the next free dashboard port, such as `18790`, and includes the port in the URL.
If OpenShell reports `sandbox is not ready`, NemoClaw waits 5 seconds and retries the dashboard forward up to three times.
These retries preserve the existing sandbox and selected port.
If the selected port becomes occupied after the sandbox build begins, onboarding rolls back the new sandbox and asks you to retry rather than print an unreachable URL.
The installation transcript does not print the gateway token.
Use `nemoclaw my-gpt-claw dashboard-url --quiet` to print the complete authenticated URL explicitly.
Expand Down
35 changes: 35 additions & 0 deletions src/lib/onboard/forward-start.test.ts
Original file line number Diff line number Diff line change
Expand Up @@ -937,6 +937,41 @@ describe("runDetachedForwardStartWithRetries", () => {
expect(spawn).toHaveBeenCalledTimes(2);
});

it("retries an OpenShell sandbox readiness rejection after a bounded settle delay", () => {
const fetchList = vi
.fn()
.mockReturnValueOnce(forwardListWith([]))
.mockReturnValue(forwardListWith([{ sandbox: "my-sandbox", port: 18789 }]));
const spawn = vi
.fn()
.mockImplementationOnce(({ stderr }: { stderr: number }) => {
fs.writeSync(
stderr,
"Error: code: 'The system is not in a state required for the operation's execution', message: \"sandbox is not ready\"\n",
);
return { pid: 784 };
})
.mockReturnValueOnce({ pid: 785 });
const beforeRetry = vi.fn();
const sleep = vi.fn();

const result = runDetachedForwardStartWithRetries(
spawn,
fetchList,
{ port: 18789, sandboxName: "my-sandbox" },
beforeRetry,
{
sleepMs: sleep,
isPortListening: vi.fn().mockReturnValue(false),
},
);

expect(result.ok).toBe(true);
expect(beforeRetry).not.toHaveBeenCalled();
expect(spawn).toHaveBeenCalledTimes(2);
expect(sleep).toHaveBeenCalledWith(5_000);
});
Comment on lines +940 to +973

Copy link
Copy Markdown
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

🎯 Functional Correctness | 🟡 Minor | ⚡ Quick win

Assert that the settle delay occurs before the second spawn.

expect(sleep).toHaveBeenCalledWith(5_000) proves only that a matching call occurred. It does not prove that the call happened between the failed and successful spawn, or that the test used only one settle delay. Record the injected events and assert spawn-1, sleep(5_000), spawn-2; also assert that sleep was called once.

As per path instructions, verify behavioral confidence at the public boundary rather than only the presence of a mock call.

Suggested test adjustment
+    const events: string[] = [];
     const spawn = vi
       .fn()
       .mockImplementationOnce(({ stderr }: { stderr: number }) => {
+        events.push("spawn-1");
         fs.writeSync(
           stderr,
           "Error: code: 'The system is not in a state required for the operation's execution', message: \"sandbox is not ready\"\n",
         );
         return { pid: 784 };
       })
-      .mockReturnValueOnce({ pid: 785 });
+      .mockImplementationOnce(() => {
+        events.push("spawn-2");
+        return { pid: 785 };
+      });
     const beforeRetry = vi.fn();
-    const sleep = vi.fn();
+    const sleep = vi.fn((ms: number) => {
+      events.push(`sleep-${ms}`);
+    });
...
+    expect(sleep).toHaveBeenCalledTimes(1);
     expect(sleep).toHaveBeenCalledWith(5_000);
+    expect(events).toEqual(["spawn-1", "sleep-5000", "spawn-2"]);
📝 Committable suggestion

‼️ IMPORTANT
Carefully review the code before committing. Ensure that it accurately replaces the highlighted code, contains no missing lines, and has no issues with indentation. Thoroughly test & benchmark the code to ensure it meets the requirements.

Suggested change
it("retries an OpenShell sandbox readiness rejection after a bounded settle delay", () => {
const fetchList = vi
.fn()
.mockReturnValueOnce(forwardListWith([]))
.mockReturnValue(forwardListWith([{ sandbox: "my-sandbox", port: 18789 }]));
const spawn = vi
.fn()
.mockImplementationOnce(({ stderr }: { stderr: number }) => {
fs.writeSync(
stderr,
"Error: code: 'The system is not in a state required for the operation's execution', message: \"sandbox is not ready\"\n",
);
return { pid: 784 };
})
.mockReturnValueOnce({ pid: 785 });
const beforeRetry = vi.fn();
const sleep = vi.fn();
const result = runDetachedForwardStartWithRetries(
spawn,
fetchList,
{ port: 18789, sandboxName: "my-sandbox" },
beforeRetry,
{
sleepMs: sleep,
isPortListening: vi.fn().mockReturnValue(false),
},
);
expect(result.ok).toBe(true);
expect(beforeRetry).not.toHaveBeenCalled();
expect(spawn).toHaveBeenCalledTimes(2);
expect(sleep).toHaveBeenCalledWith(5_000);
});
it("retries an OpenShell sandbox readiness rejection after a bounded settle delay", () => {
const fetchList = vi
.fn()
.mockReturnValueOnce(forwardListWith([]))
.mockReturnValue(forwardListWith([{ sandbox: "my-sandbox", port: 18789 }]));
const events: string[] = [];
const spawn = vi
.fn()
.mockImplementationOnce(({ stderr }: { stderr: number }) => {
events.push("spawn-1");
fs.writeSync(
stderr,
"Error: code: 'The system is not in a state required for the operation's execution', message: \"sandbox is not ready\"\n",
);
return { pid: 784 };
})
.mockImplementationOnce(() => {
events.push("spawn-2");
return { pid: 785 };
});
const beforeRetry = vi.fn();
const sleep = vi.fn((ms: number) => {
events.push(`sleep-${ms}`);
});
const result = runDetachedForwardStartWithRetries(
spawn,
fetchList,
{ port: 18789, sandboxName: "my-sandbox" },
beforeRetry,
{
sleepMs: sleep,
isPortListening: vi.fn().mockReturnValue(false),
},
);
expect(result.ok).toBe(true);
expect(beforeRetry).not.toHaveBeenCalled();
expect(spawn).toHaveBeenCalledTimes(2);
expect(sleep).toHaveBeenCalledTimes(1);
expect(sleep).toHaveBeenCalledWith(5_000);
expect(events).toEqual(["spawn-1", "sleep-5000", "spawn-2"]);
});
🤖 Prompt for AI Agents
Verify each finding against current code. Fix only still-valid issues, skip the
rest with a brief reason, keep changes minimal, and validate.

In `@src/lib/onboard/forward-start.test.ts` around lines 940 - 973, Update the
test around runDetachedForwardStartWithRetries to record ordered events from the
injected spawn and sleep mocks, then assert the sequence is spawn-1,
sleep(5_000), spawn-2. Also assert sleep was called exactly once while
preserving the existing public-boundary result and retry assertions.

Source: Path instructions


it("preserves a ControlMaster listener created by the current attempt (#6099)", () => {
const fetchList = vi.fn().mockReturnValue(forwardListWith([]));
const spawn = vi.fn().mockImplementation(({ stderr }: { stderr: number }) => {
Expand Down
11 changes: 11 additions & 0 deletions src/lib/onboard/forward-start.ts
Original file line number Diff line number Diff line change
Expand Up @@ -120,8 +120,12 @@ export function looksLikeUntrackedForward(diagnostic: string): boolean {
* once OpenShell either keeps the attempt alive until the listener is ready or
* exposes a structured retryable outcome. Keep the fragments narrow so an
* unrelated SSH or gateway failure cannot enter the listener-retry path.
* OpenShell 0.0.101 can also reject a forward during the sandbox readiness
* handoff. That command has already exited, so list polling cannot recover it;
* the retry wrapper below gives the OpenShell gateway a bounded settle interval.
*/
export function looksLikeForwardListenerStartFailure(diagnostic: string): boolean {
if (/\bsandbox is not ready\b/i.test(diagnostic)) return true;
Comment on lines +123 to +128

Copy link
Copy Markdown
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

🎯 Functional Correctness | 🟡 Minor | ⚡ Quick win

🧩 Analysis chain

🏁 Script executed:

#!/usr/bin/env bash
set -euo pipefail

rg -n -C 6 \
  'sandbox is not ready|looksLikeForwardListenerStartFailure|runDetachedForwardStartWithRetries|Permission denied|gateway transport unavailable' \
  src

Repository: NVIDIA/NemoClaw

Length of output: 45490


🏁 Script executed:

#!/usr/bin/env bash
set -euo pipefail

printf '%s\n' '--- implementation ---'
cat -n src/lib/onboard/forward-start.ts | sed -n '110,145p;490,550p'

printf '%s\n' '--- tests ---'
cat -n src/lib/onboard/forward-start.test.ts | sed -n '940,1050p;1080,1110p'

printf '%s\n' '--- exact diagnostic constants and related contracts ---'
rg -n -C 5 \
  'OPENSHELL_SANDBOX_NOT_READY|sandbox is not ready|listener-start-failure|runDetachedForwardStartWithDiagnostics' \
  src/lib/onboard src/lib/actions/sandbox

Repository: NVIDIA/NemoClaw

Length of output: 50371


🏁 Script executed:

#!/usr/bin/env bash
set -euo pipefail

printf '%s\n' '--- diagnostic construction ---'
cat -n src/lib/onboard/forward-start.ts | sed -n '321,490p'

printf '%s\n' '--- focused listener-start tests ---'
rg -n -C 8 \
  'listener-start-failure|access denied|no forward list result|sandbox readiness rejection|sandbox is not ready' \
  src/lib/onboard/forward-start.test.ts

printf '%s\n' '--- deterministic matcher probe ---'
python3 - <<'PY'
import re

matcher = re.compile(r"\bsandbox is not ready\b", re.I)
cases = {
    "completed command": (
        "Error: code: 'The system is not in a state required for the operation's "
        'execution\', message: "sandbox is not ready"'
    ),
    "composite authentication diagnostic": (
        'forward start failed: Permission denied (publickey); '
        'previous attempt reported "sandbox is not ready"'
    ),
    "composite gateway diagnostic": (
        'gateway transport unavailable while handling sandbox is not ready'
    ),
    "unrelated forwarding diagnostic": (
        'forward start failed: local target unavailable; sandbox is not ready was '
        'reported by a different operation'
    ),
}
for name, diagnostic in cases.items():
    print(f"{name}: {bool(matcher.search(diagnostic))}")
PY

Repository: NVIDIA/NemoClaw

Length of output: 14602


Match the full completed-command readiness diagnostic in both retry checks. The current substring match classifies composite authentication or gateway diagnostics as retryable and applies the 5-second delay. Keep unrelated forwarding failures terminal, and add a negative retry test for a composite diagnostic.

🤖 Prompt for AI Agents
Verify each finding against current code. Fix only still-valid issues, skip the
rest with a brief reason, keep changes minimal, and validate.

In `@src/lib/onboard/forward-start.ts` around lines 123 - 128, Update
looksLikeForwardListenerStartFailure to match the full completed-command
“sandbox is not ready” readiness diagnostic rather than any substring, while
preserving terminal handling for unrelated forwarding failures. Apply the same
exact diagnostic matching in both retry checks, and add a negative test covering
a composite authentication or gateway diagnostic that must not trigger the
5-second retry delay.

return /ssh exited before local forward listener opened|local forward listener did not open\b/i.test(
diagnostic,
);
Expand Down Expand Up @@ -178,6 +182,7 @@ function blockingSleepMs(ms: number): void {
// supported OpenShell version either stops retaining persistent dead rows or
// exposes an atomic recovery operation.
const DEAD_FORWARD_GRACE_MS = 2_000;
const SANDBOX_READY_RETRY_SETTLE_MS = 5_000;

/**
* Build a `DetachedForwardSpawnRunner` that spawns the given argv as a
Expand Down Expand Up @@ -502,6 +507,7 @@ export function runDetachedForwardStartWithRetries(
options: DetachedForwardStartOptions = {},
): DetachedForwardStartOutcome {
const maxRetries = options.maxRetries ?? 3;
const sleepImpl = options.sleepMs ?? blockingSleepMs;
let deadForwardRecoveryAvailable = true;
const isPortListening = options.isPortListening ?? probeLocalPortListening;
const runAttempt = (): DetachedForwardStartOutcome =>
Expand All @@ -528,6 +534,11 @@ export function runDetachedForwardStartWithRetries(
if (looksLikeForwardPortConflict(attempt.diagnostic)) {
beforeRetryCleanup();
}
if (/\bsandbox is not ready\b/i.test(attempt.diagnostic)) {
// Keep the existing sandbox and port ownership intact while the
// OpenShell gateway finishes the readiness handoff.
sleepImpl(SANDBOX_READY_RETRY_SETTLE_MS);
}
standardRetries++;
}
attempt = runAttempt();
Expand Down
Loading