Skip to content
Merged
Show file tree
Hide file tree
Changes from all commits
Commits
File filter

Filter by extension

Filter by extension


Conversations
Failed to load comments.
Loading
Jump to
Jump to file
Failed to load files.
Loading
Diff view
Diff view
3 changes: 0 additions & 3 deletions .github/workflows/e2e.yaml
Original file line number Diff line number Diff line change
Expand Up @@ -2781,9 +2781,6 @@ jobs:
npx vitest run --project e2e-live \
test/e2e/live/full-e2e.test.ts \
--silent=false --reporter=default
npx vitest run --project e2e-live \
test/e2e/live/onboard-progress-budget.test.ts \
--silent=false --reporter=default

- name: Upload full-e2e artifacts
if: always()
Expand Down
2 changes: 2 additions & 0 deletions docs/deployment/install-openclaw-plugins.mdx
Original file line number Diff line number Diff line change
Expand Up @@ -74,6 +74,8 @@ To run a second sandbox alongside an existing one, use a dedicated build directo
## Build Performance

Custom plugin images are normal Docker builds, so build time depends on the build context size and the Docker layer cache rather than on NemoClaw.
NemoClaw sends user-supplied `--from` contexts to the OpenShell gateway builder and reserves its host-side local BuildKit prebuild for contexts that NemoClaw generates itself.
On a local Docker-driver gateway, a `Local BuildKit build skipped` notice is expected and the custom image build continues through the gateway.

Keep the build context small and dedicated.
The Dockerfile's parent directory is staged as the build context before the Docker build starts, so a broad directory can make onboarding look stuck while Docker is only preparing context.
Expand Down
2 changes: 2 additions & 0 deletions docs/manage-sandboxes/install-plugins-hermes.mdx
Original file line number Diff line number Diff line change
Expand Up @@ -54,6 +54,8 @@ Put the custom Dockerfile and every file it needs to `COPY` in one directory.
`nemohermes onboard --from <Dockerfile>` sends the Dockerfile's parent directory as the Docker build context.
Add a `.dockerignore` next to the Dockerfile to keep local caches, generated artifacts, model files, or other unneeded paths out of the staged context.
NemoClaw still excludes credential-like paths such as `.env*`, `.ssh/`, `.aws/`, `.npmrc`, `secrets/`, `*.pem`, and `*.key`, even if `.dockerignore` tries to include them.
NemoClaw sends user-supplied `--from` contexts to the OpenShell gateway builder and reserves its host-side local BuildKit prebuild for contexts that NemoClaw generates itself.
On a local Docker-driver gateway, a `Local BuildKit build skipped` notice is expected and the custom image build continues through the gateway.

```text
my-hermes-plugin-sandbox/
Expand Down
7 changes: 7 additions & 0 deletions docs/reference/commands-nemohermes.mdx
Original file line number Diff line number Diff line change
Expand Up @@ -342,6 +342,12 @@ If the staged context is larger than 100 MB, onboarding prints a warning before
Move the Dockerfile into a smaller dedicated directory or add `.dockerignore` entries for generated artifacts to shrink the context.
If the directory contains unreadable files (for example, Windows system files visible in WSL), onboarding exits with an error suggesting you move the Dockerfile to a dedicated directory.

<Note>
NemoClaw builds user-supplied `--from` contexts with the OpenShell gateway builder.
The host-side local BuildKit prebuild is limited to build contexts generated entirely by NemoClaw.
On a local Docker-driver gateway, a `Local BuildKit build skipped` notice is expected and onboarding continues with the custom image.
</Note>

```bash
nemohermes onboard --from path/to/Dockerfile
```
Expand Down Expand Up @@ -400,6 +406,7 @@ Combining `--from <Dockerfile>` with non-interactive onboarding requires one of

Use a custom Dockerfile for the sandbox image.
This variant of `nemohermes onboard` accepts a `--from <Dockerfile>` argument to build the sandbox from a user-supplied Dockerfile instead of the default NemoClaw image.
The user-supplied context uses the OpenShell gateway builder instead of NemoClaw's host-side local BuildKit prebuild.

```bash
nemohermes onboard --from ./Dockerfile.custom
Expand Down
7 changes: 7 additions & 0 deletions docs/reference/commands.mdx
Original file line number Diff line number Diff line change
Expand Up @@ -446,6 +446,12 @@ If the staged context is larger than 100 MB, onboarding prints a warning before
Move the Dockerfile into a smaller dedicated directory or add `.dockerignore` entries for generated artifacts to shrink the context.
If the directory contains unreadable files (for example, Windows system files visible in WSL), onboarding exits with an error suggesting you move the Dockerfile to a dedicated directory.

<Note>
NemoClaw builds user-supplied `--from` contexts with the OpenShell gateway builder.
The host-side local BuildKit prebuild is limited to build contexts generated entirely by NemoClaw.
On a local Docker-driver gateway, a `Local BuildKit build skipped` notice is expected and onboarding continues with the custom image.
</Note>

```bash
$$nemoclaw onboard --from path/to/Dockerfile
```
Expand Down Expand Up @@ -504,6 +510,7 @@ Combining `--from <Dockerfile>` with non-interactive onboarding requires one of

Use a custom Dockerfile for the sandbox image.
This variant of `$$nemoclaw onboard` accepts a `--from <Dockerfile>` argument to build the sandbox from a user-supplied Dockerfile instead of the default NemoClaw image.
The user-supplied context uses the OpenShell gateway builder instead of NemoClaw's host-side local BuildKit prebuild.

```bash
$$nemoclaw onboard --from ./Dockerfile.custom
Expand Down
Original file line number Diff line number Diff line change
Expand Up @@ -40,6 +40,7 @@ describe("preflightRebuildImage", () => {
buildCtx: "/tmp/rebuild-managed-context",
stagedDockerfile: "/tmp/rebuild-managed-context/Dockerfile",
cleanupBuildCtx,
origin: "generated" as const,
}));
const result = await preflightRebuildImage(input(null), {
stageBuildContext,
Expand Down
1 change: 1 addition & 0 deletions src/lib/actions/sandbox/rebuild-gpu-opt-out.test.ts
Original file line number Diff line number Diff line change
Expand Up @@ -244,6 +244,7 @@ describe("buildRebuildRecreateOnboardOpts", () => {
stagedDockerfile: "/tmp/dcode-rebuild/Dockerfile",
buildId: "dcode-build",
cleanupBuildCtx: () => true,
origin: "generated" as const,
},
gatewayName: "nemoclaw",
};
Expand Down
Original file line number Diff line number Diff line change
Expand Up @@ -61,6 +61,7 @@ describe("managed DCode rebuild image preflight", () => {
buildCtx,
stagedDockerfile,
cleanupBuildCtx,
origin: "generated" as const,
}));
const prepareDockerfilePatch = vi.fn(async () => ({
buildId: "dcode-build-1",
Expand Down Expand Up @@ -260,7 +261,12 @@ describe("managed DCode rebuild image preflight", () => {
return true;
});
const result = await prepareManagedDcodeRebuildImage(dcodeInput(), {
stageBuildContext: vi.fn(() => ({ buildCtx, stagedDockerfile, cleanupBuildCtx })),
stageBuildContext: vi.fn(() => ({
buildCtx,
stagedDockerfile,
cleanupBuildCtx,
origin: "generated" as const,
})),
prepareDockerfilePatch: vi.fn(async () => ({
buildId: "dcode-build-cleanup",
resolvedBaseImage: null,
Expand Down Expand Up @@ -292,6 +298,7 @@ describe("managed DCode rebuild image preflight", () => {
buildCtx,
stagedDockerfile,
cleanupBuildCtx,
origin: "generated" as const,
})),
prepareDockerfilePatch: vi.fn(async () => ({
buildId: "dcode-build-failure",
Expand Down
3 changes: 2 additions & 1 deletion src/lib/agent/base-image.ts
Original file line number Diff line number Diff line change
Expand Up @@ -15,6 +15,7 @@ import {
dockerTag,
} from "../adapters/docker";
import { ROOT } from "../runner";
import { SANDBOX_BUILD_CONTEXT_PREFIX } from "../sandbox/build-context";
import {
buildLocalBaseTag,
createSandboxBaseImageResolutionKey,
Expand Down Expand Up @@ -324,7 +325,7 @@ export function createAgentSandbox(
}

const { imageTag: baseImageRef, resolutionMetadata } = ensureAgentBaseImage(agent, options);
const buildCtx = fs.mkdtempSync(path.join(os.tmpdir(), "nemoclaw-build-"));
const buildCtx = fs.mkdtempSync(path.join(os.tmpdir(), SANDBOX_BUILD_CONTEXT_PREFIX));
fs.cpSync(ROOT, buildCtx, {
recursive: true,
filter: (src) => {
Expand Down
10 changes: 5 additions & 5 deletions src/lib/onboard.ts
Original file line number Diff line number Diff line change
Expand Up @@ -2771,7 +2771,7 @@ async function createSandboxWithBaseImageResolution(
// in env args, so it must not persist in /tmp after a failed sandbox create.
// run() calls process.exit() on failure (bypassing normal control flow), so
// we register a process 'exit' handler to guarantee cleanup in all cases.
const { buildCtx, stagedDockerfile, cleanupBuildCtx } =
const { buildCtx, stagedDockerfile, origin, cleanupBuildCtx } =
preparedDcodeRebuild.resolveSandboxBuildContext(
{
preparedBuildContext,
Expand All @@ -2798,6 +2798,7 @@ async function createSandboxWithBaseImageResolution(
"openclaw-sandbox.yaml",
);
const basePolicyPath = (agent && agentOnboard.getAgentPolicyPath(agent)) || defaultPolicyPath;
const dockerDriverGateway = isLinuxDockerDriverGatewayEnabled();
const {
activeMessagingChannels,
initialSandboxPolicy,
Expand All @@ -2818,7 +2819,7 @@ async function createSandboxWithBaseImageResolution(
extraProviders: registry.listExtraProviders(),
hermesToolGateways,
sandboxGpuConfig: effectiveSandboxGpuConfig,
dockerDriverGateway: isLinuxDockerDriverGatewayEnabled(),
dockerDriverGateway,
appendResourceFlags: (args) =>
appendResourceFlagsForProfile(args, resourceProfile, getOpenshellBinary(), {
isNonInteractive,
Expand Down Expand Up @@ -2884,8 +2885,7 @@ async function createSandboxWithBaseImageResolution(
hermesDashboardState,
manageDashboard,
openshellShellCommand,
// Transitional BuildKit handoff removal is tracked by #6258.
prebuild: { buildCtx, buildId, dockerDriverGateway: isLinuxDockerDriverGatewayEnabled() },
prebuild: { buildCtx, buildId, dockerDriverGateway, origin },
});
const dockerGpuCreatePatch = dockerGpuSandboxCreate.createDockerGpuSandboxCreatePatch({
enabled: useDockerGpuPatch,
Expand Down Expand Up @@ -3007,7 +3007,7 @@ async function createSandboxWithBaseImageResolution(
// when applicable, then gates host-network local inference reachability (#4509).
dockerGpuLocalInference.verifyGpuSandboxAfterReady(effectiveSandboxGpuConfig, provider, {
sandboxName,
dockerDriverGateway: isLinuxDockerDriverGatewayEnabled(),
dockerDriverGateway,
useDockerGpuPatch,
verifyDirectSandboxGpu,
verifyGpuOrExit: dockerGpuCreatePatch.verifyGpuOrExit,
Expand Down
3 changes: 3 additions & 0 deletions src/lib/onboard/build-context-stage.test.ts
Original file line number Diff line number Diff line change
Expand Up @@ -53,6 +53,7 @@ describe("stageCreateSandboxBuildContext", () => {
` Docker build context: ${buildContextDir}`,
]);
expect(fs.readFileSync(result.stagedDockerfile, "utf-8")).toBe("FROM scratch\n");
expect(result.origin).toBe("custom");
expect(fs.existsSync(path.join(result.buildCtx, "extra.txt"))).toBe(true);
expect(fs.existsSync(path.join(result.buildCtx, ".ssh"))).toBe(false);
expect(result.cleanupBuildCtx()).toBe(true);
Expand Down Expand Up @@ -198,6 +199,7 @@ describe("stageCreateSandboxBuildContext", () => {
});

expect(agentResult.buildCtx).toBe(agentBuild.buildCtx);
expect(agentResult.origin).toBe("generated");
expect(createAgentSandbox).toHaveBeenCalledWith({ name: "hermes" });
expect(stageDefaultSandboxBuildContext).not.toHaveBeenCalled();

Expand All @@ -210,6 +212,7 @@ describe("stageCreateSandboxBuildContext", () => {
});

expect(defaultResult.buildCtx).toBe(defaultBuild.buildCtx);
expect(defaultResult.origin).toBe("generated");
expect(stageDefaultSandboxBuildContext).toHaveBeenCalledWith("/repo");
});
});
7 changes: 6 additions & 1 deletion src/lib/onboard/build-context-stage.ts
Original file line number Diff line number Diff line change
Expand Up @@ -9,6 +9,8 @@ import type { AgentDefinition } from "../agent/defs";
import { isErrnoException } from "../core/errno";
import {
collectBuildContextStats,
SANDBOX_BUILD_CONTEXT_PREFIX,
type SandboxBuildContextOrigin,
type StagedBuildContext,
stageOptimizedSandboxBuildContext,
} from "../sandbox/build-context";
Expand All @@ -31,6 +33,7 @@ export interface CreateSandboxBuildContextInput {
}

export interface CreateSandboxBuildContextResult extends StagedBuildContext {
origin: SandboxBuildContextOrigin;
cleanupBuildCtx(): boolean;
}

Expand All @@ -57,6 +60,7 @@ export function stageCreateSandboxBuildContext(
const warn = input.warn ?? console.warn;
const error = input.error ?? console.error;
const exit = input.exit ?? ((code?: number): never => process.exit(code));
const origin = input.fromDockerfile ? "custom" : "generated";

let build: StagedBuildContext;

Expand Down Expand Up @@ -92,7 +96,7 @@ export function stageCreateSandboxBuildContext(
" The --from flag sends the Dockerfile's parent directory to Docker; use a dedicated directory if this is not intentional.",
);
}
const buildCtx = fs.mkdtempSync(path.join(os.tmpdir(), "nemoclaw-build-"));
const buildCtx = fs.mkdtempSync(path.join(os.tmpdir(), SANDBOX_BUILD_CONTEXT_PREFIX));
const stagedDockerfile = path.join(buildCtx, "Dockerfile");
const cleanupCustomBuildCtx = (): void => {
try {
Expand Down Expand Up @@ -133,6 +137,7 @@ export function stageCreateSandboxBuildContext(

return {
...build,
origin,
cleanupBuildCtx: createCleanupBuildContext(build.buildCtx),
};
}
49 changes: 48 additions & 1 deletion src/lib/onboard/machine/live-flow-slice.test.ts
Original file line number Diff line number Diff line change
@@ -1,7 +1,7 @@
// SPDX-FileCopyrightText: Copyright (c) 2026 NVIDIA CORPORATION & AFFILIATES. All rights reserved.
// SPDX-License-Identifier: Apache-2.0

import { describe, expect, it, vi } from "vitest";
import { afterEach, describe, expect, it, vi } from "vitest";

import { createSession, type Session } from "../../state/onboard-session";
import {
Expand Down Expand Up @@ -69,6 +69,11 @@ function phase(
}

describe("runLiveOnboardFlowSlice", () => {
afterEach(() => {
vi.useRealTimers();
vi.restoreAllMocks();
});

it("uses the strict slice runner for fresh matching entry states", async () => {
const runSlice = vi.fn(async ({ context }) => ({
context: { value: context.value + 1 },
Expand Down Expand Up @@ -178,6 +183,48 @@ describe("runLiveOnboardFlowSlice", () => {
expect(applyCompatibleResult).toHaveBeenCalledOnce();
});

it("keeps compatibility phases visible through the default heartbeat reporter", async () => {
vi.useFakeTimers();
vi.setSystemTime(0);
const log = vi.spyOn(console, "log").mockImplementation(() => {});
let markPhaseStarted!: () => void;
let releasePhase!: () => void;
const phaseStarted = new Promise<void>((resolve) => {
markPhaseStarted = resolve;
});
const phaseReleased = new Promise<void>((resolve) => {
releasePhase = resolve;
});
const liveRuntime = runtime("provider_selection");
const pendingGateway: OnboardSequencePhase<Context> = {
state: "gateway",
async run(context) {
markPhaseStarted();
await phaseReleased;
return { context, result: advanceTo("inference") };
},
};

const running = runLiveOnboardFlowSlice({
context: { value: 1 },
runtime: liveRuntime.runtime,
phases: [pendingGateway],
runWhenState: ["gateway"],
compatibilityWhenState: ["provider_selection"],
runSlice: vi.fn(),
applyCompatibleResult: (result) => liveRuntime.applyResult(result),
});
await phaseStarted;
try {
await vi.advanceTimersByTimeAsync(30_000);
expect(log).toHaveBeenCalledWith(" ⏳ Still working on Gateway startup… (30s elapsed)");
} finally {
releasePhase();
await running;
}
expect(vi.getTimerCount()).toBe(0);
});

it("rejects non-resume states before the slice entry before running side effects", async () => {
const liveRuntime = runtime("init");
const blocked = phase("provider_selection", 2);
Expand Down
2 changes: 2 additions & 0 deletions src/lib/onboard/prepared-dcode-rebuild.test.ts
Original file line number Diff line number Diff line change
Expand Up @@ -19,6 +19,7 @@ const preparedBuildContext: PreparedSandboxBuildContext = {
stagedDockerfile: "/tmp/prepared-dcode/Dockerfile",
buildId: "6195-prepared",
cleanupBuildCtx: () => true,
origin: "generated",
};
const preparedOptions: PreparedDcodeRebuildOptions = {
resume: true,
Expand Down Expand Up @@ -119,6 +120,7 @@ describe("prepared DCode rebuild adapter", () => {
buildCtx: "/tmp/ordinary",
stagedDockerfile: "/tmp/ordinary/Dockerfile",
cleanupBuildCtx: () => true,
origin: "generated" as const,
}));
const onExit = vi.fn();

Expand Down
Loading
Loading