Skip to content
Merged
Show file tree
Hide file tree
Changes from 18 commits
Commits
Show all changes
19 commits
Select commit Hold shift + click to select a range
cf8b72e
fix(snapshot): name the python3 prerequisite when sanitization cannot…
harjothkhara Aug 17, 2026
ae90932
fix(snapshot): cover the remaining prerequisite branches and drop the…
harjothkhara Aug 17, 2026
823f2e7
docs(snapshot): clarify verified interpreter recovery
harjothkhara Aug 17, 2026
6cc7a01
fix(snapshot): report retained sanitization path
cv Aug 17, 2026
eaea9f6
fix(snapshot): preserve sanitizer failure evidence
harjothkhara Aug 17, 2026
b4030d6
test(snapshot): cover helper failure classification
harjothkhara Aug 17, 2026
d4158b1
chore(snapshot): format sanitizer boundary
harjothkhara Aug 17, 2026
cd7c7f6
test(snapshot): pass the retained path to failure mapping
harjothkhara Aug 17, 2026
c0bfc13
test(snapshot): assert the canonical retained backup path
harjothkhara Aug 17, 2026
efc5f5d
test(snapshot): keep prerequisite case linear
cv Aug 17, 2026
4e0fc94
fix(logger): preserve aggregate error details
harjothkhara Aug 17, 2026
dc92863
docs(snapshot): scope interpreter guidance to live commands
harjothkhara Aug 17, 2026
ba59f8d
test(snapshot): harden sanitizer failure coverage
harjothkhara Aug 17, 2026
50f09bf
test(snapshot): check python before sanitizer scan
apurvvkumaria Aug 17, 2026
df34207
test(snapshot): distinguish helper execution failures
harjothkhara Aug 17, 2026
5ed9f0e
Merge branch 'main' into oss-find/nemoclaw-2026-08-16
prekshivyas Aug 18, 2026
cdb3e71
Merge branch 'main' into oss-find/nemoclaw-2026-08-16
prekshivyas Aug 18, 2026
edb39f2
test(logger): cover aggregate error redaction
apurvvkumaria Aug 18, 2026
937e9f7
refactor(security): simplify sanitizer prerequisite errors
apurvvkumaria Aug 18, 2026
File filter

Filter by extension

Filter by extension

Conversations
Failed to load comments.
Loading
Jump to
Jump to file
Failed to load files.
Loading
Diff view
Diff view
43 changes: 43 additions & 0 deletions docs/reference/troubleshooting.mdx
Original file line number Diff line number Diff line change
Expand Up @@ -1631,6 +1631,49 @@ $$nemoclaw <name> snapshot create
$$nemoclaw <name> rebuild
```

### Snapshot Sanitization Requires a Verified Python Interpreter

The CLI operations listed below stop when NemoClaw cannot resolve a verified interpreter.
The error begins with this text:

```text
python3 is required for snapshot sanitization; install python3 and rerun
```

NemoClaw removes credentials from copied state with an isolated `python3` helper.
The operation fails closed when that helper cannot run.
These operations can report the error:

- `$$nemoclaw <name> snapshot create`
- `$$nemoclaw <name> rebuild`, including rebuilds started by `$$nemoclaw upgrade-sandboxes`
- `$$nemoclaw backup-all`, including the installer's pre-upgrade backup and an eligible stopped Docker-driver sandbox

A host that already has `python3` can still report this message.
NemoClaw does not search `PATH` for this credential-bearing helper.
NemoClaw accepts `python3` only at these locations:

- `/usr/bin/python3`
- `/usr/local/bin/python3`
- `/opt/homebrew/bin/python3`
- `/opt/local/bin/python3`
- A `python3` executable beside the canonical Node.js executable

NemoClaw rejects a candidate that fails its ownership, permission, or executable checks.
For more information about the interpreter requirement, refer to [Prerequisites](../get-started/prerequisites).

If NemoClaw reports that it removed the incomplete snapshot, install or repair `python3` at an accepted location.
Then rerun the complete command.

If cleanup fails, treat the exact reported directory as retained until you confirm that it is absent.

<Warning title="Retained Incomplete Snapshot">
A retained incomplete snapshot can contain unsanitized credentials.
The directory must remain owner-only.
You must not restore, copy, or share the directory.
You must remove the directory only by the exact path that NemoClaw reports.
You must confirm that the exact path no longer exists before you rerun the complete command.
</Warning>

### Sandbox shows as stopped

When status reports `sandbox_container_stopped`, Docker still has a container for the sandbox, but the container is not running.
Expand Down
50 changes: 50 additions & 0 deletions nemoclaw/src/security/snapshot-sanitizer-failure.test.ts
Original file line number Diff line number Diff line change
Expand Up @@ -17,13 +17,17 @@ import path from "node:path";
import { afterEach, describe, expect, it, vi } from "vitest";
import {
applyDescriptorSnapshotActions,
applyDescriptorSnapshotActionsResult,
decodeDescriptorSnapshotContent,
inspectDescriptorSnapshotRoot,
installDescriptorSnapshotFile,
resolveTrustedSnapshotSanitizerPythonPath,
SnapshotSanitizerPrerequisiteError,
type SnapshotFileIdentity,
scanDescriptorSnapshot,
scanDescriptorSnapshotResult,
setSnapshotSanitizerPythonPathForTest,
snapshotSanitizerFailure,
} from "../shared/snapshot-sanitizer-boundary.cjs";
import { sanitizeMigrationDirectory, sanitizeOpenClawConfigFile } from "./snapshot-sanitizer.js";

Expand Down Expand Up @@ -68,6 +72,13 @@ function requireTrustedPython(): string {
return python as string;
}

function requireApplyFailure(
result: ReturnType<typeof applyDescriptorSnapshotActionsResult>,
): Extract<typeof result, { ok: false }> {
expect(result.ok).toBe(false);
return result as Extract<typeof result, { ok: false }>;
}

afterEach(() => {
setSnapshotSanitizerPythonPathForTest(undefined);
vi.unstubAllEnvs();
Expand Down Expand Up @@ -149,6 +160,45 @@ describe("migration snapshot sanitizer fallbacks", () => {
).toBe(false);
});

it("reports the interpreter state recorded by the apply attempt (#8202)", () => {
const rootPath = makeRoot();
const configPath = path.join(rootPath, "config.json");
writeFileSync(configPath, "original");
const root = inspectDescriptorSnapshotRoot(rootPath)!;
const python = requireTrustedPython();
const scan = scanDescriptorSnapshot(root, new Set())!;
const config = scan.files.find((file) => file.path === "config.json")!;
setSnapshotSanitizerPythonPathForTest(null);

const result = applyDescriptorSnapshotActionsResult(root, scan, [
{ kind: "remove", path: config.path, metadata: config.metadata },
]);
expect(result).toEqual({ ok: false, reason: "python-unavailable" });

setSnapshotSanitizerPythonPathForTest(python);
const failure = requireApplyFailure(result);
expect(
snapshotSanitizerFailure(failure.reason, "generic helper failure", root.canonicalPath),
).toBeInstanceOf(SnapshotSanitizerPrerequisiteError);
});

it("keeps the generic error when the helper ran and failed (#8202)", () => {
const error = snapshotSanitizerFailure("helper-failed", "generic helper failure", makeRoot());

expect(error).not.toBeInstanceOf(SnapshotSanitizerPrerequisiteError);
expect(error.message).toBe("generic helper failure");
});

it("classifies a running helper failure separately from a missing interpreter (#8202)", () => {
const root = inspectDescriptorSnapshotRoot(makeRoot())!;
writePythonWrapper(["exit 1"]);

expect(scanDescriptorSnapshotResult(root, new Set())).toEqual({
ok: false,
reason: "helper-failed",
});
});

it("fails closed when the descriptor install helper is unavailable", () => {
const root = inspectDescriptorSnapshotRoot(makeRoot());
expect(root).not.toBeNull();
Expand Down
79 changes: 64 additions & 15 deletions nemoclaw/src/shared/snapshot-sanitizer-boundary.cts
Original file line number Diff line number Diff line change
Expand Up @@ -55,9 +55,7 @@ export function resolveTrustedSnapshotSanitizerPythonPath(): string | null {
}

/** @visibleForTesting Install an explicit helper substitute without weakening production lookup. */
export function setSnapshotSanitizerPythonPathForTest(
pythonPath: string | null | undefined,
): void {
export function setSnapshotSanitizerPythonPathForTest(pythonPath: string | null | undefined): void {
if (process.env.VITEST !== "true") {
throw new Error("Snapshot sanitizer Python substitution is only available under Vitest");
}
Expand All @@ -67,13 +65,42 @@ export function setSnapshotSanitizerPythonPathForTest(
snapshotSanitizerPythonPathForTest = pythonPath;
}

/** Resolve the interpreter the helpers will actually run, including the test substitute. */
function snapshotSanitizerPythonPath(): string | null {
if (process.env.VITEST === "true" && snapshotSanitizerPythonPathForTest !== undefined) {
return snapshotSanitizerPythonPathForTest;
}
return resolveTrustedSnapshotSanitizerPythonPath();
}

/** No trusted python3 interpreter resolved, so no descriptor-relative helper can run. (#8202) */
export class SnapshotSanitizerPrerequisiteError extends Error {
readonly snapshotPath: string;

constructor(snapshotPath: string) {
super("python3 is required for snapshot sanitization; install python3 and rerun");
this.name = "SnapshotSanitizerPrerequisiteError";
this.snapshotPath = snapshotPath;
}
}

export type SnapshotSanitizerHelperFailure = "python-unavailable" | "helper-failed";

export type SnapshotSanitizerHelperResult<T> =
| { readonly ok: true; readonly value: T }
| { readonly ok: false; readonly reason: SnapshotSanitizerHelperFailure };

/** Map the helper attempt's recorded failure without probing the host again. */
export function snapshotSanitizerFailure(
reason: SnapshotSanitizerHelperFailure,
message: string,
validatedSnapshotPath: string,
): Error {
return reason === "python-unavailable"
? new SnapshotSanitizerPrerequisiteError(validatedSnapshotPath)
: new Error(message);
}

export interface SnapshotFileIdentity {
readonly dev: string;
readonly ino: string;
Expand Down Expand Up @@ -712,15 +739,15 @@ export function inspectDescriptorSnapshotRoot(rootPath: string): DescriptorSnaps
};
}

/** Read a bounded snapshot tree through pinned directory descriptors. */
export function scanDescriptorSnapshot(
/** Read a bounded snapshot tree and retain why the helper could not run. */
export function scanDescriptorSnapshotResult(
root: DescriptorSnapshotRoot,
sensitiveNames: ReadonlySet<string>,
targetName?: string,
): DescriptorSnapshotScan | null {
): SnapshotSanitizerHelperResult<DescriptorSnapshotScan> {
const mode = targetName === undefined ? "scan-tree" : "scan-file";
const pythonPath = snapshotSanitizerPythonPath();
if (pythonPath === null) return null;
if (pythonPath === null) return { ok: false, reason: "python-unavailable" };
const result = spawnSync(
pythonPath,
[
Expand All @@ -740,19 +767,30 @@ export function scanDescriptorSnapshot(
timeout: HELPER_TIMEOUT_MS,
},
);
if (result.status !== 0 || result.error) return null;
return parseScanResult(result.stdout);
if (result.status !== 0 || result.error) return { ok: false, reason: "helper-failed" };
const scan = parseScanResult(result.stdout);
return scan === null ? { ok: false, reason: "helper-failed" } : { ok: true, value: scan };
}

/** Install or remove sanitized artifacts through their pinned parent descriptors. */
export function applyDescriptorSnapshotActions(
/** Read a bounded snapshot tree through pinned directory descriptors. */
export function scanDescriptorSnapshot(
root: DescriptorSnapshotRoot,
sensitiveNames: ReadonlySet<string>,
targetName?: string,
): DescriptorSnapshotScan | null {
const result = scanDescriptorSnapshotResult(root, sensitiveNames, targetName);
return result.ok ? result.value : null;
}

/** Apply sanitized artifacts and retain why the helper could not run. */
export function applyDescriptorSnapshotActionsResult(
root: DescriptorSnapshotRoot,
scan: DescriptorSnapshotScan,
actions: readonly SnapshotSanitizationAction[],
): boolean {
if (actions.length === 0) return true;
): SnapshotSanitizerHelperResult<void> {
if (actions.length === 0) return { ok: true, value: undefined };
const pythonPath = snapshotSanitizerPythonPath();
if (pythonPath === null) return false;
if (pythonPath === null) return { ok: false, reason: "python-unavailable" };
const result = spawnSync(
pythonPath,
["-I", "-c", SNAPSHOT_SANITIZER_PYTHON, "apply", root.canonicalPath],
Expand All @@ -764,7 +802,18 @@ export function applyDescriptorSnapshotActions(
timeout: HELPER_TIMEOUT_MS,
},
);
return result.status === 0 && !result.error;
return result.status === 0 && !result.error
? { ok: true, value: undefined }
: { ok: false, reason: "helper-failed" };
}

/** Install or remove sanitized artifacts through their pinned parent descriptors. */
export function applyDescriptorSnapshotActions(
root: DescriptorSnapshotRoot,
scan: DescriptorSnapshotScan,
actions: readonly SnapshotSanitizationAction[],
): boolean {
return applyDescriptorSnapshotActionsResult(root, scan, actions).ok;
}

/** Create one direct child through a pinned directory descriptor without replacing an entry. */
Expand Down
46 changes: 35 additions & 11 deletions src/lib/cli/logger.test.ts
Original file line number Diff line number Diff line change
Expand Up @@ -39,17 +39,14 @@ describe("Logger", () => {
expect(log.level).toBe("debug");
});

it.each([
"1",
"true",
"y",
"yes",
"TRUE",
])("enables debug for the supported NEMOCLAW_DEBUG value %s", async (value) => {
vi.stubEnv("NEMOCLAW_DEBUG", value);
const { log } = await freshLogger();
expect(log.level).toBe("debug");
});
it.each(["1", "true", "y", "yes", "TRUE"])(
"enables debug for the supported NEMOCLAW_DEBUG value %s",
async (value) => {
vi.stubEnv("NEMOCLAW_DEBUG", value);
const { log } = await freshLogger();
expect(log.level).toBe("debug");
},
);

it("does not treat the framework DEBUG variable as a NemoClaw logging control", async () => {
vi.stubEnv("DEBUG", "*");
Expand Down Expand Up @@ -224,6 +221,33 @@ describe("Logger", () => {
expect(output()).toContain('"name": "Error"');
});

it("serializes nested AggregateError entries without exposing credentials", async () => {
const apiKey = `nvapi-${"a".repeat(40)}`;
const authorizationToken = "opaque-aggregate-authorization-token";
const urlToken = "opaque-aggregate-url-token";
const { log } = await freshLogger();
log.setDebug(true);
const sanitizerError = new Error(`sanitizer failed with ${apiKey}`);
sanitizerError.stack = `Error: sanitizer failed\nAuthorization: Bearer ${authorizationToken}`;
Object.assign(sanitizerError, {
endpoint: `https://example.test/path?access_token=${urlToken}`,
});
const cause = new AggregateError(
[sanitizerError, new Error("cleanup failed")],
"both failed",
);

log.debugObject("context", new Error("outer failure", { cause }));

expect(output()).toContain('"errors"');
expect(output()).toContain("sanitizer failed");
expect(output()).toContain("cleanup failed");
expect(output()).not.toContain(apiKey);
expect(output()).not.toContain(authorizationToken);
expect(output()).not.toContain(urlToken);
expect(output()).toContain("<REDACTED>");
});

it("does not let a synchronous stderr failure escape", async () => {
const { log } = await freshLogger();
stderrSpy.mockImplementation(() => {
Expand Down
3 changes: 3 additions & 0 deletions src/lib/cli/logger.ts
Original file line number Diff line number Diff line change
Expand Up @@ -65,6 +65,9 @@ function normalizeForSerialization(value: unknown, seen = new WeakSet<object>())
if (value.cause !== undefined) {
normalized.cause = normalizeForSerialization(value.cause, seen);
}
if (value instanceof AggregateError) {
normalized.errors = normalizeForSerialization(value.errors, seen);
}
for (const [key, entry] of Object.entries(value)) {
normalized[key] = normalizeForSerialization(entry, seen);
}
Expand Down
28 changes: 21 additions & 7 deletions src/lib/security/snapshot-sanitizer.ts
Original file line number Diff line number Diff line change
Expand Up @@ -5,12 +5,13 @@ import path from "node:path";
import { parse as parseYaml } from "yaml";

import {
applyDescriptorSnapshotActions,
applyDescriptorSnapshotActionsResult,
decodeDescriptorSnapshotContent,
inspectDescriptorSnapshotRoot,
type SnapshotSanitizationAction,
type SnapshotScannedFile,
scanDescriptorSnapshot,
scanDescriptorSnapshotResult,
snapshotSanitizerFailure,
} from "../../../nemoclaw/dist/shared/snapshot-sanitizer-boundary.cjs";

import {
Expand All @@ -23,6 +24,9 @@ import {
valueLooksLikeSecret,
} from "./credential-filter";

/** Re-exported so CLI callers identify the prerequisite failure without importing the plugin boundary module. (#8202) */
export { SnapshotSanitizerPrerequisiteError } from "../../../nemoclaw/dist/shared/snapshot-sanitizer-boundary.cjs";

const MAX_SANITIZATION_PASSES = 3;

const VENDORED_DEPENDENCY_DIRECTORY = "node_modules";
Expand Down Expand Up @@ -184,16 +188,26 @@ export function sanitizeSnapshotDirectory(rootPath: string): void {
throw new Error(`Failed to inspect snapshot artifacts safely: ${rootPath}`);
}

const scan = scanDescriptorSnapshot(root, CREDENTIAL_SENSITIVE_BASENAMES);
if (scan === null) {
throw new Error(`Failed to inspect snapshot artifacts safely: ${rootPath}`);
const scanResult = scanDescriptorSnapshotResult(root, CREDENTIAL_SENSITIVE_BASENAMES);
if (!scanResult.ok) {
throw snapshotSanitizerFailure(
scanResult.reason,
`Failed to inspect snapshot artifacts safely: ${rootPath}`,
root.canonicalPath,
);
}
const scan = scanResult.value;
const actions = scan.files
.map((file) => actionForScannedFile(file))
.filter((action): action is SnapshotSanitizationAction => action !== null);
if (actions.length === 0) return;
if (!applyDescriptorSnapshotActions(root, scan, actions)) {
throw new Error(`Failed to sanitize snapshot artifacts safely: ${rootPath}`);
const applyResult = applyDescriptorSnapshotActionsResult(root, scan, actions);
if (!applyResult.ok) {
throw snapshotSanitizerFailure(
applyResult.reason,
`Failed to sanitize snapshot artifacts safely: ${rootPath}`,
root.canonicalPath,
);
}
}
throw new Error(`Snapshot artifacts did not reach a stable sanitized state: ${rootPath}`);
Expand Down
Loading
Loading