Skip to content
Merged
Show file tree
Hide file tree
Changes from all commits
Commits
File filter

Filter by extension

Filter by extension

Conversations
Failed to load comments.
Loading
Jump to
Jump to file
Failed to load files.
Loading
Diff view
Diff view
4 changes: 4 additions & 0 deletions Dockerfile
Original file line number Diff line number Diff line change
Expand Up @@ -39,6 +39,10 @@ FROM ${BASE_IMAGE}
ARG OPENCLAW_VERSION=2026.5.27
ARG OPENCLAW_2026_5_27_INTEGRITY=sha512-2N93zhdAo88KAbHt6T7KvYXf4s7XIkYXBgv1npYpn7e1Y9FvrtgtpsA38my9rtFW+70uXEojRPX5/OqnuDqJPw==

# OpenShell blocks the link-local EC2 Instance Metadata Service. Keep AWS SDK
# credential chains from attempting an impossible metadata discovery path.
ENV AWS_EC2_METADATA_DISABLED=true

# OpenClaw 2026.5.27 loads some generated source through jiti. Disable its
# filesystem transform cache so source fragments that mention provider marker
# names do not persist under /tmp/jiti inside the sandbox.
Expand Down
4 changes: 4 additions & 0 deletions Dockerfile.base
Original file line number Diff line number Diff line change
Expand Up @@ -49,6 +49,10 @@

FROM node:22-trixie-slim@sha256:2d9f5c76c8f4dd36e8f253bee5d828a83a6c09f36188f0b0414325232e0b175d

# OpenShell blocks the link-local EC2 Instance Metadata Service. Keep AWS SDK
# credential chains from attempting an impossible metadata discovery path.
ENV AWS_EC2_METADATA_DISABLED=true

ENV DEBIAN_FRONTEND=noninteractive

RUN apt-get update && apt-get install -y --no-install-recommends \
Expand Down
35 changes: 35 additions & 0 deletions docs/reference/troubleshooting.mdx
Original file line number Diff line number Diff line change
Expand Up @@ -837,6 +837,41 @@ Instead:
2. If you build NemoClaw from source, bump the pinned `openclaw` version in `Dockerfile.base` and rebuild the sandbox base image.
3. Run `$$nemoclaw <name> rebuild` to recreate the sandbox with the updated image. The rebuild command automatically backs up workspace state before destroying the old sandbox and restores it afterward.

<AgentOnly variant="openclaw">

### AWS EC2 Instance-Role Credential Discovery Is Unavailable

This is expected in an OpenClaw sandbox.
NemoClaw forces `AWS_EC2_METADATA_DISABLED=true` because OpenShell blocks the link-local EC2 Instance Metadata Service endpoint.
Existing sandboxes must use a current NemoClaw image before they receive this environment invariant.

Upgrade NemoClaw and rebuild the sandbox:

```bash
$$nemoclaw update --yes
$$nemoclaw <name> rebuild --yes
```

Reconnect and verify the value:

```bash
$$nemoclaw <name> connect
printenv AWS_EC2_METADATA_DISABLED
```

Expected output:

```text
true
```

Only EC2 instance-role discovery is disabled.
Static access keys, bearer tokens, shared profiles, SSO and process credentials, web identity, and ECS container credentials remain eligible.
NemoClaw's host-local Amazon Bedrock adapter is outside the sandbox credential-discovery boundary and remains available.
Do not add `169.254.169.254` to a network policy or override the variable.

</AgentOnly>

### Inference requests time out

Verify that the inference provider endpoint is reachable from the host.
Expand Down
43 changes: 43 additions & 0 deletions docs/security/best-practices.mdx
Original file line number Diff line number Diff line change
Expand Up @@ -107,6 +107,49 @@ The sandbox blocks all outbound connections unless you explicitly list the endpo
| Risk if relaxed | Each allowed endpoint is a potential data exfiltration path. The agent can send workspace content, credentials, or conversation history to any reachable host. |
| Recommendation | Add only endpoints the agent needs for its task. Prefer operator approval for one-off requests over permanently widening the baseline. |

### Credential Discovery Boundaries

Credential probes are active network behavior, not harmless fallback checks.
NemoClaw allows a credential source only in the process that is intended to hold those credentials and can reach that source.

| Execution path | Rule |
|---|---|
| Host-brokered provider | Discover and use host credentials only in the host adapter. The sandbox receives a narrow `inference.local` interface, not the host credentials. |
| Sandbox-direct provider | Keep only credential sources that are deliberately provisioned and reachable inside the sandbox. |
| Ambient metadata provider | Disable SDK discovery when the metadata endpoint cannot exist across the sandbox boundary. Do not make an internal or link-local endpoint reachable merely to satisfy an automatic probe. |
| New agent or provider | Declare the enabled credential sources and add agent-specific tests proving that unavailable sources are not probed. |

Network policy remains a second enforcement layer, not a substitute for disabling an impossible credential provider.
Adding support for a new metadata-backed source requires a separately designed broker and explicit credential-boundary review.

<AgentOnly variant="openclaw">

#### OpenClaw EC2 Instance Metadata Credential Discovery

NemoClaw forces `AWS_EC2_METADATA_DISABLED` to `true` for every OpenClaw process because OpenShell blocks the link-local EC2 Instance Metadata Service endpoint.
This disables only EC2 instance-role credential discovery inside the sandbox.
Static access keys, bearer tokens, shared profiles, SSO and process credentials, web identity, and ECS container credentials remain eligible.
NemoClaw's host-local Amazon Bedrock adapter runs outside this sandbox credential-discovery boundary and remains unaffected.

| Aspect | Detail |
|---|---|
| Default | OpenClaw images, gateway processes, child processes, cron jobs, and connected shells receive `AWS_EC2_METADATA_DISABLED=true`. |
| What you can change | This is a NemoClaw sandbox invariant, not a user-facing setting. |
| Risk if relaxed | Probing `169.254.169.254` cannot succeed through the OpenShell network boundary, and approving the link-local address would weaken SSRF protection. |
| Recommendation | Do not add `169.254.169.254` to a network policy or override this environment variable. Removing the defense requires a separately designed, brokered IMDS credential flow. |

</AgentOnly>

<AgentOnly variant="hermes">

#### Hermes Managed Inference Credential Discovery

NemoClaw configures Hermes with a custom provider through `inference.local`, and the shipped Hermes image does not install the optional native Bedrock dependency.
AWS credential discovery for supported Bedrock inference therefore remains in NemoClaw's host-local adapter, outside the sandbox.
If direct Hermes Bedrock support is added later, it must apply the same unavailable-metadata rule and include Hermes-specific credential-probe acceptance tests before it is enabled.

</AgentOnly>

### Binary-Scoped Endpoint Rules

Each network policy entry uses the `binaries` field to restrict which executables can reach the endpoint.
Expand Down
6 changes: 6 additions & 0 deletions scripts/nemoclaw-start.sh
Original file line number Diff line number Diff line change
Expand Up @@ -231,6 +231,11 @@ case "${1:-}" in
esac
NEMOCLAW_CMD=("$@")

# OpenShell blocks the link-local EC2 Instance Metadata Service. Force this
# after self-wrapper normalization so injected or inherited values cannot make
# OpenClaw processes probe an impossible credential source.
export AWS_EC2_METADATA_DISABLED=true

# Marker file the Docker HEALTHCHECK reads to decide whether an in-container
# gateway liveness check is meaningful. Its presence means this container has
# entered the OpenClaw gateway launch path (standalone deployments and the #3975
Expand Down Expand Up @@ -2886,6 +2891,7 @@ export NO_PROXY="$_NO_PROXY_VAL"
export http_proxy="$_PROXY_URL"
export https_proxy="$_PROXY_URL"
export no_proxy="$_NO_PROXY_VAL"
export AWS_EC2_METADATA_DISABLED="true"
export JITI_FS_CACHE="false"
PROXYEOF
local _openclaw_env_name _openclaw_env_value _escaped_openclaw_env_value
Expand Down
3 changes: 2 additions & 1 deletion test/hermes-start.test.ts
Original file line number Diff line number Diff line change
Expand Up @@ -760,7 +760,7 @@ function runRuntimeShellEnvBootstrap() {
const result = spawnSync("bash", [scriptPath], {
encoding: "utf-8",
timeout: 5000,
env: process.env,
env: { ...process.env, AWS_EC2_METADATA_DISABLED: "false" },
});
const envFileContent = fs.existsSync(envFile) ? fs.readFileSync(envFile, "utf-8") : "";
const envFileMode = fs.existsSync(envFile)
Expand Down Expand Up @@ -805,6 +805,7 @@ describe("agents/hermes/start.sh runtime shell env", () => {
expect(run.envFileMode).toBe("444");
expect(run.envFileContent).toContain(`export HERMES_HOME="${run.hermesHome}"`);
expect(run.envFileContent).toContain('export HERMES_TUI_DIR="/opt/hermes/ui-tui"');
expect(run.envFileContent).not.toContain("AWS_EC2_METADATA_DISABLED");
expect(run.envFileContent).not.toContain('HERMES_TUI_DIR="${HERMES_TUI_DIR:-');
expect(run.envFileContent).toContain(`export SSL_CERT_FILE=${escapedCaFile}`);
expect(run.envFileContent).toContain("# nemoclaw-configure-guard begin");
Expand Down
45 changes: 44 additions & 1 deletion test/runner.test.ts
Original file line number Diff line number Diff line change
Expand Up @@ -898,7 +898,7 @@ describe("regression guards", () => {
});
});

describe("OpenClaw runtime cache hardening", () => {
describe("OpenClaw runtime hardening", () => {
const repoRoot = path.join(import.meta.dirname, "..");

it("disables jiti filesystem cache in base, runtime, and connect shells", () => {
Expand All @@ -913,6 +913,49 @@ describe("regression guards", () => {
expect(runtimeSrc).toContain("ENV JITI_FS_CACHE=false");
expect(startSrc).toContain('export JITI_FS_CACHE="false"');
});

it("disables EC2 metadata credential discovery across image, startup, and shell boundaries", () => {
const baseSrc = fs.readFileSync(path.join(repoRoot, "Dockerfile.base"), "utf-8");
const runtimeSrc = fs.readFileSync(path.join(repoRoot, "Dockerfile"), "utf-8");
const startSrc = fs.readFileSync(
path.join(repoRoot, "scripts", "nemoclaw-start.sh"),
"utf-8",
);
const hermesBaseSrc = fs.readFileSync(
path.join(repoRoot, "agents", "hermes", "Dockerfile.base"),
"utf-8",
);
const hermesRuntimeSrc = fs.readFileSync(
path.join(repoRoot, "agents", "hermes", "Dockerfile"),
"utf-8",
);
const hermesStartSrc = fs.readFileSync(
path.join(repoRoot, "agents", "hermes", "start.sh"),
"utf-8",
);

expect(baseSrc).toContain("ENV AWS_EC2_METADATA_DISABLED=true");
expect(runtimeSrc).toContain("ENV AWS_EC2_METADATA_DISABLED=true");
const runtimeStageStart = runtimeSrc.indexOf("# Stage 3: Runtime image");
expect(runtimeStageStart).toBeGreaterThan(-1);
for (const [source, stageStart] of [
[baseSrc, 0],
[runtimeSrc, runtimeStageStart],
] as const) {
const fromIndex = source.indexOf("\nFROM ", stageStart);
expect(fromIndex).toBeGreaterThan(-1);
const firstRunIndex = source.indexOf("\nRUN ", fromIndex);
expect(firstRunIndex).toBeGreaterThan(-1);
const metadataEnvIndex = source.indexOf("ENV AWS_EC2_METADATA_DISABLED=true", fromIndex);
expect(metadataEnvIndex).toBeGreaterThan(fromIndex);
expect(metadataEnvIndex).toBeLessThan(firstRunIndex);
}
Comment thread
coderabbitai[bot] marked this conversation as resolved.
expect(startSrc).toContain("export AWS_EC2_METADATA_DISABLED=true");
expect(startSrc).toContain('export AWS_EC2_METADATA_DISABLED="true"');
expect(hermesBaseSrc).not.toContain("AWS_EC2_METADATA_DISABLED");
expect(hermesRuntimeSrc).not.toContain("AWS_EC2_METADATA_DISABLED");
expect(hermesStartSrc).not.toContain("AWS_EC2_METADATA_DISABLED");
});
});

describe("sandbox ships tmux for the bundled tmux-session flow (#4513)", () => {
Expand Down
65 changes: 65 additions & 0 deletions test/service-env.test.ts
Original file line number Diff line number Diff line change
Expand Up @@ -39,6 +39,18 @@ function extractRuntimeShellEnvSnippet() {
return `${src.slice(start, end).trimEnd()}\nwrite_runtime_shell_env`;
}

function extractOpenClawBootstrapEnvSnippet() {
const src = readFileSync(NEMOCLAW_START_SCRIPT, "utf-8");
const start = src.indexOf("# Normalize the sandbox-create bootstrap wrapper");
const end = src.indexOf("# Marker file the Docker HEALTHCHECK reads", start);
const extractionFailure =
"Failed to extract OpenClaw bootstrap environment normalization from " +
"scripts/nemoclaw-start.sh";
expect(start, extractionFailure).not.toBe(-1);
expect(end, extractionFailure).toBeGreaterThan(start);
return src.slice(start, end).trimEnd();
}

function extractRuntimeShellEnvShimSnippet() {
const src = readFileSync(NEMOCLAW_START_SCRIPT, "utf-8");
const start = src.indexOf("ensure_runtime_shell_env_shim() {");
Expand All @@ -53,6 +65,34 @@ function extractRuntimeShellEnvShimSnippet() {
}

describe("service environment", () => {
describe("OpenClaw EC2 metadata discovery", () => {
it("overrides ambient and sandbox-create wrapper false values before startup", () => {
const tmpFile = join(tmpdir(), `nemoclaw-imds-bootstrap-${process.pid}.sh`);
try {
const wrapper = [
"#!/usr/bin/env bash",
"set -euo pipefail",
"set -- env AWS_EC2_METADATA_DISABLED=false nemoclaw-start openclaw agent",
extractOpenClawBootstrapEnvSnippet(),
'printf "%s\\n" "$AWS_EC2_METADATA_DISABLED"',
].join("\n");
writeFileSync(tmpFile, wrapper, { mode: 0o700 });

const out = execFileSync("bash", [tmpFile], {
encoding: "utf-8",
env: { ...process.env, AWS_EC2_METADATA_DISABLED: "false" },
});
expect(out.trim()).toBe("true");
} finally {
try {
unlinkSync(tmpFile);
} catch {
/* ignore */
}
}
});
});

describe("start-services behavior", () => {
const scriptPath = join(import.meta.dirname, "../scripts/start-services.sh");

Expand Down Expand Up @@ -577,6 +617,7 @@ describe("service environment", () => {
expect(envFile).toContain("export NO_PROXY=");
expect(envFile).not.toContain("inference.local");
expect(envFile).toContain("10.200.0.1");
expect(envFile).toContain('export AWS_EC2_METADATA_DISABLED="true"');
expect(envFile).toContain("export OPENCLAW_GATEWAY_TOKEN='test-token-123'");
expect(envFile).toContain("nemoclaw-configure-guard begin");
expect(envFile).toContain('command openclaw "$@"');
Expand Down Expand Up @@ -611,6 +652,18 @@ describe("service environment", () => {
}).trim();
}
expect(perms).toBe("444");

const connectedValue = execFileSync(
"bash",
[
"--noprofile",
"--norc",
"-c",
`export AWS_EC2_METADATA_DISABLED=false; source ${JSON.stringify(join(fakeDataDir, "proxy-env.sh"))}; printf "%s" "$AWS_EC2_METADATA_DISABLED"`,
],
{ encoding: "utf-8" },
);
expect(connectedValue).toBe("true");
} finally {
try {
unlinkSync(tmpFile);
Expand Down Expand Up @@ -974,11 +1027,15 @@ describe("service environment", () => {
const fakeDataDir = join(tmpdir(), `nemoclaw-idempotent-test-${process.pid}`);
execFileSync("mkdir", ["-p", fakeDataDir]);
const tmpFile = join(tmpdir(), `nemoclaw-idempotent-write-test-${process.pid}.sh`);
const chownLog = join(fakeDataDir, "chown.log");
try {
const persistBlock = extractRuntimeShellEnvSnippet();
const toolRedirects = extractToolRedirects();
const wrapper = [
"#!/usr/bin/env bash",
'id() { if [ "${1:-}" = "-u" ]; then printf "0\\n"; else command id "$@"; fi; }',
'chown() { printf "%s\\n" "$*" >> "$CHOWN_LOG"; }',
`export CHOWN_LOG=${JSON.stringify(chownLog)}`,
sandboxInitSource,
toolRedirects,
'PROXY_HOST="10.200.0.1"',
Expand All @@ -1000,6 +1057,14 @@ describe("service environment", () => {
// HTTP_PROXY line — no duplication from repeated runs.
const httpProxyCount = (envFile.match(/export HTTP_PROXY=/g) || []).length;
expect(httpProxyCount).toBe(1);
const metadataCount = (envFile.match(/export AWS_EC2_METADATA_DISABLED=/g) || []).length;
expect(metadataCount).toBe(1);
expect((lstatSync(join(fakeDataDir, "proxy-env.sh")).mode & 0o777).toString(8)).toBe("444");
const chownCalls = readFileSync(chownLog, "utf-8").trim().split("\n");
expect(chownCalls).toHaveLength(3);
expect(chownCalls.every((call) => /^root:root .*\/\.proxy-env\.sh\.tmp\./.test(call))).toBe(
true,
);
} finally {
try {
unlinkSync(tmpFile);
Expand Down
Loading