diff --git a/Dockerfile b/Dockerfile index 535f0744081..7c03a0431f4 100644 --- a/Dockerfile +++ b/Dockerfile @@ -39,6 +39,10 @@ FROM ${BASE_IMAGE} ARG OPENCLAW_VERSION=2026.5.27 ARG OPENCLAW_2026_5_27_INTEGRITY=sha512-2N93zhdAo88KAbHt6T7KvYXf4s7XIkYXBgv1npYpn7e1Y9FvrtgtpsA38my9rtFW+70uXEojRPX5/OqnuDqJPw== +# OpenShell blocks the link-local EC2 Instance Metadata Service. Keep AWS SDK +# credential chains from attempting an impossible metadata discovery path. +ENV AWS_EC2_METADATA_DISABLED=true + # OpenClaw 2026.5.27 loads some generated source through jiti. Disable its # filesystem transform cache so source fragments that mention provider marker # names do not persist under /tmp/jiti inside the sandbox. diff --git a/Dockerfile.base b/Dockerfile.base index 3840a6a320b..f3176f5187d 100644 --- a/Dockerfile.base +++ b/Dockerfile.base @@ -49,6 +49,10 @@ FROM node:22-trixie-slim@sha256:2d9f5c76c8f4dd36e8f253bee5d828a83a6c09f36188f0b0414325232e0b175d +# OpenShell blocks the link-local EC2 Instance Metadata Service. Keep AWS SDK +# credential chains from attempting an impossible metadata discovery path. +ENV AWS_EC2_METADATA_DISABLED=true + ENV DEBIAN_FRONTEND=noninteractive RUN apt-get update && apt-get install -y --no-install-recommends \ diff --git a/docs/reference/troubleshooting.mdx b/docs/reference/troubleshooting.mdx index 401f9dcbdf9..a3651d3bc96 100644 --- a/docs/reference/troubleshooting.mdx +++ b/docs/reference/troubleshooting.mdx @@ -837,6 +837,41 @@ Instead: 2. If you build NemoClaw from source, bump the pinned `openclaw` version in `Dockerfile.base` and rebuild the sandbox base image. 3. Run `$$nemoclaw rebuild` to recreate the sandbox with the updated image. The rebuild command automatically backs up workspace state before destroying the old sandbox and restores it afterward. + + +### AWS EC2 Instance-Role Credential Discovery Is Unavailable + +This is expected in an OpenClaw sandbox. +NemoClaw forces `AWS_EC2_METADATA_DISABLED=true` because OpenShell blocks the link-local EC2 Instance Metadata Service endpoint. +Existing sandboxes must use a current NemoClaw image before they receive this environment invariant. + +Upgrade NemoClaw and rebuild the sandbox: + +```bash +$$nemoclaw update --yes +$$nemoclaw rebuild --yes +``` + +Reconnect and verify the value: + +```bash +$$nemoclaw connect +printenv AWS_EC2_METADATA_DISABLED +``` + +Expected output: + +```text +true +``` + +Only EC2 instance-role discovery is disabled. +Static access keys, bearer tokens, shared profiles, SSO and process credentials, web identity, and ECS container credentials remain eligible. +NemoClaw's host-local Amazon Bedrock adapter is outside the sandbox credential-discovery boundary and remains available. +Do not add `169.254.169.254` to a network policy or override the variable. + + + ### Inference requests time out Verify that the inference provider endpoint is reachable from the host. diff --git a/docs/security/best-practices.mdx b/docs/security/best-practices.mdx index c050b05d28a..dd303e6bdd1 100644 --- a/docs/security/best-practices.mdx +++ b/docs/security/best-practices.mdx @@ -107,6 +107,49 @@ The sandbox blocks all outbound connections unless you explicitly list the endpo | Risk if relaxed | Each allowed endpoint is a potential data exfiltration path. The agent can send workspace content, credentials, or conversation history to any reachable host. | | Recommendation | Add only endpoints the agent needs for its task. Prefer operator approval for one-off requests over permanently widening the baseline. | +### Credential Discovery Boundaries + +Credential probes are active network behavior, not harmless fallback checks. +NemoClaw allows a credential source only in the process that is intended to hold those credentials and can reach that source. + +| Execution path | Rule | +|---|---| +| Host-brokered provider | Discover and use host credentials only in the host adapter. The sandbox receives a narrow `inference.local` interface, not the host credentials. | +| Sandbox-direct provider | Keep only credential sources that are deliberately provisioned and reachable inside the sandbox. | +| Ambient metadata provider | Disable SDK discovery when the metadata endpoint cannot exist across the sandbox boundary. Do not make an internal or link-local endpoint reachable merely to satisfy an automatic probe. | +| New agent or provider | Declare the enabled credential sources and add agent-specific tests proving that unavailable sources are not probed. | + +Network policy remains a second enforcement layer, not a substitute for disabling an impossible credential provider. +Adding support for a new metadata-backed source requires a separately designed broker and explicit credential-boundary review. + + + +#### OpenClaw EC2 Instance Metadata Credential Discovery + +NemoClaw forces `AWS_EC2_METADATA_DISABLED` to `true` for every OpenClaw process because OpenShell blocks the link-local EC2 Instance Metadata Service endpoint. +This disables only EC2 instance-role credential discovery inside the sandbox. +Static access keys, bearer tokens, shared profiles, SSO and process credentials, web identity, and ECS container credentials remain eligible. +NemoClaw's host-local Amazon Bedrock adapter runs outside this sandbox credential-discovery boundary and remains unaffected. + +| Aspect | Detail | +|---|---| +| Default | OpenClaw images, gateway processes, child processes, cron jobs, and connected shells receive `AWS_EC2_METADATA_DISABLED=true`. | +| What you can change | This is a NemoClaw sandbox invariant, not a user-facing setting. | +| Risk if relaxed | Probing `169.254.169.254` cannot succeed through the OpenShell network boundary, and approving the link-local address would weaken SSRF protection. | +| Recommendation | Do not add `169.254.169.254` to a network policy or override this environment variable. Removing the defense requires a separately designed, brokered IMDS credential flow. | + + + + + +#### Hermes Managed Inference Credential Discovery + +NemoClaw configures Hermes with a custom provider through `inference.local`, and the shipped Hermes image does not install the optional native Bedrock dependency. +AWS credential discovery for supported Bedrock inference therefore remains in NemoClaw's host-local adapter, outside the sandbox. +If direct Hermes Bedrock support is added later, it must apply the same unavailable-metadata rule and include Hermes-specific credential-probe acceptance tests before it is enabled. + + + ### Binary-Scoped Endpoint Rules Each network policy entry uses the `binaries` field to restrict which executables can reach the endpoint. diff --git a/scripts/nemoclaw-start.sh b/scripts/nemoclaw-start.sh index d5cb5687799..52657cb9d74 100755 --- a/scripts/nemoclaw-start.sh +++ b/scripts/nemoclaw-start.sh @@ -231,6 +231,11 @@ case "${1:-}" in esac NEMOCLAW_CMD=("$@") +# OpenShell blocks the link-local EC2 Instance Metadata Service. Force this +# after self-wrapper normalization so injected or inherited values cannot make +# OpenClaw processes probe an impossible credential source. +export AWS_EC2_METADATA_DISABLED=true + # Marker file the Docker HEALTHCHECK reads to decide whether an in-container # gateway liveness check is meaningful. Its presence means this container has # entered the OpenClaw gateway launch path (standalone deployments and the #3975 @@ -2886,6 +2891,7 @@ export NO_PROXY="$_NO_PROXY_VAL" export http_proxy="$_PROXY_URL" export https_proxy="$_PROXY_URL" export no_proxy="$_NO_PROXY_VAL" +export AWS_EC2_METADATA_DISABLED="true" export JITI_FS_CACHE="false" PROXYEOF local _openclaw_env_name _openclaw_env_value _escaped_openclaw_env_value diff --git a/test/hermes-start.test.ts b/test/hermes-start.test.ts index f9513fe079b..9c0a3579871 100644 --- a/test/hermes-start.test.ts +++ b/test/hermes-start.test.ts @@ -760,7 +760,7 @@ function runRuntimeShellEnvBootstrap() { const result = spawnSync("bash", [scriptPath], { encoding: "utf-8", timeout: 5000, - env: process.env, + env: { ...process.env, AWS_EC2_METADATA_DISABLED: "false" }, }); const envFileContent = fs.existsSync(envFile) ? fs.readFileSync(envFile, "utf-8") : ""; const envFileMode = fs.existsSync(envFile) @@ -805,6 +805,7 @@ describe("agents/hermes/start.sh runtime shell env", () => { expect(run.envFileMode).toBe("444"); expect(run.envFileContent).toContain(`export HERMES_HOME="${run.hermesHome}"`); expect(run.envFileContent).toContain('export HERMES_TUI_DIR="/opt/hermes/ui-tui"'); + expect(run.envFileContent).not.toContain("AWS_EC2_METADATA_DISABLED"); expect(run.envFileContent).not.toContain('HERMES_TUI_DIR="${HERMES_TUI_DIR:-'); expect(run.envFileContent).toContain(`export SSL_CERT_FILE=${escapedCaFile}`); expect(run.envFileContent).toContain("# nemoclaw-configure-guard begin"); diff --git a/test/runner.test.ts b/test/runner.test.ts index c72c36d20d0..ece5f7bc0c0 100644 --- a/test/runner.test.ts +++ b/test/runner.test.ts @@ -898,7 +898,7 @@ describe("regression guards", () => { }); }); - describe("OpenClaw runtime cache hardening", () => { + describe("OpenClaw runtime hardening", () => { const repoRoot = path.join(import.meta.dirname, ".."); it("disables jiti filesystem cache in base, runtime, and connect shells", () => { @@ -913,6 +913,49 @@ describe("regression guards", () => { expect(runtimeSrc).toContain("ENV JITI_FS_CACHE=false"); expect(startSrc).toContain('export JITI_FS_CACHE="false"'); }); + + it("disables EC2 metadata credential discovery across image, startup, and shell boundaries", () => { + const baseSrc = fs.readFileSync(path.join(repoRoot, "Dockerfile.base"), "utf-8"); + const runtimeSrc = fs.readFileSync(path.join(repoRoot, "Dockerfile"), "utf-8"); + const startSrc = fs.readFileSync( + path.join(repoRoot, "scripts", "nemoclaw-start.sh"), + "utf-8", + ); + const hermesBaseSrc = fs.readFileSync( + path.join(repoRoot, "agents", "hermes", "Dockerfile.base"), + "utf-8", + ); + const hermesRuntimeSrc = fs.readFileSync( + path.join(repoRoot, "agents", "hermes", "Dockerfile"), + "utf-8", + ); + const hermesStartSrc = fs.readFileSync( + path.join(repoRoot, "agents", "hermes", "start.sh"), + "utf-8", + ); + + expect(baseSrc).toContain("ENV AWS_EC2_METADATA_DISABLED=true"); + expect(runtimeSrc).toContain("ENV AWS_EC2_METADATA_DISABLED=true"); + const runtimeStageStart = runtimeSrc.indexOf("# Stage 3: Runtime image"); + expect(runtimeStageStart).toBeGreaterThan(-1); + for (const [source, stageStart] of [ + [baseSrc, 0], + [runtimeSrc, runtimeStageStart], + ] as const) { + const fromIndex = source.indexOf("\nFROM ", stageStart); + expect(fromIndex).toBeGreaterThan(-1); + const firstRunIndex = source.indexOf("\nRUN ", fromIndex); + expect(firstRunIndex).toBeGreaterThan(-1); + const metadataEnvIndex = source.indexOf("ENV AWS_EC2_METADATA_DISABLED=true", fromIndex); + expect(metadataEnvIndex).toBeGreaterThan(fromIndex); + expect(metadataEnvIndex).toBeLessThan(firstRunIndex); + } + expect(startSrc).toContain("export AWS_EC2_METADATA_DISABLED=true"); + expect(startSrc).toContain('export AWS_EC2_METADATA_DISABLED="true"'); + expect(hermesBaseSrc).not.toContain("AWS_EC2_METADATA_DISABLED"); + expect(hermesRuntimeSrc).not.toContain("AWS_EC2_METADATA_DISABLED"); + expect(hermesStartSrc).not.toContain("AWS_EC2_METADATA_DISABLED"); + }); }); describe("sandbox ships tmux for the bundled tmux-session flow (#4513)", () => { diff --git a/test/service-env.test.ts b/test/service-env.test.ts index ab73f6e57a7..dfc06914563 100644 --- a/test/service-env.test.ts +++ b/test/service-env.test.ts @@ -39,6 +39,18 @@ function extractRuntimeShellEnvSnippet() { return `${src.slice(start, end).trimEnd()}\nwrite_runtime_shell_env`; } +function extractOpenClawBootstrapEnvSnippet() { + const src = readFileSync(NEMOCLAW_START_SCRIPT, "utf-8"); + const start = src.indexOf("# Normalize the sandbox-create bootstrap wrapper"); + const end = src.indexOf("# Marker file the Docker HEALTHCHECK reads", start); + const extractionFailure = + "Failed to extract OpenClaw bootstrap environment normalization from " + + "scripts/nemoclaw-start.sh"; + expect(start, extractionFailure).not.toBe(-1); + expect(end, extractionFailure).toBeGreaterThan(start); + return src.slice(start, end).trimEnd(); +} + function extractRuntimeShellEnvShimSnippet() { const src = readFileSync(NEMOCLAW_START_SCRIPT, "utf-8"); const start = src.indexOf("ensure_runtime_shell_env_shim() {"); @@ -53,6 +65,34 @@ function extractRuntimeShellEnvShimSnippet() { } describe("service environment", () => { + describe("OpenClaw EC2 metadata discovery", () => { + it("overrides ambient and sandbox-create wrapper false values before startup", () => { + const tmpFile = join(tmpdir(), `nemoclaw-imds-bootstrap-${process.pid}.sh`); + try { + const wrapper = [ + "#!/usr/bin/env bash", + "set -euo pipefail", + "set -- env AWS_EC2_METADATA_DISABLED=false nemoclaw-start openclaw agent", + extractOpenClawBootstrapEnvSnippet(), + 'printf "%s\\n" "$AWS_EC2_METADATA_DISABLED"', + ].join("\n"); + writeFileSync(tmpFile, wrapper, { mode: 0o700 }); + + const out = execFileSync("bash", [tmpFile], { + encoding: "utf-8", + env: { ...process.env, AWS_EC2_METADATA_DISABLED: "false" }, + }); + expect(out.trim()).toBe("true"); + } finally { + try { + unlinkSync(tmpFile); + } catch { + /* ignore */ + } + } + }); + }); + describe("start-services behavior", () => { const scriptPath = join(import.meta.dirname, "../scripts/start-services.sh"); @@ -577,6 +617,7 @@ describe("service environment", () => { expect(envFile).toContain("export NO_PROXY="); expect(envFile).not.toContain("inference.local"); expect(envFile).toContain("10.200.0.1"); + expect(envFile).toContain('export AWS_EC2_METADATA_DISABLED="true"'); expect(envFile).toContain("export OPENCLAW_GATEWAY_TOKEN='test-token-123'"); expect(envFile).toContain("nemoclaw-configure-guard begin"); expect(envFile).toContain('command openclaw "$@"'); @@ -611,6 +652,18 @@ describe("service environment", () => { }).trim(); } expect(perms).toBe("444"); + + const connectedValue = execFileSync( + "bash", + [ + "--noprofile", + "--norc", + "-c", + `export AWS_EC2_METADATA_DISABLED=false; source ${JSON.stringify(join(fakeDataDir, "proxy-env.sh"))}; printf "%s" "$AWS_EC2_METADATA_DISABLED"`, + ], + { encoding: "utf-8" }, + ); + expect(connectedValue).toBe("true"); } finally { try { unlinkSync(tmpFile); @@ -974,11 +1027,15 @@ describe("service environment", () => { const fakeDataDir = join(tmpdir(), `nemoclaw-idempotent-test-${process.pid}`); execFileSync("mkdir", ["-p", fakeDataDir]); const tmpFile = join(tmpdir(), `nemoclaw-idempotent-write-test-${process.pid}.sh`); + const chownLog = join(fakeDataDir, "chown.log"); try { const persistBlock = extractRuntimeShellEnvSnippet(); const toolRedirects = extractToolRedirects(); const wrapper = [ "#!/usr/bin/env bash", + 'id() { if [ "${1:-}" = "-u" ]; then printf "0\\n"; else command id "$@"; fi; }', + 'chown() { printf "%s\\n" "$*" >> "$CHOWN_LOG"; }', + `export CHOWN_LOG=${JSON.stringify(chownLog)}`, sandboxInitSource, toolRedirects, 'PROXY_HOST="10.200.0.1"', @@ -1000,6 +1057,14 @@ describe("service environment", () => { // HTTP_PROXY line — no duplication from repeated runs. const httpProxyCount = (envFile.match(/export HTTP_PROXY=/g) || []).length; expect(httpProxyCount).toBe(1); + const metadataCount = (envFile.match(/export AWS_EC2_METADATA_DISABLED=/g) || []).length; + expect(metadataCount).toBe(1); + expect((lstatSync(join(fakeDataDir, "proxy-env.sh")).mode & 0o777).toString(8)).toBe("444"); + const chownCalls = readFileSync(chownLog, "utf-8").trim().split("\n"); + expect(chownCalls).toHaveLength(3); + expect(chownCalls.every((call) => /^root:root .*\/\.proxy-env\.sh\.tmp\./.test(call))).toBe( + true, + ); } finally { try { unlinkSync(tmpFile);