Skip to content
Merged
Show file tree
Hide file tree
Changes from all commits
Commits
File filter

Filter by extension

Filter by extension

Conversations
Failed to load comments.
Loading
Jump to
Jump to file
Failed to load files.
Loading
Diff view
Diff view
9 changes: 2 additions & 7 deletions agents/hermes/start.sh
Original file line number Diff line number Diff line change
Expand Up @@ -29,13 +29,8 @@ fi
# shellcheck source=scripts/lib/sandbox-init.sh
source "$_SANDBOX_INIT"

# Harden: limit process count to prevent fork bombs
if ! ulimit -Su 512 2>/dev/null; then
echo "[SECURITY] Could not set soft nproc limit (container runtime may restrict ulimit)" >&2
fi
if ! ulimit -Hu 512 2>/dev/null; then
echo "[SECURITY] Could not set hard nproc limit (container runtime may restrict ulimit)" >&2
fi
# Harden RLIMITs (nproc #809 + nofile #4527) as root PID 1, before any step-down.
harden_resource_limits

# SECURITY: Lock down PATH
export PATH="/usr/local/sbin:/usr/local/bin:/usr/sbin:/usr/bin:/sbin:/bin"
Expand Down
22 changes: 22 additions & 0 deletions docs/deployment/sandbox-hardening.mdx
Original file line number Diff line number Diff line change
Expand Up @@ -31,6 +31,23 @@ The startup script (`nemoclaw-start.sh`) applies the same limit.

Adjust the value with the `--ulimit nproc=512:512` flag if you launch with `docker run` directly.

## Open File Descriptor Limits

The same ENTRYPOINT also sets `ulimit -n 65536` to cap the number of open file
descriptors a sandbox user can hold. Without this cap the sandbox inherits the
Docker daemon default (`nofile` ~1048576), which can exceed the host runtime
limit and lets a runaway process exhaust file descriptors. The startup script
(`nemoclaw-start.sh`) applies the same limit.

Adjust the value via the `--ulimit nofile=65536:65536` flag if launching with
`docker run` directly.

Like the process limit, this is applied to the PID 1 entrypoint process tree
(gateway + agent). `openshell sandbox connect` shells are spawned outside that
tree and still inherit the runtime default (tracked upstream in
NVIDIA/OpenShell#1452), so enforce both limits at the container runtime when
that residual matters to you.

## Dropping Linux Capabilities

The NemoClaw entrypoint drops dangerous capabilities from the process bounding set before it starts agent services.
Expand All @@ -51,6 +68,7 @@ when you launch the image directly:
docker run --rm \
--cap-drop=ALL \
--ulimit nproc=512:512 \
--ulimit nofile=65536:65536 \
nemoclaw-sandbox
```

Expand All @@ -68,6 +86,9 @@ services:
nproc:
soft: 512
hard: 512
nofile:
soft: 65536
hard: 65536
security_opt:
- no-new-privileges:true
read_only: true
Expand Down Expand Up @@ -132,4 +153,5 @@ The `test/e2e/e2e-cloud-experimental/checks/04-landlock-readonly.sh` script vali
- [#807](https://github.com/NVIDIA/NemoClaw/issues/807): gcc in sandbox image
- [#808](https://github.com/NVIDIA/NemoClaw/issues/808): netcat in sandbox image
- [#809](https://github.com/NVIDIA/NemoClaw/issues/809): No process limit
- [#4527](https://github.com/NVIDIA/NemoClaw/issues/4527): Cap open file descriptors (nofile)
- [#797](https://github.com/NVIDIA/NemoClaw/issues/797): Drop Linux capabilities
15 changes: 15 additions & 0 deletions docs/security/best-practices.mdx
Original file line number Diff line number Diff line change
Expand Up @@ -346,6 +346,21 @@ This behavior is best effort: if the container runtime restricts `ulimit` modifi
| Risk if relaxed | Removing or raising the limit makes the sandbox vulnerable to fork-bomb attacks, where a runaway process spawns children until the host runs out of resources. If the entrypoint cannot set the limit (logs `[SECURITY] Could not set soft/hard nproc limit`), the container runs without process limits. |
| Recommendation | Keep the default at 512. If the agent runs workloads that spawn many child processes (such as parallel test runners), increase to 1024 and monitor host resource usage. If the entrypoint logs a warning about ulimit restrictions, set the limit through the container runtime instead. |

### Open File Descriptor Limit

An open file descriptor limit caps the number of files, sockets, and pipes the
sandbox user can hold open at once. The entrypoint sets both soft and hard
limits using `ulimit -n 65536`. This is best-effort: if the container runtime
restricts `ulimit` modification, the entrypoint logs a security warning and
continues without the limit.

| Aspect | Detail |
|---|---|
| Default | 65536 open files, soft and hard (`ulimit -n 65536`), best-effort. |
| What you can change | Increase or decrease the limit with `--ulimit nofile=N:N` in `docker run` or the `ulimits` section in Compose. The runtime-level ulimit takes precedence over the entrypoint's setting. |
| Risk if relaxed | Without this cap the sandbox inherits the Docker daemon default (`nofile` ~1048576). A runaway or hostile process can then open file descriptors until it exhausts them — a denial-of-service that can starve the gateway, the agent, or the host of file handles. If the entrypoint cannot set the limit (logs `[SECURITY] Could not set soft/hard nofile limit`), the container runs without a file-descriptor cap. Ref [#4527](https://github.com/NVIDIA/NemoClaw/issues/4527). |
| Recommendation | Keep the default at 65536. If the agent legitimately keeps many connections or files open, raise it deliberately and monitor host file-descriptor usage. If the entrypoint logs a warning about ulimit restrictions, set the limit through the container runtime instead. |

### Non-Root User

The sandbox runs agent processes as a dedicated `sandbox` user and group.
Expand Down
28 changes: 28 additions & 0 deletions scripts/lib/sandbox-init.sh
Original file line number Diff line number Diff line change
Expand Up @@ -220,6 +220,34 @@ lock_config_after_write() {
done
}

# ── Resource limits (RLIMITs) ────────────────────────────────────
# Harden RLIMITs at PID 1 (root) so the caps are inherited by every descendant
# and cannot be raised after the privilege step-down. nproc (#809) prevents
# fork bombs; nofile (#4527) caps open file descriptors so the sandbox no
# longer inherits the Docker daemon default (~1048576), which can exceed the
# host runtime limit. Best-effort: some container runtimes (e.g. brev, Docker
# Desktop, WSL) reject ulimit changes with "Invalid argument"; warn but do not
# block startup. Set the soft limit before the hard limit (ordering matters;
# see #951). Hard==soft makes the cap unraisable: raising the hard RLIMIT
# requires CAP_SYS_RESOURCE, which the unprivileged stepped-down agent never
# holds. NOTE: this only covers the PID 1 entrypoint process tree (gateway +
# agent); 'openshell sandbox connect' shells are spawned outside this tree and
# still inherit the runtime default (NVIDIA/OpenShell#1452).
harden_resource_limits() {
if ! ulimit -Su 512 2>/dev/null; then
echo "[SECURITY] Could not set soft nproc limit (container runtime may restrict ulimit)" >&2
fi
if ! ulimit -Hu 512 2>/dev/null; then
echo "[SECURITY] Could not set hard nproc limit (container runtime may restrict ulimit)" >&2
fi
if ! ulimit -Sn 65536 2>/dev/null; then
echo "[SECURITY] Could not set soft nofile limit (container runtime may restrict ulimit)" >&2
fi
if ! ulimit -Hn 65536 2>/dev/null; then
echo "[SECURITY] Could not set hard nofile limit (container runtime may restrict ulimit)" >&2
fi
}

# ── Capability dropping ──────────────────────────────────────────
# CIS Docker Benchmark 5.3: containers should not run with default caps.
# OpenShell manages the container runtime so we cannot pass --cap-drop=ALL
Expand Down
12 changes: 3 additions & 9 deletions scripts/nemoclaw-start.sh
Original file line number Diff line number Diff line change
Expand Up @@ -92,15 +92,9 @@ fi
# shellcheck source=scripts/lib/sandbox-init.sh
source "$_SANDBOX_INIT"

# Harden: limit process count to prevent fork bombs (ref: #809)
# Best-effort: some container runtimes (e.g., brev) restrict ulimit
# modification, returning "Invalid argument". Warn but don't block startup.
if ! ulimit -Su 512 2>/dev/null; then
echo "[SECURITY] Could not set soft nproc limit (container runtime may restrict ulimit)" >&2
fi
if ! ulimit -Hu 512 2>/dev/null; then
echo "[SECURITY] Could not set hard nproc limit (container runtime may restrict ulimit)" >&2
fi
# Harden RLIMITs (nproc #809 + nofile #4527) as root PID 1, before the capsh
# drop and the setpriv step-down, so the caps are inherited and unraisable.
harden_resource_limits

# PATH was already locked down at the top of this script (before the
# early stderr capture). This comment marks the original location.
Expand Down
76 changes: 75 additions & 1 deletion test/sandbox-init.test.ts
Original file line number Diff line number Diff line change
Expand Up @@ -667,6 +667,77 @@ EOF
});
});

describe("harden_resource_limits", () => {
// Shim `ulimit` (a bash builtin) by overriding it with a function inside the
// sourced body. The function records each invocation so we can assert both
// the nproc (#809) and nofile (#4527) caps are applied, soft-before-hard.
it("applies nproc and nofile soft+hard limits in order", () => {
const { stdout } = runWithLib(
[
// Override the ulimit builtin to record args and succeed.
"ulimit() { printf 'ulimit %s\\n' \"$*\"; return 0; }",
"harden_resource_limits",
].join("\n"),
);
const calls = stdout.split("\n").filter((line) => line.startsWith("ulimit "));
expect(calls).toEqual([
"ulimit -Su 512",
"ulimit -Hu 512",
"ulimit -Sn 65536",
"ulimit -Hn 65536",
]);
});

it("is best-effort: exits 0 and warns when ulimit fails", () => {
// Shim ulimit to always fail. The function must not abort (best-effort)
// and must emit a [SECURITY] warning for each of the four limits.
const { stdout } = runWithLib(
[
"ulimit() { return 1; }",
"harden_resource_limits 2>&1",
'echo "HARDEN_OK"',
].join("\n"),
);
expect(stdout).toContain("HARDEN_OK");
expect(stdout).toContain("Could not set soft nproc limit");
expect(stdout).toContain("Could not set hard nproc limit");
expect(stdout).toContain("Could not set soft nofile limit");
expect(stdout).toContain("Could not set hard nofile limit");
});
});

describe("entrypoints call harden_resource_limits", () => {
// Both entrypoints must delegate RLIMIT hardening to the shared helper and
// must no longer carry the pre-#4527 raw inline `ulimit -Su 512` block.
for (const rel of ["../scripts/nemoclaw-start.sh", "../agents/hermes/start.sh"]) {
it(`${rel} calls harden_resource_limits and has no raw inline nproc block`, () => {
const src = readFileSync(join(import.meta.dirname, rel), "utf-8");
expect(src).toContain("harden_resource_limits");
expect(src).not.toContain("ulimit -Su 512");
expect(src).not.toContain("ulimit -Hu 512");
});
}

// SECURITY (#4527): the RLIMIT caps are only unraisable if they are set
// while still root PID 1, BEFORE drop_capabilities (capsh) and the
// setpriv/gosu step-down. A refactor that moved the harden call after the
// privilege drop would turn it into dead code (cap set as the unprivileged
// agent, hard limit no longer lowered) while every other test stayed green.
// Pin the ordering so that regression is caught.
for (const rel of ["../scripts/nemoclaw-start.sh", "../agents/hermes/start.sh"]) {
it(`${rel} calls harden_resource_limits before drop_capabilities`, () => {
const src = readFileSync(join(import.meta.dirname, rel), "utf-8");
// Anchor to executable command lines, not free-text, so a comment
// mentioning either name cannot satisfy (or break) the ordering check.
const hardenIdx = src.match(/^\s*harden_resource_limits\s*$/m)?.index ?? -1;
const dropIdx = src.match(/^\s*drop_capabilities\b.*$/m)?.index ?? -1;
expect(hardenIdx).toBeGreaterThanOrEqual(0);
expect(dropIdx).toBeGreaterThanOrEqual(0);
expect(hardenIdx).toBeLessThan(dropIdx);
});
}
});

describe("init_step_down_prefixes", () => {
it("falls back to gosu when setpriv is unavailable", () => {
// Source-time init runs before our test body, so re-run it with a
Expand Down Expand Up @@ -832,7 +903,10 @@ EOF
it("nemoclaw-start.sh sources sandbox-init.sh", () => {
const src = readFileSync(join(import.meta.dirname, "../scripts/nemoclaw-start.sh"), "utf-8");
const start = src.indexOf("_SANDBOX_INIT=");
const end = src.indexOf("# Harden: limit process count", start);
// Bound the source block at the harden_resource_limits call line itself
// (executable, stable) rather than a free-text comment that may be reworded.
const hardenCallFromStart = src.slice(start).match(/^\s*harden_resource_limits\s*$/m);
const end = hardenCallFromStart ? start + (hardenCallFromStart.index ?? 0) : -1;
if (start === -1 || end === -1 || end <= start) {
throw new Error("Expected sandbox-init source block in scripts/nemoclaw-start.sh");
}
Expand Down
Loading