feat(egress): host hardening survey — follow-up to #30179 - #35187
Closed
Bartok9 wants to merge 10 commits into
Closed
feat(egress): host hardening survey — follow-up to #30179#35187Bartok9 wants to merge 10 commits into
Bartok9 wants to merge 10 commits into
Conversation
Adds a TLS-intercepting egress proxy for remote terminal sandboxes (Docker
v1; Modal/SSH to follow). When enabled, the sandbox holds opaque proxy
tokens; iron-proxy swaps them for real provider API keys at the egress
boundary. Compromising the sandbox leaks tokens that only work from behind
the proxy.
Wraps ironsh/iron-proxy (Apache-2.0, Go binary). Same lazy-install pattern
as the recently merged Bitwarden Secrets Manager integration — pinned
version, SHA-256 verified download into ~/.hermes/bin/iron-proxy, no apt
or sudo required.
Disabled by default. Run `hermes egress setup` to mint tokens and
`hermes egress start` to launch. The Docker backend then automatically
mounts the CA, sets HTTPS_PROXY + CA-bundle env vars, and adds the
host-gateway hostmap.
New surfaces:
hermes egress install — download the pinned iron-proxy binary
hermes egress setup — interactive wizard (supports --from-bitwarden)
hermes egress start — spawn the managed proxy daemon
hermes egress stop — SIGTERM (+SIGKILL after 5s grace)
hermes egress status — binary + config + pid + listening + mappings
hermes egress disable — flip proxy.enabled = false
hermes egress config — print the path to the generated proxy.yaml
Optional Bitwarden integration: `--from-bitwarden` sources the real
upstream credentials from a BSM project at proxy startup, so rotating a
key in the Bitwarden web app propagates to sandboxes on the next proxy
start without touching .env.
Hermes-side scope (v1):
agent/proxy_sources/iron_proxy.py — install + CA + config + lifecycle
hermes_cli/proxy_cli.py — `hermes egress` subcommand tree
hermes_cli/config.py — "proxy:" section in DEFAULT_CONFIG
hermes_cli/main.py — argparse wiring (uses 'egress'
because 'proxy' is the existing
inbound OAuth reverse proxy)
tools/environments/docker.py — CA mount, HTTPS_PROXY, CA-bundle
env vars, --add-host wiring
Hermetic tests cover the full lifecycle: token mint, mapping discovery,
config + mappings I/O, install pipeline (HTTP + tar + checksum all mocked),
subprocess lifecycle (Popen mocked), Docker backend arg builder.
A live E2E test (gated on HERMES_RUN_E2E=1) downloads the real iron-proxy
binary, spawns it, routes a curl request through it against a local fake
upstream, and verifies the Authorization header was swapped from the proxy
token to the real secret value (and the proxy token did NOT leak through
to upstream).
Failures (binary missing, port collision, bad token) never block agent
startup — they emit a warning and continue. The Docker backend refuses to
start a sandbox when proxy.enabled=true but the daemon is dead, unless
proxy.enforce_on_docker is explicitly set to false.
Docs: website/docs/user-guide/egress/{index,iron-proxy}.md
Tests: tests/test_iron_proxy.py (35), tests/test_iron_proxy_e2e.py (1)
P0 — must-fix
- iron_proxy: emit default upstream_deny_cidrs (loopback, IMDS
169.254.0.0/16, RFC1918) when caller passes None. Honours the docs
promise that cloud-metadata IPs are refused regardless of allowlist.
- iron_proxy: bind 127.0.0.1 (+ docker0 bridge IP on Linux) instead of
INADDR_ANY (':9090'). LAN peers with a leaked sandbox token could
otherwise spend the operator's API quota against any allowlisted
upstream.
- ensure_ca_cert: write the CA private key via os.open(..., 0o600)
instead of shutil.copy2+os.chmod — closes the TOCTOU window where
the key existed under the default umask.
- discover_uncovered_providers + proxy.fail_on_uncovered_providers
config: refuse to start (when strict) if env vars for non-bearer
providers (Anthropic native x-api-key, AWS SigV4, Azure OpenAI,
etc.) are present. Surfaces a wizard warning in non-strict mode.
P1 — should-fix
- start_proxy: build a minimal subprocess env (PATH/HOME/locale +
only the env names referenced by mappings) instead of os.environ
.copy(). Strips proxy-recursion vars (HTTPS_PROXY etc.). Stops
the proxy's /proc/<pid>/environ from leaking every host secret
to same-uid local processes.
- start_proxy: optional Bitwarden refresh path
(refresh_secrets_from_bitwarden=True, bitwarden_config=...).
When credential_source=bitwarden, cmd_start wires it in — that's
what delivers the rotation guarantee the docs make.
- build_proxy_config: wire audit_log into the rendered yaml
(log.audit_path). Parameter was accepted but never used.
- ensure_audit_log: pre-create the audit log with 0o600 perms so
iron-proxy inherits tight permissions instead of relying on umask.
- Rename 'hermes proxy ...' → 'hermes egress ...' in user-facing
strings (docstring, RuntimeError messages, post-setup banner).
- start_proxy: open log file with 0o600 perms and close the parent
fd immediately after Popen — fixes the per-restart fd leak.
- DockerEnvironment: detect collisions between docker_env and the
egress-controlling env vars (HTTPS_PROXY, SSL_CERT_FILE, etc.).
When enforce_on_docker=true, fail loud rather than silently
inverting the isolation; when false, warn and let docker_env win.
- proxy_cli: merge_mappings preserves existing tokens on re-setup;
--rotate-tokens flag re-mints all of them. Stops re-running
`hermes egress setup` from invalidating tokens baked into
already-running sandboxes.
- proxy_cli: --from-bitwarden fail-loud on disabled BW config,
missing access token, or empty vault. Previously fell through to
the env path while still writing credential_source: bitwarden.
- docker.py: narrow `except Exception` → `except ImportError`;
iron_proxy._read_tunnel_port_from_config: same. Bare excepts
were masking real config-load bugs.
- start_proxy: write pidfile via os.open with O_NOFOLLOW + 0o600
+ st_uid check. Refuses to follow a pre-existing symlink at the
pidfile path.
- mint_proxy_token docstring: document the 128-bit suffix entropy
explicitly (sha256 truncated to 32 hex chars).
P2 — follow-up
- start_proxy: poll-with-timeout (100ms cadence on _port_listening)
instead of an unconditional 5s sleep. Saves several seconds per
Docker container create when enforce_on_docker=true.
- docker.py: apply enforce_on_docker semantics when CA file vanishes
between status.configured check and CA mount. Previously returned
empty args silently.
- docker.py: refuse to mount when mappings.json is empty/corrupt
(was indistinguishable from upstream outage from inside the
sandbox).
- install_iron_proxy: tarfile.extract(..., filter='data') to silence
the PEP 706 deprecation and opt into the 3.14+ default.
- _proxy_state_dir: chmod 0o700 unconditionally; add
_proxy_state_dir_ro() so read-only callers don't create the dir.
- stop_proxy: re-verify pid before SIGKILL via /proc/<pid>/stat
starttime AND _pid_alive. Prevents SIGKILL'ing a recycled pid.
- _pid_alive: tightened cmdline check — basename match on argv[0]
plus an in-process nonce env var ('iron-proxy' in cmdline matched
'tail iron-proxy.log' and editors with the log open).
- docker.py: NODE_OPTIONS=--use-openssl-ca so Node.js routes through
the OpenSSL CA store SSL_CERT_FILE controls, narrowing the
Python/curl-replace vs Node-add asymmetry waefrebeorn flagged.
P3 — polish
- proxy_cli: dest='egress_command' (was 'proxy_command' which
collided lexically with the inbound OAuth subparser).
- iron_proxy_version: cache by binary path — get_status is called
per Docker container create, version is constant per binary.
- Drop unused `import sys` from iron_proxy.
- proxy_cli: `is not None` check on --tunnel-port (was treating 0
as falsy and silently substituting the default).
- proxy_cli cmd_disable: use get_status().pid instead of reaching
into ip._read_pid() (stale pidfile from a crashed run would have
fired a spurious "still running" warning).
- Tests: replace hardcoded /tmp/ca.* paths with tmp_path-derived
fixtures so tests are hermetic across hosts.
CI
- Windows footguns scanner: os.kill(pid, 0) is now gated behind
platform.system() != 'Windows' with a windows-footgun: ok marker;
signal.SIGKILL falls back to SIGTERM on Windows via
getattr(signal, 'SIGKILL', signal.SIGTERM).
- docs MDX compilation: replace bare `<https://…>` URLs with
`[text](url)` syntax (MDX-jsx parser rejects the angle-bracket
form).
Tests
- 32 new tests covering default deny CIDRs, bind policy, audit log
wiring, subprocess env minimization, CA TOCTOU 0o600, state dir
0o700, empty-mappings refusal, CA-vanished refusal, docker_env
collision detection, token preservation/rotate, uncovered provider
detection, and the proxy_cli command handlers + argparse wiring.
- All 156 tests in test_iron_proxy + test_iron_proxy_cli +
test_docker_environment + test_config pass locally.
Acknowledged but not addressed in this revision
- E2E test for HTTPS CONNECT + TLS-MITM path: existing E2E exercises
plain HTTP; full MITM coverage needs separate CI infra (real iron-
proxy binary + curl with custom CA). Tracked as follow-up.
- Cosign-style supply-chain verification for the binary checksum:
upstream iron-proxy doesn't sign releases yet. Accepted pattern
(same as Bitwarden integration); tracked as follow-up.
- CA rotation CLI (`hermes egress rotate-ca`): scope-cut to a
follow-up.
Reviewers: @annguyenNous @waefrebeorn @GodsBoy @erhnysr
The bws helper's warnings list contains non-secret status messages
('rate limited', 'project not found', etc.), but CodeQL's taint
analyzer can't distinguish those from the secrets dict returned by
the same call. Log the count instead of the strings — the warnings
are still observable via 'hermes secrets bitwarden status'.
…ings
GodsBoy 2nd-round P1 (all 4 addressed):
- _detect_docker_bridge_ip: replace `ip.count('.') == 3` heuristic with
ipaddress.IPv4Address validation + reject unspecified/loopback/multicast/
reserved/link-local/global addresses. Hostile `ip` shim on PATH used to
be able to inject 0.0.0.0 here and re-open INADDR_ANY binding.
- cmd_setup credential_source preservation: re-running `hermes egress
setup` without --from-bitwarden no longer silently downgrades a previous
bitwarden config back to env. Require --no-bitwarden to switch
explicitly; otherwise preserve the existing mode and surface the
decision.
- fail_on_uncovered_providers docstring/default mismatch: docstring used
to claim default=True; behavior was default=False. Resolved by
truth-in-advertising — docstring now correctly states default=False —
AND splitting providers into a strict LLM-specific tier
(_LLM_SPECIFIC_NON_BEARER_PROVIDERS, used by start blocking) and a
generic uncovered tier (used by wizard warnings). Generic cloud creds
(AWS_*, GOOGLE_APPLICATION_CREDENTIALS) no longer trip refuse-start
for operators using terraform/gcloud alongside Hermes. New
discover_blocked_providers() returns the strict subset.
- start_proxy poll-loop must verify listening before pidfile:
previously fell through deadline-expired as success and wrote a
pidfile for a non-listening daemon. Refactored into a do-while
shape, require `listening=True` for success, kill the child + unlink
the pidfile on failure paths.
GodsBoy 2nd-round P2 (the worth-keeping subset):
- O_NOFOLLOW + 0o600 + st_uid check on iron-proxy.log open (symmetric
with the pidfile and audit-log paths the same PR hardens).
- pidfile O_EXCL: refactored pidfile-write into _write_pidfile_safely
which uses O_EXCL to detect concurrent starts. EEXIST with a live
pid means "another start in progress" — refuse with actionable
message; EEXIST with a dead pid means "stale crash" — unlink and
retry once. Discriminates rather than racing.
- _VERSION_CACHE: invalidate on install_iron_proxy success;
don't cache empty stdout (would poison `hermes egress status` for
the lifetime of the process if first probe hit a corrupt binary).
- ensure_audit_log now RAISES on OSError instead of swallowing it as
a warning. Previous behavior let the daemon create the file under
the default umask, exactly the world-readable scenario the helper
was built to prevent. cmd_setup catches the new RuntimeError and
surfaces "✗" with the actionable message.
- SIGINT/SIGTERM handler scoped around the start_proxy poll loop:
Ctrl-C while waiting for `hermes egress start` no longer leaks an
orphan daemon with the port bound. Handler kills the child +
unlinks the pidfile before re-raising.
- pidfile written IMMEDIATELY after Popen, BEFORE the listening
verification. Parent dying during the poll loop now leaves a
pidfile pointing at the orphan so the next `hermes egress stop` can
clean up. Failure paths in the poll loop explicitly unlink.
- _DEFAULT_UPSTREAM_DENY_CIDRS: add ::ffff:0:0/96 (IPv4-mapped IPv6 —
closes the v6-resolved IMDS bypass), 100.64.0.0/10 (CGNAT / cloud
overlays / K8s pod networks), 198.18.0.0/15 (RFC2544 benchmark).
- _NON_BEARER_PROVIDERS split into LLM-specific (Anthropic / Azure /
Gemini — block when strict) vs generic-cloud (AWS_*, GCP appdefault
— warn-only).
- docker.py except narrowing: load_config can raise yaml.YAMLError on
a malformed config.yaml, not just ImportError. Two callsites
(collision check + precedence resolution) now catch yaml.YAMLError
via a sentinel `import yaml` and fail-safe to enforced mode.
GodsBoy 2nd-round P3:
- _reset_for_tests: was a no-op claiming symmetry with bitwarden;
now actually clears _VERSION_CACHE and _proxy_nonce so in-process
callers (notebooks, pytest -p no:xdist) don't see state leakage.
- tests/test_iron_proxy_cli.py: replaced hardcoded Path("/tmp/...")
with hermes_home/-derived fixtures. Matches the same cleanup we
did for test_iron_proxy.py in the previous round.
- --rotate-tokens confirmation gate: when there are existing tokens,
prompt for "rotate" confirmation (skipped when stdin isn't a tty
so CI/scripted use still works) AND back up the mappings to a
timestamped sibling before overwriting. Surface a no-op note when
rotate is requested with no existing tokens.
stephenschoettler (runtime-boundary review):
- #1 BWS silent degrade at proxy start: when credential_source=bitwarden
but the BWS access token or project_id is missing OR the fetch
returns no values for mapped providers, raise instead of silently
falling back to host env. cmd_start also pre-checks at the wizard
layer for actionable error messages. Opt-in escape hatch via new
`proxy.allow_env_fallback: true` config for migration scenarios.
- #2 docker_env collision detection extended: `docker_env:
{OPENROUTER_API_KEY: sk-real}` in config.yaml with enforce_on_docker:
true now raises just like an HTTPS_PROXY collision would. The
collision check pulls mapped provider names from load_mappings() at
call time.
- #3 PID nonce persisted to disk: cross-CLI-invocation stale-pidfile
defense now works. start_proxy writes the nonce next to the pidfile
(sibling 0o600), stop_proxy reads it back via _read_persisted_nonce()
and uses it as a _pid_alive signal in the new process. Falls back
to argv0 basename matching when the file is missing (legacy install).
arshkumarsingh:
- #1 NODE_OPTIONS append-merge: egress dict no longer sets NODE_OPTIONS
directly (would clobber the operator's --max-old-space-size etc.).
Carry the egress flag in a sentinel key
_HERMES_EGRESS_NODE_OPTIONS_APPEND; DockerEnvironment merges into the
existing NODE_OPTIONS in env_args computation with de-duplication.
- #2 docs: structured per-request audit log is at audit.log, not
iron-proxy.log (the latter is daemon stdout/stderr). Diagram and
step-7 text corrected; both file roles are now documented separately.
Tests
- Added 12 new tests in test_iron_proxy.py covering bridge-IP rejection
(parametrized over 8 dangerous inputs), default deny-list adjacency
(IPv4-mapped-v6 + CGNAT), blocked-providers strict-subset property,
_pid_proc_starttime parser with paren-containing comm,
stop_proxy SIGKILL suppression on starttime drift, _reset_for_tests
clear behavior, iron_proxy_version don't-cache-empty, NODE_OPTIONS
sentinel verification, ensure_audit_log raise-on-OSError, and
persisted-nonce roundtrip.
- Added 1 new test in test_iron_proxy_cli.py covering cmd_start
BWS-token-missing fail-loud.
- All 100 tests in test_iron_proxy + test_iron_proxy_cli pass; all 78
tests in test_docker_environment + test_config still pass.
Acknowledged but not addressed:
- GodsBoy P3 dead-code `extra_env` kwarg: kept (removing is a breaking
change for any out-of-tree caller; the kwarg is documented and works).
- Residual risks GodsBoy called out: iron-proxy in-memory secret
zeroisation (Go-binary territory, out of scope); _PROXY_SUBPROCESS_ENV
_ALLOWLIST cosmetic gaps (RUST_LOG, GOMAXPROCS); follow-up.
internals reference Pre-v3 the egress docs were 175 lines covering the basics: quick start, slash commands, security model, failure modes. After three rounds of PR review we added a half-dozen new config knobs, two new flags, a strict/warn tier split for uncovered providers, persisted-nonce cross-process defense, audit-log + log-file separation, NODE_OPTIONS append-merge, docker_env collision detection, etc. — none of which the user-facing doc reflected. This commit closes that gap end-to-end: website/docs/user-guide/egress/iron-proxy.md (175 → 567 lines) - Configuration section expanded with every new knob: fail_on_uncovered_providers, allow_env_fallback, upstream_deny_cidrs. - Tables for default allowed hosts + default deny CIDRs. - Bind policy section (loopback + docker bridge, NOT 0.0.0.0) with the operator-facing "why can't I hit the proxy from my LAN" answer. - Uncovered providers section with the strict tier (Anthropic / Azure / Gemini — block when fail_on_uncovered_providers=true) vs warn tier (AWS, GCP appdefault — present on every dev laptop, never block). - Bitwarden integration expanded: rotation semantics, fail-loud at start, the allow_env_fallback escape hatch, --no-bitwarden flag, the preserve-existing-source rule on plain re-setup. - Slash commands section with --no-bitwarden, --rotate-tokens, and the token-rotation operator playbook (confirmation gate, backup file naming, restart-required caveat). - State directory layout table covering all 9 files we create + their modes. - Audit log vs daemon log distinction (the arshkumarsingh #2 fix that motivated the corrected diagram). - CA distribution into the sandbox: full table of injected env vars, the Python/curl REPLACE vs Node ADD asymmetry caveat with the NODE_OPTIONS=--use-openssl-ca mitigation. - docker_env collision detection: what gets blocked, what gets warned, the migration escape hatch. - PID + nonce defense section explaining how iron-proxy.nonce works cross-CLI and the SIGKILL-suppress-on-recycle path. - Security model expanded with the new defenses (IPv4-mapped-v6 IMDS bypass closure, env-var leakage prevention, LAN-peer-with-token-leak coverage). - Failure modes extended for every new refuse-start path. - Troubleshooting section (180 new lines) with grep-friendly error matchers for each common failure: BWS token missing, uncovered provider refused, port collision, slow bind, 403 from proxy, SSL verification errors inside the sandbox, 401 from upstreams, address- in-use orphan recovery, per-request audit log inspection. website/docs/getting-started/quickstart.md - One-paragraph mention of the egress proxy under "Sandboxed terminal" so operators discover the feature when they enable Docker isolation. website/docs/reference/cli-commands.md - Top-level command table now lists `hermes egress` alongside `hermes proxy` (different purpose, different direction — call it out). - New `## hermes egress` section with full subcommand syntax, common flows (first-time setup, switching credential source, rotating tokens, adding upstream), and diagnostic shortcuts. website/docs/reference/environment-variables.md - New "Egress proxy (sandbox-injected)" section documenting every env var the Docker backend injects: HERMES_EGRESS_PROXY, HERMES_PROXY_TOKEN_<NAME>, HTTPS_PROXY/HTTP_PROXY/NO_PROXY, REQUESTS_CA_BUNDLE/SSL_CERT_FILE/CURL_CA_BUNDLE/NODE_EXTRA_CA_CERTS, NODE_OPTIONS append-merge, HERMES_IRON_PROXY_NONCE. - Also fixes a stale layout issue with the Persistent Shell table that had two trailing rows getting orphaned in the v3 commit. website/docs/developer-guide/egress-internals.md (NEW, 363 lines) - Module layout map (which file owns what). - Full lifecycle walkthrough for install / setup / start / stop with the actual function calls in order. - "Security invariants" section enumerating every load-bearing property with the regression test name that guards it. These are the rules contributors must preserve when touching the module: - filesystem perms (0o700 dir, 0o600 secrets, O_NOFOLLOW everywhere) - subprocess env minimisation (no os.environ.copy) - bind policy (loopback + docker bridge, never 0.0.0.0) - default deny CIDR coverage - audit log fail-loud - bitwarden fail-loud - docker_env collision detection - PID recycling defense - token preservation on re-setup - credential_source preservation - Extension points: adding a bearer-token provider, adding a non-bearer provider, wiring iron-proxy into a non-Docker backend, subscribing to per-request audit events. - Testing recipe (hermetic + E2E + CLI smoke). website/sidebars.ts - New `developer-guide/egress-internals` entry under Developer Guide → Internals (alongside acp-internals, cron-internals, trajectory-format). Build verification - `cd website && npm install && npx docusaurus build` succeeds locally. - All three new pages render to static HTML in all three locales (en + zh-Hans + ko). - No new broken links or broken anchors introduced (pre-existing warnings on translation stubs are unrelated).
Single content conflict in hermes_cli/config.py — kept BOTH the paste_collapse_threshold knobs from main and the proxy section from this branch (they're independent additions to DEFAULT_CONFIG). All 187 tests in test_iron_proxy.py + test_iron_proxy_cli.py + test_config.py pass post-merge.
propagate handler exit codes
Live testing the full wizard against the real v0.39.0 binary
(downloaded + extracted via our own install_iron_proxy()) surfaced
three real bugs that the unit tests couldn't catch:
1. `proxy.http_listens` (plural) — NOT a field in v0.39's config struct.
Our code emitted both `http_listen` (string) and `http_listens`
(list) believing v0.39 accepts both forms. The binary actually
rejects with "field http_listens not found in type config.Proxy"
at YAML unmarshal time, so the daemon fails to start. Confirmed
via strings(1) audit of the v0.39 binary — only `http_listen` is
tagged.
2. `log.audit_path` — NOT a field in v0.39's config.Log struct. Same
class of error: "field audit_path not found in type config.Log".
Per-request audit-log records are not separable from server-level
logs at this binary version.
3. `metrics.listen` defaults to ":9090" — which is the SAME port as
our default `tunnel_port: 9090`. Result: every operator who runs
`hermes egress setup` followed by `hermes egress start` gets
"bind: address already in use" because the proxy listener and the
metrics listener fight for port 9090. We now explicitly pin
`metrics.listen: 127.0.0.1:0` to give it an ephemeral loopback
port that can never collide with tunnel_port regardless of what
operator sets.
Plus a fourth bug — pre-existing but surfaced by the egress live
test — that affects every Hermes subcommand:
4. `hermes_cli/main.py` calls `args.func(args)` at the bottom of
main() but discards the return value. Every subcommand handler
that returns a non-zero exit code (cmd_start refusing because
`fail_on_uncovered_providers=true`, cmd_setup refusing because
--from-bitwarden but BWS unreachable, etc.) was silently exiting 0.
Fix: capture the handler's return value and `sys.exit(rc)` when
it's a non-zero int. Other subcommands' contracts unchanged
because they either return 0/None or don't return at all.
Validation:
- 188/188 in test_iron_proxy.py + test_iron_proxy_cli.py +
test_config.py pass post-fix.
- 5333/5337 in tests/hermes_cli/ pass; the 4 unrelated failures
(test_managed_installs.py + test_update_hangup_protection.py)
are pre-existing on main, not touched by this PR.
- Manual wizard run end-to-end with the v0.39.0 binary in an
isolated HERMES_HOME:
* `egress install` — downloads + SHA-256 verifies + extracts
* `egress setup` — generates CA, mints tokens, writes
proxy.yaml that the binary now accepts (no http_listens,
no audit_path, metrics pinned to 127.0.0.1:0)
* `egress start` — daemon binds 127.0.0.1:9090, listens=yes
* `egress status` — shows pid + listening + mappings
* `egress stop` — clean shutdown, pidfile + nonce removed
* Idempotent re-start returns the running pid without spawning
* curl through the proxy with the openrouter token gets
forwarded; an attacker host gets HTTP 403 (allowlist works);
169.254.169.254 gets HTTP 403 (deny CIDR works)
* Refuse-start paths exit 1 with actionable messages:
- `fail_on_uncovered_providers=true` + ANTHROPIC_API_KEY set
- `credential_source=bitwarden` + BWS_ACCESS_TOKEN unset
* `--rotate-tokens` confirmation gate fires via pty:
typing 'cancel' aborts; typing 'rotate' proceeds and
creates a mappings.json.rotated-<timestamp> backup
Test updates:
- `test_default_bind_is_loopback_not_zero_zero` — asserts the
singular `http_listen` is loopback AND asserts `http_listens`
(plural) is NOT in the rendered yaml.
- `test_default_bind_uses_loopback_on_linux` — replaces
`test_default_bind_includes_docker_bridge_on_linux`. v0.39
only supports one bind per daemon process, so the docker bridge
augmentation is dropped from the rendered config; sandboxes
reach the daemon via host.docker.internal -> host-gateway
mapping, so loopback-only is functional.
- `test_metrics_listener_pinned_to_loopback_ephemeral` — new
regression test asserting `metrics.listen == "127.0.0.1:0"`.
- `test_audit_log_kwarg_does_not_inject_audit_path_v039` —
replaces `test_audit_log_path_lands_in_yaml`. audit_log kwarg
is still accepted for forward compatibility but does NOT emit
log.audit_path until upstream supports it.
…vior The previous docs round (906b1da) described the integration the way we wanted it to work — `http_listens` plural with a docker bridge bind, dedicated `audit.log` for per-request JSON records. Live testing against the real v0.39.0 binary in [905ce58] surfaced that neither field exists in v0.39's config schema, and the docs were making promises the daemon couldn't keep. This commit walks every claim in the docs back to what the binary actually does today, while keeping the upgrade path explicit so the docs stay coherent when the pinned `_IRON_PROXY_VERSION` bumps: website/docs/user-guide/egress/iron-proxy.md - Bind policy section: rewritten. Was "loopback + docker bridge IP on Linux"; now "loopback only" with an explicit explanation that v0.39 only supports one bind per daemon and that host.docker.internal -> host-gateway mapping is what sandboxes use to reach the loopback bind. - Bind policy section adds a note on the metrics-port pin that the previous round of docs didn't even mention. - State directory layout table: `audit.log` description rewritten to acknowledge it's a pre-created sentinel for future binary versions, NOT something the v0.39 daemon writes to. - New section "Logging on iron-proxy v0.39" replaces the old "Audit log vs daemon log" section. Explicitly tells operators the daemon log is the single source of truth for both audiences on v0.39, with the upgrade path called out. - Data-flow diagram step 7: rewritten to send per-request records to `iron-proxy.log` on v0.39 with cross-link to the new logging section. - Diagram caption updated. - Security-model "allowlisted-host exfiltration" line: "audit log captures" -> "daemon log captures". - Security-model "LAN peer leak" line: removed the docker-bridge claim. - Troubleshooting section's per-request-inspection recipes: rewritten to use `iron-proxy.log` and explain when the split stream will land. - Limitations list gets a new bullet calling out the single-bind + combined-log v0.39 constraints + the auto-upgrade posture. website/docs/developer-guide/egress-internals.md - Bind policy invariant: documents the singular `http_listen` v0.39 schema constraint + dead-code-until-upgrade status of the bridge-bind path. - New "Metrics port collision" invariant documenting why `metrics.listen: 127.0.0.1:0` is non-negotiable. - Audit log fail-loud invariant adds the v0.39 schema constraint note + the new `test_audit_log_kwarg_does_not_inject_audit_path_v039` regression test. - "Subscribing to per-request audit events" section updated to send watchers at `iron-proxy.log` for v0.39 with the upgrade pivot called out. website/docs/reference/cli-commands.md - Diagnostic shortcut for tailing the audit log: `tail audit.log | jq` -> `tail iron-proxy.log | jq` with the v0.39 note inline. Build verification: - `npx docusaurus build` succeeds across all three locales (en + zh-Hans + ko). - New `#logging-on-iron-proxy-v039` anchor lands in the rendered HTML and the in-page cross-references resolve. - No new broken anchors introduced (pre-existing warnings on unrelated zh-Hans pages are unchanged). - No leftover stale `#audit-log-vs-daemon-log` or `#http_listens` references anywhere on the egress pages.
Read-only survey of host perimeter controls — firewall (UFW / firewalld /
nftables), Tailscale, fail2ban, SSH config, Docker seccomp — alongside two
iron-proxy runtime signals (via the existing `get_status()`). Shows the
whole defense-in-depth stack in one table.
`hermes egress harden` is informational; it always exits 0 and never gates
a deploy. The `--baseline` flag (`minimal` / `catalin` / `paranoid`)
colors the summary line only; the `--all` flag includes passing signals
in the table; `--json` emits a stable schema for SIEM/dashboard ingest.
The 10 signals: tailscale, ufw, firewalld, nftables, fail2ban,
ssh-password-auth, ssh-root-login, iron-proxy-enabled, iron-proxy-running,
docker-seccomp. Each probe is best-effort with a 4 s timeout — missing
binary / non-Linux host → skip (not fail), so the survey never wedges.
Stdlib only, no new dependencies. Reuses `get_status()` from the
iron-proxy core (the only touch outside the new module).
Inspired by @catalinmpit's Hetzner+Tailscale+UFW+Cloudflare+fail2ban
deployment that prompted Teknium's "secure hermes" question on X — the
`catalin` baseline encodes that posture as a first-class target.
Bugbot review fixes baked in (caught on the original fork PR, retargeted
to upstream here as one clean commit):
* firewalld substring match -> exact-equality match
"running" in "not running".lower() was True, falsely reporting PASS
when firewalld is stopped (high severity). Regression test added:
test_firewalld_stopped_is_fail_not_pass.
* cmd_harden table column 3 now shows s.detail (e.g. "ufw installed
but inactive") instead of s.status (which is already rendered as a
glyph in column 1). Mirrors cmd_doctor. Regression test added:
test_cmd_harden_table_shows_detail_not_status.
Validation:
27 passed in 1.23 s (tests/test_iron_proxy_harden.py)
Author: Bartok9 (Daniel Pike), opened in response to Teknium's invitation
to @catalinmpit for a security review of NousResearch#30179. Complements NousResearch#30179
(sandbox-egress isolation) by surveying the perimeter side of the stack.
alt-glitch
added
type/feature
Contributor
Author
Polish — improved description bullets• |
This was referenced Jun 17, 2026
teknium1
force-pushed
the
feat/iron-proxy
branch
3 times, most recently
from
June 26, 2026 08:11
cfa96dd to
6ff7e78
Compare
This file contains hidden or bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
Sign up for free
to join this conversation on GitHub.
Already have an account?
Sign in to comment
3 participants
Add this suggestion to a batch that can be applied as a single commit.This suggestion is invalid because no changes were made to the code.Suggestions cannot be applied while the pull request is closed.Suggestions cannot be applied while viewing a subset of changes.Only one suggestion per line can be applied in a batch.Add this suggestion to a batch that can be applied as a single commit.Applying suggestions on deleted lines is not supported.You must change the existing code in this line in order to create a valid suggestion.Outdated suggestions cannot be applied.This suggestion has been applied or marked resolved.Suggestions cannot be applied from pending reviews.Suggestions cannot be applied on multi-line comments.Suggestions cannot be applied while the pull request is queued to merge.Suggestion cannot be applied right now. Please check back later.
Why this complements #30179
#30179 introduced the iron-proxy sandbox-egress layer (swap opaque proxy tokens for real upstream creds at the network boundary, so a prompt-injected agent never sees real keys). That stops credential exfil from inside the sandbox.
This PR adds the host-perimeter side of the picture.
hermes egress hardenis a read-only survey that probes the host's firewall (UFW / firewalld / nftables), Tailscale, fail2ban, SSH config, and Docker seccomp — alongside two iron-proxy runtime signals viaget_status()— so an operator sees the whole defense-in-depth stack in one table.The two layers solve different threats; neither substitutes for the other:
OPENAI_API_KEYto an attacker. iron-proxy stops that.A companion CA-rotation PR (
hermes egress rotate-ca) is open separately against the same base.What lands
agent/proxy_sources/host_hardening.py— stdlib-only, no side effects, every probe graceful (missing binary / non-Linux host →skip, neverfail). The only touch to the existing iron-proxy code is afrom .iron_proxy import get_statusfor the two runtime signals.hermes egress harden(one sub-parser entry + one handler inhermes_cli/proxy_cli.py):--baseline {minimal,catalin,paranoid}(defaultminimal)--json→{"signals":[{name,status,detail,fix}],"baseline":str,"satisfied":bool,"missing":[str]}--all→ show passing signals too (default: gaps-only)tests/test_iron_proxy_harden.py(mirrorstest_iron_proxy_doctor.pystyle if/when doctor lands).website/docs/user-guide/egress/hardening-baselines.md+ a "Host hardening" section iniron-proxy.mdand ahardenblock incli-commands.md.The 10 signals
tailscaletailscale status --json→BackendState=Runningufwufw status verbose→ active + default-deny incomingfirewalldfirewall-cmd --state→ exact match "running" (see Bugbot fix below)nftablesnft list ruleset→ non-emptyfail2banfail2ban-client status→ ≥1 jailssh-password-auth^PasswordAuthentication noin sshd_configssh-root-loginiron-proxy-enabledget_status()iron-proxy-runningget_status()(pid alive + listening)docker-seccompdocker infoSecurityOptions includesseccompBaselines
Color the summary line only — never force a fail:
Failure modes considered
_run()helper; a hang →skip, never a wedged survey.ufw / firewalld / nftables / fail2banskip cleanly on macOS; Tailscale / SSH-config / iron-proxy / Docker still run./etc/ssh/sshd_config— both SSH signalsskip(notfail); many minimal containers don't run sshd.get_status()raising — wrapped; the iron-proxy signals degrade toskip, never crash the survey.tailscalenon-JSON →warn;nft/dockernon-zero exit →warnwith an actionable hint.Bugbot review fixes baked in
Two issues Cursor Bugbot caught on the original cross-fork iteration, fixed inline here (no separate cleanup PR needed):
🔴 High severity —
firewalldsubstring match false-positives a stopped daemon"running" in out.strip().lower()evaluatesTruefor"not running"— meaning the probe would report PASS for a stopped firewalld, the worst-possible failure mode for a hardening probe. Fix: exact-equality match on the stripped output. Regression test added:test_firewalld_stopped_is_fail_not_pass.🟡 Medium severity — table column 3 showed redundant status text
The third column rendered
s.status("fail"/"warn"/"skip") which is already encoded in the glyph in column 1, while the usefuls.detail("ufw installed but inactive", "BackendState=Running") was never shown in non-JSON mode. Fix: column renamed to "Detail" and now renderss.detail. Regression test added:test_cmd_harden_table_shows_detail_not_status.Validation
The 1 skip is the existing E2E test gated behind
HERMES_RUN_E2E=1(unchanged).feat/iron-proxy)tests/test_iron_proxy_harden.py)hermes egress harden --jsonruns clean (exit 0) on a macOS dev host with Linux-only signals degrading toskip— confirms graceful cross-platform behavior.Coverage gaps
enabledsignal treats a generated CA +proxy.yamlas "enabled" becauseget_status()doesn't readconfig.yaml'sproxy.enabled. An explicitenabled=Truealso passes. Documented inline.nft list rulesetwithout root returns a non-zero exit →warn(we can't distinguish "no ruleset" from "needs root"); the hint says to re-run withsudo.Ambiguity flags
get_status()exposes.enabled(dataclass defaultFalse, not populated from config) and.configured(CA + proxy.yaml present). I treatedenabled OR configuredas PASS so the signal is useful without a full config load. If you want strictconfig.yaml: proxy.enabledsemantics, that's a one-line change.hardenas its own command vs. a doctor check — kept as a separate command tree (hermes egress harden). They're complementary:doctorchecks the proxy is healthy;hardenchecks the host is locked down.missinglist reports all acceptable members so the operator sees every valid fix. Open to reporting just a representative one if that's noisy.Cut from scope (known follow-ups)
Documented in
hardening-baselines.mdunder "Future enhancements":iptables -Lbaselineuserns-remap)Attribution
Opened by Bartok9 (Daniel Pike) at Daniel's request, in response to Teknium's invitation on X to @catalinmpit for a security review of #30179. Teknium's prompt was essentially "can we get a security review of how Hermes' egress proxy holds up in a real deployment?" — and Catalin's public deployment described the shape this PR encodes as the
catalinbaseline:This PR turns that ad-hoc perimeter into a first-class, surveyable baseline that composes with the existing sandbox-egress isolation.