fix(gateway): stop revoking managed-mode group access on the flush dir

[ZHJay]

What does this PR do?

gateway/shutdown_flush.py::_get_flush_dir() creates $HERMES_HOME/pending_messages with an unconditional mode=0o700 and chmods it 0700 again:

flush_dir.mkdir(parents=True, exist_ok=True, mode=0o700)
if os.name == "posix":
    os.chmod(flush_dir, 0o700)

On a managed (NixOS) install, both mechanisms override group permissions the module deliberately configures. This is pre-existing upstream code, not a regression from a recent PR.

The NixOS evidence (nix/nixosModules.nix on current main):

• Lines 711-719 pre-create only stateDir, .hermes, cron, sessions, logs, memories, plugins at 2770 (setgid, group-rwx) via systemd.tmpfiles, mirrored by the activation loop at 740-743. pending_messages is absent from both lists — so on a managed host it is always created lazily at runtime, and these two lines are the only thing setting its mode.
• Line 907 sets UMask = "0007", commented "files created by the gateway should be group-writable so interactive users in the hermes group can read/write them."
• Lines 135-136 avoid chown -R specifically because it strips setgid, "destroying the 2770 permissions … for group access by hostUsers."
• container.hostUsers get a ~/.hermes symlink to that same stateDir, so the gateway service and an interactive CLI share one $HERMES_HOME at two different uids.

Measured under real managed conditions (2770 parent, umask 0007, dir absent):

	dir	payload
before	0o700 (no group)	0o600
after	0o770 (group-rwx)	0o600

The payload files stay 0o600 either way — only the directory widens, which is the part that gates the second process. Measured consequence: mkdir(exist_ok=True) on a dir owned by the other uid succeeds silently, then the mkstemp inside it fails EACCES. So the flush is lost, not merely unreadable, and because every caller wraps this module in except Exception: pass it fails silently.

Blast radius, stated honestly

The automatic reader recover_pending_to_db() is called from exactly one place, gateway/run.py:26505 — gateway-only. In the common case writer and reader are the same service, so this is narrower than a user-facing directory.

What makes it more than cosmetic: flush_agent_history_to_file payloads are deliberately skipped by automatic recovery (if payload.get("reason") == "shutdown-with-unpersisted-agent-history": continue) and documented as left for a human — "so an operator can salvage the conversation after repairing state.db." A 0700 dir owned by the service uid locks that operator out of their own transcript. Plus, on managed hosts the two uids sharing $HERMES_HOME are both gateway processes, so the EACCES write-loss path is reachable without any CLI involvement.

Related Issue

Refs #77472 — the pending_messages/ item in cluster R-DUMP ("flush files raw unredacted; 0600 skipped on Windows"). This PR addresses the directory-mode half of that item on managed installs; the payload files were already 0600 on main and stay 0600 here. The sibling file-mode items in that cluster are #77520 (trajectories / MoA traces / /save) and #77655 (a2a logs, batch trajectories, sessions/saved); its unbounded-growth item is #78395 and its redactor-correctness item is #78379.

Note on that issue's framing: the report characterises this path as an at-rest exposure. It is not one on POSIX — the payloads are already 0o600 and the directory 0o700 on main. The real defect runs the opposite direction, which is what this PR fixes: 0o700 is forced even on a managed install, revoking the group access nix/nixosModules.nix deliberately configures (UMask = "0007" at nixosModules.nix:907, ~/.hermes at 0o2770 setgid).

No dedicated issue for this specific defect. Related context: #72680 (the incident that introduced this module), and #14181 tracks the same managed/shared-runtime permission class for SKILL.md. Sibling PR #77579 (head 4195f1e98) is precedent for the fix shape, not a parent — the two touch disjoint files and can land in either order.

• fix(gateway): resolve session_key in shutdown-flush recovery #75536 (fix(gateway): resolve session_key in shutdown-flush recovery, open, head b1db79117) — shares two files with this PR. The production hunks are disjoint: fix(gateway): resolve session_key in shutdown-flush recovery #75536 edits recover_pending_to_db (from line 168) while this edits _get_flush_dir (lines 36-46), so gateway/shutdown_flush.py should merge cleanly. tests/gateway/test_shutdown_flush.py will conflict textually — both changes are pure appends at the same anchor (line 132 of a 134-line file), fix(gateway): resolve session_key in shutdown-flush recovery #75536 adding 110 lines and this one 293. Semantically compatible; the conflict is positional only. Happy to rebase behind whichever lands first.

Type of Change

• 🐛 Bug fix (non-breaking change that fixes an issue)

Changes Made

• gateway/shutdown_flush.py
  • _get_flush_dir() branches on is_managed() at the creation site, mirroring ensure_hermes_home (hermes_cli/config.py:896) and its logs/curator bare-mkdir precedent. Managed installs get mkdir(parents=True, exist_ok=True) so the inherited setgid + UMask=0007 lands 2770.
  • New _managed_install() helper — local import, failure means "not managed" (an unimportable config module is the single-user source install, where owner-only is right).
  • The hand-rolled os.chmod(flush_dir, 0o700) is replaced by _reconcile_flush_dir_mode(), which delegates to hermes_cli.config._secure_dir — the single owner of this policy for HERMES_HOME and every subdir ensure_hermes_home creates.
• tests/gateway/test_shutdown_flush.py — 5 → 16 tests, three new classes. (Baseline measured at merge base cb11a7e25: 5 test functions, no parametrize; current upstream/main also has 5.)

Why delegate the chmod rather than guard it inline

The hand-rolled chmod diverged from the shared policy on three axes, each independently a defect:

1. it ran in managed mode, where the policy is to stand down;
2. it ignored HERMES_HOME_MODE, which every sibling dir honours;
3. it was unguarded — chmod by a non-owner raises EPERM, which propagated straight out of _get_flush_dir() into callers that all swallow exceptions, silently degrading the flush to a no-op.

17 gateway/ modules already import from hermes_cli.config (counted on upstream/main), so the dependency direction is established, not new.

Not using #77655's secure_mkdir: it does not exist on main (#77655 is separately open), so importing it would couple two open PRs. Following that author's own precedent, this branch stands alone.

HERMES_HOME_MODE now applies (0o701 instead of 0o700) as a deliberate consequence of delegating. #77655 argued the hatch exists for traversal to a served subdir and nothing is served here. I went the other way for this site: pending_messages is a peer of sessions/ and logs/, which all honour it, the payload files stay 0600, and uuid4().hex filenames aren't guessable from an execute-only directory. Happy to gate it if a maintainer prefers consistency with #77655 instead.

Relationship to #74897

#74897 moved write_file new files from 0600 to umask-derived 0644. Reading it, the real story is that _atomic_write's chmod branch only ran when the target already existed, so new files silently kept mktemp's 0600 — an accident, not a policy — and #74897 restored umask-derived permissions for a path the user named with a documented interop contract (#70856: Obsidian LiveSync, Docker volumes, NAS mounts). This change moves in the same direction for managed mode: removing a hardcoded mode in favour of the configured umask.

How to Test

1. Managed-mode fresh creation (the case at issue):

   ./scripts/run_tests.sh tests/gateway/test_shutdown_flush.py -q

   16 passed. TestFlushDirManagedPermissions reproduces all three real conditions — HERMES_MANAGED=nixos, parent chmod'd 2770, umask 0007, dir absent.

2. Teeth, per mechanism (each reverted individually, restored byte-identically, sha256 re-confirmed):

   reverted mechanism	result
   managed-branch mkdir(..., mode=0o700) restored	2 failures
   unmanaged mkdir mode= stripped	1 failure (isolation test only)
   managed early-return deleted	3 failures
   reconciler try/except removed	1 failure

   The mkdir mode and the chmod do mask each other, as suspected. Stripping mode= from the unmanaged mkdir left both plain "is it 0700?" tests green, because _secure_dir alone produced 0700. Only test_creation_mode_alone_is_owner_only, which stubs the reconciler to a recording no-op, caught it. That's why the isolation fixture exists rather than a second final-mode assertion.

3. Full octal matrix, dir and payload, before/after, measured under a temp HERMES_HOME:

   scenario	dir before	dir after	file before	file after
   unmanaged, umask 022	0o700	0o700	0o600	0o600
   unmanaged, umask 077	0o700	0o700	0o600	0o600
   managed, umask 0007, parent 2770, dir absent	0o700	0o770	0o600	0o600
   managed, dir pre-existing 0750	0o700	0o750	0o600	0o600
   HERMES_HOME_MODE=0701, umask 022	0o700	0o701	0o600	0o600

On atomic_json_write(..., mode=0o600) in _write_payload — left untouched

Measured, so it doesn't get "fixed" needlessly: a fresh file is 0o600 with or without the argument under umask 022/077/007, because mkstemp creates at 0600 and _restore_file_mode no-ops with no prior mode. The argument only changes the overwrite path, which uuid4().hex filenames make unreachable. It does close a real fchmod-before-rename window for the overwrite case, so removing it would be churn. No change made.

Checklist

Code

• I've read the Contributing Guide
• My commit messages follow Conventional Commits
• I searched for existing PRs to make sure this isn't a duplicate
• My PR contains only changes related to this fix
• I've run pytest tests/ -q and all tests pass — not run. I ran the canonical ./scripts/run_tests.sh on the changed file (16/16) plus the 77-file regression set matching shutdown_flush|pending_messages: 558 passed, 1 failed. The one failure is test_telegram_thread_fallback.py::test_send_image_upload_fallback_blocks_connect_time_rebind, a pre-existing SSRF-guard baseline failure — confirmed baseline by reproducing it with gateway/shutdown_flush.py reverted to upstream/main. test_shutdown_forensics.py (Linux-only) is likewise baseline-failing in this worktree, verified the same way.
• I've added tests for my changes
• I've tested on my platform: macOS 27.0 (Darwin 27.0.0, arm64), Python 3.11.15

Documentation & Housekeeping

• I've updated relevant documentation (docstrings on both new helpers and _get_flush_dir)
• N/A — no config keys added or changed. Behavior follows existing HERMES_MANAGED / HERMES_HOME_MODE; no new HERMES_* env var
• N/A — no architecture or workflow change
• Cross-platform: the reconciler keeps its original non-POSIX early return, and the new tests are POSIX-gated (os.name != "posix"). Windows ACL enforcement is deliberately left to fix(security): enforce owner-only ACLs on Windows in _secure_file #77527 rather than duplicated here. macOS verified directly; setgid inheritance confirmed on Darwin (2770 parent → 0770 child under umask 0007)
• N/A — no tool schema change

Screenshots / Logs

=== Summary: 1 files, 16 tests passed, 0 failed (100% complete) in 3.5s

ruff 0.16.1 — All checks passed!
scripts/check-windows-footguns.py --diff upstream/main — ✓ No Windows footguns found
scripts/check_subprocess_stdin.py — ✅ All TUI-context subprocess calls have explicit stdin=

[Copilot]

Copilot was unable to review this pull request because the user who requested the review has reached their quota limit.

[ZHJay]

Answering the three sweeper:risk-* labels with measurements, since they're the only review signal on this PR so far and each is checkable. All numbers below were produced against branch head 97a5e6daa in a clean worktree, on macOS (POSIX), with the real _get_flush_dir() / _write_payload() code paths — no stubs.

sweeper:risk-message-delivery — this PR removes a message-loss path; it does not add one.

The label is the right instinct on this file, but the direction is inverted. The loss mechanism on main is that the failure is silent:

dir mode: 0o500
mkdir(exist_ok=True) on non-writable existing dir: SUCCEEDS silently
mkstemp inside it: FAILS errno=EACCES -> the flush is lost, not merely unreadable

_get_flush_dir() returns happily, then the mkstemp inside atomic_json_write raises, and every caller of this module wraps it in except Exception: pass. So a 0700 directory owned by the service uid doesn't degrade the flush to "written but unreadable" — it drops the payload. Widening the directory is what restores delivery for the second uid.

sweeper:risk-session-state — the payload mode is untouched; only the directory mode moves.

Measured on the real save/_write_payload path:

scenario	pending_messages/	payload
unmanaged, umask 022	0o700	0o600
managed (HERMES_MANAGED=nixos), parent 2770, umask 0007	0o770	0o600

Payloads stay owner-only in both. The verbatim user content in them is not more exposed by this change; the directory is what gates whether the other process can write at all.

sweeper:risk-compatibility — one intended behavior change, plus one that follows from delegating.

• Managed installs: 0o700 → 0o770. This is the fix. pending_messages is not in nix/nixosModules.nix's systemd.tmpfiles.rules (only stateDir, .hermes, cron, sessions, logs, memories, plugins at 2770), so on a managed host it is always created lazily here and these lines were the only thing setting its mode. Confirmed absent: rg -n "pending_messages" nix/ returns nothing. That module runs the gateway with UMask = "0007" (nixosModules.nix:907) precisely so group members can share state.
• Unmanaged installs: unchanged at 0o700 under both umask 022 and umask 077.
• HERMES_HOME_MODE now applies (0701 yields 0o701 rather than 0o700). That is a deliberate consequence of replacing the hand-rolled os.chmod(flush_dir, 0o700) with hermes_cli.config._secure_dir, which every sibling HERMES_HOME subdir already goes through. Payloads remain 0o600 and the filenames are uuid4().hex, so an execute-only directory exposes nothing.
• The delegation also picks up _secure_dir's _chown_to_hermes_uid. That is a no-op unless HERMES_UID/HERMES_GID are set (_resolve_hermes_uid_gid returns (None, None) and _chown_to_hermes_uid returns early), so it adds no behavior on a normal install; where those are set (ensure_hermes_home() creates root-owned dirs in profile subdirectories when kanban workers are dispatched #34107 Docker deployments), matching the sibling dirs is the intended outcome.

tests/gateway/test_shutdown_flush.py is 6 → 16 tests:

$ ./scripts/run_tests.sh tests/gateway/test_shutdown_flush.py
=== Summary: 1 files, 16 tests passed, 0 failed (100% complete) in 11.3s (24 workers) ===

Two things worth flagging rather than leaving implicit:

• No workflow run has ever executed on this PR — the check-suites sit at conclusion=action_required, which is this repo's approval gate for contributors without a merged PR, not a failure. Everything above is local evidence only. If a maintainer approves the runs, CI can speak for itself.
• _managed_install() is duplicated across this PR and fix(security): create plaintext transcript artifacts owner-only #77520 (and fix(security): create browser-profile and media-cache artifacts owner-only #77579). Each branch carried its own copy so it could merge independently of the others — that's the honest reason, not a design argument. Given the "extend, don't duplicate" line in AGENTS.md, consolidating onto one shared helper once the first of them lands is a legitimate follow-up, and I'm happy to do it in whichever order you prefer.

This branch merges cleanly into current upstream/main (verified by local test-merge; the mergeable: UNKNOWN GitHub reports is staleness, and BLOCKED is the review-approval gate rather than a conflict), so no rebase is needed unless you want one.