fix(security): create plaintext transcript artifacts owner-only

[ZHJay]

What does this PR do?

/save snapshots, agent trajectories, and MoA traces each persist verbatim conversation content — message text, tool results, tool-call arguments — through a bare open() / json.dump(). Permissions therefore came from the process umask, i.e. 0o644 on a default install.

Trajectories are the sharpest case: save_trajectory() appends to the CWD, not to the 0o700 HERMES_HOME. A run inside a checkout drops a world-readable full transcript next to the source, and neither filename was gitignored.

Scoped to file modes only. Content is deliberately not redacted:

• These artifacts export the same history the session DB replays, and masking a credential in a replayed path poisons the replay — that's Passwords get replaced by *** but model reads back its own conversation history and fails on second tool call. #43083, guarded by tests/agent/test_tool_call_arg_no_redaction.py.
• Trajectories (training data) and MoA traces (offline end-to-end audit) exist to be full-fidelity; redacting them unconditionally would defeat the feature.
• Trace redaction is already available opt-in as moa.privacy_filter: display.

Related Issue

Refs #77472 (partial — the file-mode items only; the "exact-value redaction on every persistence path" ask is deliberately not implemented, for the reasons above)

Type of Change

• 🔒 Security fix

Changes Made

• utils.py — new open_private_append() for append-mode artifacts. Only the creating open applies the mode: a file the user deliberately relaxed (to share, or to feed a training pipeline) keeps its permissions. Mode is advisory on Windows, where at-rest protection is ACL-based.
• cli.py (save_conversation) — write the snapshot via atomic_json_write(..., mode=0o600). This also closes a truncated-file window: the old json.dump wrote in place, so a serialization failure mid-write left a partial snapshot behind. Now the temp file is discarded and the same (x_x) Failed to save: message is printed.
• agent/trajectory.py — create trajectory_samples.jsonl / failed_trajectories.jsonl owner-only.
• agent/moa_trace.py — create the trace JSONL owner-only and moa-traces/ as 0o700, except on a managed (NixOS) install, where the mode is omitted so the module's configured setgid + umask decide. See "Managed-mode carve-out" below.
• .gitignore — ignore both trajectory filenames.
• tests/test_transcript_artifact_file_modes.py — new.

How to Test

1. scripts/run_tests.sh tests/test_transcript_artifact_file_modes.py tests/hermes_cli/test_atomic_json_write.py — 14 tests (was 8: 5 + 3). They assert the contract (no group/other bits on a freshly created artifact) rather than freezing an octal, run under a deliberately permissive umask 0o022, and exercise the real write paths so a regression to umask-derived modes fails here.

2. Regression check on the paths touched: scripts/run_tests.sh tests/cli/test_save_conversation_location.py tests/agent/test_moa_trace_streamed_capture.py tests/test_trajectory_compressor.py tests/test_trajectory_compressor_async.py — 33 passed.

3. Shared-helper blast radius: scripts/run_tests.sh tests/hermes_cli/test_atomic_json_write.py tests/test_atomic_replace_symlinks.py tests/hermes_cli/test_atomic_yaml_write.py tests/test_stale_utils_module_import.py tests/test_utils_truthy_values.py — 22 passed. Plus scripts/run_tests.sh -k moa (2999 files) — exit 0.

4. Manual, real config gate (not monkeypatched) — temp HERMES_HOME, moa.save_traces: true, umask 022:

   resolved trace dir: <tmp>/.hermes/moa-traces
   dir  mode: 0o700
   file mode: 0o600
   full-fidelity input preserved: token sk-proj-aaaaaa ...
   

5. Manual /save: snapshot lands at 0o600; with non-serializable content it prints (x_x) Failed to save: Object of type Unserializable is not JSON serializable and leaves no partial or .tmp file behind.

Review follow-up (second commit): the /save test no longer purges
sys.modules before importing cli. The filter was m.startswith("cli"),
which also matched click and its 11 submodules and broke class identity for
the rest of the suite. It was unnecessary — the autouse _hermetic_environment
fixture (tests/conftest.py:441) already redirects HERMES_HOME,
get_hermes_home() re-reads the env var on every call with no caching, and
save_conversation resolves it at call time. Teeth check after the edit:
restoring the pre-fix open(path, "w") + json.dump makes
test_save_conversation_snapshot_owner_only fail with
hermes_conversation_*.json is group/other-accessible (0o644).

Pre-existing unrelated failure, present on clean main before this branch and unchanged by it: tests/agent/test_compression_concurrent_fork.py::test_fence_cancelled_compression_leaves_lock_reacquirable.

Managed-mode carve-out, and the mode= wiring gap (third commit)

A full-suite verification pass (2514 files / 23624 passed on this branch at
merge base 3572d4bca, 2026-08-01; 39 failures, all 39 reproducing
identically at that same merge base) found three defects in this PR's own
work. None of them were catchable by the suite as it stood. upstream/main
has since advanced past that base — 2551 raw tests/ files at 3572d4bca
versus 2630 today — so absolute counts from a re-run will not match these.

1. moa-traces/ was revoking NixOS group access. mode=0o700 was passed
to base.mkdir unconditionally, with no managed-mode carve-out.
nix/nixosModules.nix pre-creates only stateDir, .hermes, cron, sessions,
logs, memories, plugins (2770 — setgid, group-rwx) via systemd.tmpfiles;
moa-traces is not among them, so on a managed host it is created lazily
here and that mode was the only thing setting it. The same module runs the
gateway with UMask = "0007" specifically so "files created by the gateway
should be group-writable so interactive users in the hermes group can
read/write them", and avoids chown -R to keep the setgid bit alive "for group
access by hostUsers" — who get a ~/.hermes symlink to that same stateDir.
Gateway and interactive CLI therefore share one $HERMES_HOME, and a 0700
dir created by whichever ran first locked the other out with EACCES — the
tracing feature itself, not just its permissions.

ensure_hermes_home already branches on is_managed() at its own creation
site, and its logs/curator lazy mkdir deliberately lets the configured umask
decide inside an already-secured parent. This follows that precedent at the
creation site rather than only at reconciliation.

Measured on the real save_moa_turn path under a temp HERMES_HOME:

scenario	merge base	before this commit	after
unmanaged, umask 022	0o755	0o700	0o700
unmanaged, umask 077	0o700	0o700	0o700
managed, umask 0007, parent 2770, dir absent	0o770	0o700	0o770
managed, dir pre-existing 0750	0o750	0o750	0o750
HERMES_HOME_MODE=0701 (unmanaged)	0o755	0o700	0o700

The trace JSONL is 0o600 in every row after the fix. HERMES_HOME_MODE is
deliberately not honored: that hatch exists so a web server can traverse
HERMES_HOME to reach a served subdirectory, and nothing is served out of
moa-traces.

2. mode=0o600 in save_conversation was untested. Stripping it left
2514 files / 23624 tests green. mkstemp creates at 0600 and passing mode
forces original_mode = None so _restore_file_mode returns early — so on a
fresh file the argument does nothing, and /save's only test wrote a fresh
file:

target	no mode=	mode=0o600
fresh file	0o600	0o600 (identical — no-op)
existing 0o644	0o644	0o600 (the only case it changes)

What the argument buys is the overwrite path, where atomic_json_write
otherwise preserves the existing mode. That path is reachable: the snapshot
name has second resolution (hermes_conversation_%Y%m%d_%H%M%S.json), so two
/save calls in the same second land on the same file, and a 0644 snapshot
left by a pre-fix Hermes would keep 0644 while receiving a fresh full
transcript. Now tested at both levels — on the helper (covering every caller
that passes a mode, including a 0o640 case mkstemp cannot produce, so only
a real chmod reaches it) and on the /save wiring itself with a frozen
timestamp. Also names failed_trajectories.jsonl (the completed=False
branch), hardened by the same line as trajectory_samples.jsonl but previously
asserted only by implication.

3. Two comments failed measurement. "mode applies to dirs this call
creates" is false for intermediates: pathlib.mkdir(parents=True, mode=…)
applies the mode to the final component only, so a nested moa.trace_dir
override leaves intermediates at 0o755 (measured on shared/audit/moa: leaf
0o700, audit/ and shared/ 0o755). No security impact — the leaf stays
0o700 and the JSONL 0o600, so traversal into the traces is still blocked —
but the guarantee was overstated. In cli.py, 2770 is HERMES_HOME's managed
mode (0750 applies to ${stateDir}/home, a different path), HERMES_HOME_MODE
does not apply in managed mode at all since _secure_dir returns early, and
saved_dir is not 0755 "either way" — it measures 0o755 by default and
0o770 under the managed unit's UMask. Both load-bearing conclusions stand;
only the parenthetical octals were wrong.

sessions/saved is still 0o755 and is deliberately left for a follow-up —
it is a directory-mode issue, not a file mode, and out of this PR's scope.

Teeth, per mechanism. Each was reverted individually and confirmed to fail
with a diagnostic naming the cause, then restored byte-identically (checksums
verified):

reverted mechanism	result
unconditional mode=0o700 restored	managed test fails: "dropped group access (0o700) … locks the other out with EACCES"
mode= dropped entirely	unmanaged test fails: "moa-traces dir is group/other-accessible (0o755)"
mode=0o600 removed from /save	only the new overwrite test fails (0o644); the pre-existing fresh-file test stays green — the reported gap, reproduced
open_private_append → bare open	trajectory, failed-trajectory and moa-trace tests fail
both chmods disabled in atomic_json_write	0o640 assertion fails ("just mkstemp's 0o600 default surviving")
tmpfiles skeleton removed from the fixture	managed test fails on "must still create the trace dir" — the fixture is load-bearing, not decoration

The managed test reproduces all three real conditions together
(HERMES_MANAGED=nixos, parent 2770, umask 0007, dir absent) and stubs
nothing: a real config.yaml opens the gate, real load_config() reads it, and
real _traces_enabled_and_dir() derives the path. Setting only the env var
would pass for the wrong reason, since a default umask 022 yields 0755 and
satisfies "group access survived" by ambient umask rather than by the carve-out.

_managed_install() is a deliberate local helper: hermes_cli.config's
secure_mkdir (#77655) centralizes exactly this branch, but it is not on main
yet, so importing it would couple this PR's merge to that one. Consolidate when
it lands.

Verified: ruff 0.16.1 clean on all four changed files,
check-windows-footguns.py --diff upstream/main and check_subprocess_stdin.py
clean, 9-file helper regression set 67/67, and a 68-file sweep of
moa / save_conversation / managed-mode tests at 1397 passed / 1 failed — that
one (Linux-only systemd restart) reproducing identically at the untouched merge
base.

Precedent for this exact pattern

This isn't a new convention — gateway/shutdown_flush.py already writes a
secret-bearing artifact the same way:

• line 44: flush_dir.mkdir(parents=True, exist_ok=True, mode=0o700) (72024950c, "fix(gateway): harden shutdown message flush")
• line 70: atomic_json_write(final_path, payload, mode=0o600, default=str) (720cdd1d1, "refactor: use atomic_json_write instead of hand-rolled _write_payload")

Same two primitives, same modes, same reasoning: pending gateway messages are
verbatim user content, so the directory is created 0o700 and each payload
0o600. atomic_json_write's mode= parameter exists for precisely this
("avoiding chmod-after-write TOCTOU exposure for secret-bearing files" — its
own docstring). This PR applies the established pattern to three artifact
paths that were missed, plus one append-mode helper for the JSONL cases
atomic_json_write can't cover.

Why this isn't a reversal of #74897

#74897 moved write_file's new files off a hardcoded 0600 and onto
umask-derived 0644 (then #74918 hardened the arithmetic into chmod "=rw").
Both are already in this branch's base, so the direction looks opposite. It
isn't the same class of file:

	#74897 (write_file)	this PR
Who chose the path	the user	the agent (fixed filenames / HERMES_HOME)
What it is	a user document written on request	a transcript artifact emitted as a side effect
Who reads it	other processes, by design (Obsidian LiveSync, Docker volumes, NAS mounts)	the owner, replaying or auditing their own session
Cost of 0600	breaks a documented interop contract — the bug in #70856	none known

write_file is a general-purpose file writer; forcing 0600 there broke
cross-process readers, which is exactly why #74897 was right. These four
paths are agent-generated secret-bearing artifacts — save_trajectory()
appends a full transcript to the CWD under a fixed filename the user never
named. No cross-process reader is implied by that, and there was no interop
contract to break.

The two changes also agree on the invariant #74897 actually protected:
never override permissions the user set. #74897's regression guard is
"overwrite 0755 file → preserved". open_private_append() applies its mode
only on the creating os.open, so an existing file is never re-tightened —
asserted directly by
test_trajectory_existing_relaxed_file_is_not_retightened (writes the file,
chmods it 0644, appends, asserts still 0644). A user who widens a
trajectory to feed a training pipeline keeps that. The /save snapshot is a
fresh timestamped filename every time, so it has no pre-existing mode to
preserve.

Narrow reading: this PR only changes what mode a file is born with on paths
where the agent picked the name.

Checklist

Code

• I've read the Contributing Guide
• My commit messages follow Conventional Commits
• I searched for existing PRs to make sure this isn't a duplicate
• My PR contains only changes related to this fix
• I've run the relevant suites via scripts/run_tests.sh and they pass (see above)
• I've added tests for my changes
• I've tested on my platform: macOS 27.0 (Darwin 27.0.0, arm64), Python 3.11.15

Documentation & Housekeeping

• I've updated relevant documentation (docstrings on the new helper explain the create-only-mode contract) — behavior is unchanged for users, no user-facing doc edit needed
• cli-config.yaml.example — N/A (no config keys added; this is not env/config surface)
• CONTRIBUTING.md / AGENTS.md — N/A
• Cross-platform: POSIX mode bits are advisory on Windows (os.open honours only the read-only bit), so the tests are skipif(os.name != "posix") and the helper's docstring says so. Windows at-rest protection is ACL-based and is deliberately left to the sibling PR fix(security): enforce owner-only ACLs on Windows in _secure_file #77527 (enforce owner-only ACLs on Windows in _secure_file) rather than duplicated here — one Windows ACL implementation, in the shared helper, not two. No behavior change on any platform beyond the permission bits.

[Copilot]

Pull request overview

Hardens plaintext transcript artifacts (CLI /save snapshots, trajectory JSONL, and MoA trace JSONL) to be created with owner-only permissions to avoid umask-derived world-readable transcripts, aligning with Hermes’ full-fidelity persistence/replay requirements.

Changes:

• Added utils.open_private_append() to create new append-mode transcript artifacts as owner-only while preserving permissions on existing files.
• Updated /save, trajectory export, and MoA trace persistence paths to create files/dirs with private modes (0o600 files, 0o700 trace dir) and to write snapshots atomically.
• Added a POSIX-only test suite asserting the “no group/other bits on newly created artifacts” contract and ignored trajectory artifacts in .gitignore.

Reviewed changes

Copilot reviewed 5 out of 6 changed files in this pull request and generated 1 comment.

Show a summary per file

File	Description
utils.py	Adds open_private_append() helper for private-by-default append artifacts.
cli.py	Writes /save snapshots via atomic_json_write(..., mode=0o600) to enforce private perms + atomicity.
agent/trajectory.py	Uses open_private_append() for trajectory JSONL creation (owner-only on first create).
agent/moa_trace.py	Creates moa-traces/ as 0o700 and uses open_private_append() for trace JSONL.
.gitignore	Ignores trajectory JSONL artifacts that are written into the CWD.
tests/test_transcript_artifact_file_modes.py	Adds contract tests for artifact permissions (POSIX).

---

💡 Add Copilot custom instructions for smarter, more guided reviews. Learn how to get started.

[ZHJay]

Fixed in 3d831557d — the purge is gone entirely rather than narrowed.

The mechanism you flagged was real, and worse than order-dependence. Measured in this repo's venv:

modules matching startswith('cli'): 12
['click', 'click._compat', 'click._utils', 'click.core', 'click.decorators',
 'click.exceptions', 'click.formatting', 'click.globals', 'click.parser',
 'click.termui', 'click.types', 'click.utils']
identity preserved after purge: False

All 12 matches were click; zero were cli. The loop evicted only unrelated modules and never did the thing it was there for. Popping them mid-suite breaks class identity for anything already holding a reference — a reimported click.exceptions.UsageError is a different object, so a live except UsageError no longer catches it.

On the suggested narrowing (clear the home cache, pop only cli): also not needed, because nothing is resolved at import time and there is no cache.

• cli.py:8312 — saved_dir = get_hermes_home() / "sessions" / "saved" sits inside save_conversation, so it resolves per call, not at import.
• hermes_constants.py:114 get_hermes_home() → hermes_constants.py:62 _hermes_home_from_env() → hermes_constants.py:71 val = os.environ.get("HERMES_HOME", "").strip(). No lru_cache, no module-level memo. The old _hermes_home_cache reset was dead code — no such attribute exists on hermes_constants.
• tests/conftest.py:402 autouse _hermetic_environment already redirects HERMES_HOME per test (conftest.py:441).

So the local monkeypatch.setenv is sufficient however early cli was imported. The Path.home patch was redundant with it and went in the same commit.

Current state, tests/test_transcript_artifact_file_modes.py:198-203:

    # No sys.modules purge and no Path.home patch here: save_conversation
    # resolves the directory through get_hermes_home() at call time, and that
    # reads HERMES_HOME from the environment on every call with no caching, so
    # the setenv above is sufficient no matter how early cli was imported.
    # Purging modules by ``startswith("cli")`` would also evict click and its
    # submodules, breaking module identity for the rest of the suite.
    import cli

Green at branch head 3f34df1d5:

$ ./scripts/run_tests.sh tests/test_transcript_artifact_file_modes.py
=== Summary: 1 files, 8 tests passed, 0 failed (100% complete) in 0.4s (24 workers) ===

3d831557d also touches cli.py, but only the comment block above the atomic_json_write call — the four file-mode fixes are untouched, so there is no production behavior change in that commit.