Skip to content

fix(cli): stop profile rename widening Honcho credential mode and detaching a symlinked config - #81296

Open
briandevans wants to merge 6 commits into
NousResearch:mainfrom
briandevans:fix/cli-honcho-profile-rename-atomic-write
Open

fix(cli): stop profile rename widening Honcho credential mode and detaching a symlinked config#81296
briandevans wants to merge 6 commits into
NousResearch:mainfrom
briandevans:fix/cli-honcho-profile-rename-atomic-write

Conversation

@briandevans

Copy link
Copy Markdown
Contributor

What does this PR do?

hermes profile rename calls _migrate_honcho_profile_host to move the profile's Honcho host block to its new key. That function is the last writer of the Honcho config files that does not go through the shared atomic helper — it rewrote them with a bare tmp.write_text(...) followed by tmp.replace(path).

That matters because of what those files hold. Its three candidate paths are:

candidates = [
    new_dir / "honcho.json",
    _get_default_hermes_home() / "honcho.json",
    Path.home() / ".honcho" / "config.json",
]

Each host block carries apiKey, which plugins/memory/honcho/README.md documents as the auto-refreshing access token under a browser OAuth grant, and the third candidate is the global ~/.honcho/config.json that plugins/memory/honcho/client.py documents as "global, shared across all Honcho-enabled apps". So a profile rename rewrites a live credential file, including one shared outside Hermes.

The old write carried three defects:

  1. The credential mode was widened. The temp file is created at the process umask — typically 0644 — and Path.replace carries that mode onto the target. A honcho.json the user (or plugins/memory/honcho/oauth.py) had left at 0600 came back group- and world-readable after a rename, with no output saying so.
  2. A symlinked config was detached. Path.replace replaces the symlink with a regular file, so a deployment that symlinks honcho.json out to a dotfiles repo or a profile package silently stopped tracking it — the same failure mode atomic_replace was introduced to fix in [Bug]: atomic writes to HERMES_HOME files replace symlinked targets (config.yaml/SOUL.md) #16743.
  3. No fsync. A crash between the rename and the flush can leave a truncated credential config, which the reader at the top of this same loop then discards as unparseable.

The fix is one substitution:

try:
    atomic_json_write(path, raw, mode=0o600)
except OSError:
    continue

utils.atomic_json_write fchmods the descriptor before the replace (so there is no chmod-after-write TOCTOU window on a secret-bearing file), flushes and fsyncs, and replaces through atomic_replace, whose comment reads "Preserve symlinks — swap in-place on the real file (GitHub #16743)". One call closes all three defects. The fail-soft except OSError: continue is kept so the loop still advances to the remaining candidate paths, and the helper's own cleanup replaces the temp-file unlink the old inline handler did by hand.

mode=0o600 is this file's own established convention, not new policy. Every other writer of these exact files already enforces it:

Writer How
plugins/memory/honcho/__init__.py atomic_json_write(config_path, existing, mode=0o600)
plugins/memory/honcho/cli.py atomic_json_write(path, cfg, mode=0o600)
plugins/memory/honcho/oauth.py os.open(tmp, ..., 0o600)
cli.py os.chmod(config_path, 0o600)
hermes_cli/backup.py os.chmod(target, 0o600) when restoring external provider configs
hermes_cli/profiles.py nothing — this PR

One intentional byte-level difference: the helper does not append a trailing newline, where the old inline write did. This makes the rename path byte-consistent with the two Honcho plugin writers that already use atomic_json_write, and nothing reads these files other than json.loads.

Coverage

I swept every writer of the Honcho config files rather than fixing only the reported one. hermes_cli/memory_oauth.py only resolves and reads them; hermes_cli/backup.py already chmods 0600 on restore; the three plugin writers are listed above. hermes_cli/profiles.py::_migrate_honcho_profile_host was the only site missing the enforcement, so this is the complete set for this root cause. This PR deliberately does not widen into a mechanical repo-wide chmod-after-write sweep — those are different files with different owners and sensitivities, and several are already correct or already claimed.

Precedent

  • The in-file precedent is a commit already on main: 649ce1f8113 "fix(cli): make profile.yaml and skin writes atomic to stop silent field loss" routed this same file's profile.yaml write through atomic_yaml_write. This change follows it exactly, including the lazy from utils import ... idiom the module uses to stay import-light.
  • fix(cli): route the remaining destructive user-file rewrites through atomic writes (salvage #79323) #79746 "fix(cli): route the remaining destructive user-file rewrites through atomic writes" merged on 2026-08-05 and covered profile_distribution.py, uninstall.py, web_routers/profiles.py, xai_retirement.py, and utils.py. hermes_cli/profiles.py was not in that list, so this is a remaining site of a class that has already been accepted.
  • utils.atomic_write_text's own docstring states the invariant this restores: "every destructive file rewrite in the codebase shares one implementation."

Related Issue

No filed issue. #16743 is referenced as context for the symlink-detachment failure mode only; it is already closed and this PR does not reopen or claim it.

Type of Change

  • 🐛 Bug fix (non-breaking change that fixes an issue)
  • ✨ New feature (non-breaking change that adds functionality)
  • 🔒 Security fix
  • 📝 Documentation update
  • ✅ Tests (adding or improving test coverage)
  • ♻️ Refactor (no behavior change)
  • 🎯 New skill (bundled or hub)

Changes Made

hermes_cli/profiles.py

  • _migrate_honcho_profile_host now writes through utils.atomic_json_write(path, raw, mode=0o600). The import is function-local at the top of the function, matching the module's existing lazy-import idiom (import yaml, from utils import atomic_yaml_write) so the module stays import-light; it is hoisted above the candidate loop rather than sitting inside it.
  • The inline temp-file handler is removed — the helper cleans up its own temp file on failure. The except OSError: continue fail-soft is unchanged, so an unwritable candidate still lets the remaining candidates migrate.
  • indent is not passed: the helper already defaults to 2, matching the previous output. ensure_ascii=False is not passed either — the helper hardcodes it internally, and forwarding it through **dump_kwargs would raise TypeError: got multiple values for keyword argument 'ensure_ascii'.

tests/hermes_cli/test_profiles.py

  • New TestMigrateHonchoProfileHostWrite class, placed directly after the existing TestRenameProfile cluster. Nothing else in the file is reordered or reformatted.
  • test_rewrite_keeps_credential_config_owner_only — seeds a 0600 config holding an apiKey, pins umask to 0o022 so the pre-fix behaviour is deterministic rather than dependent on the runner's umask, and asserts the mode is still 0600 afterwards.
  • test_rewrite_preserves_a_symlinked_config — points ~/.hermes/honcho.json at a file in another directory and asserts the path is still a symlink, still resolves to the same real file, that the real file received the migrated block, and that its mode survived.
  • test_unwritable_candidate_still_advances_to_the_next — forces OSError on the first candidate and asserts the second is still migrated and no temp file is left behind. This is the guard on the unchanged fail-soft behaviour.
  • The two mode/symlink tests are skipif-gated on sys.platform == "win32". No new imports were added to the file.

How to Test

Reproduce the mode widening on a POSIX box:

export HERMES_HOME=~/.hermes
hermes profile create oldname
cat > ~/.hermes/honcho.json <<'JSON'
{"hosts": {"hermes_oldname": {"apiKey": "secret", "aiPeer": "oldname", "enabled": true}}}
JSON
chmod 600 ~/.hermes/honcho.json
ln -s ~/dotfiles/honcho.json ~/.hermes/honcho.json   # optional, for defect 2

hermes profile rename oldname newname
stat -c '%a %N' ~/.hermes/honcho.json

Before this change: 644, and the symlink is gone. After: 600, and the symlink still points at the real file.

Automated, with the before/after both verified:

uv run --with pytest --with pytest-xdist --with pytest-asyncio python3 -m pytest \
  tests/hermes_cli/test_profiles.py -k TestMigrateHonchoProfileHostWrite -v

With the production hunk reverted and the tests kept, all three fail on exactly the defects described:

AssertionError: assert (33188 & 511) == 384        # 0o100644 -> mode 0o644, expected 0o600
AssertionError: assert False                        # link.is_symlink() -- the symlink was replaced
AssertionError: assert 'hermes_oldname' in {'hermes_newname': ...}   # fail-soft skip
3 failed, 48 deselected

With the fix restored:

tests/hermes_cli/test_profiles.py ................................................... 51 passed

Adjacent helper and consumer suites, unchanged:

tests/hermes_cli/test_profiles.py tests/hermes_cli/test_atomic_json_write.py
tests/hermes_cli/test_atomic_yaml_write.py tests/hermes_cli/test_update_zip_atomic_replace.py
  -> 58 passed

tests/honcho_plugin -> 216 passed, 12 skipped

Checklist

Code

  • I've read the Contributing Guide
  • My commit messages follow Conventional Commits (fix(scope):, feat(scope):, etc.)
  • I searched for existing PRs to make sure this isn't a duplicate
  • My PR contains only changes related to this fix/feature (no unrelated commits)
  • I've run pytest tests/ -q and all tests pass
  • I've added tests for my changes (required for bug fixes, strongly encouraged for features)
  • I've tested on my platform: macOS 15 (Darwin 25.4). Not verified on Linux or Windows; the two mode/symlink tests are skipif-gated off Windows.

Documentation & Housekeeping

  • I've updated relevant documentation (README, docs/, docstrings) — or N/A
  • I've updated cli-config.yaml.example if I added/changed config keys — or N/A
  • I've updated CONTRIBUTING.md or AGENTS.md if I changed architecture or workflows — or N/A
  • I've considered cross-platform impact (Windows, macOS) per the compatibility guide — or N/A
  • I've updated tool descriptions/schemas if I changed tool behavior — or N/A

…aching a symlinked config

`_migrate_honcho_profile_host` is the last writer of the Honcho config
files that does not go through the shared atomic helper. It rewrote them
with a bare `tmp.write_text(...)` followed by `tmp.replace(path)`, which
carries three defects into files that hold the Honcho `apiKey` — under a
browser OAuth grant that key is the auto-refreshing access token, and one
of the three candidate paths is the global `~/.honcho/config.json` shared
with every other Honcho-enabled app:

1. The temp file is created at the process umask (typically 0644) and
   `Path.replace` carries that mode onto the target, so renaming a profile
   silently widened a 0600 credential file to group/world-readable.
2. `Path.replace` swaps the symlink itself, detaching a `honcho.json` that
   a managed deployment symlinks out to a dotfiles or profile package.
3. There is no fsync, so a crash after the rename can leave a truncated
   credential config.

Routing the write through `utils.atomic_json_write(path, raw, mode=0o600)`
closes all three: the helper fchmods the descriptor before the replace
(no chmod-after-write TOCTOU), flushes and fsyncs, and replaces via
`atomic_replace`, which resolves a symlinked target and swaps in place on
the real file. The fail-soft `except OSError: continue` is preserved so
the loop still advances to the next candidate path, and the helper's own
cleanup removes the temp file that the old inline handler unlinked by hand.

`mode=0o600` matches the convention already established for these exact
files by every other writer: `plugins/memory/honcho/__init__.py` and
`plugins/memory/honcho/cli.py` both call `atomic_json_write(..., mode=0o600)`,
`plugins/memory/honcho/oauth.py` opens with 0600, and `cli.py` chmods 0600
after writing. This is not new policy, it is the one site that missed it.
Copilot AI lite review requested due to automatic review settings August 7, 2026 19:48

Copilot AI left a comment

Copy link
Copy Markdown
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Pull request overview

This PR hardens hermes profile rename by fixing _migrate_honcho_profile_host to rewrite Honcho credential/config JSON via the shared atomic JSON writer, preventing permission widening, symlink detachment, and truncated writes for files that may contain Honcho OAuth access tokens (apiKey).

Changes:

  • Switch Honcho host-block migration writes to utils.atomic_json_write(..., mode=0o600) to preserve owner-only permissions and symlink semantics while also adding fsync durability.
  • Remove the bespoke temp-file write/replace logic from _migrate_honcho_profile_host, keeping the existing fail-soft “try next candidate” behavior on OSError.
  • Add focused regression tests covering mode preservation, symlink preservation, and advancing to later candidates after an OSError.

Reviewed changes

Copilot reviewed 2 out of 2 changed files in this pull request and generated no comments.

File Description
hermes_cli/profiles.py Routes Honcho config rewrites through atomic_json_write(mode=0o600) to preserve secret-file permissions, symlinks, and durability during profile renames.
tests/hermes_cli/test_profiles.py Adds regression tests ensuring the migration rewrite preserves 0600, preserves symlinks, and keeps fail-soft candidate advancement.

💡 Add Copilot custom instructions for smarter, more guided reviews. Learn how to get started.

@alt-glitch alt-glitch added type/security Security vulnerability or hardening comp/cli CLI entry point, hermes_cli/, setup wizard area/auth Authentication, OAuth, credential pools area/profiles Multi-profile isolation, HERMES_HOME scoping P2 Medium — degraded but workaround exists sweeper:risk-security-boundary Sweeper risk: may affect sandboxing, auth, credentials, or sensitive data labels Aug 7, 2026
@briandevans

Copy link
Copy Markdown
Contributor Author

CI audit — the single test failure on this PR is a known intermittent on main, already filed as #75955. Zero failures are in touched code.

Test Symptom Root cause on main
tests/test_tui_gateway_server.py::test_write_json_serializes_concurrent_writes (slice 3/12) assert len(lines) == 8assert 9 == 8 #75955, open since 2026-08-01: two earlier profile-scoped agent-build tests wait only on the built event, so their daemon threads can still be emitting session.info when this test patches the module-global _real_stdout. The stale frame lands in this test's buffer as a ninth line. The issue records this exact signature — "the concurrent-writer test's assert len(lines) == 8 receiving a ninth complete JSON frame" — and notes the production stdio transport still serializes correctly.

This PR cannot reach that test. It changes exactly two files — hermes_cli/profiles.py (the body of _migrate_honcho_profile_host) and tests/hermes_cli/test_profiles.py — and touches neither tui_gateway/ nor utils.py, so there is no import path from the diff to server.write_json or to the _real_stdout patching the failing test depends on. The failure mode is thread interleaving of captured stdout, which nothing in a Honcho-config write can influence.

Verification run against this branch, where tests/test_tui_gateway_server.py is byte-identical to origin/main (005421d888a):

pytest tests/test_tui_gateway_server.py::test_write_json_serializes_concurrent_writes
  -> 1 passed  (8 consecutive runs, no failures)

The suites this PR actually affects are green:

tests/hermes_cli/test_profiles.py                     -> 51 passed
  + test_atomic_json_write.py, test_atomic_yaml_write.py,
    test_update_zip_atomic_replace.py                 -> 58 passed
tests/honcho_plugin                                   -> 216 passed, 12 skipped

Happy to rebase once #75955 lands if that is easier than re-running the slice.

@egilewski

Copy link
Copy Markdown
Contributor

suggesting changes

The change correctly routes Honcho profile renames through the 0600, fsyncing, symlink-preserving JSON helper, and the PR-head regression coverage passes. One residual integrity gap remains: the helper's EXDEV/EBUSY fallback copies directly into the resolved credential file, so an interrupted or failed copy can truncate the existing OAuth-token configuration. Make credential writes use a same-filesystem rename or fail closed on a non-atomic fallback before merge.

  • [P2] Cross-device atomic_replace fallback can truncate Honcho credentials (utils.py:125)
    The new call routes Honcho JSON through atomic_json_write, but that helper resolves a symlink and, when os.replace raises EXDEV or EBUSY, calls shutil.copyfile(tmp_path, real_path) before copystat and fsync. The destination is therefore opened and truncated in place; a disk-full/write error or process crash can leave the prior file as partial JSON. A symlinked Honcho config on another mount or a busy bind mount can consequently lose its OAuth apiKey and break Honcho users, contrary to the claimed crash-safe invariant. The PR tests cover a successful forced fallback and a same-filesystem symlink, but not interruption or partial-copy failure.
    Remediation: Create the temporary file beside the resolved target and complete the credential update with a real rename; alternatively disable the EXDEV/EBUSY copy fallback for mode=0o600 writes and leave the old target untouched on those errors. Add a fault injection test that interrupts the fallback and verifies the original JSON and mode remain intact.

Security evidence:

  • trust boundary: Profile rename runs with the local user's filesystem authority, reads candidate Honcho JSON files in the renamed profile, Hermes home, and the shared Honcho directory, and writes files containing apiKey OAuth credentials. The helper is the write boundary; filesystem errors and symlink targets are untrusted inputs to that boundary.
  • source/sink/invariant: Validated profile names and host-key checks constrain the rewrite, while the PR's helper call fchmods the temporary file to 0600, flushes and fsyncs it, and preserves symlinks. The invariant is incomplete because the helper's EXDEV/EBUSY branch copies into the resolved credential target instead of replacing it atomically.
  • current-main reproduction: Current main still writes a sibling .tmp with Path.write_text and replaces the candidate path directly, so its source behavior retains the mode-widening and symlink-detachment defects. The PR-head helper path supplies the intended 0600 and link-preservation behavior.
  • PR-head or patch-replay validation: The bound PR head is checked out coherently. The changed profile function uses the shared atomic JSON helper, and the added tests exercise permission tightening, symlink retention, legacy host migration, and fail-soft candidate advancement.
  • positive/negative cases: Positive cases cover owner-only mode, a symlinked target, apiKey retention, aiPeer behavior, and advancement after a candidate write error. Negative guards cover missing or malformed host data and duplicate destination hosts; the non-atomic EXDEV/EBUSY fallback is the remaining negative path.
  • residual bypass search: The direct profile write sites now use the helper and no alternate Honcho rewrite was found. The helper's cross-device and busy-mount copy fallback remains a bypass of the atomic and crash-safe invariant for the symlinked credential configurations this PR intentionally adds.
  • reviewer validation: The full profile regression suite and the atomic JSON/symlink helper suites pass at the PR head, and the reviewed diff has no whitespace errors.

Signed: GPT-5.6-luna-max in Codex

@briandevans

Copy link
Copy Markdown
Contributor Author

On the atomic_replace truncation gap: it's real, and it's worth stating precisely where it lives.

The truncation is in utils.atomic_replace, not in this diff. When os.replace fails with EXDEV or EBUSY, the helper calls shutil.copyfile(tmp, real_path) — and copyfile opens its destination "wb", truncating the target before the first replacement byte exists. That was introduced by merged #43852 (bf8effad023) and is still unfixed on main: git log -S 'shutil.copyfile' -- utils.py returns that one commit. It reaches every atomic_replace caller, not just this one.

This PR's diff is hermes_cli/profiles.py plus its tests. It doesn't touch utils.py, so it isn't the origin of the gap, and closing it here would mean editing a shared writer from a profile-rename PR.

It's also still a strict improvement over main for the file it does touch. Before this PR, _migrate_honcho_profile_host wrote via Path.write_text to a sibling .tmp and then called tmp.replace(path) on the candidate path directly. That temp inherited the process umask, so a 0600 credential file came back 0644; Path.replace swapped a symlinked config for a regular file (#16743); and there was no fsync. Routing it through atomic_json_write(path, raw, mode=0o600) fixes all three. The residual cross-device window is strictly smaller than what was there before, not newly introduced by it.

I've filed #81384 to close the helper-level gap at the source. It stages the new content into the resolved target's own directory and renames from there, so the EXDEV path becomes a genuine same-filesystem os.replace; the in-place copy is kept only for an inode that cannot be renamed onto at all, and a failed copy now damages the staging temp instead of the target. Every atomic_replace caller, including this one, inherits that with no caller changes.

On CI: the single failure is tests/test_tui_gateway_server.py::test_write_json_serializes_concurrent_writes in slice 3/12 (assert 9 == 8; 1 failed, 2709 passed). That's the agent-build thread-leak flake tracked in #75955, which is still open. This diff touches neither tui_gateway/ nor utils.py, so it isn't causal here.

@egilewski

Copy link
Copy Markdown
Contributor

suggesting changes

The follow-up scope argument does not remove the blocker: this PR newly routes the symlinked credential path through atomic_replace(), which is what exposes the existing fallback to the real credential target. On current main, the migration replaces the symlink itself and leaves its target unchanged. On a clean replay of this patch onto current main, the migration instead follows the symlink; when I fault-injected EXDEV from os.replace() and then a partial copyfile() followed by ENOSPC, the symlink survived but the real credential file was left truncated to {"hosts":. The migration swallowed that OSError and continued, so the claimed crash-safe credential preservation does not hold for the newly supported symlink case.

PR #81384 addresses the helper-level fallback. Please either land and rebase onto that fix before this change, integrate the equivalent fail-safe behavior here, or make this credential migration fail closed without an in-place copy. A regression should force the fallback copy to fail after a partial write and assert that the real symlink target remains byte-for-byte unchanged.

Security evidence:

  • trust boundary: ~/.honcho/config.json carries API credentials and may be a symlink to a separately managed credential file
  • source/sink/invariant: the migration rewrites the configured host through atomic_json_write(); any failed replacement must leave the real credential target valid and unchanged
  • current-main reproduction: current main widens the regular-file mode in this environment and detaches a migrated symlink, but it leaves the symlink's real credential target unchanged
  • PR-head or patch-replay validation: the two-file patch replayed cleanly onto current main; 69 focused atomic-write and profile tests passed, while the injected fallback failure truncated the real target to {"hosts":
  • positive/negative cases: ordinary files retain mode 0600, and a successful symlink migration preserves the link and updates its target; the negative EXDEV plus partial-copy/ENOSPC case corrupts that target
  • residual bypass search: atomic_json_write() creates the temporary file beside the symlink, atomic_replace() resolves the real target, and its EXDEV/EBUSY branch uses an in-place shutil.copyfile() with no rollback
  • reviewer validation: exact-module provenance was checked at PR head and on the current-main replay; the focused suites and both diff checks passed, and the decisive fault probe reproduced on both trees containing the patch

Not checked:

  • CodeRabbit review

Signed: GPT-5.6-sol-xhigh in Codex

…tory

_migrate_honcho_profile_host already resolves each candidate to dedup the
loop, then hands the unresolved candidate to atomic_json_write. The helper
stages its temp with tempfile.mkstemp(dir=path.parent), so a symlinked
config is staged on the link's filesystem and renamed onto the real file's.

~/.honcho/config.json is routinely linked into a dotfiles repo, an encrypted
volume or a mounted secrets share. Where those are separate filesystems the
rename fails EXDEV, and the fallback copies straight onto the resolved
target with the destination opened "wb" -- emptying the file that holds the
Honcho apiKey (the auto-refreshing OAuth access token under a browser grant)
before a single replacement byte exists. A copy that then fails part-way
(ENOSPC, I/O error) leaves it truncated, and this loop's
`except OSError: continue` swallows the error, so the profile rename
finishes without ever reporting that the credentials were destroyed.

Passing the resolved target makes the staging directory the real file's own
directory, so the rename is same-filesystem by construction and EXDEV is
unreachable from this call site. The symlink survives because it is never
written through, and the user-facing output still names the candidate path.
…ailed replace

Models the filesystem boundary the way the kernel does: os.replace raises
EXDEV exactly when the staged temp is not in the destination's own
directory, so the staging directory alone decides whether the fallback is
reachable. With the rewrite staged beside the symlink it fires, and an
ENOSPC injected mid-copy leaves the credential file holding 8 bytes.

Two invariants, deliberately separate:

- the outcome -- the apiKey survives, the host migration applied, the mode
  is still 0600, the symlink still resolves to the real file, and no staged
  temp leaks into either directory;
- the construction -- every rename this rewrite issues stays inside one
  directory. That is what pins the fix itself, and it stays meaningful
  regardless of how the cross-device fallback is implemented.

Red on the parent commit with the target truncated to '{\n  "hos'.
@briandevans

Copy link
Copy Markdown
Contributor Author

Self-audit: this diff moved one exposure while fixing three. Fix pushed.

Swapping the hand-rolled tmp.write_text() + tmp.replace(path) for atomic_json_write(path, raw, mode=0o600) fixes the 0600 mode, the symlink detachment (#16743) and the missing fsync. But path here is the candidate, which may itself be a symlink, and the chain that follows is:

  1. atomic_json_write stages with tempfile.mkstemp(dir=str(path.parent), prefix=f".{path.stem}_", suffix=".tmp") — the parent of whatever path it is handed, i.e. the link's directory;
  2. atomic_replace then does real_path = os.path.realpath(target_str) if os.path.islink(target_str) else target_str;
  3. os.replace(tmp_str, real_path) — for a config symlinked onto another filesystem that is a cross-device rename, so it fails EXDEV;
  4. the fallback is shutil.copyfile(tmp_str, real_path), and copyfile opens its destination "wb"truncating the real file before a single replacement byte exists. If that copy then fails part-way (ENOSPC, I/O error) the file stays truncated, and this loop's except OSError: continue swallows the error, so the profile rename finishes and reports nothing.

The file on the receiving end holds the Honcho apiKey — the auto-refreshing OAuth access token under a browser grant — and one candidate is the global ~/.honcho/config.json shared with every other Honcho app.

Relative to main that is a regression, and I am not going to argue it away. On main, tmp.replace(path) targets the candidate: it detaches the symlink, which is the bug this PR set out to fix, but it leaves the real target byte-intact. This PR fixed the detachment and put the real file in the blast radius instead. atomic_write_text's own docstring states the invariant this helper family exists to hold — "Ensures the target file is never left in a partially-written state" — and on this path it did not hold.

Fix — head a8d769ed6fdda7629c4e68559972e11217dc6f4b

The loop already resolves each candidate for its dedup set, so the production change is one token:

-        atomic_json_write(path, raw, mode=0o600)
+        atomic_json_write(resolved, raw, mode=0o600)

The temp is then staged in the real target's own directory, so os.replace is same-filesystem by construction and EXDEV is unreachable from this call site — no fallback, no truncating copy. The symlink survives because it is never written through, which is a shorter route to the same user-visible outcome than preserving it by resolution. raw is still read through the link (identical bytes), and the ✓ Honcho host updated line still names the candidate path, so user-facing output is unchanged.

Two commits: bffed64a98e (production) and a8d769ed6fd (test).

Test — test_symlinked_config_on_another_filesystem_survives_a_failed_replace, tests/hermes_cli/test_profiles.py:642

It models the filesystem boundary the way the kernel does: os.replace raises EXDEV exactly when the staged temp is not in the destination's own directory, so the staging directory alone decides whether the fallback is reachable. An ENOSPC is injected mid-copy. On the parent commit it fails with the credential file truncated to '{\n "hos'; after the fix, os.replace is never asked to cross a boundary and the injected faults never fire at all.

It asserts the outcome — the apiKey survives, the host migration applied, the mode is still 0600, the symlink still resolves to the real file, no staged temp leaks into either directory — and, separately, the construction: every rename this rewrite issues stays inside one directory. The second assertion is what pins the fix itself, and it stays meaningful regardless of how the cross-device fallback is implemented.

Residual, stated plainly

  • EBUSY still reaches the in-place copyfile (bind mounts, a busy inode). That is deliberately out of scope here: git grep -E 'atomic_(json_write|write_text|yaml_write)\(' -- '*.py' returns 68 production call sites, and hardening them one at a time is the wrong shape — the truncating fallback belongs in utils.atomic_replace. fix(utils): stage the cross-device replace so a failed rename can't truncate the target #81384 fixes it there for all 68, staging the cross-device replace in the resolved target's own directory and finishing by rename, with EXDEV, partial-copy and ENOSPC fault injection asserting the target is untouched. Once that lands this call site is covered twice over. Keeping utils.py out of this PR is what stops the two overlapping.
  • The except OSError: resolved = path fallback a few lines above means that if Path.resolve() itself fails, the candidate is passed through and the pre-fix behaviour returns. Pre-existing and unchanged by this PR, but worth naming rather than leaving to be found.

@egilewski

Copy link
Copy Markdown
Contributor

suggesting changes

Please make the Path.resolve() failure path fail closed before writing. When resolution raises, _migrate_honcho_profile_host currently falls back to the symlink candidate, so the atomic writer can stage beside the link and copy into the real credential file; a partial copy can truncate the file containing the Honcho apiKey. Skip the candidate on resolution failure and retain a regression that verifies the old target bytes survive this fault.

Security evidence:

  • trust boundary: Profile rename rewrites Honcho JSON that can contain OAuth credentials, including symlinked files.
  • source/sink/invariant: The migration must preserve the prior credential bytes, owner-only mode, and symlink while moving only the selected host block.
  • current-main reproduction: Current main reproduces mode widening under a permissive umask and detaches a symlinked candidate during this rewrite.
  • PR-head or patch-replay validation: The focused PR replay passes the new mode, migration, symlink, cleanup, and candidate-advance checks.
  • positive/negative cases: Resolved targets pass the positive checks; a resolution error followed by a partial copy still leaves the real target truncated.
  • residual bypass search: Successful resolution removes the cross-device staging mismatch, but the resolution-exception fallback still reaches the destructive copy path.
  • reviewer validation: A fault-injected credential-free reproduction confirmed the truncated-target outcome, while duplicate-candidate and malformed-host guards were checked.

Not checked:

  • CodeRabbit review

Signed: GPT-5.6-sol-xhigh in Codex

The rewrite below is only safe when it is handed the resolved target. As
its own comment states, ``atomic_json_write`` stages its temp file in the
parent of whatever path it is given, so passing a symlink stages on the
link's filesystem while ``atomic_replace`` renames onto the real file's.
When those differ — a config linked into a dotfiles repo, an encrypted
volume, a mounted secrets share — the rename fails EXDEV and the copy
fallback opens the real file "wb", emptying the user's Honcho ``apiKey``
before a replacement byte exists.

The candidate loop broke that invariant on its own error path: when
``Path.resolve`` raised — ELOOP on a symlink cycle, ENAMETOOLONG, EACCES
on a parent directory — it fell back to the unresolved candidate and
handed that straight to the writer. So the exact input the invariant
forbids was reachable, and the ``except OSError: continue`` on the write
then swallowed the resulting failure, leaving the credential file
truncated with nothing reported to the user.

Fail closed instead: a candidate whose target cannot be determined is
skipped. This also repairs the dedup set, which degraded on the same
path — an unresolved entry can never match the resolved entry for the
same file, so a config reachable under two candidate paths could be
rewritten twice.
…hrough

Drives ``Path.resolve`` to ELOOP on the symlinked ``~/.honcho/config.json``
candidate — the case a symlink cycle, an over-long path or an EACCES parent
produces — with the cross-device rename and the "wb"-then-ENOSPC copy
fallback in place, i.e. the real deployment shape where the link points into
a dotfiles repo or a mounted secrets share.

Asserts the two things that matter: the pre-existing credential file is
byte-identical afterwards, and the writer was never handed the symlink. Both
fail without the skip — the copy fallback leaves the apiKey file holding
eight bytes — and the pre-existing tests cannot detect it, because none of
them make resolution itself fail.
@briandevans

Copy link
Copy Markdown
Contributor Author

The resolution-failure path now fails closed. Pushed.

The write in _migrate_honcho_profile_host carries a stated invariant: it must be handed the resolved target, because atomic_json_write stages with tempfile.mkstemp(dir=str(path.parent), ...) and atomic_replace then renames onto os.path.realpath(target). Give it a symlink and those two are different directories — different filesystems in the deployment this PR exists for — so os.replace fails EXDEV and the fallback shutil.copyfile(tmp, real_path) opens the credential file "wb".

The candidate loop broke that invariant on its own error path:

try:
    resolved = path.resolve()
except OSError:
    resolved = path          # ← the unresolved candidate, possibly the symlink
...
atomic_json_write(resolved, raw, mode=0o600)

Path.resolve raises on ELOOP (a symlink cycle), ENAMETOOLONG, or EACCES on a parent directory. On any of those the writer received exactly the input the invariant forbids, and the except OSError: continue on the write then swallowed the resulting failure — so the apiKey file was left truncated with nothing reported.

Fix: skip the candidate instead of falling back to it. Two consequences worth naming:

  • With the skip in place, atomic_replace at this call site only ever receives a path where realpath(target) == target, so the rename is same-directory by construction and the EXDEV/EBUSY fallback is unreachable from here. The helper-level gap in utils.atomic_replace is still real and still belongs in fix(utils): stage the cross-device replace so a failed rename can't truncate the target #81384; this call site no longer reaches it.
  • The seen dedup set is repaired by the same change. An unresolved entry can never match the resolved entry for the same file, so a config reachable under two candidate paths could previously be rewritten twice.

Commits: 10f8a080154 (fix) and a53055a3e69 (test), head a53055a3e69.

Regression, which fails without the skip:
tests/hermes_cli/test_profiles.py::TestMigrateHonchoProfileHostWrite::test_unresolvable_candidate_is_skipped_not_written_through

It drives Path.resolve to ELOOP on the symlinked ~/.honcho/config.json candidate with the cross-device rename and a partial-copy-then-ENOSPC fallback injected, then asserts the pre-existing credential bytes are byte-identical and that the writer was never handed the symlink. Without the skip the file is left holding eight bytes. Full file: 53 passed.

@egilewski

Copy link
Copy Markdown
Contributor

suggesting changes

The profile rename still has one credential-integrity failure. The new code correctly resolves symlink targets and makes cross-device staging same-directory, so the mode, symlink, resolution-failure, duplicate, and malformed-candidate cases now behave as intended. However, the shared writer still handles a busy target by copying the staged JSON directly over the resolved Honcho file. A partial copy can truncate the file containing apiKey, and the migration ignores that write error. On the replay, a forced busy replacement plus partial-copy failure left the symlink intact but changed the real target bytes. Please land the helper-level safe fallback or make this credential migration fail closed for that path, and retain a regression asserting byte-for-byte target preservation.

Security evidence:

  • trust boundary: Profile rename runs with filesystem authority over Honcho JSON that can contain OAuth credentials, including symlinked files.
  • source/sink/invariant: The migration must preserve the prior credential bytes, owner-only mode, and symlink while moving only the selected host block.
  • current-main reproduction: Current main widens the rewritten credential mode under a permissive umask and detaches a symlinked candidate; its busy fallback also permits partial in-place copy.
  • PR-head or patch-replay validation: The manager-owned replay passes the new mode, migration, symlink, resolution-failure, duplicate, malformed-candidate, and candidate-advance checks.
  • positive/negative cases: Resolved targets pass the positive checks; a forced busy replacement followed by partial copy changes the resolved credential target bytes.
  • residual bypass search: The changed call site removes the cross-device staging mismatch and skips unresolved candidates, but the shared EBUSY fallback remains reachable and non-atomic.
  • reviewer validation: Exact-module probes and the full profile suite passed on the replay; the decisive EBUSY plus partial-copy fault probe reproduced the target mutation.

Not checked:

  • CodeRabbit review

Signed: GPT-5.6-sol-xhigh in Codex

@briandevans

Copy link
Copy Markdown
Contributor Author

Scope note: the residual EBUSY / EXDEV truncation is real, and it is fixed in #81384 rather than here

Agreed on the residual, and I want to be precise about where it lives so this PR's boundary is legible.

Everything this PR is responsible for is caller-side and is in the diff: the rewrite now goes through atomic_json_write(resolved, raw, mode=0o600), so the mode no longer inherits the umask, the symlink is never written through, and an unresolvable candidate is skipped rather than written unresolved.

What remains is one line in the shared helper, utils._copy_fallback:

shutil.copyfile(tmp_str, real_path)

shutil.copyfile opens its destination "wb", so the target is truncated before the first replacement byte exists — which is exactly the "partial copy changes the resolved credential target bytes" outcome. That is not reachable from this call site alone. atomic_replace is the tail of five writers in utils.pyatomic_write_text, atomic_json_write, atomic_yaml_write, atomic_roundtrip_yaml_update, atomic_roundtrip_yaml_save — so a fix made local to _migrate_honcho_profile_host would be at the wrong layer and would leave the other four exposed on the same path.

That helper-level fix is #81384, which I rebased today onto current main — it is re-derived against the _rewrite_in_place / _copy_fallback split that landed in #84852, and it is MERGEABLE and green. It stages into the resolved target's own directory and renames from there, so the EXDEV/EBUSY branch becomes genuinely atomic instead of merely narrower, and a copy that fails partway unlinks only the staging temp and re-raises with the target's previous contents intact.

It carries the byte-for-byte preservation regression:
tests/test_atomic_replace_symlinks.py::test_atomic_replace_cross_device_never_truncates_target fails the copy the way a filling disk does and asserts the target's exact prior contents, its 0600 mode, the surviving symlink, and no leaked staging temp. On unmodified main a config.yaml holding provider: openrouter / api_key: keep-me comes back as the 8 bytes provider.

That pairing is also what makes this PR's except OSError: continue genuinely fail-closed. Today the OSError can arrive after the copy has already damaged the file, so skipping the candidate preserves nothing. With #81384 the raise arrives with the file untouched, and skipping is a real fail-closed outcome.

The invariant is already main's own, one function above the site: #84852 added _rewrite_in_place specifically to close this window on the Windows-contended branch, and says so — "Unlike shutil.copyfile this never truncates the target to zero first". It applied that reasoning to the contended arm and left the EXDEV/EBUSY arm on the truncating copy. #81384 finishes it, and the consequence there is strictly worse: on the contended arm the truncation is a transient window a concurrent reader may observe, while on this arm an interrupted copy leaves the file empty or partial on disk.

So I am deliberately not re-fixing it in this PR. Doing so would duplicate #81384 and put two open PRs on one root cause in one shared helper, which is harder to review and harder to land than either on its own. This PR stands on the caller-side defects; #81384 stands on the helper. They compose in either merge order, and neither depends on the other to be correct.

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

area/auth Authentication, OAuth, credential pools area/profiles Multi-profile isolation, HERMES_HOME scoping comp/cli CLI entry point, hermes_cli/, setup wizard P2 Medium — degraded but workaround exists sweeper:risk-security-boundary Sweeper risk: may affect sandboxing, auth, credentials, or sensitive data type/security Security vulnerability or hardening

Projects

None yet

Development

Successfully merging this pull request may close these issues.

4 participants