Skip to content

fix(utils): handle EXDEV in atomic_replace for cross-filesystem symlinks - #36856

Closed
maxmilian wants to merge 1 commit into
NousResearch:mainfrom
maxmilian:fix/atomic-replace-exdev-fallback
Closed

fix(utils): handle EXDEV in atomic_replace for cross-filesystem symlinks#36856
maxmilian wants to merge 1 commit into
NousResearch:mainfrom
maxmilian:fix/atomic-replace-exdev-fallback

Conversation

@maxmilian

@maxmilian maxmilian commented Jun 1, 2026

Copy link
Copy Markdown
Contributor

Summary

atomic_replace (used by every atomic_json_write / atomic_yaml_write site) failed with OSError: [Errno 18] EXDEV when the target was a symlink whose real file lives on a different filesystem — e.g. a managed deployment that symlinks ~/.hermes/config.yaml / SOUL.md / auth.json to a git-tracked profile package on another mount. Every atomic write to such a target blew up.

Fixes #36653.

The bug

atomic_*_write stages the temp file in path.parent (next to the symlink, on filesystem A). atomic_replace then resolves the symlink to its real target (on filesystem B) and calls os.replace(tmp, real_path) — which can't rename across devices, so it raises EXDEV. The non-symlink path is unaffected (temp and target share a directory, hence a filesystem).

Changes

  • utils.py: atomic_replace now catches EXDEV and delegates to a new _replace_across_devices helper, which re-stages the bytes on the target's own filesystem (copy + fsync + os.replace within that fs) so the swap is still a real atomic rename. The original cross-device temp is always cleaned up. Non-EXDEV OSErrors propagate unchanged.

Tests

  • tests/test_atomic_replace_symlinks.py — 3 new regression tests (11 total pass):
    • cross-device symlink fallback preserves the symlink and writes the real file
    • atomic_json_write survives a cross-device symlink target
    • non-EXDEV OSError is re-raised and leaves the target untouched
  • Existing 8 symlink tests + test_atomic_json_write.py / test_atomic_yaml_write.py (17) still pass.
  • ruff check (PLW1514) clean; scripts/check-windows-footguns.py clean (binary mode + errno.EXDEV, cross-platform safe).

Scope

  • Only atomic_replace changes; no caller signatures touched. Permission-restore (_restore_file_mode) is unaffected — it still targets the returned real path.
  • EXDEV is simulated via mock (the issue's suggested approach) rather than a real second mount, so the test runs anywhere.
  • EXDEV detection keys on the POSIX errno.EXDEV semantics; on Windows the prior behavior is preserved unchanged (the fallback simply may not trigger for cross-volume moves).

When a managed deployment symlinks ~/.hermes/config.yaml (or SOUL.md /
auth.json) to a file on a different filesystem, atomic_replace resolved the
symlink to a real path on the other device while the temp file stayed staged
next to the symlink. os.replace cannot rename across devices, so it raised
EXDEV and broke every atomic write to that target.

atomic_replace now catches EXDEV and re-stages the bytes on the target's own
filesystem (copy + fsync + os.replace within that fs) before replacing there,
keeping the write atomic. Non-EXDEV OSErrors still propagate untouched.

Regression tests (5) in test_atomic_replace_symlinks.py: cross-device symlink
fallback, atomic_json_write over a cross-device symlink, staged temp lands on
the target filesystem, copy-failure leaves the target intact with no leak, and
non-EXDEV OSErrors are not swallowed.

Fixes NousResearch#36653

Co-Authored-By: Claude Opus 4.8 (1M context) <noreply@anthropic.com>
@maxmilian
maxmilian force-pushed the fix/atomic-replace-exdev-fallback branch from c013597 to a9ffb6b Compare June 1, 2026 15:04
@alt-glitch alt-glitch added type/bug Something isn't working P2 Medium — degraded but workaround exists labels Jun 1, 2026
@maxmilian
maxmilian marked this pull request as ready for review June 1, 2026 15:28
@maxmilian

Copy link
Copy Markdown
Contributor Author

Closing — atomic_replace in utils.py on main already handles this: it resolves symlinks first and falls back to copy/fsync/unlink on EXDEV/EBUSY for cross-filesystem/bind-mount deployments. The gap this PR targeted is covered upstream.

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

P2 Medium — degraded but workaround exists type/bug Something isn't working

Projects

None yet

Development

Successfully merging this pull request may close these issues.

Symlink-preserving atomic writes can fail across filesystems

2 participants