Skip to content

fix(utils): EXDEV/EBUSY copy fallback in atomic_replace (port from gemini-cli#21541) - #43852

Merged
teknium1 merged 1 commit into
mainfrom
gemini-cli-port/atomic-replace-ebusy-fallback
Jun 13, 2026
Merged

fix(utils): EXDEV/EBUSY copy fallback in atomic_replace (port from gemini-cli#21541)#43852
teknium1 merged 1 commit into
mainfrom
gemini-cli-port/atomic-replace-ebusy-fallback

Conversation

@teknium1

Copy link
Copy Markdown
Contributor

Summary

atomic_replace() now falls back to copy + fsync + unlink when os.replace() fails with EXDEV (cross-device link) or EBUSY (busy bind mount), instead of crashing every atomic write site.

Port of the rename-fallback pattern from google-gemini/gemini-cli#21541 (their policy-file persistence hit the same failure class on Docker bind mounts / Linux aarch64).

Root cause: atomic_replace resolves symlinked targets to their real path (#16743 symlink preservation) before renaming. When a managed deployment symlinks ~/.hermes/config.yaml / auth.json / jobs.json to a file on a different filesystem (Docker bind mount, NFS/SMB share, dotfiles repo on another partition), the rename crosses devices and raises OSError(EXDEV) — reproduced E2E on current main with a /dev/shm-backed symlink target. Every caller of atomic_replace / atomic_json_write / atomic_yaml_write (~24 sites: config, auth, cron jobs, webhooks, model catalog, skills sync, …) is affected.

Changes

  • utils.py: catch EXDEV/EBUSY from os.replace, fall back to shutil.copyfile + copymode + fsync + unlink. All other OSErrors still propagate. Fallback is non-atomic but strictly better than failing the write and leaking the temp file.
  • tests/test_atomic_replace_symlinks.py: 5 new tests — EXDEV/EBUSY fallback (parametrized), symlink preserved on the fallback path, non-EXDEV errors propagate untouched, and a real-filesystem E2E using a tmpfs (/dev/shm) symlink target.

Validation

Before After
atomic_replace onto cross-device symlink OSError: [Errno 18] Invalid cross-device link content lands, symlink + mode preserved, temp cleaned
tests/test_atomic_replace_symlinks.py 8 passed 13 passed
Non-EXDEV OSError (EACCES) raises still raises, target untouched

Source PR: google-gemini/gemini-cli#21541

@github-actions

github-actions Bot commented Jun 11, 2026

Copy link
Copy Markdown
Contributor

🔎 Lint report: gemini-cli-port/atomic-replace-ebusy-fallback vs origin/main

ruff

Total: 0 on HEAD, 0 on base (➖ 0)

🆕 New issues: none

✅ Fixed issues: none

Unchanged: 0 pre-existing issues carried over.

ty (type checker)

Total: 10878 on HEAD, 10878 on base (➖ 0)

🆕 New issues: none

✅ Fixed issues: none

Unchanged: 5709 pre-existing issues carried over.

Diagnostics are surfaced as warnings — this check never fails the build.

@liuhao1024

Copy link
Copy Markdown
Contributor

Verification review — reviewed the full diff (2 files, +157/-1).

Clean port of the EXDEV/EBUSY fallback from gemini-cli#21541:

  1. Error scoping — only EXDEV (cross-device) and EBUSY (bind-mount busy) trigger the fallback; other OSError codes propagate unchanged. This is the correct narrow scope.

  2. Fallback sequenceshutil.copyfileshutil.copymodeos.fsyncos.unlink. The fsync ensures data is durable before the temp file is removed. Permission copy failures are caught and ignored (correct — the file is still readable).

  3. Symlink preservation — the fallback writes to real_path (the resolved target), so symlinked configs survive cross-device writes. This maintains the [Bug]: atomic writes to HERMES_HOME files replace symlinked targets (config.yaml/SOUL.md) #16743 invariant.

  4. Test coverage — parameterized EXDEV/EBUSY, symlink preservation, other-OSError propagation, and a real cross-device E2E test using /dev/shm vs tmpdir.

No issues found. Clean utility fix.

@alt-glitch alt-glitch added type/bug Something isn't working P2 Medium — degraded but workaround exists comp/cli CLI entry point, hermes_cli/, setup wizard area/config Config system, migrations, profiles labels Jun 11, 2026
Fallback from `os.replace` on EXDEV/EBUSY using copy+fsync+unlink while preserving symlink target semantics and metadata.
@teknium1
teknium1 force-pushed the gemini-cli-port/atomic-replace-ebusy-fallback branch from 4c57760 to 3cc8909 Compare June 13, 2026 13:52
@teknium1

Copy link
Copy Markdown
Contributor Author

Refreshed against current origin/main and force-pushed a cleaner salvage commit.

Cleanup:

  • fallback only for EXDEV / EBUSY; other OSErrors still propagate
  • preserves symlink target semantics through resolved real path handling
  • uses copy + copystat + fsync + unlink for the fallback path
  • focused test coverage for symlink, metadata, non-fallback errors, and real cross-device behavior when /dev/shm is available

Validation:

  • python3 -m py_compile utils.py tests/test_atomic_replace_symlinks.py -> passed
  • pytest -q tests/test_atomic_replace_symlinks.py -> 14 passed
  • python3 -m ruff check utils.py tests/test_atomic_replace_symlinks.py -> passed

@teknium1
teknium1 merged commit bf8effa into main Jun 13, 2026
28 checks passed
@teknium1
teknium1 deleted the gemini-cli-port/atomic-replace-ebusy-fallback branch June 13, 2026 21:50
AIalliAI pushed a commit to AIalliAI/Hermes that referenced this pull request Jun 14, 2026
…arch#43852)

Fallback from `os.replace` on EXDEV/EBUSY using copy+fsync+unlink while preserving symlink target semantics and metadata.
T02200059 pushed a commit to T02200059/hermes-agent that referenced this pull request Jun 18, 2026
…arch#43852)

Fallback from `os.replace` on EXDEV/EBUSY using copy+fsync+unlink while preserving symlink target semantics and metadata.
waefrebeorn pushed a commit to waefrebeorn/slermes that referenced this pull request Jul 2, 2026
…arch#43852)

Fallback from `os.replace` on EXDEV/EBUSY using copy+fsync+unlink while preserving symlink target semantics and metadata.
Methodician added a commit to Methodician/hermes-agent that referenced this pull request Jul 4, 2026
…arch#43852)

Fallback from `os.replace` on EXDEV/EBUSY using copy+fsync+unlink while preserving symlink target semantics and metadata.
habarmc1223-sudo pushed a commit to habarmc1223-sudo/hermes-agent-fluxmem that referenced this pull request Jul 8, 2026
…arch#43852)

Fallback from `os.replace` on EXDEV/EBUSY using copy+fsync+unlink while preserving symlink target semantics and metadata.
santhreal pushed a commit to santhreal/hermes-agent that referenced this pull request Jul 13, 2026
…arch#43852)

Fallback from `os.replace` on EXDEV/EBUSY using copy+fsync+unlink while preserving symlink target semantics and metadata.
Gravezzz pushed a commit to Gravezzz/hermes-agent that referenced this pull request Jul 21, 2026
…arch#43852)

Fallback from `os.replace` on EXDEV/EBUSY using copy+fsync+unlink while preserving symlink target semantics and metadata.
leewenjie pushed a commit to leewenjie/hermes-agent that referenced this pull request Aug 7, 2026
…arch#43852)

Fallback from `os.replace` on EXDEV/EBUSY using copy+fsync+unlink while preserving symlink target semantics and metadata.
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

area/config Config system, migrations, profiles comp/cli CLI entry point, hermes_cli/, setup wizard P2 Medium — degraded but workaround exists type/bug Something isn't working

Projects

None yet

Development

Successfully merging this pull request may close these issues.

3 participants