fix(utils): retry transient atomic replace permission errors

[zapabob]

Summary

• retry transient PermissionError failures around os.replace in atomic_replace
• keep the retry bounded so real permission failures still surface
• add a regression test for transient replace failures

Why

On Windows, antivirus/indexers or another rapid writer can briefly hold the target path while atomic_json_write is replacing it. That can make otherwise-safe concurrent writes fail with PermissionError even though retrying moments later succeeds.

Validation

• python -m pytest --timeout-method=thread tests/hermes_cli/test_atomic_json_write.py -q
• python -m ruff check utils.py tests/hermes_cli/test_atomic_json_write.py

[teknium1]

Thanks for addressing a real Windows atomic-write failure mode. Current main still performs a single os.replace at utils.py:115, while its handler only recognizes EXDEV/EBUSY at utils.py:116-135, so PermissionError remains unhandled.

Problems

• The proposed loop retries every platform at utils.py:84-91, despite the Windows-specific rationale. That delays permanent POSIX permission failures.
• Current main added the EXDEV/EBUSY copy fallback in utils.py:114-135 via bf8effad0 (#43852). This branch is currently unmergeable, so salvage needs to integrate the retry without bypassing that fallback.
• The new test at tests/hermes_cli/test_atomic_json_write.py:168-188 covers only success after transient failures; it does not cover retry exhaustion or POSIX non-retry behavior. The duplicate discussion on #45022 identified the same coverage gap.

Suggested changes

• Scope bounded PermissionError retries to Windows within the current fallback structure.
• Add Windows-success, Windows-exhaustion, and POSIX-immediate-propagation tests with sleep mocked out.

Automated hermes-sweeper review.

[teknium1]

The PR rationale is Windows mandatory locking, but this loop retries every POSIX PermissionError too. Please scope the retry to the Windows path so a permanent POSIX permission denial still propagates immediately.

[teknium1]

This only proves eventual success. When the implementation is scoped to Windows, explicitly simulate that branch here and add a separate persistent-failure assertion so the bounded retry still surfaces a real denial.

[OutThisLife]

Superseded by #84852.

Your bounded-retry instinct was right and is preserved in #84852 — a retry that wins keeps the write fully atomic, which no pure-fallback approach achieves. Credited as a co-author.

Two extensions: the retry is keyed on the specific winerror codes ({5, 32, 33}) rather than PermissionError broadly, so a genuine ACL denial doesn't silently burn the budget; and a fallback handles the reader that outlives the retries, which retry alone drops.