Skip to content

fix(code-history): close apply_patch_proposal TOCTOU + crash window (Bonus 12 #7 / PR #135) - #141

Merged
Ghenghis merged 1 commit into
feat/hermes3d-7-complete-gui-repo-wiringfrom
claude/bonus12-toctou-fix
May 9, 2026
Merged

Ghenghis merged 1 commit into
feat/hermes3d-7-complete-gui-repo-wiringfrom
claude/bonus12-toctou-fix

Conversation

@Ghenghis

@Ghenghis Ghenghis commented May 9, 2026

Copy link
Copy Markdown
Owner

Summary

Closes Bonus 12 finding #7. Tightens apply_patch_proposal from a verify-after-replace TOCTOU to a fully durable, recovery-friendly write.

Pre-fix: write tmp -> os.replace -> read -> hash -> rollback. No fsync, no inflight DB record, no parent-dir fsync. SIGKILL or power loss could leave a mutated file with no audit row to reconcile against (etcd #13839 family).

Post-fix: hash bytes -> reject if mismatch -> open("wb")+write+fsync(fd) -> record code_patch.replace_inflight in proof_events -> os.replace -> fsync(parent_dir on POSIX) -> defensive post-read hash check.

Tests added (6, 5 pass + 1 POSIX-skip on Windows host)

  • Clean apply records inflight BEFORE replace, applied AFTER
  • Hash mismatch blocks BEFORE replace; target unchanged; NO inflight written on rejection
  • os.fsync called on tmp fd
  • Simulated crash via os.replace raising OSError leaves target unchanged AND inflight marker in DB
  • POSIX-only: parent-dir fsync (>=2 fsync calls)
  • Windows-only: function does not crash trying to fsync a directory

Cross-platform fix during dev

Initial draft used os.open(..., O_WRONLY|O_CREAT|O_TRUNC) for the tmp file. On Windows that does NOT include O_BINARY by default and translated \n -> \r\n, breaking the post-replace hash. Tests caught this; final patch uses open(..., "wb") which is binary on every platform.

Verification

  • py_compile: OK
  • Focused tests: 5/5 pass + 1 skipped (POSIX-only)
  • code_operator + apply_patch + recovery_ledger: 67/68 pass (1 POSIX-skip)
  • Pre-push hook: passed

Scope

Swarm provenance

Agent 8 of the 20-agent Blocker Elimination Swarm produced the diff.

References

🤖 Generated with Claude Code

…Bonus 12 #7 / PR #135)

Closes Bonus 12 finding #7 from PR #135 / bonus12-bug-finder.md.

Pre-fix order
- write tmp -> os.replace -> read target -> hash -> rollback (post-replace)
- No fsync on tmp fd; Windows write-back caching could lose bytes on
  power loss (etcd #13839 documents the same pattern producing zero-byte
  files on POSIX).
- Hash verified AFTER os.replace, so a corrupt write went live first.
- proof_events insert ran AFTER os.replace; SIGKILL between left the
  mutated file with no audit row to reconcile against.
- No fsync of the parent directory on POSIX, so the rename itself was
  not durable.

Post-fix order
1. Hash the proposed bytes BEFORE any FS mutation; reject up-front on
   mismatch with proposal manifest.
2. Open tmp via ``open(..., "wb")`` (binary, cross-platform), write,
   flush, ``os.fsync(fileno)``.
3. Record ``code_patch.replace_inflight`` proof event with proposal_id +
   pre_apply_snapshot_id + tmp_path BEFORE os.replace, so a recovery
   worker can reconcile a crash mid-replace.
4. ``os.replace`` (atomic on Windows + POSIX).
5. POSIX-only: fsync(parent_dir_fd) so the rename itself is durable;
   gracefully skipped on Windows (NTFS journals the rename and
   ``os.fsync`` on a Windows directory handle raises).
6. Defensive post-read hash check; any drift now indicates FS
   corruption, not a proposal mismatch.

Cross-platform note
- Initial draft used ``os.open(..., O_WRONLY|O_CREAT|O_TRUNC)`` for the
  tmp file, but on Windows that does NOT include ``O_BINARY`` by default
  and can translate ``\n`` -> ``\r\n``, breaking the post-replace hash.
  Tests caught this regression; fix uses ``open(..., "wb")`` which is
  binary on every platform.

Tests added (6, 5 pass + 1 POSIX-skip on Windows)
- 04_testing/pytest/unit/test_apply_patch_toctou.py
  * clean apply records inflight intent BEFORE replace, applied AFTER
  * hash mismatch blocks BEFORE replace; target file unchanged; NO
    inflight row written when manifest is rejected up-front
  * os.fsync called at least once on the tmp fd
  * simulated crash via os.replace OSError leaves target unchanged AND
    inflight marker in DB so recovery can reconcile
  * POSIX-only: parent-dir fsync makes rename durable (>=2 fsync calls)
  * Windows-only: function does not crash trying to fsync a directory

Verification
- py_compile: OK
- Focused tests: 5/5 pass + 1 skipped (POSIX-only on Windows host)
- code_operator + apply_patch + recovery_ledger: 67/68 pass (1 skipped)
- Pre-push hook: passed

Scope
- Bonus 12 finding #7 only.
- v0.13.0 update remains formally deferred.
- RC v2 commits 2-5 remain paused.

Swarm provenance
- Agent 8 of the 20-agent Blocker Elimination Swarm produced the diff.

References
- https://docs.python.org/3/library/os.html#os.fsync
- https://docs.python.org/3/library/os.html#os.replace
- etcd-io/etcd#13839
- https://lwn.net/Articles/457667/

Co-Authored-By: Claude Opus 4.7 (1M context) <noreply@anthropic.com>
@coderabbitai

coderabbitai Bot commented May 9, 2026

Copy link
Copy Markdown

Important

Review skipped

Auto reviews are disabled on base/target branches other than the default branch.

Please check the settings in the CodeRabbit UI or the .coderabbit.yaml file in this repository. To trigger a single review, invoke the @coderabbitai review command.

⚙️ Run configuration

Configuration used: defaults

Review profile: CHILL

Plan: Pro

Run ID: 245f4880-60f8-4826-8075-958e8497c0d9

You can disable this status message by setting the reviews.review_status to false in the CodeRabbit configuration file.

Use the checkbox below for a quick retry:

  • 🔍 Trigger review
✨ Finishing Touches
🧪 Generate unit tests (beta)
  • Create PR with unit tests
  • Commit unit tests in branch claude/bonus12-toctou-fix

Comment @coderabbitai help to get the list of available commands and usage tips.

@Ghenghis
Ghenghis merged commit e5ff363 into feat/hermes3d-7-complete-gui-repo-wiring May 9, 2026
1 check passed

@gemini-code-assist gemini-code-assist Bot left a comment

Copy link
Copy Markdown

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Code Review

This pull request enhances the durability and atomicity of the apply_patch_proposal function by implementing a write-ahead pattern that includes pre-write hash verification, explicit fsync calls on temporary files and parent directories, and database-backed intent logging for crash recovery. Feedback suggests improving the portability of the directory fsync implementation by handling potential OSError exceptions on unsupported POSIX systems and ensuring the rollback path follows the same durability standards as the primary write path to prevent file corruption during failures.

Comment on lines +1533 to +1541
try:
dir_fd = os.open(str(target.parent), os.O_DIRECTORY)
except (AttributeError, OSError):
dir_fd = None
if dir_fd is not None:
try:
os.fsync(dir_fd)
finally:
os.close(dir_fd)

Copy link
Copy Markdown

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

high

The directory fsync implementation is a significant durability improvement, but it may cause crashes on certain POSIX systems or filesystems (like macOS HFS+/APFS or some network mounts) where fsync on a directory handle is not supported and returns ENOTSUP or EINVAL.

Additionally, os.open should explicitly include an access mode (like os.O_RDONLY) when combined with os.O_DIRECTORY to ensure portability and correctness across all POSIX implementations.

Suggested change
try:
dir_fd = os.open(str(target.parent), os.O_DIRECTORY)
except (AttributeError, OSError):
dir_fd = None
if dir_fd is not None:
try:
os.fsync(dir_fd)
finally:
os.close(dir_fd)
try:
# Explicitly include O_RDONLY; O_DIRECTORY is a safety flag
dir_fd = os.open(str(target.parent), os.O_RDONLY | getattr(os, "O_DIRECTORY", 0))
except (AttributeError, OSError):
dir_fd = None
if dir_fd is not None:
try:
os.fsync(dir_fd)
except OSError:
# Best-effort: some POSIX systems do not support directory fsync
pass
finally:
os.close(dir_fd)

Comment on lines 1547 to 1548
rollback_path.write_bytes(current_bytes)
os.replace(rollback_path, target)

Copy link
Copy Markdown

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

medium

While this defensive rollback restores the original file content if a post-replace hash mismatch is detected, it lacks the durability guarantees implemented in the primary write path. If the system crashes during this os.replace without an fsync of the rollback file and its parent directory, the target file could be left in a corrupted or inconsistent state. Given the high durability standards established in this PR, the rollback path should ideally follow the same fsync pattern.

Ghenghis added a commit that referenced this pull request May 9, 2026
…user) (#142)

User-mandated FIRST step in the E2E Blocker Elimination Program. Captures
every known blocker preventing Hermes3D OS from working end-to-end, with
columns: id / subsystem / severity / symptom / repro / squad / files /
receipts / fix PR / tests / rollback / status.

Initial state
- 20 blockers tracked (BLK-001..BLK-020)
- 8 verified (PRs #136-#141 closed Bonus 12 #1-#7 + the staged-update
  gate fake-pass surface from PR #136)
- 1 fixing (BLK-009 = Bonus 12 #8 MCP deadlock; Agent 9 diff ready)
- 1 fixing-doc (BLK-010 = Bonus 13 docs errata; Agent 10 diff ready)
- 1 upstream-blocked (BLK-011 = v0.13.0 upstream main continuously red)
- 1 paused (BLK-012 = RC v2 active repair loop)
- 7 open (BLK-013..BLK-020)

Squad map for 8 hardest blockers (A-H) included; each squad uses up to
6 agents (research / reproducer / fix-builder / test-builder /
sec-reviewer / integrator) with MCP locks.

Discovery audit pattern table included for the 12 classes the user
called out: TODO/FIXME, mock/fake, NotImplemented, suspicious pass,
xfail/skip, broad except, subprocess/Popen, urlopen/requests, zip/tar
extraction, secrets/env, disabled tests, hidden demo states.

E2E Definition of Done (14 items) checklist included.

Co-authored-by: Claude Opus 4.7 (1M context) <noreply@anthropic.com>
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

None yet

Projects

None yet

Development

Successfully merging this pull request may close these issues.

1 participant