fix(backup): treat files deleted after the scan as vanished, not archive failures - #124710
Open
ciaomrgrey wants to merge 4 commits into
Open
ciaomrgrey wants to merge 4 commits into
ciaomrgrey wants to merge 4 commits into
Conversation
added 2 commits
September 27, 2026 04:50
…ive failures A file listed by the scan but deleted before the archive write (pruned cron output, finished process records) no longer exists to recover. It raised ENOENT, landed in errors, and made hermes backup exit 1 with a kept zip, failing nightly producers that run while agents and cron are active. Plain files whose write raises FileNotFoundError AND whose path is really gone are now reported as vanished and kept out of errors, so the run is complete and --keep pruning proceeds. Everything else still fails: other OSErrors (EACCES, EIO), ENOENT for a path that still exists, and any *.db snapshot failure (a vanished state.db is still an incomplete backup). Byte totals use the archived entry size so a post-write delete cannot fault.
…tree databases Review found the external memory-provider write loop excused ENOENT for provider-declared *.db files, so a deleted memory.db produced a complete backup and rotated older zips. The .db exclusion now lives in _vanished_since_scan itself, so every archive path enforces it. Adds external text-positive / DB-negative regressions including prune guard.
alt-glitch
added
type/bug
added 2 commits
September 27, 2026 05:21
…e is vanished, not fatal A finished task's scratch workspace (e.g. a throwaway Chrome profile) can be cleaned up between scan and archive write. Its *.db files then failed the snapshot and made the whole backup incomplete (27 Sep producer run: 5 chrome-prof *.db under kanban/workspaces/t_d123f935/). Treat a failed *.db snapshot as vanished only when the file sits inside kanban/workspaces/<task>/ or kanban/boards/<slug>/workspaces/<task>/ and the path is really gone. Hermes-owned databases (state.db, kanban.db, board DBs, cron/executions.db), external provider databases, and any workspace DB that still exists stay fatal.
…nd parent Round-1 review: os.path.lexists() returns False on EACCES/EIO, so an inaccessible, still-existing workspace database was classified as vanished. Discriminate with os.lstat and admit only FileNotFoundError, for both the file and its containing directory; any other error, or a still-present directory, keeps the snapshot failure fatal.
This branch has not been deployed
This file contains hidden or bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
Sign up for free
to join this conversation on GitHub.
Already have an account?
Sign in to comment
2 participants
Add this suggestion to a batch that can be applied as a single commit.This suggestion is invalid because no changes were made to the code.Suggestions cannot be applied while the pull request is closed.Suggestions cannot be applied while viewing a subset of changes.Only one suggestion per line can be applied in a batch.Add this suggestion to a batch that can be applied as a single commit.Applying suggestions on deleted lines is not supported.You must change the existing code in this line in order to create a valid suggestion.Outdated suggestions cannot be applied.This suggestion has been applied or marked resolved.Suggestions cannot be applied from pending reviews.Suggestions cannot be applied on multi-line comments.Suggestions cannot be applied while the pull request is queued to merge.Suggestion cannot be applied right now. Please check back later.
What does this PR do?
hermes backupscans the home, then archives. A plain file listed by the scan but deleted before the archive write (pruned cron output, a finished process record) raises ENOENT, lands inerrors, and the run exits 1 with "Backup incomplete" and no--keeppruning. Nightly backups that run while agents and cron are active fail on files that no longer exist to recover.This PR reports such files as "vanished" instead of errors, so the run completes and pruning proceeds. It stays deliberately narrow; everything else still fails:
FileNotFoundErrorandos.path.lexists(path)is false now (ENOENT for a path that still exists stays an error);*.dbnever qualifies, on any archive path (main tree, external memory-provider files, automatic full-zip): a missing database is an incomplete backup.Byte totals use the archived entry size (
zf.infolist()[-1].file_size) so a delete right after the write cannot fault thestat().Related Issue
No existing issue found. Related but different: #82042 (Chrome-debug transient path exclusions + SQLite busy bound, currently conflicting) — this PR is the general scan/archive race, independent of Chrome.
Type of Change
Changes Made
hermes_cli/backup.py:_vanished_since_scan()helper;_write_zip_entries(..., on_vanished=None)(default behavior unchanged for callers that don't pass it);run_backupand the external-provider loop collect/print/log vanished files separately; the automatic full-zip path logs them at debug.tests/hermes_cli/test_backup.py: race (file deleted between scan and write) completes and prunes; vanished + real error still fails; ENOENT with path still present still fails; vanished*.dbstill fails and does not prune; external provider text file vanishing is benign, external*.dbvanishing stays fatal (including the prune guard).How to Test
pytest tests/hermes_cli/test_backup.pyonmain: the race / mixed / external-text tests fail (4 red incl. an external reviewer probe); the*.dbnegative guards pass on both (already correct onmain).pytest tests/hermes_cli/test_backup*.py tests/hermes_cli/test_config_backups.py tests/hermes_cli/test_setup_reset_backup.py— 113 passed, 3 skipped, plus 1 failure (TestImportHonorsHermesHomeOverride::test_import_targets_named_profile_home, "file I/O against the REAL hermes home") that fails identically on pristinemainon this host (unrelated).ruff check hermes_cli/backup.py tests/hermes_cli/test_backup.py— passed.Checklist
*.db, permission or I/O failures