Skip to content

fix(backup): serialize and atomically publish snapshots (salvage #69411) - #77913

Merged
kshitijk4poor merged 1 commit into
NousResearch:mainfrom
kshitijk4poor:salvage-69411-backup
Aug 3, 2026
Merged

fix(backup): serialize and atomically publish snapshots (salvage #69411)#77913
kshitijk4poor merged 1 commit into
NousResearch:mainfrom
kshitijk4poor:salvage-69411-backup

Conversation

@kshitijk4poor

Copy link
Copy Markdown
Collaborator

Salvages the backup-safety commit of #69411 by @hao Wang — cherry-picked with authorship preserved.

Context — what this fixes, for whom

Anyone with automatic backups + manual snapshots: run_backup, create_quick_snapshot, and the automatic full-zip path could run CONCURRENTLY (gateway auto-backup firing while a user runs /snapshot), interleaving SQLite safe-copies and half-written snapshot directories. The commit adds a shared cross-process backup lock (_backup_operation_lock, lockfile + BackupInProgressError) and atomic publication: snapshots build in a staging dir and rename into place, so a crashed backup can never leave a half-snapshot that restore would trust.

Scope decisions (bundle split — the PR carries 4 distinct concerns)

Kept: the backup commit (f0fcaf5: 241 lines backup.py + 105-line stability test file), the concern with the clearest bug and cleanest boundary.
Adaptations:

  • One conflict: main's snapshot path gained a failed-DBs abort message after the PR's base; kept main's message with the PR's staging_dir cleanup.
  • Stripped the track_activity("backup", ...) telemetry wrappers — they import hermes_cli.perf_diagnostics, a module introduced by a DIFFERENT commit of the bundle (e304e97) that isn't salvaged here. The locking (the actual fix) is fully preserved; the telemetry can ride with a future perf_diagnostics PR.
    Dropped for separate consideration (documented in the close comment):
  • Desktop startup-queue + prewarm removal (74c99e1, 0409d51): removal of the hover-intent prewarm feature is a maintainer taste call, and the desktop stores drifted on main
  • Model discovery latency + perf_diagnostics module (e304e97): touches tui_gateway/server.py + ws.py which this campaign just modified (perf(cold-start): mitigate the ~14s GIL stall during backend init (salvage #60807) #77814); needs its own review against the new warm-list

Verification

  • tests/hermes_cli/test_backup_stability.py: 6 passed (concurrent-backup serialization, atomic publish, staging cleanup)
  • Full backup suite: 46 passed
  • Mutation check: revert backup.py to main → error; restore → 6 passed
  • ruff clean

Closes #69411 (backup core superseded by this salvage — original author credited via cherry-pick authorship; remaining concerns documented for re-proposal).

@kshitijk4poor
kshitijk4poor enabled auto-merge (rebase) August 3, 2026 18:13
@kshitijk4poor
kshitijk4poor merged commit aad8f74 into NousResearch:main Aug 3, 2026
35 checks passed
@alt-glitch alt-glitch added type/bug Something isn't working P2 Medium — degraded but workaround exists comp/cli CLI entry point, hermes_cli/, setup wizard sweeper:risk-compatibility Sweeper risk: may break existing users, config, migrations, defaults, or upgrades labels Aug 3, 2026
@kshitijk4poor
kshitijk4poor deleted the salvage-69411-backup branch August 5, 2026 07:08
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

comp/cli CLI entry point, hermes_cli/, setup wizard P2 Medium — degraded but workaround exists sweeper:risk-compatibility Sweeper risk: may break existing users, config, migrations, defaults, or upgrades type/bug Something isn't working

Projects

None yet

Development

Successfully merging this pull request may close these issues.

2 participants