Skip to content

fix(state): throttle repeated VACUUM rewrites (salvage #67351) - #76839

Merged
kshitijk4poor merged 2 commits into
NousResearch:mainfrom
kshitijk4poor:salvage-67351-vacuum-throttle
Aug 2, 2026
Merged

fix(state): throttle repeated VACUUM rewrites (salvage #67351)#76839
kshitijk4poor merged 2 commits into
NousResearch:mainfrom
kshitijk4poor:salvage-67351-vacuum-throttle

Conversation

@kshitijk4poor

Copy link
Copy Markdown
Collaborator

Salvages #67351 by @nkreadly07 — both commits cherry-picked to preserve authorship, retargeted onto current main (the original branch was ~3.9K commits behind and its hermes_cli/config.py hunk targeted DEFAULT_CONFIG before it moved to hermes_cli/config_defaults.py in 1fe0611).

Context — what this fixes, for whom

Users with sessions.auto_prune: true and a large state.db pay a repeated startup stall: once a prune frees any rows (pruned > 0), main's gate (if vacuum and pruned > 0: — hermes_state.py:8574) runs a full VACUUM on every maintenance sweep. On a multi-GB DB with FTS5 shadow tables, VACUUM holds an exclusive lock for the entire rewrite — seconds per 100MB, so tens of seconds of blocked writes at CLI/gateway startup, repeating day after day even though a recent VACUUM already reclaimed the space.

What the fix does (from #67351, kept verbatim)

  • Adds min_vacuum_interval_days (default 30) to maybe_auto_prune_and_vacuum: after a successful VACUUM, stamps last_vacuum in state_meta; subsequent sweeps skip VACUUM until the interval elapses. Pruning itself keeps its normal cadence — SQLite reuses freed pages meanwhile.
  • Failed VACUUM does not stamp, so the next sweep retries (test-covered).
  • Corrupt last_vacuum meta falls through to "vacuum due" (test-covered).
  • Wired through both production callers (cli.py _run_state_db_auto_maintenance, gateway/run.py maintenance hook), config default, and EN+zh docs.

Retargeting done in this salvage (no behavior change vs the original)

  • Config default moved to hermes_cli/config_defaults.py (where DEFAULT_CONFIG lives now); verified hermes_cli.config.DEFAULT_CONFIG re-export picks it up.
  • Test and docs hunks re-applied against current main text.

Verification

  • tests/test_hermes_state.py + tests/test_session_vacuum_config.py: 146 passed on current main (includes the PR's 4 throttle tests: success-stamp, recent-skip, interval-expiry retry, failed-vacuum retry — plus the config-default and CLI-forwarding tests)
  • Mutation check: reverting hermes_state.py to main makes all 4 throttle tests fail; restoring goes green
  • ruff clean on all changed files
  • Attribution: contributor mapping for @nkreadly07 landed via chore: add contributor email mapping for nkreadly07 #76837

Closes #67351 (superseded by this salvage — original author credited via cherry-pick authorship).

@alt-glitch alt-glitch added type/bug Something isn't working comp/cli CLI entry point, hermes_cli/, setup wizard comp/gateway Gateway runner, session dispatch, delivery area/config Config system, migrations, profiles area/sessions Session lifecycle, resume, persistence, history P2 Medium — degraded but workaround exists sweeper:risk-session-state Sweeper risk: may lose/corrupt/mis-associate session or context state sweeper:risk-message-delivery Sweeper risk: may drop, duplicate, misroute, or suppress messages sweeper:risk-compatibility Sweeper risk: may break existing users, config, migrations, defaults, or upgrades labels Aug 2, 2026
@kshitijk4poor
kshitijk4poor merged commit 4e0a775 into NousResearch:main Aug 2, 2026
41 checks passed
@kshitijk4poor
kshitijk4poor deleted the salvage-67351-vacuum-throttle branch August 5, 2026 07:08
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

area/config Config system, migrations, profiles area/sessions Session lifecycle, resume, persistence, history comp/cli CLI entry point, hermes_cli/, setup wizard comp/gateway Gateway runner, session dispatch, delivery P2 Medium — degraded but workaround exists sweeper:risk-compatibility Sweeper risk: may break existing users, config, migrations, defaults, or upgrades sweeper:risk-message-delivery Sweeper risk: may drop, duplicate, misroute, or suppress messages sweeper:risk-session-state Sweeper risk: may lose/corrupt/mis-associate session or context state type/bug Something isn't working

Projects

None yet

Development

Successfully merging this pull request may close these issues.

3 participants