hermes doctor stops sending users with a live gateway into --fix on a large WAL (#110054 item 3) - #113055
Merged
kshitijk4poor merged 3 commits intoSep 16, 2026
Conversation
…ever as "run --fix" `hermes doctor` (without --fix) warned "WAL file is large — run 'hermes doctor --fix' to checkpoint" without checking whether Desktop or the gateway held the database; the holder scan only ran inside --fix. A large WAL is normal for a live writer, and that nudge is how users in the NousResearch#110054 threads became the second writer that the deleted-WAL guard then fired on. The holder scan now runs on the warn path too: held (or unprovable) reports the size as normal while Desktop/gateway run and orders "stop" before any --fix; no holder keeps the checkpoint suggestion, also stop-first. Refs NousResearch#110054 (item 3 of the proposed fix), NousResearch#110073.
…honest disjunction in the warning _live_writer_holds_db in hermes_state_repair already binds the repair connector (the sibling _write_health_reason uses it); the held-branch detail now says 'or state.db cannot be inspected' so the console line matches the comment above it. Test setup for a 51 MB WAL is one helper shared by all three tests.
…ow also emitted before any --fix
kshitijk4poor
enabled auto-merge (rebase)
September 16, 2026 17:22
5 tasks
kshitijk4poor
added a commit
that referenced
this pull request
Sep 16, 2026
MintFailure.as_payload() reported ceil(not_before - monotonic()). With not_before = now + 60, the subtraction is 60.000000000000455 for some values of now, and the ceil made it 61: users saw "retry in 61s" and CI failed test_the_background_loop_retries_a_transient_failure_until_it_settles (slept == [15, 61]) whenever the runner uptime landed on such a value — twice in a row on #113055. remaining() now rounds to the millisecond before the ceil. One test pins a concrete `now` that produced the dust.
This was referenced Sep 24, 2026
This file contains hidden or bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
Sign up for free
to join this conversation on GitHub.
Already have an account?
Sign in to comment
1 participant
Add this suggestion to a batch that can be applied as a single commit.This suggestion is invalid because no changes were made to the code.Suggestions cannot be applied while the pull request is closed.Suggestions cannot be applied while viewing a subset of changes.Only one suggestion per line can be applied in a batch.Add this suggestion to a batch that can be applied as a single commit.Applying suggestions on deleted lines is not supported.You must change the existing code in this line in order to create a valid suggestion.Outdated suggestions cannot be applied.This suggestion has been applied or marked resolved.Suggestions cannot be applied from pending reviews.Suggestions cannot be applied on multi-line comments.Suggestions cannot be applied while the pull request is queued to merge.Suggestion cannot be applied right now. Please check back later.
hermes doctorno longer tells a user with a live Desktop/gateway to runhermes doctor --fixon a large WAL — the nudge that made them the second writer the deleted-WAL guard then fires on.Ours (2026-09-16); item 3 of the maintainer's proposed fix on #110054, which stays open for the rest.
Why. Without
--fix, a >50 MB WAL printedLarge WAL file — run 'hermes doctor --fix' to checkpointwith no holder check (the holder scan only ran inside--fix). A large WAL is normal for a running Desktop/gateway; the Discord threads in #110054 show users following that line into--fixwhile the gateway was live.hermes_cli/doctor_state.py::_state_db_wal— the holder check (hermes_state_repair._live_writer_holds_db, the same wrapper_write_health_reasonuses) now runs on the warn path too. Held or unprovable → "normal while Desktop or the gateway is running, or state.db cannot be inspected — checkpoint only with them stopped" plus the existing stop-first issue line; no holder → the checkpoint suggestion, ordered "stop the profile's gateway, then run --fix".--fixsemantics unchanged.--fix, stop ordered before fix; not held → stop still ordered before fix. The 51 MB WAL setup is one helper shared with the existing exclusive-guard test.test_doctor_wal_checkpoint_guard.py,test_doctor_wal_holder_guard.py,test_doctor.py;test_observational_sessiondb_modes.py(same seam) green--fix)--fixBEGIN IMMEDIATEprobe inside the holder check runs only when the scan found no foreign holderRefs #110054, #110073.