Skip to content

fix(dashboard): startup schema reconcile no longer opens state.db as a second writable owner (#107688, salvage #107691 + #107701) - #107974

Closed
kshitijk4poor wants to merge 5 commits into
NousResearch:mainfrom
kshitijk4poor:salvage/107691-dashboard-readonly-reconcile
Closed

kshitijk4poor wants to merge 5 commits into
NousResearch:mainfrom
kshitijk4poor:salvage/107691-dashboard-readonly-reconcile

Conversation

@kshitijk4poor

Copy link
Copy Markdown
Contributor

The dashboard's startup schema reconcile of this process's own state.db now opens read-only, so a store the gateway already owns never gets a second writable SessionDB owner.

  • hermes_cli/web_server_lifecycle.py::_eager_reconcile_own_session_db routes through _open_session_db_for_profile(None, read_only=True) (the path every dashboard router uses) instead of the writable registry acquire(); the read path still bootstraps a missing store or heals a stale schema through ONE writable open when its read probe fails.
  • Handle is released with release_or_close; one test asserts the read-only open and the close.

Validation: tests/hermes_cli/test_web_server.py 188 passed; the new test goes red with main's web_server_lifecycle.py.

Root cause: an unconditional writable acquire() ran the full schema init plus a close-time WAL checkpoint beside the gateway's writer on every dashboard start.

Credit: fix cherry-picked from #107691 by @kokhlo (first submitter); regression test from #107701 by @Rroven (Co-authored). Both authorships preserved in git history. Refs #107688 (reviewer note: the FTS-corruption mechanism claimed in the original PR is already fenced by rebuild admission; this lands as hardening, so Refs not Closes).

Closes #107691, closes #107701.

@alt-glitch alt-glitch added type/bug Something isn't working P2 Medium — degraded but workaround exists comp/dashboard Web dashboard / control panel UI (dashboard/, landing) comp/cli CLI entry point, hermes_cli/, setup wizard area/sessions Session lifecycle, resume, persistence, history sweeper:risk-session-state Sweeper risk: may lose/corrupt/mis-associate session or context state labels Sep 11, 2026
@kshitijk4poor
kshitijk4poor force-pushed the salvage/107691-dashboard-readonly-reconcile branch from 994e9aa to 40f7b78 Compare September 11, 2026 07:08
kshitijk4poor and others added 5 commits September 11, 2026 12:56
…uption

Root cause: _eager_reconcile_own_session_db() called acquire() for an
unconditional writable open. When gateway+dashboard share state.db, this
creates two writable SessionDB owners — the documented FTS-rebuild
corruption vector (PR NousResearch#93200, issues NousResearch#89293/NousResearch#90950).

Solution: replace acquire() with _open_session_db_at_path(read_only=True),
which still bootstraps a missing store and heals a stale schema via ONE
writable open before reopening read-only, so the dashboard never holds a
long-lived write lock that races FTS rebuilds.

Fixes NousResearch#107688
…ses the handle

Regression guard for NousResearch#107688 carried over from NousResearch#107701 (Rroven); extended
to assert the startup handle is closed, not leaked.

Co-authored-by: kshitijk4poor <82637225+kshitijk4poor@users.noreply.github.com>
…opener

`_open_session_db_for_profile(None, read_only=True)` is the path every
dashboard router already takes for this process's state.db; the inline
`Path(_default_db_path())` re-derived it. Docstrings now state the verified
rationale (no second writable owner; the read path heals through one
writable open when its probe fails) instead of asserting the FTS corruption
mechanism the reviewer showed is already fenced by rebuild admission.
…s access-mode semantics to the opener

The lifecycle docstring re-narrated _open_session_db_at_path's mechanics (two copies drift); the read-only test's docstring promised a heal it patches out.

@ehz0ah ehz0ah left a comment

Copy link
Copy Markdown
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Approval conclusion (formal approval unavailable without explicit repository access)

Reviewed exact head 1caab41045b92dfe2b441a08aec157996cd742e3 against base 4f137c73fcd689e5eb3187e47b223ad293f5f419.

Motivation

The dashboard startup reconciliation opened its own state.db through the writable registry path. On a healthy store that made the dashboard a second writable owner beside the gateway and allowed close-time checkpoint work that a read-only dashboard does not need.

Approach

This change routes startup through the existing profile-aware session opener with read_only=True, then releases or closes the returned handle through the shared lifecycle helper. It reuses the existing guarded bootstrap and schema-heal behavior instead of creating a second repair path.

Concrete changes

_eager_reconcile_own_session_db() now calls _open_session_db_for_profile(None, read_only=True). Healthy stores remain read-only. Missing, zero-byte, and stale stores can still perform the existing single writable bootstrap or reconciliation before reopening read-only. release_or_close() closes the standalone read-only handle without decrementing any registry-owned writer at the same path. The regression test verifies both the access mode and deterministic final close. The contributor mapping is limited to the test co-author carried into this combined PR.

Main-branch risk

I found no blocking regression. Generic corruption is still not escalated into a writable heal. Open and repair failures remain contained by the startup wrapper, and the existing polling path retains responsibility for later retries. Independent review also found no introduced double-close or leaked-handle path. The main residual risk is platform-specific SQLite behavior outside the focused test matrix.

Overall assessment

Approval conclusion. This is the smallest coherent combination of the prior proposals: it removes the unnecessary healthy-store writer, preserves guarded repair behavior, closes the returned handle, and adds the missing regression coverage.

Validation performed:

  • Relevant session and schema suites: 222 passed, 1 skipped
  • Regression applied to the base: failed as expected because the base still used the writable registry path
  • Ruff on changed Python files: passed
  • git diff --check: passed
  • Hosted Docker amd64, Docker arm64, and Nix checks: passed

English verdict: APPROVE at exact head 1caab41045b92dfe2b441a08aec157996cd742e3. No blocking finding was identified.

@Rroven

Rroven commented Sep 11, 2026

Copy link
Copy Markdown

Thanks for folding #107701 into this salvage — the combined approach is genuinely better than what I originally submitted. Two quick notes, from the author of the regression test:

  1. The reviewer's catch was correct: my original version discarded the returned SessionDB without releasing it, which would leak a tracked connection. Routing through _open_session_db_for_profile(None, read_only=True) and releasing via release_or_close addresses that properly — thank you for tightening it.

  2. I've verified the carried-over test test_startup_eager_reconcile_opens_read_only locally against a patched tree: it goes red against the old writable acquire() and green with the read-only routing, so it does bind the fix. The added close() assertion is the right strengthening.

One small thing worth confirming for the record: _open_session_db_for_profile still performs the one-time writable bootstrap/heal when the read probe fails (missing store or stale schema), so the heal path that #80401 depends on is preserved — the read-only open is only for the healthy store. That matches the test coverage.

No further changes from my side — happy to help re-verify if needed.

@teknium1

Copy link
Copy Markdown
Collaborator

Landed on main via #110934 (d4063e6): your _eager_reconcile_own_session_db refactor is cherry-picked with authorship preserved (@kshitijk4poor; original read-only fix by Rroven via #108076, c9b71eb). Thanks both. Closing since the commits are on main.

@teknium1 teknium1 closed this Sep 14, 2026
auto-merge was automatically disabled September 14, 2026 15:10

Pull request was closed

gabrielcosi pushed a commit to gabrielcosi/home-ops that referenced this pull request Sep 14, 2026
…9.11 ➔ v2026.9.14) (#790)

This PR contains the following updates:

| Package | Update | Change |
|---|---|---|
| [ghcr.io/gabrielcosi/hermes-agent](https://github.com/NousResearch/hermes-agent) | patch | `v2026.9.11` → `v2026.9.14` |

---

### Release Notes

<details>
<summary>NousResearch/hermes-agent (ghcr.io/gabrielcosi/hermes-agent)</summary>

### [`v2026.9.14`](https://github.com/NousResearch/hermes-agent/releases/tag/v2026.9.14): Hermes Agent v0.21.3 (v2026.9.14)

[Compare Source](NousResearch/hermes-agent@v2026.9.11...v2026.9.14)

##### Hermes Agent v0.21.3 (v2026.9.14)

**Release Date:** September 14, 2026

> Patch release. This tag rolls up the \~338 PRs merged since v0.21.2 into a stable tagged release for downstream consumers (Docker images, Hermes Cloud, hosted deployments). It exists so the remote-gateway sign-in fixes below reach Cloud agents, which auto-update to the newest release tag.

##### What this patch ships for remote Desktop / Cloud users

- **Remote dashboard sessions no longer expire on refresh bursts** ([#&#8203;110061](NousResearch/hermes-agent#110061), fixes [#&#8203;55712](NousResearch/hermes-agent#55712); salvage [#&#8203;71548](NousResearch/hermes-agent#71548) [@&#8203;Doud-FR](https://github.com/Doud-FR), [#&#8203;55717](NousResearch/hermes-agent#55717) [@&#8203;liuhao1024](https://github.com/liuhao1024)). Both refresh paths on the gateway (cookie gate and the desktop's native bearer route) now coalesce concurrent requests carrying the same rotating refresh token, so a Desktop wake burst can no longer replay an already-rotated token into the Portal's reuse detection and revoke the whole session. Refresh also runs off the event loop, so a slow identity provider no longer freezes `/api/status`. Pairs with Portal-side [NousResearch/hermes-portal#1209](https://github.com/NousResearch/hermes-portal/issues/1209) (sliding 30-day idle horizon, 5-minute rotated-token grace).

##### Also requested for this tag

- **Long-lived processes stop leaking duplicate state.db writer handles** ([#&#8203;110934](NousResearch/hermes-agent#110934), fixes [#&#8203;100896](NousResearch/hermes-agent#100896) [#&#8203;103339](NousResearch/hermes-agent#103339); salvage [#&#8203;107974](NousResearch/hermes-agent#107974) [@&#8203;kshitijk4poor](https://github.com/kshitijk4poor), mapping [@&#8203;Rroven](https://github.com/Rroven)): gateway, dashboard/Desktop backend, ACP and CLI readers attach read-only and in-process writers share the registry handle, so the `N live SessionDB handles` precursor stops firing on a healthy topology.

##### About this release

Measured at commit `9b419a2d3c2657c192008e732149d61170b32c01`, the window since v0.21.2 contains **1,036 non-merge commits** across **2,642 changed files** (+131,690 / −37,096) and **338 merged PRs**.

Also in the window, undocumented here on purpose: server→client JSON-RPC requests and a Pydantic wire-contract registry with generated TS/OpenRPC for the TUI/Desktop gateway ([#&#8203;110521](NousResearch/hermes-agent#110521), [#&#8203;110522](NousResearch/hermes-agent#110522)); reasoning-effort selection on every model picker, a composer pill and per-auxiliary control in Desktop; OpenRouter OAuth PKCE login; HEIF/HEIC/AVIF image decoding; the Honcho peer-model setup rework; MCP OAuth refresh tokens bound to their issuer; a daily MCP re-auth nudge in Desktop; Wan 3.0, Kling 3.0 / Kling Image v3, MiniMax H3 Max Turbo, Gemini Omni Flash 1.1 and Meta Muse in the FAL catalogs; Slack pasted tables and the Agent Sessions API; multiplexed-profile isolation and gateway-liveness fixes; and the state.db WAL refusal on cross-VM filesystems.

**Full curated release notes for this window ship with v0.22.0**, which will document everything from v0.21.0 onward — highlights, feature areas, and complete contributor credits. Nothing in this window is skipped.

##### Updating

- `hermes update` (git installs), or re-run the installer one-liner.
- Docker / Hermes Cloud: images build from this tag (`nousresearch/hermes-agent:v2026.9.14`).

**Full changelog:** <NousResearch/hermes-agent@v2026.9.11...v2026.9.14>

</details>

---

### Configuration

📅 **Schedule**: (in timezone Europe/Berlin)

- Branch creation
  - At any time (no schedule defined)
- Automerge
  - At any time (no schedule defined)

🚦 **Automerge**: Disabled by config. Please merge this manually once you are satisfied.

♻ **Rebasing**: Whenever PR becomes conflicted, or you tick the rebase/retry checkbox.

🔕 **Ignore**: Close this PR and you won't be reminded about these updates again.

---

 - [ ] <!-- rebase-check -->If you want to rebase/retry this PR, check this box

---

This PR has been generated by [Mend Renovate CLI](https://github.com/renovatebot/renovate).
<!--renovate-debug:eyJjcmVhdGVkSW5WZXIiOiI0NC44Mi4wIiwidXBkYXRlZEluVmVyIjoiNDQuODIuMCIsInRhcmdldEJyYW5jaCI6Im1haW4iLCJsYWJlbHMiOlsicmVub3ZhdGUvY29udGFpbmVyIiwidHlwZS9wYXRjaCJdfQ==-->

Reviewed-on: https://git.xcd.dev/gabrielcosi/home-ops/pulls/790
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

area/sessions Session lifecycle, resume, persistence, history comp/cli CLI entry point, hermes_cli/, setup wizard comp/dashboard Web dashboard / control panel UI (dashboard/, landing) P2 Medium — degraded but workaround exists sweeper:risk-session-state Sweeper risk: may lose/corrupt/mis-associate session or context state type/bug Something isn't working

Projects

None yet

Development

Successfully merging this pull request may close these issues.

6 participants