fix(state): opening state.db no longer takes the write lock when nothing needs writing (salvage #101881)

[kshitijk4poor]

Opening state.db no longer takes SQLite's write lock when the file needs no writes, so a fresh hermes process is not stalled behind a sibling process holding a transaction.

Based on #101881 by @jonpol01 (cherry-picked, authorship preserved).

Root cause

Every read-write open ran three unconditional write statements plus an unconditional drop+recreate of four triggers — the db_file_generation stamp (INSERT OR IGNORE), the active IS NULL heal (UPDATE … WHERE active IS NULL) and the fts_storage_version marker upsert. SQLite takes the RESERVED lock for a write statement even when it matches zero rows, so on a settled database each open blocked for the full busy timeout whenever another process held BEGIN IMMEDIATE.

Changes

• hermes_state.py::_ensure_db_file_generation — read the stamp first; insert only when absent (first opener still wins, re-read after insert).
• hermes_state_schema.py — the active heal runs only when a NULL row exists; the FTS layout marker is written only when its value differs.
• hermes_state_schema.py — the four display triggers in DEFERRED_INDEX_SQL were dropped and re-created on every open (DROP TRIGGER IF EXISTS on an existing trigger takes the write lock; the CREATE … IF NOT EXISTS forms do not); each pair now runs only when the stored trigger body differs from the desired one, so a changed trigger still rolls out. Found by review: without this the open still blocked and the lock test could not pass.
• tests/test_hermes_state.py — trimmed from the PR's 71 lines to two invariants: a settled open issues no INSERT/UPDATE/DELETE against the main tables; an open completes while a sibling process holds the write lock. Both proven RED with the gates reverted (statement gates AND the trigger gate), GREEN restored.

Comment on the stamp path tightened to the why.

Measured impact

Temp HERMES_HOME, database settled by two prior opens, sibling subprocess holding BEGIN IMMEDIATE for 3 s, 5 runs each:

	main	this PR
SessionDB(path) open under a sibling write lock (median)	2845 ms	16 ms
write-lock statements per settled open (INSERT/UPDATE/DELETE/DROP TRIGGER)	8	0 (only the temp. FTS5 capability probe remains)

CREATE TABLE IF NOT EXISTS statements are unchanged (SQLite resolves them against the schema before taking the lock). Read-only opens and the _fts_stale path are untouched.

Pre-existing on this host (fails identically on a clean origin/main, environmental): TestPerformancePragmasEndToEnd::test_configured_pragmas_reach_all_connection_types — linked SQLite 3.46 falls back to journal_mode=DELETE under the WAL-reset-bug guard.

[Enough1122]

AI code review — automated review for reference; please use your judgment.

PR #106751 — fix(state): opening state.db no longer takes the write lock when nothing needs writing

Summary: SessionDB open path was issuing write statements (INSERT OR IGNORE stamp, DROP+recreate triggers, unconditional UPDATE, meta re-stamp) that take the write lock even when changing nothing, blocking opens behind a sibling's transaction. Now: read-before-write everywhere + DDL that skips settled triggers. Pinned by a write-tracing test and a sibling-hold test.

Findings (all Non-blocking):

1. Non-blocking — hermes_state.py:17 SELECT-then-INSERT still races (two openers, both SELECT empty, both INSERT OR IGNORE) — but convergence is via re-read, and INSERT OR IGNORE is idempotent. Worst case one loser takes the write lock briefly; the common settled case takes none. Correct trade.
2. Non-blocking — hermes_state_schema.py:41 _execute_ddl_skipping_settled_triggers: parses DDL via sqlite3.complete_statement line accumulation and compares stored trigger SQL verbatim. Fragile only if trigger definitions gain formatting drift (stored SQL is verbatim CREATE text). Comparison normalizes IF NOT EXISTS and trailing ; but not whitespace/case — if a future migration reformats a trigger body identically-semantic-but-textually-different, it falls through to drop+recreate (correct, just takes the lock once). Fail-safe direction. Good.
3. Non-blocking — hermes_state_schema.py:68 deferred pending_drop execution: a DROP without a following CREATE (or CREATE that doesn't match the settled-skip) still executes. Preserves ordering semantics. If DDL ends with a dangling DROP, it's silently dropped (pending_drop discarded) — check whether any DDL script ends with a bare DROP TRIGGER; if so it would now be skipped. Worth a grep, likely fine.
4. Non-blocking — hermes_state_schema.py:94 NULL-active probe is index-served on legacy DBs, short-circuited on modern NOT NULL columns. The "deliberately not INDEXED BY" note shows the right sqlite quirk awareness.

Solid lock-hygiene fix with honest tests. No blocking issues.

[teknium1]

Landed on main in #108067 (ff59fcd). All three commits cherry-picked with authorship intact (and #101881 by @jonpol01 underneath); thank you @kshitijk4poor.