Skip to content

perf(session): set SQLite PRAGMA for large state.db performance #71755

New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

Sign up for GitHub

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Closed
crayfish-ai wants to merge 2 commits into from
Closed

perf(session): set SQLite PRAGMA for large state.db performance #71755

Changes from all commits
Commits
Show all changes
2 commits
Select commit Hold shift + click to select a range
6c4d21c
perf(session): route SQLite PRAGMAs through central apply_database_pr…
crayfish-ai Jul 30, 2026
72949a7
fix(pr): remove remnant local PRAGMAs from PR branch
crayfish-ai Jul 30, 2026
File filter

Filter by extension

Filter by extension
Conversations
Failed to load comments.
Loading
Jump to
Jump to file
Failed to load files.
Loading
Diff view
Diff view
37 changes: 27 additions & 10 deletions hermes_state.py
View file
Open in desktop
Original file line number Diff line number Diff line change
Expand Up @@ -851,16 +851,23 @@ def apply_database_pragmas(
*,
db_label: str = "state.db",
) -> None:
"""Apply optional WAL-sizing PRAGMAs from ``config.yaml``.
"""Apply optional performance and WAL-sizing PRAGMAs from ``config.yaml``.

Reads the ``database:`` section and applies ``wal_autocheckpoint``
and ``journal_size_limit`` when set to integer values. The journal
mode itself is NOT handled here — ``database.journal_mode`` is owned
by :func:`resolve_journal_mode` inside :func:`apply_wal_with_fallback`,
which layers the operator setting under all the safety guards
(never live-downgrading an on-disk WAL DB, filesystem fallback,
WAL-reset-bug gating). Keeping a single owner prevents a second,
unguarded journal-mode switch path.
Reads the ``database:`` section and applies configurable PRAGMAs when set
to integer values. The journal mode itself is NOT handled here —
``database.journal_mode`` is owned by :func:`resolve_journal_mode` inside
:func:`apply_wal_with_fallback`, which layers the operator setting under
all the safety guards (never live-downgrading an on-disk WAL DB,
filesystem fallback, WAL-reset-bug gating).

Supported keys under ``database:`` in config.yaml:

* ``cache_size`` — negative value = KiB, positive = pages
(e.g. ``-262144`` = 256 MB page cache)
* ``mmap_size`` — max bytes for memory-mapped I/O (0 = disabled)
* ``temp_store`` — 0=DEFAULT(file), 1=FILE, 2=MEMORY, 3=ALWAYS
* ``wal_autocheckpoint`` — WAL auto-checkpoint threshold in pages
* ``journal_size_limit`` — max journal/WAL size in bytes

Best-effort: config load or pragma failures are ignored so DB init
never breaks on a malformed ``database:`` section.
Expand All @@ -873,7 +880,15 @@ def apply_database_pragmas(
except Exception:
return

for pragma_name in ("wal_autocheckpoint", "journal_size_limit"):
# Performance PRAGMAs (applied to ALL connection types: writer, read_only,
# and WAL per-thread readers).
for pragma_name in (
"cache_size",
"mmap_size",
"temp_store",
"wal_autocheckpoint",
"journal_size_limit",
):
raw_value = cfg_get(cfg, "database", pragma_name, default=None)
if raw_value is None:
continue
Expand Down Expand Up @@ -1855,6 +1870,7 @@ def __init__(self, db_path: Path = None, read_only: bool = False):
isolation_level=None,
)
self._conn.row_factory = sqlite3.Row
apply_database_pragmas(self._conn, db_label="state.db")
return

self.db_path.parent.mkdir(parents=True, exist_ok=True)
Expand Down Expand Up @@ -2035,6 +2051,7 @@ def _get_read_conn(self) -> Optional[sqlite3.Connection]:
isolation_level=None,
)
conn.row_factory = sqlite3.Row
apply_database_pragmas(conn, db_label="state.db")
# Load the CJK tokenizer extension on this connection so
# messages_fts_cjk queries work on the read path. The .so
# registers the tokenizer in the connection's in-memory
Expand Down
Loading