Skip to content

fix: retry holographic memory sqlite locks - #40167

Closed
JimGat wants to merge 1 commit into
NousResearch:mainfrom
JimGat:fix/holographic-memory-sqlite-lock-retry
Closed

fix: retry holographic memory sqlite locks#40167
JimGat wants to merge 1 commit into
NousResearch:mainfrom
JimGat:fix/holographic-memory-sqlite-lock-retry

Conversation

@JimGat

@JimGat JimGat commented Jun 5, 2026

Copy link
Copy Markdown

Summary

  • increase holographic memory SQLite writer-lock wait time with explicit PRAGMA busy_timeout
  • retry transient database is locked failures in fact_store and fact_feedback after a safe rollback
  • add tests for busy-timeout configuration and lock retry/rollback behavior

Why

On a live Hermes install using holographic memory, both the always-on gateway and an active CLI session can touch the same SQLite database. Under writer contention, fact_store/fact_feedback writes can fail quickly with database is locked even though the DB is healthy and WAL mode is enabled.

This change makes those write paths tolerate short-lived SQLite writer contention instead of surfacing a transient tool error to the model/user.

Test Plan

  • venv/bin/python -m py_compile plugins/memory/holographic/store.py plugins/memory/holographic/__init__.py tests/agent/test_memory_provider.py
  • venv/bin/python -m pytest tests/agent/test_memory_provider.py -q -o 'addopts=' --tb=short
  • venv/bin/python -m pytest tests -k 'holographic or memory' -q -o 'addopts=' --tb=short

Increase the holographic memory SQLite busy timeout and retry transient database-is-locked write failures after rolling back any pending transaction. Add coverage for busy_timeout plus fact_store/fact_feedback retry behavior.
@alt-glitch alt-glitch added type/bug Something isn't working P3 Low — cosmetic, nice to have tool/memory Memory tool and memory providers comp/plugins Plugin system and bundled plugins labels Jun 5, 2026
TheoLong pushed a commit to TheoLong/hermes-agent that referenced this pull request Jun 21, 2026
Carry-patch of upstream PR NousResearch#40167 (JimGat).
Explicit PRAGMA busy_timeout=60000 + retry-on-locked with rollback for
fact_store/fact_feedback writes, so the always-on gateway and a
concurrent CLI/cron self-reflect subprocess no longer fast-fail with
'database is locked' under SQLite writer contention.

Applied locally ahead of upstream merge; drop on the next hermes update
once NousResearch#40167 lands. Diagnosed via the daily-self-reflect cron timeout
investigation (fact_store writes from the headless subprocess failing
while the gateway held the WAL write lock).
TheoLong pushed a commit to TheoLong/hermes-agent that referenced this pull request Jul 4, 2026
Carry-patch of upstream PR NousResearch#40167 (JimGat).
Explicit PRAGMA busy_timeout=60000 + retry-on-locked with rollback for
fact_store/fact_feedback writes, so the always-on gateway and a
concurrent CLI/cron self-reflect subprocess no longer fast-fail with
'database is locked' under SQLite writer contention.

Applied locally ahead of upstream merge; drop on the next hermes update
once NousResearch#40167 lands. Diagnosed via the daily-self-reflect cron timeout
investigation (fact_store writes from the headless subprocess failing
while the gateway held the WAL write lock).

(cherry picked from commit 9d8f60c)
TheoLong pushed a commit to TheoLong/hermes-agent that referenced this pull request Jul 6, 2026
Carry-patch of upstream PR NousResearch#40167 (JimGat).
Explicit PRAGMA busy_timeout=60000 + retry-on-locked with rollback for
fact_store/fact_feedback writes, so the always-on gateway and a
concurrent CLI/cron self-reflect subprocess no longer fast-fail with
'database is locked' under SQLite writer contention.

Applied locally ahead of upstream merge; drop on the next hermes update
once NousResearch#40167 lands. Diagnosed via the daily-self-reflect cron timeout
investigation (fact_store writes from the headless subprocess failing
while the gateway held the WAL write lock).

(cherry picked from commit 9d8f60c)
@teknium1

Copy link
Copy Markdown
Contributor

Closing as superseded: the database is locked contention this addressed was fixed at the root by PR #61726 (salvage of #43819 by @adambiggs, merged as a801046) — instead of retrying around the lock, all MemoryStore instances in a process now share one refcounted SQLite connection + lock per database, and autocommit prevents a failed write from pinning the write lock at all. Thanks for working this cluster — the retry approach treated the symptom, but your report of the failure mode contributed to pinning down the root cause.

@teknium1 teknium1 closed this Jul 10, 2026
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

comp/plugins Plugin system and bundled plugins P3 Low — cosmetic, nice to have tool/memory Memory tool and memory providers type/bug Something isn't working

Projects

None yet

Development

Successfully merging this pull request may close these issues.

3 participants