fix(engine): don't hold pooled DB connections across embedder/LLM calls (consolidation + mental model) - #3082
Merged
Conversation
Several memory-engine paths held a pooled PostgreSQL connection checked out for the entire duration of a slow external call (embedder/LLM). The pools are already bounded and per-process, so this is saturation, not a leak: enough concurrent operations park the pool on multi-second calls and everything else blocks on acquire. This covers the two paths that can be fixed without widening the read→write window unsafely: - update_mental_model: compute the embedding BEFORE acquiring a connection. The embedding text depends only on the incoming name/content, never on DB state, so it needs no connection. - consolidation: _process_memory_batch and its executors/dedup helpers no longer receive a long-lived connection. Recall, the batch LLM call, every per-action embed, and dedup adjudication run with NO connection held; each helper self-acquires a short-lived connection only around its own SQL. Moving the slow calls off the connection widens the decision→write window, so the held-transaction serialization is replaced with explicit guards: each source-liveness check (FOR SHARE) is paired with its write in one short transaction, and dedup CREATE/UPDATE folds are RETURNING-gated and re-filter live sources inside the fold transaction (sources-before-observation lock order, matching the normal write paths) so a twin or source deleted during the now connection-free window can't drop a CREATE or fold a dead source id. A cheap non-locking preflight restores the pre-refactor "skip before embed when every source is already gone" short-circuit. The separate-store (non-SQL) branches and the Oracle-safe search_vector clause are preserved. Deterministic no-DB tests pin the fold guards (RETURNING gate, live-source filtering, created/skipped propagation) and the pre-embed short-circuits; live-DB curation/invalidation/document-transfer tests are updated to the new short-acquire signatures. The update_memory_unit hold-across-embed path is intentionally left for a follow-up: its two-phase re-lock/abort/retry has to be reconciled with the pluggable memories store's cross-store transaction coordinator and validated against a real database. Refs #2434
nicoloboschi
added a commit
that referenced
this pull request
Jul 31, 2026
…led connection (#3083) Split update_memory_unit into a read/resolve/embed phase (no pooled connection held) and a short write transaction that re-reads the row, applies the precomputed embedding, and re-embeds in-txn only on a concurrent entity-set change; orphan entities from a failed edit are reclaimed by a forced graph-maintenance sweep. Preserves the pluggable store's begin_txn/decide_txn write-group. Validated by CI (all three test-api shards incl. the live-PG curation suite pass). Follow-up to #3082. Refs #2434.
This was referenced Aug 5, 2026
This file contains hidden or bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
Sign up for free
to join this conversation on GitHub.
Already have an account?
Sign in to comment
Add this suggestion to a batch that can be applied as a single commit.This suggestion is invalid because no changes were made to the code.Suggestions cannot be applied while the pull request is closed.Suggestions cannot be applied while viewing a subset of changes.Only one suggestion per line can be applied in a batch.Add this suggestion to a batch that can be applied as a single commit.Applying suggestions on deleted lines is not supported.You must change the existing code in this line in order to create a valid suggestion.Outdated suggestions cannot be applied.This suggestion has been applied or marked resolved.Suggestions cannot be applied from pending reviews.Suggestions cannot be applied on multi-line comments.Suggestions cannot be applied while the pull request is queued to merge.Suggestion cannot be applied right now. Please check back later.
Problem
Several memory-engine paths held a pooled PostgreSQL connection checked out for the entire duration of a slow external call — an embedder or LLM round-trip. The pools are already bounded (
asyncpg, defaultmax_size=20) and per-process, so this is not a leak — it's saturation: enough concurrent operations park the pool on multi-second external calls and the poller / remaining work then block untilacquire_timeout.This is a reimplementation of the idea behind #2434 (which was opened before #2917 rewrote these paths around the pluggable memories store) onto current
main.What this PR fixes
update_mental_model— the embedding is now computed before acquiring a pooled connection. Its text depends only on the incomingname/content, never on DB state.Consolidation —
_process_memory_batchand its executors / dedup helpers no longer receive a long-lived connection. Recall, the batch LLM call, every per-action embed, and dedup adjudication run with no connection held; each helper self-acquires a short-lived connection only around its own SQL.Moving the slow calls off the connection widens the decision→write window, so the held-transaction serialization is replaced with explicit guards:
FOR SHARE) is paired with its write in one short transaction, so a concurrent delete can't orphan an observation between check and write.RETURNING-gated and re-filter live sources inside the fold transaction (sources-before-observation lock order, matching the normal write paths). A twin deleted during the now connection-free window → the caller still CREATEs; a source deleted → it's dropped from the fold, never written back.The separate-store (non-SQL) branches and the Oracle-safe
search_vectorclause (#3021) are preserved throughout.Scope / deliberately deferred
update_memory_unit(andupdate_mental_model's sibling edit path) still embed inside the acquired connection, but their fix requires a two-phase read/embed-off-connection + short write transaction with re-lock/abort/retry that must be reconciled with the pluggable store'sbegin_txn/decide_txncross-store transaction coordinator — and validated against a real database. That's a follow-up PR (linked once opened) rather than bundling an unvalidated concurrency rewrite of the interactive-edit path here.Verification
ruff+tyclean.RETURNING-gating, live-source filtering, twin-vanished → still-CREATE, created/skipped propagation) and the pre-embed short-circuits/rejections directly.FOR SHAREguard skips the update). These run against live PostgreSQL in CI.Refs #2434