fix(state): bound reusable SQLite WAL readers - #81082
Closed
HenryGHJ wants to merge 2 commits into
Closed
Conversation
Collaborator
8 tasks
Contributor
Author
This file contains hidden or bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
Sign up for free
to join this conversation on GitHub.
Already have an account?
Sign in to comment
Add this suggestion to a batch that can be applied as a single commit.This suggestion is invalid because no changes were made to the code.Suggestions cannot be applied while the pull request is closed.Suggestions cannot be applied while viewing a subset of changes.Only one suggestion per line can be applied in a batch.Add this suggestion to a batch that can be applied as a single commit.Applying suggestions on deleted lines is not supported.You must change the existing code in this line in order to create a valid suggestion.Outdated suggestions cannot be applied.This suggestion has been applied or marked resolved.Suggestions cannot be applied from pending reviews.Suggestions cannot be applied on multi-line comments.Suggestions cannot be applied while the pull request is queued to merge.Suggestion cannot be applied right now. Please check back later.
What does this PR do?
A long-lived shared
SessionDBcurrently retains one SQLite WAL reader for every thread that has ever touched the read path. Finished workers can therefore accumulate database and WAL descriptors until final shutdown.This change replaces permanent per-thread readers with an exclusive reusable pool capped at eight read-only connections:
check_same_thread=False;SessionDB.close()seals the pool, closes idle readers immediately, and closes active leases when returned.This is an alternative implementation to #75352. That PR preserves per-thread caching and reaps connections after their owner thread exits. This PR instead applies a fixed process-level reader ceiling, making descriptor use independent of historical or current worker-thread count.
Related Issue
Fixes #75269
Related alternative: #75352
Type of Change
Changes Made
hermes_state.pywith a bounded reusable WAL-reader pool and exclusive lease lifecycle.tests/test_session_db_read_path_split.pywith regression coverage for finished-thread reuse, exclusive concurrent leases, hard-cap saturation, non-SQL exception return, initialization failure, broken-reader disposal, and close/return behavior.How to Test
Run the focused SessionDB, search, gateway, and delegation regression suite:
uv run --with pytest --with pytest-timeout --with pytest-asyncio \ python -m pytest \ tests/test_session_db_read_path_split.py \ tests/test_hermes_state.py \ tests/tools/test_session_search.py \ tests/gateway/test_async_session_db.py \ tests/gateway/test_async_delegation_session_binding.py \ tests/tools/test_delegate.py \ -q -o 'addopts='Confirm the result:
312 passed, 4 skipped.Run the cross-platform static guard:
Result:
No Windows footguns found.Run
git diff --check origin/main...HEADand confirm no whitespace errors.The repository's preferred
scripts/run_tests.shwrapper was also attempted. The isolated worktree did not have a project dev virtualenv with pytest and full extras, so the wrapper could not provide a valid CI-parity run there; the focused suite above was run directly with explicit pytest dependencies, as permitted by the contributing guide when the wrapper is unavailable.Manual validation used a long-running gateway workload after the patch. A short-lived-thread stress run plateaued at eight pooled readers and returned to zero after
SessionDB.close(); the restarted gateway remained connected with a stable descriptor count. No deployment logs or identifying deployment details are included in this PR.Checklist
Code
fix(scope):,feat(scope):, etc.)pytest tests/ -qand all tests pass — focused affected suites passed; the full repository suite was not runDocumentation & Housekeeping
cli-config.yaml.example: N/A — no configuration keys added or changedCONTRIBUTING.md/AGENTS.md: N/A — no contributor workflow changeScreenshots / Logs
Not applicable. Validation results are listed above; no deployment logs, session content, credentials, profile data, or local paths are included.