Skip to content

fix(gateway): bound SQLite file descriptor usage - #78287

Open
injooinjoo wants to merge 1 commit into
NousResearch:mainfrom
injooinjoo:fix/gateway-fd-leak-20260804
Open

fix(gateway): bound SQLite file descriptor usage#78287
injooinjoo wants to merge 1 commit into
NousResearch:mainfrom
injooinjoo:fix/gateway-fd-leak-20260804

Conversation

@injooinjoo

@injooinjoo injooinjoo commented Aug 4, 2026

Copy link
Copy Markdown

Summary

  • reuse the gateway SessionStore's existing SessionDB instead of creating a second long-lived instance
  • cap per-thread WAL reader connections at 16 and fall back to the locked writer connection after the cap
  • make worker-created read connections cross-thread closable so SessionDB.close() actually releases their descriptors
  • de-duplicate SessionDB shutdown when the runner and store share one instance

Reproduction

A long-lived macOS gateway with launchd's soft RLIMIT_NOFILE=256 reached 252 numeric descriptors. 145 were state.db / WAL descriptors. GatewayRunner created two SessionDB instances, and each retained one WAL reader connection per worker thread. In addition, readers were opened with SQLite's default check_same_thread=True, so the owner-thread close loop raised ProgrammingError and silently left worker-created descriptors open.

Verification

  • scripts/run_tests.sh tests/test_hermes_state.py tests/test_session_db_read_path_split.py tests/gateway/test_session.py tests/gateway/test_gateway_shutdown.py tests/gateway/test_runner_session_db_fd_budget.py -q
  • 257 tests passed on current main
  • 100 short-lived reader-thread stress: 16 tracked readers at peak; process FD count returned from 39 to 4 after SessionDB.close()
  • local launchd gateway after hotfix restart: 39 numeric FDs / 6 state.db FDs, stable for 60 seconds; Slack auth remained healthy

@alt-glitch alt-glitch added type/bug Something isn't working comp/agent Core agent runtime: loop, agent_init, prompt builder, context-compression, responses endpoint comp/gateway Gateway runner, session dispatch, delivery P2 Medium — degraded but workaround exists needs-decision Awaiting maintainer decision before any implementation sweeper:risk-message-delivery Sweeper risk: may drop, duplicate, misroute, or suppress messages sweeper:risk-session-state Sweeper risk: may lose/corrupt/mis-associate session or context state labels Aug 4, 2026
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

comp/agent Core agent runtime: loop, agent_init, prompt builder, context-compression, responses endpoint comp/gateway Gateway runner, session dispatch, delivery needs-decision Awaiting maintainer decision before any implementation P2 Medium — degraded but workaround exists sweeper:risk-message-delivery Sweeper risk: may drop, duplicate, misroute, or suppress messages sweeper:risk-session-state Sweeper risk: may lose/corrupt/mis-associate session or context state type/bug Something isn't working

Projects

None yet

Development

Successfully merging this pull request may close these issues.

2 participants