Skip to content

fix(gateway): resolve the session DB inside the active profile scope - #88632

Closed
jackulau wants to merge 1 commit into
NousResearch:mainfrom
jackulau:fix/multiplex-session-db-scope-88532
Closed

fix(gateway): resolve the session DB inside the active profile scope#88632
jackulau wants to merge 1 commit into
NousResearch:mainfrom
jackulau:fix/multiplex-session-db-scope-88532

Conversation

@jackulau

Copy link
Copy Markdown
Contributor

What does this PR do?

A multiplexed gateway serves every profile from one process, but SessionStore bound a single SessionDB during __init__:

# gateway/session.py
self._db = SessionDB()

SessionDB(db_path=None) falls back to _default_db_path(), which resolves get_hermes_home() at call time and correctly follows the context-local override installed by set_hermes_home_override. So the path machinery was already right. The bug is when it ran: at construction, on the process's own root home, long before any inbound event enters a profile scope.

The per-profile scoping on the inbound path is also already correct:

# gateway/run.py — _make_profile_message_handler
with _profile_runtime_scope(profile_home):
    return await self._handle_message(event)

and _profile_runtime_scope documents itself as redirecting get_hermes_home() for "config, skills, memory, SOUL, sessions". Sessions were meant to follow the profile. The frozen handle is what prevented it: nothing re-resolved the DB at that point, so the write went through the handle pinned to the root home at startup.

Because profile_name is stamped from source.profile by that same handler, the row is correct in every column and simply lands in the wrong file. That is why the only visible symptom is a display one: the desktop reads per profile,

# hermes_cli/web_server.py
def _open_session_db_for_profile(profile, *, read_only):
    if profile:
        _name, home = _cron_profile_home(profile)
        db_path = Path(home) / "state.db"

so /api/sessions?profile=fitness opens profiles/fitness/state.db, legitimately does not find the session, and the row shows up under the default/hermes bot instead.

The fix looks the handle up through a property that resolves the active scope per access, caching one handle per resolved path. The scoping the inbound path already performs then does the routing on its own.

Why this approach

Per-profile physical stores plus read-only cross-profile aggregation already look like the intended architecture rather than something invented here: _open_session_db_for_profile opens other profiles' stores directly, and SessionDB(read_only=True) documents itself as the non-contending attach for exactly that aggregation. On that reading the writer was the side out of line, so this moves the writer into the existing design rather than adding a new mechanism.

Two details worth calling out:

  • Caching is per resolved path, so a hot inbound path opens SQLite once per profile rather than once per message, and two profiles never share a handle. Construction happens under the cache lock so a concurrent first message on the same profile cannot open (and then leak) a second handle for the same path.
  • Assignment is preserved as an explicit pin. A large number of existing suites install a fake handle or disable the DB with store._db = None, and None alone cannot express "unpinned" versus "deliberately disabled", so an _DB_UNPINNED sentinel distinguishes them. A pin keeps winning across scope changes.

Scope and limits (please confirm these are the calls you want)

  • Inert without an active profile scope. Single-profile gateways resolve exactly the path they did before; there is a test pinning that.
  • Forward-only. Rows already written into the root store are not migrated and will keep displaying under the default bot until moved. I did not want to write a data migration unasked; happy to add one, or to leave it for a follow-up.
  • This changes where a multiplexed gateway physically writes, which is a policy call. If you would rather keep one shared store and fix the display side instead, this is the wrong patch and I will close it. Both options are allowed by the issue text and they are mutually exclusive.
  • #76584 frames default being a hardcoded alias for the root home as the underlying design problem. This is deliberately the narrow writer fix, not that decoupling.
  • I deliberately did not touch the attribution paths already covered by open work on the neighbouring seam (#88437 adapter-derived session keys, #88387 profile fences in lookup/inheritance, #75198 peer fallback scoping). All three are about which profile a row is labeled with; this report confirms the label was already correct, so there is no overlap in behavior or in files beyond gateway/session.py being untouched by the first and third.

Related Issue

Fixes #88532

Type of Change

  • 🐛 Bug fix (non-breaking change that fixes an issue)
  • ✨ New feature (non-breaking change that adds functionality)
  • 🔒 Security fix
  • 📝 Documentation update
  • ✅ Tests (adding or improving test coverage)
  • ♻️ Refactor (no behavior change)
  • 🎯 New skill (bundled or hub)

Changes Made

  • gateway/session.py
    • SessionStore.__init__ no longer binds self._db. It primes the handle for the current scope instead, which keeps startup diagnostics exactly where they were: the live-DB isolation guard still raises during construction, and the JSONL-fallback warning is still printed once at startup rather than on first use.
    • New SessionStore._open_session_db_for_active_scope() resolves _default_db_path() per call and caches one SessionDB per resolved path under a lock. A construction failure is cached as None for that path, matching the previous behavior where a failed startup left _db None for the life of the store and callers fell back to JSONL. The live-guard RuntimeError is deliberately re-raised and not cached, so it fires again on the next attempt.
    • SessionStore._db is now a property with a setter; the setter pins the value.
    • Module-level _DB_UNPINNED sentinel.
  • tests/gateway/test_multiplex_session_db_profile_scope.py (new, 5 tests).

How to Test

pytest tests/gateway/test_multiplex_session_db_profile_scope.py -q

Proof the tests encode the bug. Reverting only gateway/session.py to main and re-running fails 3 of the 5, including the one that reproduces the report by hand exactly as the issue does with sqlite3:

>       assert _session_ids(profile / "state.db") == {"20260817_233028_542fda58"}
E       AssertionError: assert set() == {'20260817_233028_542fda58'}
E         Extra items in the right set:
E         '20260817_233028_542fda58'

3 failed, 2 passed

The row is in the root store and the profile store is empty, which is the reported condition. With the fix, 5 passed.

The other two are guardrails, not reproductions, and pass either way by design: one pins that no-scope resolution is unchanged (single-profile gateways), the other that an explicit store._db = ... still wins across scope changes.

Note for anyone writing tests near this. The suite-wide conftest fixture re-points hermes_state.DEFAULT_DB_PATH, which trips the deliberate escape hatch in _default_db_path() (a re-pointed constant beats everything, including the context-local override). Under that fixture every lookup is pinned to one path and scope assertions become vacuous. The fixture here restores the import-time snapshot to close the hatch, while HERMES_HOME keeps resolution inside tmp_path, so no real store is ever opened.

Regression sweep. All 45 test files mentioning SessionStore:

436 passed, 1 failed in 105.53s

The one failure is tests/gateway/test_session_store_prune.py::test_session_store_default_db_uses_runtime_hermes_home and it is pre-existing and unrelated to this PR. It passes in isolation both with and without the change, and fails identically in the combined run with gateway/session.py reverted to main (1 failed, 431 passed), so it is a cross-file fixture-ordering interaction that already exists on main. Flagging it only so a CI run does not get misread as a regression here.

Checklist

Code

  • I've read the Contributing Guide
  • My commit messages follow Conventional Commits (fix(scope):, feat(scope):, etc.)
  • I searched for existing PRs to make sure this isn't a duplicate
  • My PR contains only changes related to this fix/feature (no unrelated commits)
  • I've added tests for my changes (required for bug fixes, strongly encouraged for features)
  • I've tested on my platform: Windows 11, Python 3.12
  • I've run pytest tests/ -q and all tests pass — not claimed. I ran the 45 SessionStore suites (436 passed, 1 pre-existing failure documented above) plus the new file, not the full tree. Leaving this unchecked rather than overstating it; CI covers the rest.

Documentation & Housekeeping

  • I've updated relevant documentation (README, docs/, docstrings) — the reasoning lives in the docstrings and comments on the new resolver and property, since that is where the next reader will look
  • I've updated cli-config.yaml.example if I added/changed config keys — N/A, no config keys
  • I've updated CONTRIBUTING.md or AGENTS.md if I changed architecture or workflows — N/A
  • I've considered cross-platform impact (Windows, macOS) per the compatibility guide — no platform-specific code; paths go through pathlib and the existing get_hermes_home(), which already resolves %LOCALAPPDATA% on Windows. Developed and tested on Windows 11.
  • I've updated tool descriptions/schemas if I changed tool behavior — N/A

Fixes NousResearch#88532.

A multiplexed gateway serves every profile from one process, but
SessionStore bound a single SessionDB during __init__:

    self._db = SessionDB()

SessionDB(db_path=None) resolves _default_db_path() at call time and
does follow the context-local HERMES_HOME override, so the path
machinery was already correct. The problem was when it ran: at
construction, on the process's own root home, long before any inbound
event enters _profile_runtime_scope. Every profile's rows therefore
landed in the root state.db, even though the scope had redirected
get_hermes_home() correctly for the turn (that helper's own docstring
lists "sessions" among what it scopes).

The rows still carry the right profile_name, stamped from
source.profile by the same handler, so nothing in the data looks wrong.
The only visible symptom is the desktop listing a profile's session
under the default bot: _open_session_db_for_profile opens
profiles/<name>/state.db, which never received the write.

Look the handle up through a property instead, resolving the active
scope per access and caching one handle per resolved path so a hot
inbound path opens SQLite once per profile rather than once per
message. Construction stays under the cache lock so a concurrent first
message on a profile cannot open and then leak a second handle.

Assignment is preserved as an explicit pin, which is what the existing
suites rely on when they install a fake handle or disable the DB with
store._db = None, and a pin keeps winning across scope changes.

Behavior is unchanged when no profile scope is active, so single-profile
gateways resolve exactly the path they did before. This does not migrate
rows that already landed in the root store; those stay where they are.
@alt-glitch alt-glitch added type/bug Something isn't working comp/gateway Gateway runner, session dispatch, delivery area/profiles Multi-profile isolation, HERMES_HOME scoping P2 Medium — degraded but workaround exists sweeper:risk-session-state Sweeper risk: may lose/corrupt/mis-associate session or context state labels Aug 17, 2026
teknium1 added a commit that referenced this pull request Aug 17, 2026
…nd release cached handles

Follow-up to the salvaged #88632 (Jack Lau) fix for #88532, extending the
same repair to the sibling frozen-at-init handle and closing the handle
lifecycle gap the per-path cache introduces.

1. GatewayRunner._session_db had the identical bug class: bound once as
   AsyncSessionDB(SessionDB()) in __init__ on the root home, while /resume,
   /title, /history and session search all execute inside
   _profile_runtime_scope on a multiplexed gateway. Convert it to the same
   property-with-pin pattern: per-access resolution of _default_db_path(),
   one cached AsyncSessionDB per resolved path under a lock, and assignment
   preserved as an explicit pin (many suites install fakes or None).
   Construction-time priming keeps the #88235 init-failure broadcast at
   startup.

2. Handle lifecycle: the per-path caches accumulate one open SessionDB per
   profile served, but the shutdown path closed only store._db /
   runner._session_db - which now resolve just the shutdown task's own
   (root) scope. Secondary profiles' handles would strand their WAL write
   locks until process exit, recreating the abandoned-handle leak
   b454e4d fixed and breaking --replace restarts with 'database is
   locked'. Add close_all_db_handles() / close_all_session_db_handles()
   sweeps and call both from the gateway teardown path. SessionDB.close()
   is idempotent, so the root handle being closed by both the legacy loop
   and the sweep is safe.

Tests: sweep coverage plus a runner-property scope/pin/cache test in
tests/gateway/test_multiplex_session_db_profile_scope.py.
@teknium1

Copy link
Copy Markdown
Contributor

Merged in #88734 with your commit cherry-picked and authorship preserved — excellent diagnosis (the frozen SessionDB handle at SessionStore.__init__ vs the already-correct context-local path resolution) and a careful fix. We E2E-verified writes land in the owning profile's state.db under a live profile scope. Closing this PR in favor of the salvage branch. Fixes #88532.

@teknium1 teknium1 closed this Aug 17, 2026
lisajlau pushed a commit to lisajlau/hermes-agent that referenced this pull request Aug 20, 2026
…nd release cached handles

Follow-up to the salvaged NousResearch#88632 (Jack Lau) fix for NousResearch#88532, extending the
same repair to the sibling frozen-at-init handle and closing the handle
lifecycle gap the per-path cache introduces.

1. GatewayRunner._session_db had the identical bug class: bound once as
   AsyncSessionDB(SessionDB()) in __init__ on the root home, while /resume,
   /title, /history and session search all execute inside
   _profile_runtime_scope on a multiplexed gateway. Convert it to the same
   property-with-pin pattern: per-access resolution of _default_db_path(),
   one cached AsyncSessionDB per resolved path under a lock, and assignment
   preserved as an explicit pin (many suites install fakes or None).
   Construction-time priming keeps the NousResearch#88235 init-failure broadcast at
   startup.

2. Handle lifecycle: the per-path caches accumulate one open SessionDB per
   profile served, but the shutdown path closed only store._db /
   runner._session_db - which now resolve just the shutdown task's own
   (root) scope. Secondary profiles' handles would strand their WAL write
   locks until process exit, recreating the abandoned-handle leak
   b454e4d fixed and breaking --replace restarts with 'database is
   locked'. Add close_all_db_handles() / close_all_session_db_handles()
   sweeps and call both from the gateway teardown path. SessionDB.close()
   is idempotent, so the root handle being closed by both the legacy loop
   and the sweep is safe.

Tests: sweep coverage plus a runner-property scope/pin/cache test in
tests/gateway/test_multiplex_session_db_profile_scope.py.
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

area/profiles Multi-profile isolation, HERMES_HOME scoping comp/gateway Gateway runner, session dispatch, delivery P2 Medium — degraded but workaround exists sweeper:risk-session-state Sweeper risk: may lose/corrupt/mis-associate session or context state type/bug Something isn't working

Projects

None yet

Development

Successfully merging this pull request may close these issues.

Bug: a profile's sessions are physically stored in the default/root state.db (profile_name correct) — desktop shows them under the default/hermes bot

3 participants