perf(state): replace search_sessions full-table aggregate with correlated subquery - #21
Open
spfcraze wants to merge 1 commit into
Open
perf(state): replace search_sessions full-table aggregate with correlated subquery#21spfcraze wants to merge 1 commit into
spfcraze wants to merge 1 commit into
Conversation
…ated subquery search_sessions() computed last_active via a LEFT JOIN over a derived table aggregating MAX(timestamp) GROUP BY session_id across EVERY message in the DB — O(all messages) on every call, reached on each 'hermes -c'/'--resume' launch and ACP session-list. The correlated scalar subquery resolves MAX(timestamp) per session row via the messages(session_id, ...) index — the same pattern list_sessions_rich already uses for its last_active. Measured (real schema, 2k sessions / 120k messages, median of 15): 10.03 ms -> 1.85 ms per LIMIT-20 call (5.4x), with the OLD plan materializing a SCAN of all messages and the NEW plan doing indexed per-row lookups. Results byte-identical (page and full-table). Tests: MRU ordering + COALESCE fallback behavior, byte-exact parity against the pre-fix join form (unfiltered/filtered/paginated), and a plan pin asserting the issued query never materializes a GROUP BY over the messages table.
spfcraze
force-pushed
the
fix/search-sessions-subquery
branch
from
August 1, 2026 13:18
547a31a to
6ea5a58
Compare
This file contains hidden or bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
Sign up for free
to join this conversation on GitHub.
Already have an account?
Sign in to comment
Add this suggestion to a batch that can be applied as a single commit.This suggestion is invalid because no changes were made to the code.Suggestions cannot be applied while the pull request is closed.Suggestions cannot be applied while viewing a subset of changes.Only one suggestion per line can be applied in a batch.Add this suggestion to a batch that can be applied as a single commit.Applying suggestions on deleted lines is not supported.You must change the existing code in this line in order to create a valid suggestion.Outdated suggestions cannot be applied.This suggestion has been applied or marked resolved.Suggestions cannot be applied from pending reviews.Suggestions cannot be applied on multi-line comments.Suggestions cannot be applied while the pull request is queued to merge.Suggestion cannot be applied right now. Please check back later.
What does this PR do?
search_sessions() computed last_active via a LEFT JOIN over a derived table aggregating MAX(timestamp) GROUP BY session_id across EVERY message in the DB — O(all messages) per call, reached on every 'hermes -c'/'--resume' launch and ACP session-list. Replace with a correlated scalar subquery that resolves MAX(timestamp) per session row via the messages(session_id, ...) index — the exact pattern list_sessions_rich already uses for its last_active.
Related Issue
No direct issue — discovered via code review and reproduced live (see below).
Related PRs reviewed during the duplicate check (none covers this change):
Changes Made
fix/search-sessions-subquery— 2 file(s) changed vs base:hermes_state.pytests/test_hermes_state.pyhermes_state.py: one query rewrite in search_sessions() (derived-table LEFT JOIN -> correlated scalar subquery), same COALESCE fallback and ORDER BY. tests/test_hermes_state.py: +4 tests — MRU ordering with COALESCE-to-started_at fallback for message-less sessions, byte-exact parity vs the pre-fix join form across unfiltered/filtered/paginated calls, and a plan pin that drives the real method and asserts the issued query never materializes a GROUP BY over the messages table.
How to Test
Measured on a scratch DB with the real schema and indexes (2k sessions / 120k messages, median of 15 runs, repo venv): OLD 10.03 ms vs NEW 1.85 ms per LIMIT-20 call (5.4x). EXPLAIN QUERY PLAN: OLD materializes a SCAN of all 120k messages plus an AUTOMATIC COVERING INDEX for the join; NEW does per-row indexed lookups (idx_messages_session). Results byte-identical for both the page query and the full unbounded query. Gap grows with DB size — the aggregate cost is O(total messages), the correlated form is O(sessions).
Validation completed (recorded by prp):
tests/test_hermes_state.py.tests/test_hermes_state.py: branch 143 passed / 0 failed vs baseline 140 passed / 0 failed — zero branch-only failures.Logs
Sabotage verification output: