perf(db): cache SQLite schema init and optimize descendant resolution by sebastianlutycz · Pull Request #39140 · NousResearch/hermes-agent

sebastianlutycz · 2026-06-04T14:56:00Z

This Pull Request introduces three key performance optimizations to the database and terminal streaming layers:

1. SQLite Schema Init Caching (`hermes_state.py`)

Problem: SessionDB.__init__ executed _init_schema() (running DDLs and in-memory schema diffs) on every single connection instantiation. Under concurrent API requests (e.g. status polling), this caused database lock contention and blocked Python's single-threaded event loop.
Solution: Introduced a process-level cache (_initialized_dbs) and thread lock (_db_init_lock) to skip schema reconciliation after first-time initialization.
Test Safe: Bypassed under pytest/unittest environments to keep test connection mocking functional.

2. Recursive Descendant Traversal (`web_server.py`)

Problem: _session_latest_descendant queried all sessions in the database without filters, construction of the tree in Python on every invocation causing large memory/CPU load.
Solution: Rewrote the query using a recursive CTE (WITH RECURSIVE) to load only the descendant branch of the target session.

3. High-Performance Rust PTY Bridge (`hermes_cli/pty_bridge.py` & `pty_bridge_rust/`)

Problem: High-throughput terminal streaming (e.g., build/test execution) in Python PTY bridge requires heavy thread switching and GIL contention due to synchronous run_in_executor reads.
Solution: Implemented a lightweight, native Rust PTY bridge subprocess. Python communicates with it asynchronously via stdin/stdout pipes, bypassing GIL/executor bottlenecks completely.
Fallback & Compatibility:
- Maintained automatic fallback to the pure-Python PTY bridge (ptyprocess) if the Rust binary is missing or on unsupported platforms.
- Bypassed in test environments to ensure compatibility with unit test mocks.

Verification

Tested with the complete web server test suite (tests/hermes_cli/test_web_server.py): 227 passed.
Tested with the state database test suite (tests/test_hermes_state.py): 251 passed.

- Cache SQLite schema initialization status process-wide in hermes_state.py to prevent redundant column reconciliations and FTS probe queries on every SessionDB connection creation. This eliminates SQLite lock contention under high concurrent requests (e.g. status polls). Bypassed in pytest/unittest environments for mock compatibility. - Optimize the _session_latest_descendant query in hermes_cli/web_server.py using a recursive CTE (WITH RECURSIVE) to query only descendants of the target session. This avoids downloading and parsing all historical sessions in Python.

… Python fallback Why introduce Rust to the stack? - High-throughput streaming of terminal/PTY logs to WebSockets (e.g. during compilation, test execution) creates heavy CPU and thread-switching bottlenecks in Python due to the GIL and ThreadPoolExecutor. - Moving terminal I/O, resizing, and PTY process monitoring to a native Rust subprocess completely avoids blocking Python's event loop and reduces context-switching latency. - Future work can port other performance-critical components (such as text diffing, trajectory compression, and tokenizers) to Rust (via PyO3/Maturin) to maximize speed and stability. Implementation: - Introduced a lightweight Rust PTY bridge in pty_bridge_rust using portable-pty and tokio. It handles input, resizes (\x1b[8;<rows>;<cols>t), and outputs on separate native threads. - Added RustPtyBridge to hermes_cli/pty_bridge.py which communicates with the compiled Rust executable via pipes (stdin/stdout). - Integrated RustPtyBridge in hermes_cli/web_server.py. - Seamless Python PtyBridge (pure-Python fallback using ptyprocess) is maintained automatically if the Rust binary is missing or if running under a testing framework (e.g. pytest/unittest), ensuring 100% compatibility for standard Python-only environments and CI.

tonydwb

Code Review Summary

Verdict: Approved

Small targeted DB perf + correctness fix: caches SQLite schema initialization per-path (skips redundant _init_schema() on subsequent SessionDB opens in the same process) and switches one descendant lookup to a recursive CTE.

Looks Good

Thread-safe via _db_init_lock.
Test-aware (is_testing bypass) so pytest suites don't share state.
Recursive CTE replaces what was previously an in-memory loop; cleaner and more correct for deeper trees.

Reviewed by Hermes Agent

…ad EOF and using std::process::exit(0) to bypass tokio stdin shutdown bug

…escendant _session_latest_descendant fetched EVERY sessions row and built the parent->children tree in Python on each call. Replace with a recursive CTE that loads only the target session's descendant branch. Hand-applied from PR NousResearch#39140 (the schema-init cache and Rust PTY bridge parts of that PR are intentionally NOT salvaged here); main's function signature gained a db parameter since the PR was cut. (cherry picked from commit 8ed5e54)

NousResearch#53966/NousResearch#39140)

The NousResearch#39140 CTE used UNION ALL, which recurses forever if a corrupted parent chain loops (a -> b -> a) — reproduced: query never returns. The old Python walk was cycle-safe via a seen-set. UNION dedups the working set and terminates. Regression test added and mutation-verified (UNION ALL hangs the test, UNION passes).

kshitijk4poor · 2026-07-08T20:11:39Z

Partial salvage: your recursive-CTE descendant lookup landed on main as d7e4d94 via #60884 (rebase), with your authorship preserved — plus a follow-up switching UNION ALL -> UNION so a corrupted parent-chain cycle can't recurse forever (the old Python walk was cycle-safe via its seen-set; the CTE needed the dedup to match). Leaving this PR open for the remaining two parts: the SQLite schema-init cache (needs a design that doesn't sniff pytest in sys.modules — a process-level init registry keyed off the resolved path could work, but cross-process invalidation needs thought) and the Rust PTY bridge (a new native component — maintainer call). If you want to split those into separate PRs they'll be easier to land.

…escendant _session_latest_descendant fetched EVERY sessions row and built the parent->children tree in Python on each call. Replace with a recursive CTE that loads only the target session's descendant branch. Hand-applied from PR NousResearch#39140 (the schema-init cache and Rust PTY bridge parts of that PR are intentionally NOT salvaged here); main's function signature gained a db parameter since the PR was cut. (cherry picked from commit 8ed5e54)

NousResearch#53966/NousResearch#39140)

The NousResearch#39140 CTE used UNION ALL, which recurses forever if a corrupted parent chain loops (a -> b -> a) — reproduced: query never returns. The old Python walk was cycle-safe via a seen-set. UNION dedups the working set and terminates. Regression test added and mutation-verified (UNION ALL hangs the test, UNION passes).

teknium1

Thanks for the performance work. The descendant-query portion has already landed as d7e4d94e2, with the necessary cycle correction in 1e2ad17afd.

Problems

hermes_cli/web_server.py:4484 uses UNION ALL; 1e2ad17afd demonstrates that a cyclic parent chain can recurse indefinitely. Main now uses UNION and has coverage at tests/hermes_cli/test_web_server.py:1330.
hermes_cli/pty_bridge.py:299 looks for an ignored /bin/hermes-pty-refactor, but this PR adds no build/install integration. A repository search of packaging, CI, Docker, and scripts finds no path that produces that binary.
pty_bridge_rust/src/main.rs:97-111 exits after try_wait() without draining the PTY reader, risking loss of trailing output. Its resize parsing at :69 also assumes a pipe read is one complete resize frame.
hermes_state.py:429 disables the cache under pytest/unittest, and this PR changes no tests, leaving the new behavior unexercised.

Suggested changes

Keep the already-salvaged CTE out of any follow-up.
Rework the cache around current SessionDB._init_schema() responsibilities (hermes_state.py:994) with explicit validity/invalidation semantics and multi-connection tests.
Add a supported native-helper distribution path plus framed I/O, EOF-drain/reap behavior, and integration tests before reconsidering the Rust bridge.

Automated hermes-sweeper review.

teknium1 · 2026-07-14T01:01:57Z

+                """
+                WITH RECURSIVE descendants(id, parent_session_id, started_at) AS (
+                    SELECT id, parent_session_id, started_at FROM sessions WHERE id = ?
+                    UNION ALL


UNION ALL is unsafe here: a corrupted a -> b -> a parent cycle never reaches a fixed point. Main's follow-up 1e2ad17afd changed this to UNION and added a regression test; retain that deduplication in any salvage.

teknium1 · 2026-07-14T01:01:57Z

+        if sys.platform.startswith("win"):
+            return False
+        # Locate binary relative to hermes-agent root directory
+        bin_path = Path(__file__).parent.parent / "bin" / "hermes-pty-refactor"


This checks an ignored /bin/hermes-pty-refactor artifact, but this PR contains no package, CI, Docker, or install step that builds or copies it there. Standard installs will therefore always take the Python fallback; add a supported distribution path before selecting this implementation.

teknium1 · 2026-07-14T01:01:57Z

+                    }
+                    Ok(n) => {
+                        let data = &buf[..n];
+                        if let Some(caps) = resize_re.captures(data) {


stdin is a byte stream, not a message-framed transport. A resize escape can be fragmented or coalesced with terminal input, causing this exact-match parser to forward control bytes into the TUI. Use explicit framing or buffered incremental parsing.

teknium1 · 2026-07-14T01:01:57Z

+                    break;
+                }
+                match child.try_wait() {
+                    Ok(Some(_status)) => {


Breaking on child exit reaches std::process::exit(0) without waiting for the PTY reader thread to drain stdout. Large or trailing output can be lost; wait for reader EOF/drain and reap cleanly before helper exit.

teknium1 · 2026-07-14T01:01:57Z


-            self._init_schema()
+            db_path_str = str(self.db_path.resolve())
+            is_testing = "pytest" in sys.modules or "unittest" in sys.modules


Disabling the cache whenever pytest or unittest is imported prevents the new branch from being covered by normal tests. Prefer explicit cache reset/test fixtures and add multi-connection tests for path scoping and invalidation behavior.

…ck contention SessionDB.__init__ ran _init_schema on every connection construction. _init_schema calls executescript(SCHEMA_SQL) (15+ CREATE TABLE/INDEX), _reconcile_columns (PRAGMA + potential ALTER TABLE), and UPDATE messages SET active=1 — all acquiring a SQLite reserved lock. When multiple SessionDB instances were created concurrently (gateway + cron + channel directory), the DDL writes competed for the reserved lock, freezing the gateway. py-spy dumps consistently showed threads stuck in _init_schema/_connect_and_init across 3 separate freeze events. Add a process-level _initialized_dbs set guarded by _db_init_lock. After the first successful _init_schema for a given db_path, all subsequent connections skip the full DDL + reconciliation entirely. Bypassed in test environments (PYTEST_CURRENT_TEST) so test fixtures that mock connections still work. Same approach as upstream PR NousResearch#39140 (still open, mixed with unrelated Rust PTY + recursive CTE changes). This is the schema-cache part only. Tests: 339 state tests + 112 gateway/cron tests — all pass.

…escendant _session_latest_descendant fetched EVERY sessions row and built the parent->children tree in Python on each call. Replace with a recursive CTE that loads only the target session's descendant branch. Hand-applied from PR NousResearch#39140 (the schema-init cache and Rust PTY bridge parts of that PR are intentionally NOT salvaged here); main's function signature gained a db parameter since the PR was cut. (cherry picked from commit 8ed5e54)

NousResearch#53966/NousResearch#39140)

The NousResearch#39140 CTE used UNION ALL, which recurses forever if a corrupted parent chain loops (a -> b -> a) — reproduced: query never returns. The old Python walk was cycle-safe via a seen-set. UNION dedups the working set and terminates. Regression test added and mutation-verified (UNION ALL hangs the test, UNION passes).

…escendant _session_latest_descendant fetched EVERY sessions row and built the parent->children tree in Python on each call. Replace with a recursive CTE that loads only the target session's descendant branch. Hand-applied from PR NousResearch#39140 (the schema-init cache and Rust PTY bridge parts of that PR are intentionally NOT salvaged here); main's function signature gained a db parameter since the PR was cut. (cherry picked from commit 8ed5e54)

NousResearch#53966/NousResearch#39140)

The NousResearch#39140 CTE used UNION ALL, which recurses forever if a corrupted parent chain loops (a -> b -> a) — reproduced: query never returns. The old Python walk was cycle-safe via a seen-set. UNION dedups the working set and terminates. Regression test added and mutation-verified (UNION ALL hangs the test, UNION passes).

kshitijk4poor · 2026-08-02T13:42:50Z

Thanks @sebastianlutycz for digging into these hot paths. The descendant-resolution optimization has since landed on main as a recursive CTE in _session_latest_descendant via d7e4d94, so that part is implemented-on-main; and the schema-init caching targets the old monolithic hermes_state.py, which has been split into mixins (21c7ae8), so the branch no longer applies cleanly. The Rust PTY bridge currently has no build/CI/distribution pipeline (the binary path is gitignored), so as bundled it can't run in any deployment. Closing this one, but if you'd like to pursue either idea, a fresh PR for schema-init caching against hermes_state_schema.py (with schema-version invalidation and test coverage), or a separate proposal for the Rust bridge including a build story, would be very welcome.

…escendant _session_latest_descendant fetched EVERY sessions row and built the parent->children tree in Python on each call. Replace with a recursive CTE that loads only the target session's descendant branch. Hand-applied from PR NousResearch#39140 (the schema-init cache and Rust PTY bridge parts of that PR are intentionally NOT salvaged here); main's function signature gained a db parameter since the PR was cut. (cherry picked from commit 8ed5e54)

NousResearch#53966/NousResearch#39140)

The NousResearch#39140 CTE used UNION ALL, which recurses forever if a corrupted parent chain loops (a -> b -> a) — reproduced: query never returns. The old Python walk was cycle-safe via a seen-set. UNION dedups the working set and terminates. Regression test added and mutation-verified (UNION ALL hangs the test, UNION passes).

…escendant _session_latest_descendant fetched EVERY sessions row and built the parent->children tree in Python on each call. Replace with a recursive CTE that loads only the target session's descendant branch. Hand-applied from PR NousResearch#39140 (the schema-init cache and Rust PTY bridge parts of that PR are intentionally NOT salvaged here); main's function signature gained a db parameter since the PR was cut. (cherry picked from commit 8ed5e54)

NousResearch#53966/NousResearch#39140)

The NousResearch#39140 CTE used UNION ALL, which recurses forever if a corrupted parent chain loops (a -> b -> a) — reproduced: query never returns. The old Python walk was cycle-safe via a seen-set. UNION dedups the working set and terminates. Regression test added and mutation-verified (UNION ALL hangs the test, UNION passes).

alt-glitch added type/perf Performance improvement or optimization P2 Medium — degraded but workaround exists comp/cli CLI entry point, hermes_cli/, setup wizard labels Jun 4, 2026

tonydwb approved these changes Jun 4, 2026

View reviewed changes

fix(rust-pty): prevent hang on process exit by monitoring reader thre…

a18427b

…ad EOF and using std::process::exit(0) to bypass tokio stdin shutdown bug

alt-glitch added comp/dashboard Web dashboard / control panel UI (dashboard/, landing) sweeper:risk-session-state Sweeper risk: may lose/corrupt/mis-associate session or context state labels Jun 26, 2026

kshitijk4poor mentioned this pull request Jul 8, 2026

perf(dashboard): keep session-DB, cron, PID-probe and log-tail I/O off the event loop (salvage #58238/#45491/#53511/#58389/#39140/#53966) #60884

Closed

liuhao1024 pushed a commit to liuhao1024/hermes-agent that referenced this pull request Jul 8, 2026

chore: AUTHOR_MAP entries for salvaged contributors (NousResearch#53511/

206c042

NousResearch#53966/NousResearch#39140)

pierrenode mentioned this pull request Jul 9, 2026

fix(state): dedup list_sessions_rich's recursive chain CTE (UNION not UNION ALL) #61201

Open

5 tasks

santhreal pushed a commit to santhreal/hermes-agent that referenced this pull request Jul 13, 2026

chore: AUTHOR_MAP entries for salvaged contributors (NousResearch#53511/

1e721ea

NousResearch#53966/NousResearch#39140)

teknium1 reviewed Jul 14, 2026

View reviewed changes

justemu pushed a commit to justemu/hermes-agent that referenced this pull request Jul 18, 2026

chore: AUTHOR_MAP entries for salvaged contributors (NousResearch#53511/

8c4d63a

NousResearch#53966/NousResearch#39140)

Gravezzz pushed a commit to Gravezzz/hermes-agent that referenced this pull request Jul 21, 2026

chore: AUTHOR_MAP entries for salvaged contributors (NousResearch#53511/

d840319

NousResearch#53966/NousResearch#39140)

kshitijk4poor closed this Aug 2, 2026

leewenjie pushed a commit to leewenjie/hermes-agent that referenced this pull request Aug 7, 2026

chore: AUTHOR_MAP entries for salvaged contributors (NousResearch#53511/

1b56a46

NousResearch#53966/NousResearch#39140)

randlee pushed a commit to randlee/hermes-agent that referenced this pull request Aug 11, 2026

chore: AUTHOR_MAP entries for salvaged contributors (NousResearch#53511/

b462fc0

NousResearch#53966/NousResearch#39140)

Navigation Menu

Uh oh!

perf(db): cache SQLite schema init and optimize descendant resolution - #39140

perf(db): cache SQLite schema init and optimize descendant resolution#39140
sebastianlutycz wants to merge 3 commits into
NousResearch:mainfrom
sebastianlutycz:perf/db-lock-and-traverse

sebastianlutycz commented Jun 4, 2026 •

edited

Loading

Uh oh!

tonydwb left a comment

Uh oh!

kshitijk4poor commented Jul 8, 2026

Uh oh!

teknium1 left a comment

Uh oh!

teknium1 Jul 14, 2026

Uh oh!

teknium1 Jul 14, 2026

Uh oh!

teknium1 Jul 14, 2026

Uh oh!

teknium1 Jul 14, 2026

Uh oh!

teknium1 Jul 14, 2026

Uh oh!

kshitijk4poor commented Aug 2, 2026

Uh oh!

Reviewers

Assignees

Labels

Projects

Milestone

Development

Uh oh!

5 participants

Uh oh!

Conversation

sebastianlutycz commented Jun 4, 2026 • edited Loading Uh oh! There was an error while loading. Please reload this page.

Uh oh!

1. SQLite Schema Init Caching (hermes_state.py)

2. Recursive Descendant Traversal (web_server.py)

3. High-Performance Rust PTY Bridge (hermes_cli/pty_bridge.py & pty_bridge_rust/)

Verification

Uh oh!

tonydwb left a comment

Choose a reason for hiding this comment

Code Review Summary

Looks Good

Uh oh!

kshitijk4poor commented Jul 8, 2026

Uh oh!

teknium1 left a comment

Choose a reason for hiding this comment

Problems

Suggested changes

Uh oh!

teknium1 Jul 14, 2026

Choose a reason for hiding this comment

Uh oh!

teknium1 Jul 14, 2026

Choose a reason for hiding this comment

Uh oh!

teknium1 Jul 14, 2026

Choose a reason for hiding this comment

Uh oh!

teknium1 Jul 14, 2026

Choose a reason for hiding this comment

Uh oh!

teknium1 Jul 14, 2026

Choose a reason for hiding this comment

Uh oh!

kshitijk4poor commented Aug 2, 2026

Uh oh!

Reviewers

Assignees

Labels

Projects

Milestone

Development

Uh oh!

5 participants

sebastianlutycz commented Jun 4, 2026 •

edited

Loading

1. SQLite Schema Init Caching (`hermes_state.py`)

2. Recursive Descendant Traversal (`web_server.py`)

3. High-Performance Rust PTY Bridge (`hermes_cli/pty_bridge.py` & `pty_bridge_rust/`)