Skip to content

perf(desktop): make session switching fast under load - #52620

Merged
OutThisLife merged 4 commits into
mainfrom
bb/desktop-session-switch-perf
Jun 25, 2026
Merged

perf(desktop): make session switching fast under load#52620
OutThisLife merged 4 commits into
mainfrom
bb/desktop-session-switch-perf

Conversation

@OutThisLife

@OutThisLife OutThisLife commented Jun 25, 2026

Copy link
Copy Markdown
Collaborator

Summary

Switching sessions in the desktop app could freeze the whole UI for several seconds on heavy, tool-rich chats — nothing scrollable, no text selection, clicks dead. Diagnosed from a user debug bundle (X1 Carbon, 14 GB RAM, a ~33k-token session stuck in retry, 5 detached backend sessions). The freeze is backend resume latency + accumulated agents + redundant renderer work, not one bug. (The originally-proposed "skip double-convert" was already implemented on main.)

  • Defer the agent build on resume (the big one). session.resume built the AIAgent (MCP discovery, prompt/skill build) before returning, and the client awaits that RPC before it paints — so the whole switch blocked on the build. session.resume now defers the build by default for every caller: return the full display transcript immediately, register an upgradable live session, pre-warm the agent on a short timer (the deferred-build contract session.create already uses); _sess() still builds on demand if the first prompt beats it. The persisted runtime identity (model/provider/base_url/api_mode/reasoning/tier) is restored on the deferred build so it can't drop the provider. A caller that needs the agent built synchronously passes eager_build: true (used by the build-race test).

  • Bound live agents with a soft LRU cap. Nothing capped how many in-memory agents accumulate; reconnecting often left detached sessions resident for the full 6h TTL (the report's detached_sessions=5). New max_live_sessions (default 16) evicts the least-recently-active detached sessions (no live client) — never a running, awaiting-input, mid-build, or live-transport one. Reopening re-resumes from disk. 0/null disables.

  • Trim redundant renderer work on switch. On the prefetch-hit cold-resume path the code rebuilt a throwaway merged-message array (+ a ~1000-entry Map) that the downstream sameMessageList ref guard already drops — pure main-thread cost. Reuse the existing array ref instead.

Parity fixes (found while making deferred the default)

The deferred path goes through _start_agent_build instead of _init_session, which was missing two things:

  • _enable_gateway_prompts() — approvals/clarify now route through gateway prompts on a deferred resume.
  • background_review_callback / memory_notifications — the self-improvement "💾 …" summary now renders in-transcript instead of leaking to stdout (also fixes session.create sessions, which build through the same path).

ACP is unaffected (it uses its own session_manager, not this RPC); the Ink TUI already consumes the same lazy info shape from session.create and upgrades on the later session.info event.

Changes

  • tui_gateway/server.py — deferred-by-default resume (eager_build opt-out); _start_agent_build restores full resume runtime overrides + wires review/memory callbacks; _enforce_session_cap / _session_is_lru_evictable / _max_live_sessions + enforcement from the idle reaper and after each new session.
  • hermes_cli/config.pymax_live_sessions: 16 default (top-level, gateway.* fallback).
  • apps/desktop/.../use-session-actions.ts — reuse $messages ref when the prefetch already painted it.

Code quality

Final commit DRYs the three deferred-session paths (cold resume / lazy watch / create) behind shared helpers — _deferred_session_record, _lazy_resume_info, _claim_or_reuse_live, _schedule_agent_build — collapsing three copies of the ~30-key session dict + lazy-info block to one each. No behavior change; suites stay green.

Test plan

  • scripts/run_tests.sh tests/tui_gateway/ — all pass (incl. new deferred-default + LRU-cap tests; build-race + projection tests pinned to eager_build)
  • scripts/run_tests.sh tests/hermes_cli/test_config*.py tests/hermes_cli/test_active_sessions.py — green
  • desktop vitest run --environment jsdom (session-actions + state-cache) — green
  • desktop tsc --noEmit + eslint — clean (0 errors)
  • Manual: cold-switch a large tool-heavy session and confirm the transcript paints immediately, the window stays interactive while the agent pre-warms, and approvals + the review summary still surface.

@github-actions

github-actions Bot commented Jun 25, 2026

Copy link
Copy Markdown
Contributor

🔎 Lint report: bb/desktop-session-switch-perf vs origin/main

ruff

Total: 0 on HEAD, 0 on base (➖ 0)

🆕 New issues: none

✅ Fixed issues: none

Unchanged: 0 pre-existing issues carried over.

ty (type checker)

Total: 11265 on HEAD, 11263 on base (🆕 +2)

🆕 New issues (2):

Rule Count
unresolved-attribute 2
First entries
run_agent.py:2989: [unresolved-attribute] unresolved-attribute: Object of type `Self@get_credits_spent_micros` has no attribute `_credits_session_start_micros`
tests/run_agent/test_credits_notices_toggle.py:76: [unresolved-attribute] unresolved-attribute: Unresolved attribute `_credits_session_start_micros` on type `AIAgent`

✅ Fixed issues (1):

Rule Count
invalid-assignment 1
First entries
tests/run_agent/test_credits_notices_toggle.py:76: [invalid-assignment] invalid-assignment: Object of type `None` is not assignable to attribute `_credits_session_start_micros` of type `int`

Unchanged: 5938 pre-existing issues carried over.

Diagnostics are surfaced as warnings — this check never fails the build.

@alt-glitch alt-glitch added type/perf Performance improvement or optimization comp/desktop Electron desktop app (apps/desktop/*) comp/tui Terminal UI (ui-tui/ + tui_gateway/) comp/cli CLI entry point, hermes_cli/, setup wizard P3 Low — cosmetic, nice to have labels Jun 25, 2026
Switching sessions in the desktop app could freeze the whole UI for
several seconds on heavy, tool-rich chats. Root causes and fixes:

- Cold `session.resume` built the AIAgent (MCP discovery, prompt/skill
  build) *before* returning, and the desktop awaits that RPC before it
  paints — so the entire switch blocked on the build. Add an opt-in
  `defer_build` resume path (the contract `session.create` already uses):
  return the full display transcript immediately, register an upgradable
  live session, and pre-warm the agent on a short timer. The persisted
  runtime identity (model/provider/base_url/api_mode/reasoning/tier) is
  restored on the deferred build so it can't drop the provider.

- Nothing bounded how many in-memory agents accumulate; a user who
  reconnects often piled up detached sessions for the full 6h TTL. Add a
  soft LRU cap (`max_live_sessions`, default 16) that evicts the
  least-recently-active DETACHED sessions (no live client) — never a
  running, awaiting-input, mid-build, or live-transport one. Reopening
  re-resumes from disk.

- On the prefetch-hit cold-resume path, skip rebuilding a throwaway
  merged-message array (and its 1000-entry Map) when the prefetch already
  painted the exact transcript; the downstream sameMessageList guard
  already drops the publish, so it was pure main-thread cost.

The desktop opts into `defer_build` for every non-watch cold resume; the
eager path stays for CLI/TUI and existing callers.
Per review: gating the faster path behind a `defer_build` flag that the
only caller always sends is pointless. Flip it — `session.resume` now
defers the agent build by default for every caller (desktop + Ink TUI);
a caller that needs the agent built synchronously passes `eager_build:
true` (used by the build-race test). The desktop no longer sends a flag.

While verifying the flip, fixed two real parity gaps the deferred path
had vs the old eager (`_init_session`) path:

- `_enable_gateway_prompts()` was never called on a deferred resume, so
  approvals/clarify wouldn't route through the gateway prompt callbacks.
- `_start_agent_build` never wired `background_review_callback` /
  `memory_notifications`, so a deferred-built session's self-improvement
  "💾 …" summary leaked to stdout instead of rendering in-transcript.
  Wiring it there also fixes it for `session.create` sessions, which
  build through the same path.

ACP is unaffected (it uses its own session_manager, not this RPC); the
Ink TUI already consumes the same lazy `info` shape from session.create
and upgrades on the later `session.info` event.
Collapse the duplicated cold-resume / lazy-watch / create scaffolding into
shared helpers: _deferred_session_record (the live-session dict minus the
agent), _lazy_resume_info (the not-yet-built session.info), _claim_or_reuse_live
(lock + double-checked register-or-reuse), and _schedule_agent_build (the
pre-warm timer). Net -12 lines, three copies of the ~30-key session dict and
the lazy-info block down to one each. No behavior change.
@OutThisLife
OutThisLife force-pushed the bb/desktop-session-switch-perf branch from c6e4a25 to 1ca1f9f Compare June 25, 2026 19:03
These three assert the eager build contract — stored runtime overrides /
profile db reach _make_agent synchronously, and the agent binds to the
compression tip. Under deferred-by-default the build runs off-thread, so
they raced the timer (green in CI, flaky locally). Pin them to
eager_build; deferred coverage lives in the protocol tests.
@OutThisLife
OutThisLife enabled auto-merge June 25, 2026 19:18
@OutThisLife
OutThisLife merged commit edf3591 into main Jun 25, 2026
32 checks passed
@OutThisLife
OutThisLife deleted the bb/desktop-session-switch-perf branch June 25, 2026 19:20
waefrebeorn pushed a commit to waefrebeorn/slermes that referenced this pull request Jul 2, 2026
habarmc1223-sudo pushed a commit to habarmc1223-sudo/hermes-agent-fluxmem that referenced this pull request Jul 8, 2026
santhreal pushed a commit to santhreal/hermes-agent that referenced this pull request Jul 13, 2026
Gravezzz pushed a commit to Gravezzz/hermes-agent that referenced this pull request Jul 21, 2026
leewenjie pushed a commit to leewenjie/hermes-agent that referenced this pull request Aug 7, 2026
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

comp/cli CLI entry point, hermes_cli/, setup wizard comp/desktop Electron desktop app (apps/desktop/*) comp/tui Terminal UI (ui-tui/ + tui_gateway/) P3 Low — cosmetic, nice to have type/perf Performance improvement or optimization

Projects

None yet

Development

Successfully merging this pull request may close these issues.

2 participants