perf(desktop): make session switching fast under load - #52620
Merged
Conversation
Contributor
🔎 Lint report:
|
| Rule | Count |
|---|---|
unresolved-attribute |
2 |
First entries
run_agent.py:2989: [unresolved-attribute] unresolved-attribute: Object of type `Self@get_credits_spent_micros` has no attribute `_credits_session_start_micros`
tests/run_agent/test_credits_notices_toggle.py:76: [unresolved-attribute] unresolved-attribute: Unresolved attribute `_credits_session_start_micros` on type `AIAgent`
✅ Fixed issues (1):
| Rule | Count |
|---|---|
invalid-assignment |
1 |
First entries
tests/run_agent/test_credits_notices_toggle.py:76: [invalid-assignment] invalid-assignment: Object of type `None` is not assignable to attribute `_credits_session_start_micros` of type `int`
Unchanged: 5938 pre-existing issues carried over.
Diagnostics are surfaced as warnings — this check never fails the build.
Switching sessions in the desktop app could freeze the whole UI for several seconds on heavy, tool-rich chats. Root causes and fixes: - Cold `session.resume` built the AIAgent (MCP discovery, prompt/skill build) *before* returning, and the desktop awaits that RPC before it paints — so the entire switch blocked on the build. Add an opt-in `defer_build` resume path (the contract `session.create` already uses): return the full display transcript immediately, register an upgradable live session, and pre-warm the agent on a short timer. The persisted runtime identity (model/provider/base_url/api_mode/reasoning/tier) is restored on the deferred build so it can't drop the provider. - Nothing bounded how many in-memory agents accumulate; a user who reconnects often piled up detached sessions for the full 6h TTL. Add a soft LRU cap (`max_live_sessions`, default 16) that evicts the least-recently-active DETACHED sessions (no live client) — never a running, awaiting-input, mid-build, or live-transport one. Reopening re-resumes from disk. - On the prefetch-hit cold-resume path, skip rebuilding a throwaway merged-message array (and its 1000-entry Map) when the prefetch already painted the exact transcript; the downstream sameMessageList guard already drops the publish, so it was pure main-thread cost. The desktop opts into `defer_build` for every non-watch cold resume; the eager path stays for CLI/TUI and existing callers.
Per review: gating the faster path behind a `defer_build` flag that the only caller always sends is pointless. Flip it — `session.resume` now defers the agent build by default for every caller (desktop + Ink TUI); a caller that needs the agent built synchronously passes `eager_build: true` (used by the build-race test). The desktop no longer sends a flag. While verifying the flip, fixed two real parity gaps the deferred path had vs the old eager (`_init_session`) path: - `_enable_gateway_prompts()` was never called on a deferred resume, so approvals/clarify wouldn't route through the gateway prompt callbacks. - `_start_agent_build` never wired `background_review_callback` / `memory_notifications`, so a deferred-built session's self-improvement "💾 …" summary leaked to stdout instead of rendering in-transcript. Wiring it there also fixes it for `session.create` sessions, which build through the same path. ACP is unaffected (it uses its own session_manager, not this RPC); the Ink TUI already consumes the same lazy `info` shape from session.create and upgrades on the later `session.info` event.
Collapse the duplicated cold-resume / lazy-watch / create scaffolding into shared helpers: _deferred_session_record (the live-session dict minus the agent), _lazy_resume_info (the not-yet-built session.info), _claim_or_reuse_live (lock + double-checked register-or-reuse), and _schedule_agent_build (the pre-warm timer). Net -12 lines, three copies of the ~30-key session dict and the lazy-info block down to one each. No behavior change.
OutThisLife
force-pushed
the
bb/desktop-session-switch-perf
branch
from
June 25, 2026 19:03
c6e4a25 to
1ca1f9f
Compare
These three assert the eager build contract — stored runtime overrides / profile db reach _make_agent synchronously, and the agent binds to the compression tip. Under deferred-by-default the build runs off-thread, so they raced the timer (green in CI, flaky locally). Pin them to eager_build; deferred coverage lives in the protocol tests.
OutThisLife
enabled auto-merge
June 25, 2026 19:18
4 tasks
1 task
waefrebeorn
pushed a commit
to waefrebeorn/slermes
that referenced
this pull request
Jul 2, 2026
…ssion-switch-perf
habarmc1223-sudo
pushed a commit
to habarmc1223-sudo/hermes-agent-fluxmem
that referenced
this pull request
Jul 8, 2026
…ssion-switch-perf
santhreal
pushed a commit
to santhreal/hermes-agent
that referenced
this pull request
Jul 13, 2026
…ssion-switch-perf
Gravezzz
pushed a commit
to Gravezzz/hermes-agent
that referenced
this pull request
Jul 21, 2026
…ssion-switch-perf
leewenjie
pushed a commit
to leewenjie/hermes-agent
that referenced
this pull request
Aug 7, 2026
…ssion-switch-perf
This file contains hidden or bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
Sign up for free
to join this conversation on GitHub.
Already have an account?
Sign in to comment
Add this suggestion to a batch that can be applied as a single commit.This suggestion is invalid because no changes were made to the code.Suggestions cannot be applied while the pull request is closed.Suggestions cannot be applied while viewing a subset of changes.Only one suggestion per line can be applied in a batch.Add this suggestion to a batch that can be applied as a single commit.Applying suggestions on deleted lines is not supported.You must change the existing code in this line in order to create a valid suggestion.Outdated suggestions cannot be applied.This suggestion has been applied or marked resolved.Suggestions cannot be applied from pending reviews.Suggestions cannot be applied on multi-line comments.Suggestions cannot be applied while the pull request is queued to merge.Suggestion cannot be applied right now. Please check back later.
Summary
Switching sessions in the desktop app could freeze the whole UI for several seconds on heavy, tool-rich chats — nothing scrollable, no text selection, clicks dead. Diagnosed from a user debug bundle (X1 Carbon, 14 GB RAM, a ~33k-token session stuck in retry, 5 detached backend sessions). The freeze is backend resume latency + accumulated agents + redundant renderer work, not one bug. (The originally-proposed "skip double-convert" was already implemented on
main.)Defer the agent build on resume (the big one).
session.resumebuilt theAIAgent(MCP discovery, prompt/skill build) before returning, and the client awaits that RPC before it paints — so the whole switch blocked on the build.session.resumenow defers the build by default for every caller: return the full display transcript immediately, register an upgradable live session, pre-warm the agent on a short timer (the deferred-build contractsession.createalready uses);_sess()still builds on demand if the first prompt beats it. The persisted runtime identity (model/provider/base_url/api_mode/reasoning/tier) is restored on the deferred build so it can't drop the provider. A caller that needs the agent built synchronously passeseager_build: true(used by the build-race test).Bound live agents with a soft LRU cap. Nothing capped how many in-memory agents accumulate; reconnecting often left detached sessions resident for the full 6h TTL (the report's
detached_sessions=5). Newmax_live_sessions(default 16) evicts the least-recently-active detached sessions (no live client) — never a running, awaiting-input, mid-build, or live-transport one. Reopening re-resumes from disk.0/nulldisables.Trim redundant renderer work on switch. On the prefetch-hit cold-resume path the code rebuilt a throwaway merged-message array (+ a ~1000-entry
Map) that the downstreamsameMessageListref guard already drops — pure main-thread cost. Reuse the existing array ref instead.Parity fixes (found while making deferred the default)
The deferred path goes through
_start_agent_buildinstead of_init_session, which was missing two things:_enable_gateway_prompts()— approvals/clarify now route through gateway prompts on a deferred resume.background_review_callback/memory_notifications— the self-improvement "💾 …" summary now renders in-transcript instead of leaking to stdout (also fixessession.createsessions, which build through the same path).ACP is unaffected (it uses its own
session_manager, not this RPC); the Ink TUI already consumes the same lazyinfoshape fromsession.createand upgrades on the latersession.infoevent.Changes
tui_gateway/server.py— deferred-by-default resume (eager_buildopt-out);_start_agent_buildrestores full resume runtime overrides + wires review/memory callbacks;_enforce_session_cap/_session_is_lru_evictable/_max_live_sessions+ enforcement from the idle reaper and after each new session.hermes_cli/config.py—max_live_sessions: 16default (top-level,gateway.*fallback).apps/desktop/.../use-session-actions.ts— reuse$messagesref when the prefetch already painted it.Code quality
Final commit DRYs the three deferred-session paths (cold resume / lazy watch / create) behind shared helpers —
_deferred_session_record,_lazy_resume_info,_claim_or_reuse_live,_schedule_agent_build— collapsing three copies of the ~30-key session dict + lazy-info block to one each. No behavior change; suites stay green.Test plan
scripts/run_tests.sh tests/tui_gateway/— all pass (incl. new deferred-default + LRU-cap tests; build-race + projection tests pinned toeager_build)scripts/run_tests.sh tests/hermes_cli/test_config*.py tests/hermes_cli/test_active_sessions.py— greenvitest run --environment jsdom(session-actions + state-cache) — greentsc --noEmit+eslint— clean (0 errors)