perf(tui): cut visible cold start ~57% with lazy agent init

[OutThisLife]

Summary

This PR makes hermes --tui feel much closer to plain hermes on cold start.

The main finding from profiling: plain hermes feels instant because it shows the prompt before constructing AIAgent. The TUI was doing more work up front: Node/Ink imports, Python gateway imports, MCP discovery, tool discovery, and eager full agent construction for the tools/skills panel.

This PR attacks both classes of delay:

1. Remove avoidable cold-start imports from the TUI path.
2. Match classic CLI behavior by showing a usable TUI + skeleton panel before full AIAgent construction.

Headline numbers

Measured on macOS Terminal.app / Apple Silicon using a PTY harness, with warmups and repeated measured runs.

Milestone	Before	After	Delta
Full tools/skills panel p50	1843ms	deferred	—
Visible usable TUI surface p50	1843ms	794ms	−1049ms / −56.9%
Import/gateway-only full-ready p50	1843ms	1537–1655ms	−188ms to −306ms
Python gateway gateway.ready	252–365ms	105–151ms	~−180ms
_make_agent()	~1452ms	~1239ms	~−213ms

The new UX target is skeleton-ready rather than full tools/skills panel ready. The full panel hydrates later when session.info arrives.

How the numbers were measured

PTY startup harness

I measured the user-visible TUI with a Python harness that:

1. opens a real PTY with pty.openpty() so Ink behaves like an actual terminal,
2. spawns:

   hermes --tui

3. reads raw stdout/stderr from the PTY,
4. records timestamps when byte markers appear.

Markers:

Metric	Marker
first_out	first non-empty chunk from PTY
banner	frame containing summoning…
old full ready	full tools/skills counts / hydrated panel (31 tools, skills count)
new skeleton ready	skeleton rows visible (browser:, terminal:, apple: placeholder rows)

Each benchmark does warmup runs first, then reports p50/mean/min/max from repeated measured runs.

Backend RPC harness

For backend-only timings, I spawned the Python TUI gateway directly:

python -m tui_gateway.entry

Then sent newline-delimited JSON-RPC over stdio and measured:

• process start → gateway.ready
• session.create request → response
• session.create → session.info
• prompt.submit → first session.info/message.start when lazy init is triggered

Key observation:

session.create response: ~0.4–0.5ms
old session.create → session.info: ~1167ms
new session.create: no eager session.info; full agent build is deferred

CPU/import profiling

For CPU/import attribution, I used:

python -X importtime -c "import run_agent"
python -X importtime -c "import tui_gateway.entry"

and targeted cProfile around:

tui_gateway.server._make_agent(...)

This identified the biggest remaining costs in agent construction and the avoidable classic-CLI import from tools/code_execution_tool.py.

What profiling found

1. Static @hermes/ink import blocked gateway start

ui-tui/src/lib/memoryMonitor.ts statically imported @hermes/ink just to access evictInkCaches.

That pulled the full ~414KB Ink bundle — React, renderer, components, hooks — before gw.start() could spawn Python.

Fix: lazy dynamic import inside the memory-pressure tick. That tick only fires after startup and only needs the function under memory pressure.

2. MCP SDK imported even with no MCP servers

tui_gateway/entry.py unconditionally imported tools.mcp_tool before gateway.ready.

That transitively loaded:

mcp + pydantic + httpx + jsonschema + starlette formparsers

Cost was about ~200ms on this machine, wasted when mcp_servers is absent.

Fix: cheap config probe first; only import MCP tooling when servers are configured.

3. execute_code schema imported the classic CLI

During tool discovery:

tools.code_execution_tool
→ module-level EXECUTE_CODE_SCHEMA
→ _get_execution_mode()
→ from cli import CLI_CONFIG

That imported classic cli.py, prompt_toolkit/Rich/classic-REPL setup, etc. into TUI startup.

Fix: read code_execution config using cached hermes_cli.config.read_raw_config() instead.

Measured impact:

Import / path	Before	After
import run_agent	~466ms	~347ms
model_tools import	~418ms	~272ms
_make_agent()	~1452ms	~1239ms

4. Biggest architectural mismatch: TUI eagerly built AIAgent

Classic CLI:

show banner/prompt first
build AIAgent on first prompt

Old TUI:

show shell
call session.create
start _make_agent immediately
wait for hydrated tools/skills/session.info panel

So TUI visibly paid the agent construction cost that classic CLI hides until first prompt.

Fix: session.create now returns a lightweight lazy session immediately, and the frontend renders skeleton rows. The full AIAgent is constructed by _start_agent_build() only when an agent-backed method is first called.

User-visible behavior after this PR

Startup now shows compact unicode-animations braille loaders instead of fake skeleton rows or literal loading copy:

Available Tools
⠋ discovering tools

Available Skills
⠋ scanning skills

When the real session.info event arrives, this hydrates automatically into the full tools/skills/session panel — no first prompt required.

First prompt can still pay agent construction if submitted before hydration completes. That is intentional and matches classic CLI’s deferred-cost model.

Implementation details

• ui-tui/src/lib/memoryMonitor.ts

  • lazy-loads @hermes/ink for evictInkCaches.

• tui_gateway/entry.py

  • skips MCP SDK import when no mcp_servers are configured.

• tools/code_execution_tool.py

  • no longer imports cli.CLI_CONFIG while building module-level schema.

• tui_gateway/server.py

  • session.create returns placeholder { tools: {}, skills: {}, lazy: true } immediately.
  • _start_agent_build() centralizes real agent construction.
  • _sess() triggers lazy build for agent-backed methods.
  • prompt.submit is treated as a long RPC handler so first-build wait does not block the stdio dispatcher.

• ui-tui/src/components/branding.tsx

  • renders skeleton rows while info.lazy is true.

• ui-tui/src/app/useConfigSync.ts

  • skips startup /voice status probe; uses HERMES_VOICE env for initial UI bit.

• ui-tui/src/app/useSubmission.ts

  • skips input.detect_drop RPC for ordinary plain-text prompts.

Tests / verification

Ran:

python -m py_compile tui_gateway/server.py
cd ui-tui && npm run type-check && npm run build
scripts/run_tests.sh tests/tui_gateway/test_protocol.py::test_sess_found tests/tools/test_code_execution_modes.py tests/tools/test_code_execution.py
cd ui-tui && npm test -- --run src/__tests__/useSessionLifecycle.test.ts src/__tests__/useConfigSync.test.ts

Results:

101 pytest passed
22 vitest passed
TypeScript type-check clean
TUI build clean

Earlier in the PR:

scripts/run_tests.sh tests/tui_gateway/ tests/tools/test_mcp_tool.py → 195 passed
ui-tui full vitest suite → 430 passed

Known unrelated note: test_session_resume_returns_hydrated_messages has a pre-existing mock DB kwarg mismatch reproduced outside this change path.

[OutThisLife]

Follow-up commit pushed: 9e398e180 perf(tui): avoid importing classic CLI during tool discovery

Additional profiling found the remaining perceived lag was mostly session readiness after gateway.ready, not Ink rendering:

• session.create placeholder returns in ~0.5ms
• real session.info was arriving ~1167ms later
• cProfile of tui_gateway.server::_make_agent showed AIAgent construction at ~1452ms
• major avoidable cost: tools.code_execution_tool building module-level EXECUTE_CODE_SCHEMA called _get_execution_mode(), which imported cli.CLI_CONFIG and pulled the classic REPL stack (prompt_toolkit/Rich/etc.) into TUI startup.

Fix: read code_execution via lightweight cached hermes_cli.config.read_raw_config() instead of importing cli.

New measurements on macOS Terminal.app:

Metric	Before follow-up	After follow-up
import run_agent	~466ms	~347ms
model_tools import	~418ms	~272ms
_make_agent	~1452ms	~1239ms
session.create → session.info	~1167ms	~999ms
full hermes --tui → ready p50	~1655ms	~1537ms

Cumulative PR improvement vs original baseline is now roughly:

• ready p50: ~1843ms → ~1537ms (~306ms / ~16.6% faster)
• ready stdev: 126ms → 53ms (less jitter)

Tests:
scripts/run_tests.sh tests/tools/test_code_execution_modes.py tests/tools/test_code_execution.py → 100 passed.

[OutThisLife]

Follow-up pushed: b66cbb7b4 perf(tui): defer agent construction until first prompt

This implements the lazy/skeleton approach:

• session.create now returns a lightweight placeholder immediately with lazy: true, empty tools/skills, model/cwd only.
• The TUI intro panel renders skeleton rows (no "loading tools..." text) for tools/skills until real session.info arrives.
• Real AIAgent construction moves to _start_agent_build(), triggered by _sess() the first time a method actually needs the agent.
• prompt.submit is routed through the RPC worker pool so waiting for first agent construction doesn't block the stdio dispatcher.
• Startup no longer probes voice.toggle status (optional audio/STT deps) and plain prompts skip the pre-submit input.detect_drop RPC.

Numbers on macOS Terminal.app:

Metric	Prior PR state	Lazy skeleton
first output p50	~614ms	~667ms
banner p50	~642ms	~698ms
visible/skeleton ready p50	~1537ms full-ready	~794ms skeleton-ready

Compared to the original baseline (~1843ms full ready), the visible startup surface is now about ~1.05s faster. First prompt still pays agent construction if it races readiness, which matches classic CLI's deferred behavior.

Tests run:

• python -m py_compile tui_gateway/server.py
• cd ui-tui && npm run type-check && npm run build
• scripts/run_tests.sh tests/tui_gateway/test_protocol.py::test_sess_found tests/tools/test_code_execution_modes.py tests/tools/test_code_execution.py → 101 passed
• cd ui-tui && npm test -- --run src/__tests__/useSessionLifecycle.test.ts src/__tests__/useConfigSync.test.ts → 22 passed

[Copilot]

Pull request overview

Improves perceived cold-start performance of hermes --tui by deferring expensive imports and shifting full AIAgent construction off the initial “first paint” path, while keeping the UI usable via a skeleton session panel until session.info hydrates.

Changes:

• Lazily initialize the backend agent in the TUI gateway (fast session.create, build on first agent-backed RPC) and treat prompt.submit as a long handler.
• Reduce cold-start work by deferring heavy imports (Ink cache eviction helper in Node; MCP discovery in Python when unconfigured; avoid importing classic CLI during tool schema build).
• Update TUI UI to render a tools/skills skeleton while backend session info is still “lazy”, and skip certain startup RPCs on common paths.

Reviewed changes

Copilot reviewed 10 out of 10 changed files in this pull request and generated 2 comments.

Show a summary per file

File	Description
ui-tui/src/types.ts	Adds SessionInfo.lazy flag to support skeleton/hydration flow.
ui-tui/src/lib/memoryMonitor.ts	Defers @hermes/ink import via dynamic import to keep cold-start path lighter.
ui-tui/src/components/branding.tsx	Renders skeleton tool/skill rows when info.lazy is true.
ui-tui/src/components/appLayout.tsx	Shows SessionPanel whenever info exists (not only when version exists).
ui-tui/src/app/useSubmission.ts	Skips input.detect_drop RPC for common plain-text prompts.
ui-tui/src/app/useConfigSync.ts	Avoids startup /voice status RPC; initializes voice UI bit via env.
tui_gateway/server.py	Implements lazy agent build (_start_agent_build), triggers build from _sess(), and marks prompt.submit as a long handler.
tui_gateway/entry.py	Avoids importing MCP tooling when mcp_servers are not configured.
tools/code_execution_tool.py	Uses hermes_cli.config.read_raw_config() to avoid importing interactive CLI during schema construction.
tests/tools/test_code_execution.py	Adds/adjusts tests around _load_config() behavior and avoiding CLI dependency.

Comments suppressed due to low confidence (1)

tui_gateway/server.py:150

• Adding prompt.submit to _LONG_HANDLERS avoids blocking the dispatcher while the lazy agent build runs, but other “control-plane” methods (e.g. session.interrupt at ~2130) still call _sess() and can now block on _wait_agent() for up to 30s during first-time agent init. That reintroduces the stdin backpressure problem described above (interrupts/approvals sitting unread) in the exact window users are most likely to mash interrupt. Consider either adding session.interrupt (and any other methods that can be invoked during init) to _LONG_HANDLERS, or special-casing _sess()/session.interrupt to avoid waiting when agent_ready isn’t set yet (e.g., return a fast “initializing” response or set an interrupt-pending flag).

_LONG_HANDLERS = frozenset(
    {
        "cli.exec",
        "prompt.submit",
        "session.branch",
        "session.resume",
        "shell.exec",
        "skills.manage",
        "slash.exec",
    }

---

💡 Add Copilot custom instructions for smarter, more guided reviews. Learn how to get started.

[OutThisLife]

Fix pushed: 0a6ecea67 fix(tui): hydrate lazy startup panel and use animated loaders

Addresses the manual QA finding:

• The lazy placeholder could stay stuck on starting agent… / empty tools+skills if no first prompt was submitted, because the real agent build was only triggered by _sess().
• session.create still returns the lightweight placeholder immediately, but now schedules _start_agent_build() shortly after the response is flushed, so tools/skills hydrate automatically.
• Replaced the ugly skeleton bar rows with compact unicode-animations braille loaders:
  • ⠋ discovering tools
  • ⠋ scanning skills

Verification:

• manual PTY run: placeholder appears, spinner animates, full tools/skills panel hydrates automatically
• python -m py_compile tui_gateway/server.py
• cd ui-tui && npm run type-check && npm run build
• cd ui-tui && npm test -- --run src/__tests__/useSessionLifecycle.test.ts src/__tests__/useConfigSync.test.ts → 22 passed
• scripts/run_tests.sh tests/tui_gateway/test_protocol.py::test_sess_found tests/tools/test_code_execution_modes.py tests/tools/test_code_execution.py → 101 passed

[OutThisLife]

Addressed Copilot review in a2819e182 fix(tui): address lazy startup review races.

Fixes:

1. ui-tui/src/lib/memoryMonitor.ts

   • Cached the lazy @hermes/ink dynamic import promise.
   • Added per-level inFlight tracking so overlapping interval ticks cannot run duplicate evictions/heap dumps while an import/dump is awaiting.

2. tui_gateway/server.py

   • Added a per-session agent_build_lock around the lazy build check/set.
   • Prevents concurrent RPC handlers from starting duplicate agent builders / slash workers / approval registrations.

Verification:

• python -m py_compile tui_gateway/server.py
• cd ui-tui && npm run type-check && npm run build
• cd ui-tui && npm test -- --run src/__tests__/useSessionLifecycle.test.ts src/__tests__/useConfigSync.test.ts → 22 passed
• scripts/run_tests.sh tests/tui_gateway/test_protocol.py::test_sess_found tests/tools/test_code_execution_modes.py tests/tools/test_code_execution.py → 101 passed

[Copilot]

Pull request overview

Improves hermes --tui cold-start responsiveness by deferring expensive imports and lazy-initializing the backend agent so the TUI can render a usable surface sooner, then hydrate tools/skills asynchronously.

Changes:

• Add a “lazy session” mode where session.create returns immediately and agent construction runs deferred/on-demand.
• Reduce cold-start work by deferring heavy imports (Ink bundle, MCP SDK) and avoiding classic-CLI imports during tool discovery.
• Update TUI UI/UX to show loader/skeleton states and skip non-essential startup RPCs.

Reviewed changes

Copilot reviewed 10 out of 10 changed files in this pull request and generated 4 comments.

Show a summary per file

File	Description
ui-tui/src/types.ts	Adds SessionInfo.lazy to support lazy session hydration in the TUI.
ui-tui/src/lib/memoryMonitor.ts	Defers @hermes/ink import until memory-pressure handling needs it.
ui-tui/src/components/branding.tsx	Renders an inline spinner while tools/skills are still “lazy”/loading.
ui-tui/src/components/appLayout.tsx	Shows the session panel when info exists (not only when version is present).
ui-tui/src/app/useSubmission.ts	Avoids an extra input.detect_drop RPC for ordinary plain-text prompts.
ui-tui/src/app/useConfigSync.ts	Removes startup /voice status probe; initializes UI state from env.
tui_gateway/server.py	Implements deferred agent build, placeholder session.create, and long-handler treatment for prompt.submit.
tui_gateway/entry.py	Skips MCP discovery import unless MCP servers appear configured.
tools/code_execution_tool.py	Uses read_raw_config() instead of importing classic CLI config during schema/tool discovery.
tests/tools/test_code_execution.py	Adds/adjusts tests validating _load_config() behavior with read_raw_config().

---

💡 Add Copilot custom instructions for smarter, more guided reviews. Learn how to get started.

[OutThisLife]

Clean pass pushed: 72a3af63d fix(tui): keep prompt submit off the RPC pool.

I re-read the diff with the /clean instruction in mind and tightened the one awkward async shape the independent review called out:

• removed prompt.submit from _LONG_HANDLERS
• prompt.submit now returns immediately again
• it does a no-wait session lookup, starts the lazy build, and spawns a tiny waiter thread that blocks on agent_ready before entering the existing turn dispatcher
• avoids occupying a bounded RPC pool worker for a normal chat turn while preserving lazy-start hydration

Verification:

• direct JSON-RPC harness: prompt.submit response ~0.14ms, then session.info + message.start arrive asynchronously
• python -m py_compile tui_gateway/server.py
• cd ui-tui && npm run type-check && npm run build
• scripts/run_tests.sh tests/tui_gateway/test_protocol.py::test_sess_found tests/tools/test_code_execution_modes.py tests/tools/test_code_execution.py → 101 passed
• cd ui-tui && npm test -- --run src/__tests__/useSessionLifecycle.test.ts src/__tests__/useConfigSync.test.ts → 22 passed

[OutThisLife]

Final /clean follow-up pushed: 88a9efdb1 fix(tui): tighten cold-start edge cases after review.

Addressed remaining non-blocking review nits:

• memoryMonitor.ts: lazy @hermes/ink import now clears the cached promise on rejection so a transient import failure can retry.
• memoryMonitor.ts: per-level inFlight cleanup now lives in finally, so future exceptions cannot suppress that memory level forever.
• tui_gateway/entry.py: MCP cold-start probe now uses cheap read_raw_config() instead of full load_config().
• useSubmission.ts: plain prompts still skip input.detect_drop, but explicit relative path prefixes (./, ../) still go through file-drop detection.

Verification:

• python -m py_compile tui_gateway/server.py tui_gateway/entry.py
• cd ui-tui && npm run type-check && npm run build
• scripts/run_tests.sh tests/tui_gateway/test_protocol.py::test_sess_found tests/tools/test_code_execution_modes.py tests/tools/test_code_execution.py → 101 passed
• cd ui-tui && npm test -- --run src/__tests__/useSessionLifecycle.test.ts src/__tests__/useConfigSync.test.ts → 22 passed

[Copilot]

Pull request overview

This PR reduces hermes --tui cold-start latency by deferring expensive imports and moving full AIAgent construction off the initial “paint the TUI” critical path, while still hydrating tools/skills shortly after startup.

Changes:

• Add lazy-session semantics (info.lazy) and render a loader/skeleton panel until session.info hydrates.
• Defer heavyweight imports (Ink bundle eviction helpers, MCP SDK discovery) until actually needed/configured.
• Avoid extra startup RPCs (voice status probe, file-drop detection for plain prompts) to keep the stdio RPC pipe responsive.

Reviewed changes

Copilot reviewed 10 out of 10 changed files in this pull request and generated 3 comments.

Show a summary per file

File	Description
ui-tui/src/types.ts	Adds SessionInfo.lazy to represent placeholder/lazy-hydrated session state.
ui-tui/src/lib/memoryMonitor.ts	Switches @hermes/ink import to a dynamic import inside the memory-pressure tick.
ui-tui/src/components/branding.tsx	Adds inline loader UI for tools/skills while info.lazy is true.
ui-tui/src/components/appLayout.tsx	Shows SessionPanel whenever info exists (enables early skeleton rendering).
ui-tui/src/app/useSubmission.ts	Skips input.detect_drop RPC for common plain-text prompts.
ui-tui/src/app/useConfigSync.ts	Avoids startup voice.toggle status probe; initializes voice bit from env.
tui_gateway/server.py	Implements lazy agent build + immediate session.create response; makes prompt.submit non-blocking.
tui_gateway/entry.py	Avoids importing MCP tooling unless mcp_servers are configured.
tools/code_execution_tool.py	Reads code_execution config via read_raw_config() to avoid importing classic CLI at module import time.
tests/tools/test_code_execution.py	Adds/adjusts tests for _load_config behavior with read_raw_config.

---

💡 Add Copilot custom instructions for smarter, more guided reviews. Learn how to get started.

[OutThisLife]

All Copilot review threads are resolved.

Additional fixes pushed while resolving:

• cc5efb6fc fix(tui): keep non-agent session RPCs lazy

  • session.title, session.history, and session.usage now use no-wait session lookup where possible instead of forcing agent init.

• d341af22c fix(tui): preserve busy and init error signaling

  • prompt.submit now claims session.running before returning success, preserving RPC-level session busy so the frontend can queue.
  • agent-init timeout/failure now emits a normal error event instead of writing a second response for an already-settled request id.

Previously pushed fixes covered the other Copilot threads:

• import-promise retry / in-flight cleanup in memoryMonitor.ts
• read_raw_config() for MCP cold-start probe
• daemon timer for deferred build
• per-session lazy-build lock

Verification rerun after the last fix:

• python -m py_compile tui_gateway/server.py tui_gateway/entry.py
• cd ui-tui && npm run type-check && npm run build
• scripts/run_tests.sh tests/tui_gateway/test_protocol.py::test_sess_found tests/tools/test_code_execution_modes.py tests/tools/test_code_execution.py → 101 passed
• cd ui-tui && npm test -- --run src/__tests__/useSessionLifecycle.test.ts src/__tests__/useConfigSync.test.ts → 22 passed

[Copilot]

Pull request overview

Reduces hermes --tui cold-start latency by deferring expensive agent/tooling initialization and avoiding heavyweight imports on the critical startup path, while still hydrating the full tools/skills panel shortly after the UI becomes usable.

Changes:

• Introduces “lazy session” startup: session.create returns immediately with placeholder SessionInfo and triggers deferred AIAgent construction/hydration via background build.
• Removes/defers expensive imports on the startup path (lazy @hermes/ink import; conditional MCP tooling import; avoid importing classic CLI during tool discovery).
• Skips some non-essential early RPCs to keep the single stdio JSON-RPC pipe unblocked (voice status probe; file-drop detection for plain prompts), and updates the UI to show loaders/skeleton state.

Reviewed changes

Copilot reviewed 10 out of 10 changed files in this pull request and generated 1 comment.

Show a summary per file

File	Description
ui-tui/src/types.ts	Adds SessionInfo.lazy flag for skeleton/hydration state.
ui-tui/src/lib/memoryMonitor.ts	Defers @hermes/ink import and adds in-flight guarding around heap-dump actions.
ui-tui/src/components/branding.tsx	Shows inline loader for tools/skills while info.lazy and empty.
ui-tui/src/components/appLayout.tsx	Renders SessionPanel earlier (no longer gated on info.version).
ui-tui/src/app/useSubmission.ts	Avoids input.detect_drop RPC for typical plain-text prompts.
ui-tui/src/app/useConfigSync.ts	Removes startup voice.toggle status probe; initializes UI from HERMES_VOICE.
tui_gateway/server.py	Implements deferred agent build (_start_agent_build), lazy session.create, and async prompt.submit wait/build behavior.
tui_gateway/entry.py	Avoids MCP SDK import unless mcp_servers are configured.
tools/code_execution_tool.py	Reads code_execution config via read_raw_config() to avoid importing classic CLI.
tests/tools/test_code_execution.py	Adds tests validating _load_config() behavior and ensuring classic CLI is not imported.

---

💡 Add Copilot custom instructions for smarter, more guided reviews. Learn how to get started.