Skip to content

perf(tui): improve streaming performance by memoizing useSessionLifecycle return - #38491

Closed
stremtec wants to merge 1 commit into
NousResearch:mainfrom
stremtec:perf/tui-memoize-session-lifecycle
Closed

perf(tui): improve streaming performance by memoizing useSessionLifecycle return#38491
stremtec wants to merge 1 commit into
NousResearch:mainfrom
stremtec:perf/tui-memoize-session-lifecycle

Conversation

@stremtec

@stremtec stremtec commented Jun 3, 2026

Copy link
Copy Markdown
Contributor

Summary

Fixes cascading re-render chains that cause unnecessary GC pressure during 60fps streaming.

Root Cause

useSessionLifecycle() returned a new plain object literal on every render. This caused every useCallback/useMemo that depended on it — onEvent, slash, appActions, closeLiveSession — to be recreated every render cycle. During streaming at 60fps, this resulted in:

  • Massive GC churn from constantly recreated closures
  • Agents nudge firing multiple times (stale closure state reset)
  • Unnecessary React reconciliation in the entire component tree

Fix

  1. useSessionLifecycle.ts — wrap the return value in useMemo with the seven stable callbacks as dependencies. The returned object identity is now stable across renders.

  2. useMainApp.ts — remove clearSelection from onEvent's useMemo dependency array (it was never passed to createGatewayEventHandler, causing unnecessary re-creation).

Verification

  • TypeScript compilation: clean (no new errors)
  • All 187 TUI tests pass: tests/tui_gateway/ + tests/hermes_cli/test_tui_*.py + tests/cli/test_tui_*.py

…-renders

H6: useSessionLifecycle() returned a new plain object on every render,
causing every useCallback/useMemo that depended on it (onEvent, slash,
appActions, closeLiveSession) to re-run every render. During 60fps
streaming this caused massive GC pressure and stale-closure bugs
(e.g. agents nudge firing multiple times).

Fix: wrap the return value in useMemo with the seven stable callbacks
as dependencies.  Also remove clearSelection from onEvent's deps
array since it was never passed to createGatewayEventHandler.
@alt-glitch alt-glitch added type/perf Performance improvement or optimization comp/tui Terminal UI (ui-tui/ + tui_gateway/) P3 Low — cosmetic, nice to have labels Jun 3, 2026
@teknium1

Copy link
Copy Markdown
Contributor

Thanks for the focused TUI performance contribution. The core opportunity remains on current main: useSessionLifecycle still returns a fresh object at ui-tui/src/app/useSessionLifecycle.ts:405, while slash depends on the full session object at ui-tui/src/app/useMainApp.ts:837-888.

Suggested changes

  • During maintainer salvage, keep the lifecycle-return useMemo; the separate clearSelection deletion is already reflected in current main's onEvent dependency list (ui-tui/src/app/useMainApp.ts:767-782).
  • Add a focused regression test for stable lifecycle return identity when callback dependencies are unchanged.

Automated hermes-sweeper review.

@teknium1 teknium1 added sweeper:blast-contained Sweeper blast radius: contained — one narrow path / opt-in / few users area/streaming Streaming responses: gateway delivery, provider wire labels Jul 14, 2026
@kshitijk4poor

Copy link
Copy Markdown
Collaborator

Thanks @stremtec — re-derived onto current main in #77788 with your commit's authorship preserved (the hook moved into ui-tui/src/app/ across 10k commits, so the useMemo was re-applied by hand rather than cherry-picked). One correction folded during review: the dep array covers all nine returned handles including trimTail. All nine verified useCallback-stable so the memo actually holds. Closing in favor of the salvage.

kshitijk4poor pushed a commit that referenced this pull request Aug 3, 2026
Re-derivation of #38491 by @stremtec onto current main (the original is
10,119 commits behind; the hook moved into ui-tui/src/app/). The hook
returned a fresh object literal every render, defeating memoization in
useMainApp's consumers; useMemo over the (all-useCallback-stable)
handles makes the return referentially stable.

Dep array covers ALL nine returned handles incl. trimTail (the
re-derivation initially omitted it - stale-closure class).
vashkartik added a commit to vashkartik/hermes-agent that referenced this pull request Aug 4, 2026
* fix(agent): jittered, interrupt-aware backoff for empty-response retries

Empty content retries previously fired back-to-back with no delay,
wasting up to 3 rapid API calls, and could not be cancelled mid-wait.
Apply the same jittered_backoff() already used for rate-limit and
API-error retries, sleeping in small increments so a user interrupt
aborts the wait instead of blocking until it elapses.

Fixes #35230

* test: fake clock for the backoff-status test (was busy-spinning 7.5s)

The retry loop gates on real time.time() < sleep_end; with sleep mocked
to a no-op the test hot-spun 7.5 wall-clock seconds. Advance a fake
clock by each sleep amount instead (pattern precedent:
test_session_activity_persist.py).

* perf(providers): cache provider list snapshots

* test: pin the hit-path copy guard on the provider snapshot cache

The existing test only mutated the miss-path return; a mutation to
'return _PROVIDER_LIST_CACHE' (aliasing the global cache) survived the
suite. One line pins the cached-return copy. Mutation-checked.

* feat(gateway): add opt-in 'latency' runtime footer field

The runtime footer (`/footer`) shows what model ran and how full the context
is, but not how long the turn took. On a messaging platform there is no
progress bar and no shell timer — a turn that took 4 seconds and one that took
four minutes produce visually identical replies. Users comparing models,
providers, or reasoning levels have no at-a-glance signal for the one
dimension they most often care about, and "was that slow or did I imagine it?"
is unanswerable after the fact.

Adds a `latency` field to the existing footer machinery, rendering the
wall-clock duration of the agent run: `<1s`, `22s`, `1m05s`.

`gateway/run.py` measures with `time.monotonic()` immediately around the
`self._run_agent(...)` await in `_handle_message_with_agent` — the same
function that already builds the footer, so the value is the user-perceived
turn duration (monotonic, so it is immune to wall-clock/NTP adjustment).

`latency` is deliberately NOT in `_DEFAULT_FIELDS`. It is opt-in via
`display.runtime_footer.fields`. Every existing footer — and every footer a
user has today without touching config — renders byte-identically.

This is enforced by tests, not just asserted:

- `test_latency_not_in_default_fields` pins the default tuple.
- `test_resolve_footer_config_default_fields_exclude_latency` pins what
  config resolution produces for an untouched config.
- `test_default_footer_renders_byte_identically` pins five exact output
  strings for default-config renders **while supplying `turn_seconds`** —
  proving that even when the caller measures timing, a default-configured
  footer does not show it.
- `test_default_build_footer_line_ignores_turn_seconds` asserts
  `build_footer_line(...) == build_footer_line(..., turn_seconds=125.0)`
  under default fields.

Adding `latency` to `_DEFAULT_FIELDS` fails 11 of these tests.

No new config surface (reuses `display.runtime_footer.fields`), no new env
vars, no new core tool, no new model-facing schema. One new module-private
helper (`_format_latency`), one new keyword argument threaded through the two
existing footer functions, and 3 lines in `gateway/run.py`.

`turn_seconds` defaults to `None` and the field is skipped when it is `None`
or negative, so any call site that does not measure timing keeps working
unchanged.

`tests/gateway/test_runtime_footer.py` (+185): `_format_latency` boundary
table (sub-second, rounding at 59.4/59.6, the `m{:02d}s` zero-pad, 60m), the
render/skip/opt-in matrix, field-order placement, `build_footer_line`
threading, and the byte-stability block above.

RED-proved by mutation — each of these breaks tests:
- `latency` added to `_DEFAULT_FIELDS` → 11 failures
- dropping the `turn_seconds is not None and >= 0` guard → 2 failures
- `{sec:02d}` → `{sec}` → 6 failures
- `build_footer_line` not threading `turn_seconds` → 1 failure

51 passed in `tests/gateway/test_runtime_footer.py`; 54 passed across the
footer blast radius. `ruff check` clean.

* perf(transport): gate prompt cache keys by provider capability

* feat(transport): imply prompt_cache_key capability for api.openai.com

Review follow-up on the #56798 salvage: the gate shipped fully dormant
(no provider profile sets supports_prompt_cache_key, no production
caller passes it, and no plain 'openai' profile exists to set it on) —
AGENTS.md rejects dead code wired in without E2E proof.

Activate the one endpoint where the field is first-class: exact-host
api.openai.com (OpenAI documents prompt_cache_key; GPT-5.6+ docs
recommend it for cache routing). Deliberately NOT substring matching —
Azure/OpenAI-compat endpoints may reject unknown fields and stay
opt-in via the flag. 4 new tests (imply + 3 spoof/proxy/Azure
negatives); mutation-checked (substring-weakened host check fails the
spoof tests).

* perf(gateway): reuse loaded turn config for timestamp check

Re-derivation of PR #65645 onto current main: _build_gateway_agent_history
already runs inside a turn whose config was loaded once into
ctx.user_config; re-reading config from disk via _load_gateway_config()
per turn is redundant. Reuse the loaded turn config.

* perf(cli): add --prefer-offline to npm install during update (#39267)

Re-derivation of PR #39399 onto current main: pass --prefer-offline to
the web-UI workspace install (both silent and verbose arms of
_install_web_deps) and to the update-time Node dependency refresh in
_update_node_dependencies, so npm reuses its local cache instead of
re-fetching metadata. Test expectations updated to match, mirroring the
PR's own test-update commit.

* perf(cron): skip config load on idle scheduler ticks (idea from #33612)

Re-derivation of #33612 by @LeonSGP43 onto the rewritten scheduler (the
original is 10,692 commits behind; its tick() no longer exists in that
shape, so this is a fresh minimal fix crediting the PR's idea).

The gateway's built-in ticker calls tick(verbose=False) every 60s. The
idle early-return was gated on 'verbose and not due_jobs', so idle
GATEWAY ticks fell through to load_config() + worker-pool resolution
every minute. Return early on ANY idle tick; keep the post-tick MCP
orphan sweep (main intentionally reaps orphaned stdio children on idle
ticks).

3 new tests; mutation-checked (restoring the verbose-gated guard fails
the config-skip test). 66 scheduler tests green.

* fix(feishu): defer the lark_oapi import off the startup path

Salvage of #57657, ported onto the plugin layout (the adapter moved
from gateway/platforms/feishu.py to plugins/platforms/feishu/adapter.py
since the PR's base). lark_oapi takes seconds to import and holds the
GIL doing it; the module-level import made every gateway boot pay that
cost even with Feishu unconfigured.

- _load_lark_oapi() with double-checked locking binds the SDK globals
  on first use; connect() and _standalone_send() call it via
  asyncio.to_thread so the loop never blocks on the import.
- probe_bot() also calls _load_lark_oapi() (sync context) so the SDK
  probe path is preserved rather than silently degrading to the HTTP
  fallback before a first connect.
- check_feishu_requirements() is install-only and no longer rebinds
  globals; test_feishu.py gets a setUpModule that binds them eagerly
  for tests that inject fake clients.

Includes the dedicated lazy-import test file (check-does-not-import,
connect-loads-on-worker-thread).

* test: bind lark SDK globals session-wide, not per-file

CI exposed the whole class: feishu tests across MANY files (thread
routing, text batching, sdk executor, ...) inject a mock _client and
skip connect(), so the deferred import leaves the request-builder
globals None. Replace the single-file setUpModule with a session-scoped
autouse conftest fixture that binds the globals once when lark_oapi is
installed; when it isn't, the affected tests already skip via their own
skipUnless guards. Full tests/gateway run: zero failures beyond main's
pre-existing baseline (sorted failure-diff).

* fix(feishu): test SDK globals by None-ness, not globals() membership

The no-SDK fallback guards check '"Name" in globals()' — correct on
main where a failed module-level import leaves those names undefined,
but the deferred-import port pre-binds every SDK name to None, so the
guard was always true and the fallback paths called .builder() on None
(AttributeError) wherever lark_oapi isn't installed. Local runs passed
because lark IS installed here; CI's default env has no feishu extra.
Rewrote all 14 guards to 'is not None', which is correct under both
conditions. Verified by simulating CI with a lark-blocking meta_path
hook: 74 passed, 18 skipped (the skipUnless set), zero failures.

* chore: add contributor email mapping for WojtekMR3

* perf: replace COUNT(*) with LIMIT-based existence checks

Two places were using SELECT COUNT(*) when they only needed a boolean:
- has_any_sessions() called session_count() > 1 (full table scan)
- delete_session() used SELECT COUNT(*) WHERE id=? (full matching scan)

Fix:
- Add session_count_ge(n) to SessionDB — short-circuits via
  SELECT 1 FROM sessions LIMIT n, returns bool
- has_any_sessions() uses session_count_ge(2) instead of session_count() > 1
- delete_session() uses SELECT 1 ... LIMIT 1 with fetchone() is None
- Add tests for session_count_ge

* fix(state): take the connection lock in session_count_ge + document archived semantics

Review fold-ins on top of #56768 (@Skywind5487):
- session_count_ge ran its query without self._lock, unlike every
  sibling counter on SessionDB (session_count, session_count_by_source).
- Document the deliberate semantics change: session_count() defaults to
  archived = 0, which is both the expensive part (full index scan,
  measured 543us vs 4us on 20k sessions) and wrong for the only caller
  (has_any_sessions asks 'has this install ever had sessions' -- an
  archived session is still a created one).

* perf(state): index assistant tool-call rows for Insights queries

InsightsEngine._get_tool_usage and _get_skill_usage scan messages for
role='assistant' AND tool_calls IS NOT NULL, but no index aligns with
that predicate, so SQLite scans the full messages table on a large
state.db. Add a partial index over exactly those rows.

role and tool_calls are base columns in the messages table, so the index
lives in SCHEMA_SQL (created on both fresh and existing databases via the
executescript on every open) rather than DEFERRED_INDEX_SQL.

Adds schema regression coverage (fresh + reopened DB, plan uses the index)
and an Insights regression test proving tool/skill output is identical with
and without the index present.

Fixes #67341

* perf(insights): pin partial index on assistant tool-call queries

Review follow-up (#67341): on a freshly initialized state.db (before
ANALYZE has run) the source-filtered branches of _get_tool_usage /
_get_skill_usage did not select idx_messages_assistant_calls_by_session
— the optimizer drove from idx_sessions_source_id and probed each
session's messages via idx_messages_session_active, scanning non
tool-call rows. Pin the index with INDEXED BY on all four fixed-predicate
branches so the plan is deterministic for both the unfiltered and
source-filtered scopes without depending on statistics.

Safe because the index is declared in SCHEMA_SQL (created by every
read-write SessionDB._init_schema) and every InsightsEngine caller opens
a read-write SessionDB; read-only attachments (which skip schema init)
are never used for insights.

Extract the four queries into class constants and add tests: query-plan
coverage for both scopes without ANALYZE, row-level equivalence between
pinned and un-pinned forms, and an assertion that INDEXED BY fails loudly
if the index is absent.

* fix(insights): fall back to unpinned queries when the partial index is absent

The INDEXED BY pin is a hard dependency -- SQLite raises 'no such
index' when the named index is missing. That happens in production:
the web dashboard's usage analytics (_get_usage_analytics,
_get_models_analytics) open state.db read_only=True, which skips
_init_schema, so a DB last written by a pre-index version has no
idx_messages_assistant_calls_by_session and every insights call
crashes with OperationalError (reproduced E2E).

Probe sqlite_master once in __init__ and strip the pin from the four
prepared statements when absent -- identical rows, optimizer-chosen
plan, no crash. Replaces the change-detector test that froze the
crash as intended behavior with a fallback-equivalence test.

* refactor(insights): strip INDEXED BY pins via an attribute loop

Simplify-pass fold: the four copy-pasted .replace blocks meant a\nfifth pinned statement could forget its strip line — a hard 'no such\nindex' crash on read-only DBs, the exact bug the fallback prevents.\nLoop over the attribute names instead.

* perf(state): batch compression-tip row fetch in list_sessions_rich

list_sessions_rich()'s compression-root projection called
_get_session_rich_row() once per root — a separate single-row query per
compression root on every session-list render. Resolve every tip id
first, then fetch all tip rows in one WHERE id IN (...) query via the
new _get_session_rich_rows_batch().

_get_session_rich_row() is now a thin wrapper over the batch method, so
the enriched SELECT (preview + last_active) lives in exactly one place —
future column changes (e.g. #42196's include_system_prompt) only touch
one query.

get_compression_tip()'s chain walk is untouched; it's a genuine
per-session graph walk with branch/delegate-exclusion and race handling,
and batching it safely is out of scope here.

Co-Authored-By: Claude Fable 5 <noreply@anthropic.com>

* test(state): guard compact_rows threading through batched tip-row fetch

Adds two regression tests for the #59077 batch: (1) _get_session_rich_rows_batch(compact_rows=True) uses the schema-derived compact projection (no system_prompt, git_branch/git_repo_root kept); (2) list_sessions_rich(compact_rows=True) threads compact_rows through the compression-tip projection call site. Mutation-checked: hardcoding compact_rows=False at the call site fails test 2.

* refactor(state): chunk the batched tip-row IN clause at 900 ids

Simplify-pass fold: SQLITE_MAX_VARIABLE_NUMBER is 999 on pre-3.32\nbuilds (which the repo still supports — the trigram-availability\nmachinery exists for exactly that class), and limit=10000\nlist_sessions_rich callers exist in web_server. Chunk inside the\nbatch helper — the single choke point — so no call site can overflow.

* fix(yuanbao): clear _processing_msg_ids/_processing_msg_texts after each message

_dispatch_inbound_event() writes session_key → msg_id/raw_text into
_processing_msg_ids and _processing_msg_texts so RecallGuardMiddleware
can find and interrupt the currently-processing message.  These entries
were never removed after a message finished processing, causing both
dicts to grow unboundedly — one persistent entry per unique session key
for the lifetime of the bot.

Fix: clear both entries in the _process_message_background() finally
block, after super() returns.  The guard compares the stored msg_id
against event.message_id before popping: a concurrent pending message
may have already overwritten the entry in _dispatch_inbound_event while
we were running, in which case the drain task owns it and we must not
clear it.  When msg_id is absent (nothing was written at dispatch time)
the pop is a safe no-op.

Note: _msg_content_cache already bounds itself to 200 entries at the
same write site; _processing_msg_ids and _processing_msg_texts had no
such bound.

* fix(yuanbao): evict stale entries from _member_cache on TTL expiry

_build_msg_body_with_mentions() checks the TTL of each _member_cache
entry and returns an empty member list when the entry is stale, but
never removes the entry from the dict.  Over time every group_code the
bot has ever queried accumulates a permanent entry, retaining the full
member list (potentially thousands of records per group) until
disconnect().

Fix: delete the stale entry at the point it is detected as expired.
The next call to get_group_member_list_raw() for the same group will
repopulate the cache with fresh data as before.

Symmetric with the existing TTL pattern in MessageDeduplicator, which
evicts on access.

* fix(yuanbao): pop tracking entries only for truthy matching msg_id + regression tests

Follow-up on the salvaged pair: the original guard's `not msg_id` arm let an
id-less internal/synthetic event erase a tracking entry a concurrently-queued
id-bearing message's drain task still needs for recall matching (id-less
events never write entries in _dispatch_inbound_event, so they must never
pop). Tests cover: normal cleanup, id-less non-erasure, overwritten-entry
ownership handoff, TTL eviction + fresh-entry survival.

* chore: add frizikk to AUTHOR_MAP

* perf(zai): parallelize endpoint detection probes

Z.AI has separate billing for general vs coding plans and global vs
China endpoints. On startup, detect_zai_endpoint() probes up to 4
endpoints sequentially with 8s timeout each, taking 8-9 seconds when
the first endpoints return non-200 (rate limited) before a working one
is found.

Replace the sequential loop with concurrent.futures.ThreadPoolExecutor
to probe all 4 endpoints in parallel. Results are returned in
ZAI_ENDPOINTS priority order so the preference chain is preserved.

Benchmark on macOS M4 Max, Python 3.11, Hermes v0.8.0:
  Before: 8.8s (sequential: global=0.9s/429, cn=1.6s/429,
           coding-global=4.3s/200, coding-cn=2.0s/200)
  After:  ~4.5s (single round-trip, bounded by slowest endpoint)

Signed-off-by: Merlin <merlin@merlin.me>

* test(zai): cover parallel-probe contracts + restore candidate-model loop

Rebase fold: the original PR predates ZAI_ENDPOINTS growing per-endpoint
probe_models lists; the parallel worker now preserves that candidate-model
fallback loop (was: scalar model). Tests (both mutation-checked):
- candidate-model fallback within one endpoint worker
- ZAI_ENDPOINTS priority order wins over completion order
- all-fail returns None

* perf(zai): early-exit when the highest-priority endpoint wins (simplify finding)

The as_completed drain + `with` join made the parallel version WORSE than
sequential main in the common case (first endpoint succeeds fast, others
slow/unreachable): main returned at first success, the parallel version
waited for every straggler. Now: after each completion, walk endpoints in
priority order and return as soon as a success is unbeatable (all
higher-priority probes already finished); pool uses shutdown(wait=False) so
losers drain in the background. Mutation-checked: removing the early exit
makes the new timing test fail (8.2s vs <1.5s).

* Bound MiniMax OAuth error responses

* fix(minimax-oauth): read streamed error bodies inside the client context + real-transport tests

Follow-ups on the salvaged bounded-read fix:
- refresh flow: the non-200 branch reads a STREAMED body, which fails
  (ReadError/StreamClosed) once the httpx.Client context has exited —
  moved inside the context. Repro + regression test use a real socket
  server (MockTransport buffers in memory and cannot catch this).
- truncation guard: >limit bodies end with ...[truncated] (mutation-checked
  against the is_stream_consumed fallback).
- test mocks now model the streamed-read surface (is_stream_consumed,
  iter_bytes, client.send) so non-200 paths exercise the real bounded read.

* chore: map xaydinoktay@gmail.com to aydnOktay

* chore(contributors): map four B2 salvage author emails (#77641)

unixwzrd.register@mac.com -> unixwzrd (#74679); dai.suzuki.829@gmail.com -> hariNEzuMI928 (#75395); lexharddrive69@gmail.com -> hdd69 (#38470); coder@trevhome.local -> trevornk (#76282). Needed for the B2 desktop-renderer salvage attributions.

* perf(session-search): project fields before enrichment

* test(session-search): guard projected enrichment

* fix: skip memory prefetch on trivial user prompts (greetings)

Salvage of PR #25350 (commits 88ffede2d + 2b848a0b2 + 3136dc63a, squashed
and ported): the run_agent.py prefetch site the PR gated has since moved
into agent/turn_context.py's build_turn_context(), so the trivial-query
gate lands there instead.

- Gate the per-turn memory_manager.prefetch_all() on a trivial-prompt
  check so greetings/acknowledgements ('hi!', 'thanks', 'ok') no longer
  block the turn on provider network round-trips or inject stale context.
- Extend honcho's _TRIVIAL_PROMPT_RE with greetings and a trailing
  punctuation class so 'hey!' / 'hello.' classify as trivial.
- Add honcho classifier tests for greeting forms.

* chore: add ayushere to AUTHOR_MAP

* refactor(memory): single shared trivial-prompt classifier + gate tests

Rebase fold on the salvaged gate:
- is_trivial_prompt/TRIVIAL_PROMPT_RE move to agent/memory_provider (the
  ABC both the core gate and providers already import) — one source of
  truth; honcho's _TRIVIAL_PROMPT_RE now aliases it, turn_context and the
  queue_prefetch_all warm path (a sibling site main grew after the PR's
  base) both use it
- tests: gate tests at the prefetch call site (mutation-checked), shared
  classifier tests incl. prefix-collision guards (k8s/yolo/note/supper),
  and honcho dialectic-machinery tests re-driven with a substantive prompt
  ("hello" became trivial by design — those tests exercise thread cadence,
  not the classifier)

* refactor(honcho): delegate _is_trivial_prompt wholly to the shared classifier

Simplify-pass finding: sharing only the REGEX left the wrapper logic
(empty/strip/slash checks) duplicated, half-defeating the no-drift goal.
The classmethod now calls agent/memory_provider.is_trivial_prompt directly;
_TRIVIAL_PROMPT_RE stays as a class attr for backward compatibility with
any external referents.

* fix(desktop): measure adaptive stream flush through the deferred commit frame

scheduleDeltaFlush's adaptive floor is driven by lastFlushCostRef, but
runFlush only timed flushQueuedDeltas(), the synchronous store write.
While a session streams, syncSessionStateToView defers the $messages
publish (React commit + Streamdown re-parse) to its own rAF, so the
measured cost stayed near zero and the floor collapsed to the fixed
33ms path no matter how expensive the real commit was.

runFlush now records the write cost as a fallback, then extends the
measurement through a rAF registered after the view-sync one: it runs
in the same frame right after the deferred commit, and the rAF
timestamp marks frame start so only in-frame work is counted, not the
vsync wait. A stale callback from before a newer flush is ignored, and
a hidden renderer that never fires rAF keeps the write-cost fallback.

* fix(desktop): dedupe optimistic user turns for all wire references, not only images

* test(desktop): cover wire reference normalization edges

* fix(desktop): sort reference-kinds import per lint gate

* fix(desktop): full-jitter backoff on gateway WS reconnect loops

All three desktop reconnect loops (primary gateway boot, secondary
multi-profile gateway pool, plugin event socket) used bare exponential
backoff with no jitter. After a gateway restart every disconnected
client redials on the exact same schedule, so the reconnect attempts
land in lockstep instead of spreading out -- a burst that can starve
the gateway's file descriptors while it's still coming back up.

Add reconnect-backoff.ts implementing AWS-style full-jitter backoff
(random delay in [0, min(cap, base * 2^attempt))) and wire it into all
three call sites in place of their local Math.min/2**attempt math.
Manual reconnect paths already reset the attempt counter and bypass
the timer entirely -- unchanged.

* fix(desktop): escalate gateway reconnect on elapsed time, not attempt count

With the full-jitter backoff (300ms base) six attempts can elapse in ~9s,
so the old RECONNECT_ESCALATE_AFTER=6 attempt threshold raised the
recoverable boot error during a brief post-boot blip — breaking the
'a remote that drops post-boot keeps looping with NO boot.error' contract.
Escalate after RECONNECT_ESCALATE_AFTER_MS (45s, matching the old
deterministic 1->15s ladder's calibration) elapsed since the first failed
reconnect of the episode. Reset on clean open, manual/wake reconnect, and
soft switch, preserving the reset-on-success path.

* chore(contributors): map vittoria3103.123@gmail.com -> VittoriaLanzo (#77665)

Needed for the #62082 curator toolset-pin salvage attribution.

* fix(desktop): un-break the .btn-arc rule — '*/' inside a CSS comment ended it early

The comment above .btn-arc contained 'bg-*/', whose */ terminated the comment mid-sentence, leaving 'text-* variant utilities. */ .btn-arc {' as an invalid prelude — CSS error recovery can drop the whole .btn-arc rule. Reword so no */ appears inside the comment.

Extracted from #59352 by @rerdi92 (the rest of that PR — a month-stale icons.ts rewrite and a chunk-size warning-ceiling bump — is superseded/masking and was not salvaged).

* perf(desktop): stop idle chat re-renders — memo ChatView, stable tile props, gated adapter re-sync

Re-derive of PR #38470 on today's main (its target file desktop-controller.tsx no longer exists after the contrib/ refactor; the three surviving ideas are applied at their new homes):

- incremental-external-store-runtime: the dep-less setAdapter effect ran every render; gate on [runtime, store] — behavior-preserving because __internal_setAdapter early-exits on identical store.
- ChatView is now memo()d, and session-tile hoists its inline arrow props to useCallbacks/module constants so the memo actually holds.
- Render-count regression test (mocked Thread) proves an unrelated parent re-render no longer re-renders the chat shell.

Credit: idea and original implementation by @hdd69 in #38470.

* perf(curator): trim dead tool-schema from the LLM review fork

The curator LLM review loop (_run_llm_review) built its AIAgent without
enabled_toolsets, so it advertised the full default catalog (~30 tools plus the
context_engine lcm_* family) on every call. The fork uses only four tools, fixed
by its own system prompt, with no dispatch path to the rest, so ~26 tool schemas
shipped on every request as dead weight: ~7K input tokens per call on a loop that
makes 50-100 calls per consolidation pass.

Restrict the fork to enabled_toolsets=["skills", "terminal"], the same tools the
prompt already names. Behavior-neutral: the prompt held the model to these tools
and nothing routed calls to the others. Mirrors the background_review fork
(background_review.py:788-794). Call-site only; AIAgent already forwards the kwarg.

Adds test_review_fork_restricts_toolsets_to_skills_and_terminal (captures the
constructor kwarg) and test_review_fork_toolset_surface_is_skills_plus_terminal
(pins the resolved surface).

* fix(desktop): keep a mid-turn reply on screen when its session is reopened

Switching sessions while a turn streams (or right as it completes) could
leave the assistant reply missing until restart. Resume merges stored
history with the gateway's `inflight` projection, whose assistant row is
text-only and often an empty `assistant-stream-${sessionId}` shell; both
reconcile paths then dropped the local pending row that held the only copy
of the streamed text, reasoning and tool calls.

A shared pair of guards replaces the ad-hoc comparisons at all three sites.
`localPendingSupersedes` accepts the cached row only when it is the same
reply further along — an empty shell it has content for, or text it strictly
extends — so a longer unrelated row can no longer hijack an ordinal or reuse
a stream id, and a retained `inflight.error` snapshot is never mistaken for
an empty shell. `withAuthoritativeTurnState` then takes content from the
renderer while liveness, row id and reactions stay the backend's call, so a
settled shell cannot leave a finished reply spinning.

Co-authored-by: arimu1 <19286898+arimu1@users.noreply.github.com>

* fix(desktop): stop a finished reply rendering twice after history catches up

When a turn's reply commits under its own id, the settled local
`assistant-stream-*` row shifts one assistant ordinal earlier, so ordinal
pairing finds nothing at its slot and re-appends it — the same answer twice.

Drop a settled stream row only when the authoritative transcript already
carries that exact text. Keying `isPendingAssistant` on the explicit pending
flag alone would also have fixed this, but it discards the sibling case in
the same report: a reply that finished locally before the gateway committed
it, where the local row is the only copy that exists.

Co-authored-by: Dolverin <59100064+Dolverin@users.noreply.github.com>

* chore(contributors): map two B3 salvage author emails (#77685)

abdulsalamalotaibi86@gmail.com -> carbongotfound (#74025); soundbrokaz@kakao.com -> JeremyDev87 (#72813).

* refactor(desktop): hoist the reference-line matcher; drop dead textWithoutImageRefs

Follow-up to #77653: textWithoutReferenceLines built a fresh /g RegExp per call and hand-managed lastIndex — but it runs on both sides of every message comparison in the reconcile loops. An anchored non-global regex has no shared-lastIndex hazard and can be hoisted to module scope. Also removes textWithoutImageRefs, whose last production consumer #77653 replaced (kept IMAGE_REF_LINE_RE for extractImageRefs), and retargets its now-stale comment.

* fix(desktop): do not sandwich structured mid-turn rows with inflight dump

Skip pure-text inflight.assistant projections when the transcript already
has reasoning/tool-call structure, and only overlay journal answer text
on strict extension.

Fixes #76444

* fix(desktop): scope inflight dump suppression to the live turn tail

Only skip/graft structure for the current live assistant (stream id,
pending, or after the latest user), not completed historical tool rows.
Require live-tail identity for same-turn structure carry. Align journal
overlay with strict answer-text extension.

Addresses review + CI on #76744.

* fix(desktop): require structure-bearing row for live-tail same-turn carry

Structure-only same-turn carry used (live(previous) || live(message)), so a
new live text-only assistant at a compression-rewritten ordinal could inherit
reasoning/tool parts from an unrelated historical structured row.

Require the structure-bearing cached row itself to be live-tail (pending /
assistant-stream-* / interim). Add regressions for non-extending live dump
carry and the compression graft rejection.

Addresses salvage path on #76744 / #76444.

* refactor(desktop): one live-tail vocabulary for transcript reconciliation

Two fixes landed overlapping helpers on the same statement: the mid-turn
reply guard grew `isLiveProjectionRow` / `hasStreamedContent`, while the
inflight-dump guard grew `isLiveTailRow` / `hasStructuralParts`. Two
definitions of "is this row live" and "does it carry content" in one
function is how the next change silently reshapes one of them.

Collapse to a single module-level pair. `isLiveTailRow` now covers pending,
stream ids, inflight projections and sealed interim rows, so the reply guard
also stops treating an interim row as committed history; `hasStreamedContent`
is defined in terms of `hasStructuralParts`. Both text-extension checks route
through `isStrictAnswerTextExtension` rather than a bare `startsWith`.

Also hoists the live-tail lookup out of an inline IIFE and fixes the lint
warnings it carried.

Co-authored-by: 686f6c61 <github@00b.tech>

* fix(dashboard): cache plugins hub payload and avoid auth probes

* test(dashboard): cover install-hook invalidation of plugins hub cache

* fix(dashboard): warm cold check_fn verdicts with a background probe

On dashboard-only sessions nothing else executes check_fn warmers (they
live only in the tool-schema build), so the hub's read-only cache lookup
would report auth_required=False forever. On a cache miss, schedule a
deduplicated daemon-thread probe off the request path; the short hub TTL
surfaces the verdict on the next fetch.

* fix(desktop): cancel the pending commit-cost measurement rAF

Follow-up to #77652: each runFlush registered a fresh requestAnimationFrame and never cancelled it. Chromium parks rAF callbacks for hidden renderers, so a long hidden stream at the 33ms floor accumulates thousands of parked closures that all fire in the first frame on refocus (all but one no-oping through the stale-frame guard). Track the pending handle, cancel it before requesting a new one (only the newest flush's measurement matters), and cancel on unmount.

* perf(dashboard): skip full InsightsEngine on /api/analytics/usage (#18511)

* perf(dashboard): keep tools in focused analytics usage (#18511)

* refactor(insights): drop consumer-less get_skill_breakdown alias (simplify-pass)

The 2-line alias had zero production consumers (web_server calls
get_usage_breakdown directly). Tests rewired onto the real API; the
contracts they pin are unchanged. Stale test docstring fixed.

* fix(web): clamp dashboard pagination and analytics-days params (#39200 + #74778 salvage)

Re-derivation of aydnOktay's twin clamp PRs onto current main (the
session-list endpoints moved into web_routers/; the analytics endpoints
gained asyncio.to_thread wrappers since the originals):

- limit le=100 on /api/sessions, /api/sessions/search and the
  /api/profiles/sessions fan-out (one unbounded request could drag every
  session row + correlated-subquery preview work out of SQLite, times
  every profile's state.db on the fan-out).
- days ge=1 le=365 on /api/analytics/usage + /api/analytics/models
  (huge or non-positive values force full-history InsightsEngine work or
  inverted windows; the UI only offers 7/30/90 presets).

FastAPI Query bounds reject at the validation layer (422). 8 new tests;
both clamp classes mutation-checked (clamp removed -> its tests fail).

* fix(clamps): raise profile fan-out limit to le=500 (simplify-pass finding)

le=100 would 422 real desktop callers: sessions-settings fetches
archived at limit=200, the command palette lists at 200, and the
electron remote-merge over-fetches limit+offset (exceeds 100 at
offset>=81, and its .catch(()=>null) silently drops remote sessions).
Clamp must sit above real client maxima. New test pins limit=200 w/
offset.

* fix(web): avoid blocking provider validation

* perf(plugins): seed plugin routes from sessionStorage cache for instant render

- Plugin manifests are now cached in sessionStorage on fetch.
- On refresh, plugin routes are registered synchronously from cache, preventing unwanted redirects to /sessions.
- Removes the !pluginsLoading guard from the catch-all route in App.tsx, as plugin routes are now always available on first render.
- Background fetch always updates the cache and routes, so new/removed plugins are reflected after reload.
- Resolves the race condition where plugin pages would redirect to /sessions on hard refresh.

* fix(plugins): validate cached manifests are an array

* test(plugins): export cache helpers and add focused fallback/refresh tests

* fix(plugins): keep loading gate when cached manifests include a /chat override

The sessionStorage seed set loading=false whenever any cache existed, which
defeats App.tsx's load-bearing pluginsLoading gate: with a cached manifest
that declares tab.override === "/chat", the persistent ChatPage host must
NOT mount before plugins resolve, or it spawns a PTY and gets yanked when
the override plugin takes over the route.

Seed loading=false from the cache only when no cached manifest overrides
/chat (canSeedLoadedFromCache); manifests are still seeded either way so
plugin routes register synchronously on refresh. Adds focused tests for
the gate, including the /chat-override case.

* perf(dashboard): serve hashed /assets bundles with immutable cache headers

Every hashed bundle chunk under /assets/ was served with no caching
directives, so each dashboard load re-fetched (or at best revalidated)
every JS/CSS chunk. Those filenames carry a Vite content hash — the
bytes behind a given URL can never change; a rebuild mints new
filenames referenced by a freshly served index.html.

Mark them Cache-Control: public, max-age=31536000, immutable:
- the /assets StaticFiles mount, via a subclass that stamps the header
  on 200s only (404s stay uncached — a rebuild can create the file),
- serve_css, preserving its X-Forwarded-Prefix url() rewrites for
  /fonts/, /fonts-terminal/, /ds-assets/, /assets/.

index.html keeps no-store, no-cache, must-revalidate — it is the
mutable entry point that binds users to the current hashes.

The original PR also added hand-rolled per-request gzip compression of
asset responses; that part is deliberately dropped. This server is a
localhost-default dashboard backend: compressing every response on the
CPU to save loopback bandwidth is a pessimization, and callers that
front it with a real proxy already get compression there.

Salvaged from PR #28543 (idea by @sea-monsters; gzip groups dropped as
described above).

Co-Authored-By: Claude Fable 5 <noreply@anthropic.com>

* feat: reset-aware primary restore — stay on fallback until the rate-limit window resets

restore_primary_runtime retries the primary every turn once the 60s
transient cooldown clears. For subscription-window limits (Claude
Pro/Max 5h windows, Codex weekly caps) the reset is hours or days away,
so every retry is a guaranteed failure costing two provider switches
and two prompt-cache invalidations per turn.

Add CredentialPool.next_available_at() (earliest reset across exhausted
entries; None when available now or no reset info) and gate the restore
on it: skip while the primary's pool says nobody can serve, restore on
the first turn after the reset elapses. Fail-open: any gate error or
missing reset info falls through to the existing per-turn retry, so
recovery can never be later than today. Cross-provider fallbacks
consult the PRIMARY's pool (not the attached fallback pool), reusing
the loaded pool for the existing rebind to keep auth reads at one per
restore.

* fix(credential_pool): run next_available_at under the pool lock

Review fold on the #67642 salvage: next_available_at() called
_available_entries() — which prunes DEAD entries, syncs tokens, and
persists — and iterated self._entries with no lock, racing concurrent
select()/rotation exactly as has_available()'s comment warns. Wrap the
method body in self._lock and pin it with a non-blocking-acquire probe
test.

* fix(credential_pool): defer single-use-token refresh outside threading lock

select() and acquire_lease() held self._lock during the entire
_available_entries() loop, which for openai-codex and xai-oauth providers
includes a cross-process file lock (_auth_store_lock) plus OAuth token
refresh HTTP POST.  The lock timeout can exceed 20 seconds, blocking all
credential pool consumers across every gateway thread and subagent.

Collect single-use-token refresh entries under the lock, then execute the
refreshes outside it.  On success the refreshed entry is merged back into
the pool and re-selected.  Non-single-use providers (anthropic, nous)
continue refreshing inside the lock since their refresh is a simple HTTP
POST with no cross-process coordination.

* fix(credential_pool): serialize deferred-refresh pool mutations

Review folds on the #71775 salvage (dossier findings 1+2):

- self._lock becomes an RLock and the mutation primitives
  (_replace_entry, _persist) are now self-locking, so the deferred
  single-use-token refresh path — which deliberately runs its
  cross-process flock + OAuth network I/O OUTSIDE the pool lock —
  still serializes its pool mutations against concurrent
  select()/rotation. In-lock callers re-acquire reentrantly.
- Dropped _refresh_pending_entries' redundant second _replace_entry:
  _refresh_entry already merges the refreshed entry internally.

Adds tests/agent/test_credential_pool_deferred_refresh.py pinning both
invariants: select() must NOT hold the lock during the refresh window
(the PR's whole point), and the post-refresh mutations MUST contend on
the lock (blocking-thread probe).

* `hermes sessions optimize-storage` aborts with

```
Error: optimization failed: no such table: messages_fts_trigram
No data was lost. Re-run to resume.
```

on any install where the trigram FTS index is legitimately absent. The failure is
deterministic — re-running can never make progress, because the crash happens at the same
point every time — so the database is permanently stuck on the legacy high-footprint FTS
layout with no supported way forward.

Observed on a 5.4 GB production `state.db`. After the fix the same database optimized
successfully and shrank to 3.3 GB.

The trigram index is absent whenever the runtime cannot maintain it. On a SQLite build
without the `trigram` tokenizer, `_ensure_fts_schema()` returns `False`, so `__init__`
leaves `self._trigram_available = False` and no `messages_fts_trigram` table on disk. This
is a **supported degraded runtime**, not damage — CJK/substring search falls back to
`LIKE` and everything else works normally. `_is_fts5_unavailable_error()` and
`_warn_trigram_unavailable()` exist specifically to make this path graceful.

Two code paths write the boundary sweep for the deferred FTS rebuild, and only one of them
respects that flag:

| Function | Trigram `INSERT` guarded? |
|---|---|
| `fts_rebuild_step()` | ✅ `if include_trigram:` where `include_trigram = self._trigram_available` |
| `_fts_rebuild_finish()` | ❌ unconditional |

`_fts_rebuild_finish()` runs the boundary sweep at the *end* of the backfill. Its
unguarded `INSERT INTO messages_fts_trigram …` raises `OperationalError`, which propagates
out of `optimize_fts_storage()` and aborts the entire optimization — *after* the backfill
has already completed. Hence the characteristic output showing 100% progress immediately
before the error:

```
Rebuilding index: 100% (909,671/909,671)
Error: optimization failed: no such table: messages_fts_trigram
```

There is a second, quieter consequence. The teardown phase that reclaims the demoted
`fts_v22_trash_*` shadow tables runs *after* the backfill phase in
`optimize_fts_storage()`. Because the crash happens before teardown is ever reached, those
tables are never emptied or dropped — so the space the migration was supposed to reclaim
stays allocated indefinitely, and the leftover trash tables look (misleadingly) like
evidence of a half-finished migration.

Build a populated v23 database, set the deferred-rebuild markers, then reopen it on a
runtime where `_ensure_fts_schema('messages_fts_trigram', …)` returns `False` (exactly
what a SQLite build without the trigram tokenizer produces) and call
`optimize_fts_storage()`:

```
[precondition] trigram absent, _trigram_available=False, rebuild pending  ✓

RED  ✗ optimize_fts_storage raised OperationalError: no such table: messages_fts_trigram
```

With this patch applied, unchanged harness:

```
     optimize_fts_storage returned {'ok': True, 'vacuumed': None}
GREEN ✓ optimize ok; markers cleared; base FTS 'zebra' -> 200 hits
```

Full harness and transcripts in `TEST-EVIDENCE.md`.

Gate the sweep on `self._trigram_available`, exactly as `fts_rebuild_step()` already does:

```python
include_trigram = self._trigram_available

def _do(conn):
    ...
    if include_trigram:
        conn.execute("INSERT INTO messages_fts_trigram(...) ...")
```

The base `messages_fts` sweep and the marker cleanup are untouched, so the rebuild still
finalizes correctly and the index remains complete for every row it is responsible for.
The fix does not disable or weaken search to dodge the error — the regression tests assert
that base FTS still returns results afterwards.

`TestFtsRebuildFinishWithoutTrigram` in `tests/test_hermes_state.py`:

- `test_rebuild_finish_skips_trigram_when_unavailable` — drives `_fts_rebuild_finish()`
  directly on a trigram-less runtime; asserts it completes, clears both rebuild markers,
  and leaves base FTS searchable.
- `test_optimize_fts_storage_succeeds_without_trigram` — end-to-end through the public
  `optimize_fts_storage()` entry point; asserts `ok=True`, markers cleared, search intact.

Both use the existing `_NoTrigramConnection` helper already in the file. Both fail on
`main` with `no such table: messages_fts_trigram` and pass with this patch.

`tests/test_hermes_state.py` passes in full (463 tests → 465 with these two). `ruff` clean.

This PR is the crash only.

A companion PR narrows `_db_opens_cleanly()` so that
`hermes sessions repair --check-only` stops reporting a write-broken FTS schema as
healthy — the gap that makes this class of problem hard to diagnose in the first place.
The two are independent and can land in either order.

* perf(tools): shrink lazy tool catalog overhead

* refactor(tool-search): drop dead fallback ladder in _available_source_summary

Simplify-pass finding: _listing_group_label already falls back to 'other' for empty source names, and _classify_source guarantees source_name=='' only when source=='other' — both  legs were dead by construction. Aligns the summary path's grouping with the listing path.

* fix(credential_pool): unpack the tuple in next_available_at's gate

Cross-PR interaction fix: #77714 (salvage of #71775) changed
_available_entries to return (available, pending_refresh) while #77631
(salvage of #67642) added next_available_at() which still truthiness-
tests the bare return. A non-empty tuple is always truthy — even
([], []) — so the reset-aware gate silently returned None ('no wait
info') for every exhausted pool, disabling the feature #77631 shipped.
Unpack the tuple and test the available list.

Also adapts the lock-probe test for the RLock introduced by #77714
(same-thread non-blocking acquire always succeeds on an RLock; probe
from a helper thread instead).

* chore: map copii.list@gmail.com to stremtec

* chore(contributors): map marzukia@users.noreply.github.com -> marzukia (#77774)

Needed for the #37117 salvage (#77696 CI failure).

* chore: map bot@bkstock.dev to BKStock

* perf(session): route SQLite PRAGMAs through central apply_database_pragmas

Addresses review from @teknium1 on PR #71755:

- Extended apply_database_pragmas() to handle cache_size, mmap_size,
  and temp_store from config.yaml (alongside existing wal_autocheckpoint
  and journal_size_limit). No hardcoded defaults — all values are
  opt-in via config.yaml, avoiding policy conflicts with other PRs.
- Applied to ALL connection types: writer (_connect_and_init),
  read_only cross-profile attach, and WAL per-thread readers
  (_get_read_conn). Previously PRAGMAs only ran on the writer path.
- Removed inline PRAGMAs from _connect_and_init — single source of
  truth in apply_database_pragmas().
- Documented config keys with examples in function docstring.

* fix(pr): remove remnant local PRAGMAs from PR branch

* test(session): guard config-gated performance PRAGMAs across all connection types

E2E guard for the salvaged PR #71755: database.cache_size/mmap_size/
temp_store from config.yaml must reach the writer connection, the
read-only cross-profile attach, and the WAL per-thread reader — and a
default install (no database: keys) must keep byte-identical SQLite
defaults on every connection type. Also covers integer-coercion
rejection of garbage values for the three new keys.

cache_size uses -16000 (not the doc example -2000) because -2000 is
SQLite's compiled-in default and would not discriminate a regression.

* fix(desktop): flush queued deltas on window focus

* perf(desktop): stop scroll and status loops in busy sessions

* perf(desktop): pause hidden-pane timers in agents view, cron sidebar, and floating pet

Partial pick of the surviving renderer hunks from #75395 (perf commit
6502e441d plus fixup 3fbbc9c1d): gate the 500ms subagent now-ticker and
the cron sidebar 1s ticker/run-poll on usePaneVisible, and skip the
legacy floating-pet poll while the document is hidden. Dropped hunks
(electron/main.ts, vitest.setup.ts/config) intentionally excluded.

* style(desktop): restore alphabetical import order in agents/index.tsx

* refactor(desktop): shared pulse beat + fully-gated cron peek (simplify folds)

Two findings from the simplify pass on the final trio diff:

- status-pulse: one pause controller + one aligned period timer shared by all StatusPulse instances (ref-counted), instead of N x (document/window/bridge listeners + unsynchronized 5s wakes) — a sidebar can show dozens of pulsing dots. Pause still cancels in-flight animations so the compositor sleeps immediately.
- cron-jobs-section: the runs-peek effect created its interval even while the pane was hidden (callback no-oped but the timer still woke the renderer every 8s/60s per expanded job). Early-return when hidden — visibility is already in the dep array, so becoming visible restarts load + timer.

* fix(lint): import sort + eslint-disable for timer-handle ref clear in effect

CI-caught: cron-jobs-section had an extra blank line between sorted imports; use-message-stream's visibility-flush effect assigns flushHandleRef.current=null inside a useEffect (legitimate timer-clear, not an atom mirror) — eslint-disable-next-line per the rule's documented convention.

* fix(state): narrow FTS UPDATE triggers with AFTER UPDATE OF + migration

Retarget #73639 onto the SessionDB mixin split (hermes_state_common /
hermes_state_schema). Fresh installs create UPDATE OF content/tool_*
triggers; existing broad AFTER UPDATE triggers are inspected and
replaced under schema init without an FTS rebuild (WHEN clauses already
guarded content correctness; OF skips non-content status writes that
saturated disk I/O on large state.db).

Tests: tests/test_fts_update_of_narrowing.py (4)

* fix(state): fail closed on CJK trigger migration

* fix(state): quarantine CJK when ensure soft-fails after OF migration

_ensure_fts_cjk_schema never raises on OperationalError; post-condition
after dropping messages_fts_cjk_update now requires a narrowed UPDATE
trigger or durable fts_cjk_stale + unavailable. Covers the production
soft-fail path the raise-only handler missed.

* refactor(state): drop unreachable regex guard in trigger migration

Simplify-pass fold: to_drop names come from the literal update_names\nallowlist via IN binding, so the [A-Za-z0-9_]+ fullmatch could never\nfail — and if it somehow did, its `continue` would miscount (the\nskipped trigger stayed in len(to_drop)/the log while CREATE TRIGGER\nIF NOT EXISTS silently kept the broad variant). Delete the guard and\nits function-local re import; keep the invariant as a comment.

* fix(security): reject always-blocked OpenViking endpoints

## Summary
- Normalize OpenViking endpoints through `is_always_blocked_url` and fall back to the default local endpoint when poisoned.
- Keep intentional loopback / LAN self-host working.
- Add focused unit tests.

## Salvage / credit
Memory-provider endpoint floor sibling of RetainDB/Supermemory always-blocked hardening (avoids over-broad #4984-style private-IP bans).

(cherry picked from commit 8fa607d0aedb8c5fca398d7f112b1b25ade54fa2)

* fix(openviking): fail closed on blocked endpoints

(cherry picked from commit 389a90b81c9c2c89810f2fa7461f8faa9a5c9578)

* fix(openviking): don't spawn a second server onto a live port

`_start_local_openviking_server()` spawned `openviking-server`
unconditionally. Both callers — `initialize()` and the runtime
unreachable handler — reach it from a health probe, and that probe can
time out client-side while the server is up and serving. The spawned
process then loses the data-directory lock and exits immediately with
`DataDirectoryLocked`; because the probe keeps timing out, the cycle
repeats every cooldown window (~5 min observed).

The existing 30s `_failed_refresh` cooldown paces the loop but cannot
stop it, since it expires while the underlying condition persists.

Probe the target host:port before spawning and treat an occupied port as
already-started. This guards both call sites at their single convergence
point. The probe deliberately tests only that a listener owns the port —
enough to know a second server would lose the lock — and says nothing
about that listener's health.

The parse/probe now precedes the PATH lookup, so a reachable server is
reported as running even when `openviking-server` is not on PATH.

Fixes #74846

(cherry picked from commit b49427d85fd6628eb4a7fe099e5c390c5c4cc935)

* fix(openviking): drop stale "disabled for this Hermes run" warnings

The provider used to disable OpenViking permanently when the server was
unreachable. That was fixed: `_ensure_client()` now reconnects lazily,
with a 30s cooldown gate in `_ensure_client_locked`.

Only one of the seven user-facing warnings was updated to match. The
other six still told the user memory was "disabled for this Hermes run",
which is no longer true — every one of those paths is retried on the next
access. A user who reads the old message has no reason to retry, which is
very likely how #5721 ("never recovers") came to be filed against
behaviour that already recovers.

All six sites were traced to confirm none is terminal for the run: the
`initialize()`-time and waiter-thread failures never arm `_failed_refresh`
(only line 2439 does), so they retry on the very next access with no
cooldown at all.

The replacement wording deliberately omits the "(after cooldown)"
parenthetical used at the already-correct site — that detail is only
accurate where `_failed_refresh` was just armed. The neutral phrasing is
true at all six.

Also promotes two clause separators to periods to avoid "…; …disabled;"
collisions.

(cherry picked from commit 8346403a4b97af503d26b0f7905ff513828d821e)

* fix(openviking): re-arm the commit guard after in-place compression

`_committed_session_ids` is a permanent per-sid latch, and
`_session_needs_commit` checks it before the turn counter by design — a
racing sync_turn can re-increment `_turn_count` after commit+reset, so
the guard must win to stop a double-commit.

That is correct for a session being left behind. It is wrong for one
that keeps its id. `compress_context()` commits before rewriting the
transcript in both modes, and with `compression.in_place: true` (the
default) `on_session_switch` receives the same id and does not rotate.
The latch then rejects every later commit for a still-live session — the
next compression, /new, normal session end, startup recovery — so every
post-compression turn is silently never extracted.

Rotation mode is unaffected because a fresh child id is minted and
starts clean, which is what confirms the latch's intent was only ever to
dedupe the departing id.

Clear the latch when compression completes without rotation. Turns
arriving after that point are genuinely new, and this is a defined
moment rather than a race. The rotation path is untouched, so the old
id stays latched and its _finalize_session_async still dedupes against
the compression commit.

Fixes #74695

(cherry picked from commit d1e5c3dc33ef0d43d021662674e1a7cd5e43eecd)

* test(openviking): cover the compression lifecycle, not a hand-set latch

Review feedback: the previous test called _mark_session_committed
directly, so it verified the guard's behavior but not the wiring that
sets it — a future break in the commit_memory_session -> same-id
compression-boundary path would not be caught.

Add a lifecycle regression that drives the real sequence: on_session_end
commits through the actual path, on_session_switch(same id,
reason="compression") crosses the boundary, sync_turn records a genuinely
new turn, and a second on_session_end must produce a second commit POST.

Without the fix it fails showing exactly one commit call, which is the
reported data loss: every turn after the first compression is dropped.
The rotation and /undo tests stay as scope guards.

(cherry picked from commit 0ca5a330630a30b105cbbc32e8a23f2c5ffe0eab)

* fix(memory): read non-secret provider config from config.yaml for OpenViking and RetainDB

OpenViking is_available() only consulted env vars and use_ovcli_config, so an
endpoint saved to config.yaml (e.g. by the Dashboard) reported needs_config;
_resolve_connection_settings() likewise never folded config.yaml's non-secret
fields into its chain. RetainDB initialize() read base_url/project from the
environment only, ignoring the values the Dashboard writes to config.yaml.

Both now resolve non-secret fields as env -> (ovcli ->) config.yaml -> default;
secrets still come from the environment. Adds regression tests for both.

Fixes #68209

(cherry picked from commit dca57915b97b5705b30927a062e1d0f2f23d3841)

* fix(openviking): read recall settings from config.yaml first, env vars as fallback

_recall_config() previously read all settings (recall_limit, score_threshold,
recall_resources, etc.) exclusively from environment variables. This forced
users to store behavioural configuration in .env, violating the Hermes
convention that .env is for secrets only.

The infrastructure to load config.yaml -> memory.openviking was already in
place via _load_hermes_openviking_config(), but _recall_config() never
called it.

Fix: call _load_hermes_openviking_config() and pass its values as the
default parameter to _env_int/_env_float/_env_bool. Env vars still override
config.yaml values, preserving backward compatibility.

Closes #62540

(cherry picked from commit 6aadf1256835745e0302aa3d3b5ae0660b368637)

* test(openviking): cover config.yaml recall settings with temp-HERMES_HOME tests

Add three tests to TestOpenVikingConfigSchema:

1. test_recall_config_reads_from_config_yaml — writes memory.openviking
   settings in config.yaml and verifies _recall_config() consumes them.

2. test_recall_config_env_overrides_config_yaml — writes both config.yaml
   and OPENVIKING_RECALL_* env vars, verifies env takes precedence.

3. test_recall_config_partial_config_yaml — partially populated config.yaml
   falls back to defaults for omitted keys.

All 46 openviking_plugin tests pass (43 existing + 3 new).

(cherry picked from commit b8d7834caf06c6912004333c270faa248eaed4cd)

* fix(openviking): integrate reliability and configuration hardening

* chore(contributors): map OpenViking source authors

* test(retaindb): guard scoped secret config resolution

* fix(openviking): verify servers before sending credentials

* fix(openviking): catch endpoint errors in setup validation functions

Review follow-up for salvaged PR #76782. Three setup-wizard
validation functions called _normalize_openviking_url outside their
try/except blocks. Since _normalize_openviking_url now raises
_OpenVikingEndpointError for blocked or malformed endpoints, an
invalid endpoint would crash the wizard instead of returning a
friendly (False, message) tuple.

- _validate_openviking_auth: move _normalize_openviking_url inside try
- _validate_openviking_root_access: same
- _validate_openviking_setup_values: catch _OpenVikingEndpointError explicitly
- Remove dead ternary in _normalize_openviking_url safety check (candidate
  always has http/https scheme by that point)
- Replace redundant float('-inf') < x < float('inf') with math.isfinite()
  in _setting_float; drop the redundant infinity check from _setting_int
  (is_integer() already rejects inf/nan)

* fix(state): deduplicate session system prompts

* chore: map cicav legacy noreply email

* fix(tui): avoid writable Kanban opens on empty polls

* fix(context): dedupe subdirectory hints by content digest and skip backup/vendor dirs

SubdirectoryHintTracker re-injected identical context files whenever the same
AGENTS.md was reachable through more than one path. Symlinked shared
workspaces, hardlinks, and timestamped backup copies all alias a single file,
so a normal session could ship the same 8KB of instructions two or three
times. Nothing deduped it and nothing excluded directories that only ever
hold copies.

Two changes:

* Track a sha256 of every injected hint body. Repeat content is skipped, and
  the working directory's own context file is seeded at construction so the
  copy prompt_builder already loaded at startup is never sent again.
* Skip directories that hold copies rather than authoritative context
  (backups, node_modules, venv, site-packages, .git, .Trash, vendor, caches).
  Screening is relative to working_dir, so a project that legitimately lives
  under vendor/ keeps discovering its own subdirectory hints.

Measured on a real session that touched a symlinked shared workspace:
3 injections / ~24,000 chars before, 1 injection / 8,112 chars after.

14 new tests cover symlink aliasing, byte-identical copies, working-dir
seeding, distinct content still being injected, each excluded directory name,
excluded ancestors, and the working-dir-inside-excluded-name case.

* perf(state): batch the turn flush into one SQLite transaction

Re-derivation of #23254 (@devsart95) on today's flush loop. The turn
flush in _flush_messages_to_session_db wrote one BEGIN IMMEDIATE
transaction per message row; a typical agent turn (user + assistant +
tool results) paid 3-8 transactions -- and, off WAL (the default on
macOS while the WAL-reset guard is active), 3-8 fsyncs -- per turn.

Adds SessionDB.append_messages_batch: same row shape as append_message
(shared _prepare_message_row serializer + _MESSAGE_INSERT_SQL column
list, so the two writers cannot drift), same compression-lock and
compression-closed guards, one aggregated session-counter UPDATE, one
transaction for the whole batch. Row serialization stays outside the
write lock.

The flush loop now collects the turn's new rows and writes them in one
call. All-or-nothing pairs exactly with the persisted-marker stamping:
on failure no rows landed and no markers were stamped, so the next
flush re-writes the whole tail (same recovery contract as before,
minus the partial-prefix case that could double-count).

Measured (same harness, 5-message turn, journal_mode=DELETE,
synchronous=FULL): 2.32ms -> 0.83ms median per turn flush (64% faster,
5 fsyncs -> 1). On WAL the win is smaller but the atomicity fix holds.

* perf(tui-gateway): batch branch-seed history copies (whole-bug-class)

Sibling sites of the per-message flush pattern: both branch-seed
paths (session.branch in methods_session.py and the lazy seed persist
in server.py) copied the parent history row-by-row -- one transaction
per row, and a branch seed can be hundreds of rows. Route both through
SessionDB.append_messages_batch. The server.py path also gains real
atomicity: _branch_seed_persisted assumed every row landed, which the
per-row loop could not guarantee.

* test(run-agent): update flush-path fakes and assertions for batched writes

The flush now goes through append_messages_batch; MagicMock-based
assertions and barrier fakes that hooked append_message observed
nothing (the flush's try/except swallowed the AttributeError). Assert
on the batch payload instead.

* refactor(state): fold simplify findings — reuse _insert_message_rows, share guards, chunk seeds

Simplify-pass folds on the #23254 salvage:

- REUSE (HIGH): append_messages_batch now delegates row serialization to
  the pre-existing _insert_message_rows helper (already shared by
  replace_messages / archive_and_compact / portability import) instead
  of adding a third serialization path (_prepare_message_row +
  _MESSAGE_INSERT_SQL are gone). One row-writer for every multi-row
  path; the row-ID return was consumed by no production caller, so the
  batch returns the inserted count.

- QUALITY (HIGH): the compression-lock + compression-closed admission
  guards are extracted into _check_transcript_write_guards, shared by
  append_message and append_messages_batch (previously duplicated 23
  lines that had already needed targeted fixes, #74478). The role-gated
  reasoning filtering is no longer duplicated in run_agent.py — it
  lives at its one site inside _insert_message_rows.

- EFFICIENCY (MEDIUM, measured): unbounded seed copies hold one BEGIN
  IMMEDIATE for seconds (10k rows ~= 2.4s; FTS triggers dominate) and
  monopolize the in-process write lock. append_messages_batch grows a
  chunk_rows param; all seed/copy call sites use chunk_rows=500. Same
  recovery semantics as the old per-row loops, bounded lock holds.

- REUSE (MEDIUM): the two remaining per-row branch-copy loops found by
  the pass (gateway/slash_commands.py /branch, hermes_cli
  cli_commands_mixin.py branch) are converted to chunked batches too
  (AsyncSessionDB's generic to_thread forwarder covers the async site).

Turn-flush benchmark unchanged after the refactor: 2.43 -> 0.87 ms
median per 5-message flush (64% faster).

* fix(tests): update two more append_message.call_args assertions to append_messages_batch

CI-caught: test_verification_stop_caching and test_tui_gateway_server::test_native_vision_turn_persists_a_renderable_image_ref both assert on append_message.call_args, but the flush loop now calls append_messages_batch. Same class of test-fake fallout fixed in 5 other files — these two were missed.

* perf(tui): memoize useSessionLifecycle return (idea from #38491)

Re-derivation of #38491 by @stremtec onto current main (the original is
10,119 commits behind; the hook moved into ui-tui/src/app/). The hook
returned a fresh object literal every render, defeating…
vashkartik added a commit to vashkartik/hermes-agent that referenced this pull request Aug 4, 2026
* perf(transport): gate prompt cache keys by provider capability

* feat(transport): imply prompt_cache_key capability for api.openai.com

Review follow-up on the #56798 salvage: the gate shipped fully dormant
(no provider profile sets supports_prompt_cache_key, no production
caller passes it, and no plain 'openai' profile exists to set it on) —
AGENTS.md rejects dead code wired in without E2E proof.

Activate the one endpoint where the field is first-class: exact-host
api.openai.com (OpenAI documents prompt_cache_key; GPT-5.6+ docs
recommend it for cache routing). Deliberately NOT substring matching —
Azure/OpenAI-compat endpoints may reject unknown fields and stay
opt-in via the flag. 4 new tests (imply + 3 spoof/proxy/Azure
negatives); mutation-checked (substring-weakened host check fails the
spoof tests).

* perf(gateway): reuse loaded turn config for timestamp check

Re-derivation of PR #65645 onto current main: _build_gateway_agent_history
already runs inside a turn whose config was loaded once into
ctx.user_config; re-reading config from disk via _load_gateway_config()
per turn is redundant. Reuse the loaded turn config.

* perf(cli): add --prefer-offline to npm install during update (#39267)

Re-derivation of PR #39399 onto current main: pass --prefer-offline to
the web-UI workspace install (both silent and verbose arms of
_install_web_deps) and to the update-time Node dependency refresh in
_update_node_dependencies, so npm reuses its local cache instead of
re-fetching metadata. Test expectations updated to match, mirroring the
PR's own test-update commit.

* perf(cron): skip config load on idle scheduler ticks (idea from #33612)

Re-derivation of #33612 by @LeonSGP43 onto the rewritten scheduler (the
original is 10,692 commits behind; its tick() no longer exists in that
shape, so this is a fresh minimal fix crediting the PR's idea).

The gateway's built-in ticker calls tick(verbose=False) every 60s. The
idle early-return was gated on 'verbose and not due_jobs', so idle
GATEWAY ticks fell through to load_config() + worker-pool resolution
every minute. Return early on ANY idle tick; keep the post-tick MCP
orphan sweep (main intentionally reaps orphaned stdio children on idle
ticks).

3 new tests; mutation-checked (restoring the verbose-gated guard fails
the config-skip test). 66 scheduler tests green.

* fix(feishu): defer the lark_oapi import off the startup path

Salvage of #57657, ported onto the plugin layout (the adapter moved
from gateway/platforms/feishu.py to plugins/platforms/feishu/adapter.py
since the PR's base). lark_oapi takes seconds to import and holds the
GIL doing it; the module-level import made every gateway boot pay that
cost even with Feishu unconfigured.

- _load_lark_oapi() with double-checked locking binds the SDK globals
  on first use; connect() and _standalone_send() call it via
  asyncio.to_thread so the loop never blocks on the import.
- probe_bot() also calls _load_lark_oapi() (sync context) so the SDK
  probe path is preserved rather than silently degrading to the HTTP
  fallback before a first connect.
- check_feishu_requirements() is install-only and no longer rebinds
  globals; test_feishu.py gets a setUpModule that binds them eagerly
  for tests that inject fake clients.

Includes the dedicated lazy-import test file (check-does-not-import,
connect-loads-on-worker-thread).

* test: bind lark SDK globals session-wide, not per-file

CI exposed the whole class: feishu tests across MANY files (thread
routing, text batching, sdk executor, ...) inject a mock _client and
skip connect(), so the deferred import leaves the request-builder
globals None. Replace the single-file setUpModule with a session-scoped
autouse conftest fixture that binds the globals once when lark_oapi is
installed; when it isn't, the affected tests already skip via their own
skipUnless guards. Full tests/gateway run: zero failures beyond main's
pre-existing baseline (sorted failure-diff).

* fix(feishu): test SDK globals by None-ness, not globals() membership

The no-SDK fallback guards check '"Name" in globals()' — correct on
main where a failed module-level import leaves those names undefined,
but the deferred-import port pre-binds every SDK name to None, so the
guard was always true and the fallback paths called .builder() on None
(AttributeError) wherever lark_oapi isn't installed. Local runs passed
because lark IS installed here; CI's default env has no feishu extra.
Rewrote all 14 guards to 'is not None', which is correct under both
conditions. Verified by simulating CI with a lark-blocking meta_path
hook: 74 passed, 18 skipped (the skipUnless set), zero failures.

* chore: add contributor email mapping for WojtekMR3

* perf: replace COUNT(*) with LIMIT-based existence checks

Two places were using SELECT COUNT(*) when they only needed a boolean:
- has_any_sessions() called session_count() > 1 (full table scan)
- delete_session() used SELECT COUNT(*) WHERE id=? (full matching scan)

Fix:
- Add session_count_ge(n) to SessionDB — short-circuits via
  SELECT 1 FROM sessions LIMIT n, returns bool
- has_any_sessions() uses session_count_ge(2) instead of session_count() > 1
- delete_session() uses SELECT 1 ... LIMIT 1 with fetchone() is None
- Add tests for session_count_ge

* fix(state): take the connection lock in session_count_ge + document archived semantics

Review fold-ins on top of #56768 (@Skywind5487):
- session_count_ge ran its query without self._lock, unlike every
  sibling counter on SessionDB (session_count, session_count_by_source).
- Document the deliberate semantics change: session_count() defaults to
  archived = 0, which is both the expensive part (full index scan,
  measured 543us vs 4us on 20k sessions) and wrong for the only caller
  (has_any_sessions asks 'has this install ever had sessions' -- an
  archived session is still a created one).

* perf(state): index assistant tool-call rows for Insights queries

InsightsEngine._get_tool_usage and _get_skill_usage scan messages for
role='assistant' AND tool_calls IS NOT NULL, but no index aligns with
that predicate, so SQLite scans the full messages table on a large
state.db. Add a partial index over exactly those rows.

role and tool_calls are base columns in the messages table, so the index
lives in SCHEMA_SQL (created on both fresh and existing databases via the
executescript on every open) rather than DEFERRED_INDEX_SQL.

Adds schema regression coverage (fresh + reopened DB, plan uses the index)
and an Insights regression test proving tool/skill output is identical with
and without the index present.

Fixes #67341

* perf(insights): pin partial index on assistant tool-call queries

Review follow-up (#67341): on a freshly initialized state.db (before
ANALYZE has run) the source-filtered branches of _get_tool_usage /
_get_skill_usage did not select idx_messages_assistant_calls_by_session
— the optimizer drove from idx_sessions_source_id and probed each
session's messages via idx_messages_session_active, scanning non
tool-call rows. Pin the index with INDEXED BY on all four fixed-predicate
branches so the plan is deterministic for both the unfiltered and
source-filtered scopes without depending on statistics.

Safe because the index is declared in SCHEMA_SQL (created by every
read-write SessionDB._init_schema) and every InsightsEngine caller opens
a read-write SessionDB; read-only attachments (which skip schema init)
are never used for insights.

Extract the four queries into class constants and add tests: query-plan
coverage for both scopes without ANALYZE, row-level equivalence between
pinned and un-pinned forms, and an assertion that INDEXED BY fails loudly
if the index is absent.

* fix(insights): fall back to unpinned queries when the partial index is absent

The INDEXED BY pin is a hard dependency -- SQLite raises 'no such
index' when the named index is missing. That happens in production:
the web dashboard's usage analytics (_get_usage_analytics,
_get_models_analytics) open state.db read_only=True, which skips
_init_schema, so a DB last written by a pre-index version has no
idx_messages_assistant_calls_by_session and every insights call
crashes with OperationalError (reproduced E2E).

Probe sqlite_master once in __init__ and strip the pin from the four
prepared statements when absent -- identical rows, optimizer-chosen
plan, no crash. Replaces the change-detector test that froze the
crash as intended behavior with a fallback-equivalence test.

* refactor(insights): strip INDEXED BY pins via an attribute loop

Simplify-pass fold: the four copy-pasted .replace blocks meant a\nfifth pinned statement could forget its strip line — a hard 'no such\nindex' crash on read-only DBs, the exact bug the fallback prevents.\nLoop over the attribute names instead.

* perf(state): batch compression-tip row fetch in list_sessions_rich

list_sessions_rich()'s compression-root projection called
_get_session_rich_row() once per root — a separate single-row query per
compression root on every session-list render. Resolve every tip id
first, then fetch all tip rows in one WHERE id IN (...) query via the
new _get_session_rich_rows_batch().

_get_session_rich_row() is now a thin wrapper over the batch method, so
the enriched SELECT (preview + last_active) lives in exactly one place —
future column changes (e.g. #42196's include_system_prompt) only touch
one query.

get_compression_tip()'s chain walk is untouched; it's a genuine
per-session graph walk with branch/delegate-exclusion and race handling,
and batching it safely is out of scope here.

Co-Authored-By: Claude Fable 5 <noreply@anthropic.com>

* test(state): guard compact_rows threading through batched tip-row fetch

Adds two regression tests for the #59077 batch: (1) _get_session_rich_rows_batch(compact_rows=True) uses the schema-derived compact projection (no system_prompt, git_branch/git_repo_root kept); (2) list_sessions_rich(compact_rows=True) threads compact_rows through the compression-tip projection call site. Mutation-checked: hardcoding compact_rows=False at the call site fails test 2.

* refactor(state): chunk the batched tip-row IN clause at 900 ids

Simplify-pass fold: SQLITE_MAX_VARIABLE_NUMBER is 999 on pre-3.32\nbuilds (which the repo still supports — the trigram-availability\nmachinery exists for exactly that class), and limit=10000\nlist_sessions_rich callers exist in web_server. Chunk inside the\nbatch helper — the single choke point — so no call site can overflow.

* fix(yuanbao): clear _processing_msg_ids/_processing_msg_texts after each message

_dispatch_inbound_event() writes session_key → msg_id/raw_text into
_processing_msg_ids and _processing_msg_texts so RecallGuardMiddleware
can find and interrupt the currently-processing message.  These entries
were never removed after a message finished processing, causing both
dicts to grow unboundedly — one persistent entry per unique session key
for the lifetime of the bot.

Fix: clear both entries in the _process_message_background() finally
block, after super() returns.  The guard compares the stored msg_id
against event.message_id before popping: a concurrent pending message
may have already overwritten the entry in _dispatch_inbound_event while
we were running, in which case the drain task owns it and we must not
clear it.  When msg_id is absent (nothing was written at dispatch time)
the pop is a safe no-op.

Note: _msg_content_cache already bounds itself to 200 entries at the
same write site; _processing_msg_ids and _processing_msg_texts had no
such bound.

* fix(yuanbao): evict stale entries from _member_cache on TTL expiry

_build_msg_body_with_mentions() checks the TTL of each _member_cache
entry and returns an empty member list when the entry is stale, but
never removes the entry from the dict.  Over time every group_code the
bot has ever queried accumulates a permanent entry, retaining the full
member list (potentially thousands of records per group) until
disconnect().

Fix: delete the stale entry at the point it is detected as expired.
The next call to get_group_member_list_raw() for the same group will
repopulate the cache with fresh data as before.

Symmetric with the existing TTL pattern in MessageDeduplicator, which
evicts on access.

* fix(yuanbao): pop tracking entries only for truthy matching msg_id + regression tests

Follow-up on the salvaged pair: the original guard's `not msg_id` arm let an
id-less internal/synthetic event erase a tracking entry a concurrently-queued
id-bearing message's drain task still needs for recall matching (id-less
events never write entries in _dispatch_inbound_event, so they must never
pop). Tests cover: normal cleanup, id-less non-erasure, overwritten-entry
ownership handoff, TTL eviction + fresh-entry survival.

* chore: add frizikk to AUTHOR_MAP

* perf(zai): parallelize endpoint detection probes

Z.AI has separate billing for general vs coding plans and global vs
China endpoints. On startup, detect_zai_endpoint() probes up to 4
endpoints sequentially with 8s timeout each, taking 8-9 seconds when
the first endpoints return non-200 (rate limited) before a working one
is found.

Replace the sequential loop with concurrent.futures.ThreadPoolExecutor
to probe all 4 endpoints in parallel. Results are returned in
ZAI_ENDPOINTS priority order so the preference chain is preserved.

Benchmark on macOS M4 Max, Python 3.11, Hermes v0.8.0:
  Before: 8.8s (sequential: global=0.9s/429, cn=1.6s/429,
           coding-global=4.3s/200, coding-cn=2.0s/200)
  After:  ~4.5s (single round-trip, bounded by slowest endpoint)

Signed-off-by: Merlin <merlin@merlin.me>

* test(zai): cover parallel-probe contracts + restore candidate-model loop

Rebase fold: the original PR predates ZAI_ENDPOINTS growing per-endpoint
probe_models lists; the parallel worker now preserves that candidate-model
fallback loop (was: scalar model). Tests (both mutation-checked):
- candidate-model fallback within one endpoint worker
- ZAI_ENDPOINTS priority order wins over completion order
- all-fail returns None

* perf(zai): early-exit when the highest-priority endpoint wins (simplify finding)

The as_completed drain + `with` join made the parallel version WORSE than
sequential main in the common case (first endpoint succeeds fast, others
slow/unreachable): main returned at first success, the parallel version
waited for every straggler. Now: after each completion, walk endpoints in
priority order and return as soon as a success is unbeatable (all
higher-priority probes already finished); pool uses shutdown(wait=False) so
losers drain in the background. Mutation-checked: removing the early exit
makes the new timing test fail (8.2s vs <1.5s).

* Bound MiniMax OAuth error responses

* fix(minimax-oauth): read streamed error bodies inside the client context + real-transport tests

Follow-ups on the salvaged bounded-read fix:
- refresh flow: the non-200 branch reads a STREAMED body, which fails
  (ReadError/StreamClosed) once the httpx.Client context has exited —
  moved inside the context. Repro + regression test use a real socket
  server (MockTransport buffers in memory and cannot catch this).
- truncation guard: >limit bodies end with ...[truncated] (mutation-checked
  against the is_stream_consumed fallback).
- test mocks now model the streamed-read surface (is_stream_consumed,
  iter_bytes, client.send) so non-200 paths exercise the real bounded read.

* chore: map xaydinoktay@gmail.com to aydnOktay

* chore(contributors): map four B2 salvage author emails (#77641)

unixwzrd.register@mac.com -> unixwzrd (#74679); dai.suzuki.829@gmail.com -> hariNEzuMI928 (#75395); lexharddrive69@gmail.com -> hdd69 (#38470); coder@trevhome.local -> trevornk (#76282). Needed for the B2 desktop-renderer salvage attributions.

* perf(session-search): project fields before enrichment

* test(session-search): guard projected enrichment

* fix: skip memory prefetch on trivial user prompts (greetings)

Salvage of PR #25350 (commits 88ffede2d + 2b848a0b2 + 3136dc63a, squashed
and ported): the run_agent.py prefetch site the PR gated has since moved
into agent/turn_context.py's build_turn_context(), so the trivial-query
gate lands there instead.

- Gate the per-turn memory_manager.prefetch_all() on a trivial-prompt
  check so greetings/acknowledgements ('hi!', 'thanks', 'ok') no longer
  block the turn on provider network round-trips or inject stale context.
- Extend honcho's _TRIVIAL_PROMPT_RE with greetings and a trailing
  punctuation class so 'hey!' / 'hello.' classify as trivial.
- Add honcho classifier tests for greeting forms.

* chore: add ayushere to AUTHOR_MAP

* refactor(memory): single shared trivial-prompt classifier + gate tests

Rebase fold on the salvaged gate:
- is_trivial_prompt/TRIVIAL_PROMPT_RE move to agent/memory_provider (the
  ABC both the core gate and providers already import) — one source of
  truth; honcho's _TRIVIAL_PROMPT_RE now aliases it, turn_context and the
  queue_prefetch_all warm path (a sibling site main grew after the PR's
  base) both use it
- tests: gate tests at the prefetch call site (mutation-checked), shared
  classifier tests incl. prefix-collision guards (k8s/yolo/note/supper),
  and honcho dialectic-machinery tests re-driven with a substantive prompt
  ("hello" became trivial by design — those tests exercise thread cadence,
  not the classifier)

* refactor(honcho): delegate _is_trivial_prompt wholly to the shared classifier

Simplify-pass finding: sharing only the REGEX left the wrapper logic
(empty/strip/slash checks) duplicated, half-defeating the no-drift goal.
The classmethod now calls agent/memory_provider.is_trivial_prompt directly;
_TRIVIAL_PROMPT_RE stays as a class attr for backward compatibility with
any external referents.

* fix(desktop): measure adaptive stream flush through the deferred commit frame

scheduleDeltaFlush's adaptive floor is driven by lastFlushCostRef, but
runFlush only timed flushQueuedDeltas(), the synchronous store write.
While a session streams, syncSessionStateToView defers the $messages
publish (React commit + Streamdown re-parse) to its own rAF, so the
measured cost stayed near zero and the floor collapsed to the fixed
33ms path no matter how expensive the real commit was.

runFlush now records the write cost as a fallback, then extends the
measurement through a rAF registered after the view-sync one: it runs
in the same frame right after the deferred commit, and the rAF
timestamp marks frame start so only in-frame work is counted, not the
vsync wait. A stale callback from before a newer flush is ignored, and
a hidden renderer that never fires rAF keeps the write-cost fallback.

* fix(desktop): dedupe optimistic user turns for all wire references, not only images

* test(desktop): cover wire reference normalization edges

* fix(desktop): sort reference-kinds import per lint gate

* fix(desktop): full-jitter backoff on gateway WS reconnect loops

All three desktop reconnect loops (primary gateway boot, secondary
multi-profile gateway pool, plugin event socket) used bare exponential
backoff with no jitter. After a gateway restart every disconnected
client redials on the exact same schedule, so the reconnect attempts
land in lockstep instead of spreading out -- a burst that can starve
the gateway's file descriptors while it's still coming back up.

Add reconnect-backoff.ts implementing AWS-style full-jitter backoff
(random delay in [0, min(cap, base * 2^attempt))) and wire it into all
three call sites in place of their local Math.min/2**attempt math.
Manual reconnect paths already reset the attempt counter and bypass
the timer entirely -- unchanged.

* fix(desktop): escalate gateway reconnect on elapsed time, not attempt count

With the full-jitter backoff (300ms base) six attempts can elapse in ~9s,
so the old RECONNECT_ESCALATE_AFTER=6 attempt threshold raised the
recoverable boot error during a brief post-boot blip — breaking the
'a remote that drops post-boot keeps looping with NO boot.error' contract.
Escalate after RECONNECT_ESCALATE_AFTER_MS (45s, matching the old
deterministic 1->15s ladder's calibration) elapsed since the first failed
reconnect of the episode. Reset on clean open, manual/wake reconnect, and
soft switch, preserving the reset-on-success path.

* chore(contributors): map vittoria3103.123@gmail.com -> VittoriaLanzo (#77665)

Needed for the #62082 curator toolset-pin salvage attribution.

* fix(desktop): un-break the .btn-arc rule — '*/' inside a CSS comment ended it early

The comment above .btn-arc contained 'bg-*/', whose */ terminated the comment mid-sentence, leaving 'text-* variant utilities. */ .btn-arc {' as an invalid prelude — CSS error recovery can drop the whole .btn-arc rule. Reword so no */ appears inside the comment.

Extracted from #59352 by @rerdi92 (the rest of that PR — a month-stale icons.ts rewrite and a chunk-size warning-ceiling bump — is superseded/masking and was not salvaged).

* perf(desktop): stop idle chat re-renders — memo ChatView, stable tile props, gated adapter re-sync

Re-derive of PR #38470 on today's main (its target file desktop-controller.tsx no longer exists after the contrib/ refactor; the three surviving ideas are applied at their new homes):

- incremental-external-store-runtime: the dep-less setAdapter effect ran every render; gate on [runtime, store] — behavior-preserving because __internal_setAdapter early-exits on identical store.
- ChatView is now memo()d, and session-tile hoists its inline arrow props to useCallbacks/module constants so the memo actually holds.
- Render-count regression test (mocked Thread) proves an unrelated parent re-render no longer re-renders the chat shell.

Credit: idea and original implementation by @hdd69 in #38470.

* perf(curator): trim dead tool-schema from the LLM review fork

The curator LLM review loop (_run_llm_review) built its AIAgent without
enabled_toolsets, so it advertised the full default catalog (~30 tools plus the
context_engine lcm_* family) on every call. The fork uses only four tools, fixed
by its own system prompt, with no dispatch path to the rest, so ~26 tool schemas
shipped on every request as dead weight: ~7K input tokens per call on a loop that
makes 50-100 calls per consolidation pass.

Restrict the fork to enabled_toolsets=["skills", "terminal"], the same tools the
prompt already names. Behavior-neutral: the prompt held the model to these tools
and nothing routed calls to the others. Mirrors the background_review fork
(background_review.py:788-794). Call-site only; AIAgent already forwards the kwarg.

Adds test_review_fork_restricts_toolsets_to_skills_and_terminal (captures the
constructor kwarg) and test_review_fork_toolset_surface_is_skills_plus_terminal
(pins the resolved surface).

* fix(desktop): keep a mid-turn reply on screen when its session is reopened

Switching sessions while a turn streams (or right as it completes) could
leave the assistant reply missing until restart. Resume merges stored
history with the gateway's `inflight` projection, whose assistant row is
text-only and often an empty `assistant-stream-${sessionId}` shell; both
reconcile paths then dropped the local pending row that held the only copy
of the streamed text, reasoning and tool calls.

A shared pair of guards replaces the ad-hoc comparisons at all three sites.
`localPendingSupersedes` accepts the cached row only when it is the same
reply further along — an empty shell it has content for, or text it strictly
extends — so a longer unrelated row can no longer hijack an ordinal or reuse
a stream id, and a retained `inflight.error` snapshot is never mistaken for
an empty shell. `withAuthoritativeTurnState` then takes content from the
renderer while liveness, row id and reactions stay the backend's call, so a
settled shell cannot leave a finished reply spinning.

Co-authored-by: arimu1 <19286898+arimu1@users.noreply.github.com>

* fix(desktop): stop a finished reply rendering twice after history catches up

When a turn's reply commits under its own id, the settled local
`assistant-stream-*` row shifts one assistant ordinal earlier, so ordinal
pairing finds nothing at its slot and re-appends it — the same answer twice.

Drop a settled stream row only when the authoritative transcript already
carries that exact text. Keying `isPendingAssistant` on the explicit pending
flag alone would also have fixed this, but it discards the sibling case in
the same report: a reply that finished locally before the gateway committed
it, where the local row is the only copy that exists.

Co-authored-by: Dolverin <59100064+Dolverin@users.noreply.github.com>

* chore(contributors): map two B3 salvage author emails (#77685)

abdulsalamalotaibi86@gmail.com -> carbongotfound (#74025); soundbrokaz@kakao.com -> JeremyDev87 (#72813).

* refactor(desktop): hoist the reference-line matcher; drop dead textWithoutImageRefs

Follow-up to #77653: textWithoutReferenceLines built a fresh /g RegExp per call and hand-managed lastIndex — but it runs on both sides of every message comparison in the reconcile loops. An anchored non-global regex has no shared-lastIndex hazard and can be hoisted to module scope. Also removes textWithoutImageRefs, whose last production consumer #77653 replaced (kept IMAGE_REF_LINE_RE for extractImageRefs), and retargets its now-stale comment.

* fix(desktop): do not sandwich structured mid-turn rows with inflight dump

Skip pure-text inflight.assistant projections when the transcript already
has reasoning/tool-call structure, and only overlay journal answer text
on strict extension.

Fixes #76444

* fix(desktop): scope inflight dump suppression to the live turn tail

Only skip/graft structure for the current live assistant (stream id,
pending, or after the latest user), not completed historical tool rows.
Require live-tail identity for same-turn structure carry. Align journal
overlay with strict answer-text extension.

Addresses review + CI on #76744.

* fix(desktop): require structure-bearing row for live-tail same-turn carry

Structure-only same-turn carry used (live(previous) || live(message)), so a
new live text-only assistant at a compression-rewritten ordinal could inherit
reasoning/tool parts from an unrelated historical structured row.

Require the structure-bearing cached row itself to be live-tail (pending /
assistant-stream-* / interim). Add regressions for non-extending live dump
carry and the compression graft rejection.

Addresses salvage path on #76744 / #76444.

* refactor(desktop): one live-tail vocabulary for transcript reconciliation

Two fixes landed overlapping helpers on the same statement: the mid-turn
reply guard grew `isLiveProjectionRow` / `hasStreamedContent`, while the
inflight-dump guard grew `isLiveTailRow` / `hasStructuralParts`. Two
definitions of "is this row live" and "does it carry content" in one
function is how the next change silently reshapes one of them.

Collapse to a single module-level pair. `isLiveTailRow` now covers pending,
stream ids, inflight projections and sealed interim rows, so the reply guard
also stops treating an interim row as committed history; `hasStreamedContent`
is defined in terms of `hasStructuralParts`. Both text-extension checks route
through `isStrictAnswerTextExtension` rather than a bare `startsWith`.

Also hoists the live-tail lookup out of an inline IIFE and fixes the lint
warnings it carried.

Co-authored-by: 686f6c61 <github@00b.tech>

* fix(dashboard): cache plugins hub payload and avoid auth probes

* test(dashboard): cover install-hook invalidation of plugins hub cache

* fix(dashboard): warm cold check_fn verdicts with a background probe

On dashboard-only sessions nothing else executes check_fn warmers (they
live only in the tool-schema build), so the hub's read-only cache lookup
would report auth_required=False forever. On a cache miss, schedule a
deduplicated daemon-thread probe off the request path; the short hub TTL
surfaces the verdict on the next fetch.

* fix(desktop): cancel the pending commit-cost measurement rAF

Follow-up to #77652: each runFlush registered a fresh requestAnimationFrame and never cancelled it. Chromium parks rAF callbacks for hidden renderers, so a long hidden stream at the 33ms floor accumulates thousands of parked closures that all fire in the first frame on refocus (all but one no-oping through the stale-frame guard). Track the pending handle, cancel it before requesting a new one (only the newest flush's measurement matters), and cancel on unmount.

* perf(dashboard): skip full InsightsEngine on /api/analytics/usage (#18511)

* perf(dashboard): keep tools in focused analytics usage (#18511)

* refactor(insights): drop consumer-less get_skill_breakdown alias (simplify-pass)

The 2-line alias had zero production consumers (web_server calls
get_usage_breakdown directly). Tests rewired onto the real API; the
contracts they pin are unchanged. Stale test docstring fixed.

* fix(web): clamp dashboard pagination and analytics-days params (#39200 + #74778 salvage)

Re-derivation of aydnOktay's twin clamp PRs onto current main (the
session-list endpoints moved into web_routers/; the analytics endpoints
gained asyncio.to_thread wrappers since the originals):

- limit le=100 on /api/sessions, /api/sessions/search and the
  /api/profiles/sessions fan-out (one unbounded request could drag every
  session row + correlated-subquery preview work out of SQLite, times
  every profile's state.db on the fan-out).
- days ge=1 le=365 on /api/analytics/usage + /api/analytics/models
  (huge or non-positive values force full-history InsightsEngine work or
  inverted windows; the UI only offers 7/30/90 presets).

FastAPI Query bounds reject at the validation layer (422). 8 new tests;
both clamp classes mutation-checked (clamp removed -> its tests fail).

* fix(clamps): raise profile fan-out limit to le=500 (simplify-pass finding)

le=100 would 422 real desktop callers: sessions-settings fetches
archived at limit=200, the command palette lists at 200, and the
electron remote-merge over-fetches limit+offset (exceeds 100 at
offset>=81, and its .catch(()=>null) silently drops remote sessions).
Clamp must sit above real client maxima. New test pins limit=200 w/
offset.

* fix(web): avoid blocking provider validation

* perf(plugins): seed plugin routes from sessionStorage cache for instant render

- Plugin manifests are now cached in sessionStorage on fetch.
- On refresh, plugin routes are registered synchronously from cache, preventing unwanted redirects to /sessions.
- Removes the !pluginsLoading guard from the catch-all route in App.tsx, as plugin routes are now always available on first render.
- Background fetch always updates the cache and routes, so new/removed plugins are reflected after reload.
- Resolves the race condition where plugin pages would redirect to /sessions on hard refresh.

* fix(plugins): validate cached manifests are an array

* test(plugins): export cache helpers and add focused fallback/refresh tests

* fix(plugins): keep loading gate when cached manifests include a /chat override

The sessionStorage seed set loading=false whenever any cache existed, which
defeats App.tsx's load-bearing pluginsLoading gate: with a cached manifest
that declares tab.override === "/chat", the persistent ChatPage host must
NOT mount before plugins resolve, or it spawns a PTY and gets yanked when
the override plugin takes over the route.

Seed loading=false from the cache only when no cached manifest overrides
/chat (canSeedLoadedFromCache); manifests are still seeded either way so
plugin routes register synchronously on refresh. Adds focused tests for
the gate, including the /chat-override case.

* perf(dashboard): serve hashed /assets bundles with immutable cache headers

Every hashed bundle chunk under /assets/ was served with no caching
directives, so each dashboard load re-fetched (or at best revalidated)
every JS/CSS chunk. Those filenames carry a Vite content hash — the
bytes behind a given URL can never change; a rebuild mints new
filenames referenced by a freshly served index.html.

Mark them Cache-Control: public, max-age=31536000, immutable:
- the /assets StaticFiles mount, via a subclass that stamps the header
  on 200s only (404s stay uncached — a rebuild can create the file),
- serve_css, preserving its X-Forwarded-Prefix url() rewrites for
  /fonts/, /fonts-terminal/, /ds-assets/, /assets/.

index.html keeps no-store, no-cache, must-revalidate — it is the
mutable entry point that binds users to the current hashes.

The original PR also added hand-rolled per-request gzip compression of
asset responses; that part is deliberately dropped. This server is a
localhost-default dashboard backend: compressing every response on the
CPU to save loopback bandwidth is a pessimization, and callers that
front it with a real proxy already get compression there.

Salvaged from PR #28543 (idea by @sea-monsters; gzip groups dropped as
described above).

Co-Authored-By: Claude Fable 5 <noreply@anthropic.com>

* feat: reset-aware primary restore — stay on fallback until the rate-limit window resets

restore_primary_runtime retries the primary every turn once the 60s
transient cooldown clears. For subscription-window limits (Claude
Pro/Max 5h windows, Codex weekly caps) the reset is hours or days away,
so every retry is a guaranteed failure costing two provider switches
and two prompt-cache invalidations per turn.

Add CredentialPool.next_available_at() (earliest reset across exhausted
entries; None when available now or no reset info) and gate the restore
on it: skip while the primary's pool says nobody can serve, restore on
the first turn after the reset elapses. Fail-open: any gate error or
missing reset info falls through to the existing per-turn retry, so
recovery can never be later than today. Cross-provider fallbacks
consult the PRIMARY's pool (not the attached fallback pool), reusing
the loaded pool for the existing rebind to keep auth reads at one per
restore.

* fix(credential_pool): run next_available_at under the pool lock

Review fold on the #67642 salvage: next_available_at() called
_available_entries() — which prunes DEAD entries, syncs tokens, and
persists — and iterated self._entries with no lock, racing concurrent
select()/rotation exactly as has_available()'s comment warns. Wrap the
method body in self._lock and pin it with a non-blocking-acquire probe
test.

* fix(credential_pool): defer single-use-token refresh outside threading lock

select() and acquire_lease() held self._lock during the entire
_available_entries() loop, which for openai-codex and xai-oauth providers
includes a cross-process file lock (_auth_store_lock) plus OAuth token
refresh HTTP POST.  The lock timeout can exceed 20 seconds, blocking all
credential pool consumers across every gateway thread and subagent.

Collect single-use-token refresh entries under the lock, then execute the
refreshes outside it.  On success the refreshed entry is merged back into
the pool and re-selected.  Non-single-use providers (anthropic, nous)
continue refreshing inside the lock since their refresh is a simple HTTP
POST with no cross-process coordination.

* fix(credential_pool): serialize deferred-refresh pool mutations

Review folds on the #71775 salvage (dossier findings 1+2):

- self._lock becomes an RLock and the mutation primitives
  (_replace_entry, _persist) are now self-locking, so the deferred
  single-use-token refresh path — which deliberately runs its
  cross-process flock + OAuth network I/O OUTSIDE the pool lock —
  still serializes its pool mutations against concurrent
  select()/rotation. In-lock callers re-acquire reentrantly.
- Dropped _refresh_pending_entries' redundant second _replace_entry:
  _refresh_entry already merges the refreshed entry internally.

Adds tests/agent/test_credential_pool_deferred_refresh.py pinning both
invariants: select() must NOT hold the lock during the refresh window
(the PR's whole point), and the post-refresh mutations MUST contend on
the lock (blocking-thread probe).

* `hermes sessions optimize-storage` aborts with

```
Error: optimization failed: no such table: messages_fts_trigram
No data was lost. Re-run to resume.
```

on any install where the trigram FTS index is legitimately absent. The failure is
deterministic — re-running can never make progress, because the crash happens at the same
point every time — so the database is permanently stuck on the legacy high-footprint FTS
layout with no supported way forward.

Observed on a 5.4 GB production `state.db`. After the fix the same database optimized
successfully and shrank to 3.3 GB.

The trigram index is absent whenever the runtime cannot maintain it. On a SQLite build
without the `trigram` tokenizer, `_ensure_fts_schema()` returns `False`, so `__init__`
leaves `self._trigram_available = False` and no `messages_fts_trigram` table on disk. This
is a **supported degraded runtime**, not damage — CJK/substring search falls back to
`LIKE` and everything else works normally. `_is_fts5_unavailable_error()` and
`_warn_trigram_unavailable()` exist specifically to make this path graceful.

Two code paths write the boundary sweep for the deferred FTS rebuild, and only one of them
respects that flag:

| Function | Trigram `INSERT` guarded? |
|---|---|
| `fts_rebuild_step()` | ✅ `if include_trigram:` where `include_trigram = self._trigram_available` |
| `_fts_rebuild_finish()` | ❌ unconditional |

`_fts_rebuild_finish()` runs the boundary sweep at the *end* of the backfill. Its
unguarded `INSERT INTO messages_fts_trigram …` raises `OperationalError`, which propagates
out of `optimize_fts_storage()` and aborts the entire optimization — *after* the backfill
has already completed. Hence the characteristic output showing 100% progress immediately
before the error:

```
Rebuilding index: 100% (909,671/909,671)
Error: optimization failed: no such table: messages_fts_trigram
```

There is a second, quieter consequence. The teardown phase that reclaims the demoted
`fts_v22_trash_*` shadow tables runs *after* the backfill phase in
`optimize_fts_storage()`. Because the crash happens before teardown is ever reached, those
tables are never emptied or dropped — so the space the migration was supposed to reclaim
stays allocated indefinitely, and the leftover trash tables look (misleadingly) like
evidence of a half-finished migration.

Build a populated v23 database, set the deferred-rebuild markers, then reopen it on a
runtime where `_ensure_fts_schema('messages_fts_trigram', …)` returns `False` (exactly
what a SQLite build without the trigram tokenizer produces) and call
`optimize_fts_storage()`:

```
[precondition] trigram absent, _trigram_available=False, rebuild pending  ✓

RED  ✗ optimize_fts_storage raised OperationalError: no such table: messages_fts_trigram
```

With this patch applied, unchanged harness:

```
     optimize_fts_storage returned {'ok': True, 'vacuumed': None}
GREEN ✓ optimize ok; markers cleared; base FTS 'zebra' -> 200 hits
```

Full harness and transcripts in `TEST-EVIDENCE.md`.

Gate the sweep on `self._trigram_available`, exactly as `fts_rebuild_step()` already does:

```python
include_trigram = self._trigram_available

def _do(conn):
    ...
    if include_trigram:
        conn.execute("INSERT INTO messages_fts_trigram(...) ...")
```

The base `messages_fts` sweep and the marker cleanup are untouched, so the rebuild still
finalizes correctly and the index remains complete for every row it is responsible for.
The fix does not disable or weaken search to dodge the error — the regression tests assert
that base FTS still returns results afterwards.

`TestFtsRebuildFinishWithoutTrigram` in `tests/test_hermes_state.py`:

- `test_rebuild_finish_skips_trigram_when_unavailable` — drives `_fts_rebuild_finish()`
  directly on a trigram-less runtime; asserts it completes, clears both rebuild markers,
  and leaves base FTS searchable.
- `test_optimize_fts_storage_succeeds_without_trigram` — end-to-end through the public
  `optimize_fts_storage()` entry point; asserts `ok=True`, markers cleared, search intact.

Both use the existing `_NoTrigramConnection` helper already in the file. Both fail on
`main` with `no such table: messages_fts_trigram` and pass with this patch.

`tests/test_hermes_state.py` passes in full (463 tests → 465 with these two). `ruff` clean.

This PR is the crash only.

A companion PR narrows `_db_opens_cleanly()` so that
`hermes sessions repair --check-only` stops reporting a write-broken FTS schema as
healthy — the gap that makes this class of problem hard to diagnose in the first place.
The two are independent and can land in either order.

* perf(tools): shrink lazy tool catalog overhead

* refactor(tool-search): drop dead fallback ladder in _available_source_summary

Simplify-pass finding: _listing_group_label already falls back to 'other' for empty source names, and _classify_source guarantees source_name=='' only when source=='other' — both  legs were dead by construction. Aligns the summary path's grouping with the listing path.

* fix(credential_pool): unpack the tuple in next_available_at's gate

Cross-PR interaction fix: #77714 (salvage of #71775) changed
_available_entries to return (available, pending_refresh) while #77631
(salvage of #67642) added next_available_at() which still truthiness-
tests the bare return. A non-empty tuple is always truthy — even
([], []) — so the reset-aware gate silently returned None ('no wait
info') for every exhausted pool, disabling the feature #77631 shipped.
Unpack the tuple and test the available list.

Also adapts the lock-probe test for the RLock introduced by #77714
(same-thread non-blocking acquire always succeeds on an RLock; probe
from a helper thread instead).

* chore: map copii.list@gmail.com to stremtec

* chore(contributors): map marzukia@users.noreply.github.com -> marzukia (#77774)

Needed for the #37117 salvage (#77696 CI failure).

* chore: map bot@bkstock.dev to BKStock

* perf(session): route SQLite PRAGMAs through central apply_database_pragmas

Addresses review from @teknium1 on PR #71755:

- Extended apply_database_pragmas() to handle cache_size, mmap_size,
  and temp_store from config.yaml (alongside existing wal_autocheckpoint
  and journal_size_limit). No hardcoded defaults — all values are
  opt-in via config.yaml, avoiding policy conflicts with other PRs.
- Applied to ALL connection types: writer (_connect_and_init),
  read_only cross-profile attach, and WAL per-thread readers
  (_get_read_conn). Previously PRAGMAs only ran on the writer path.
- Removed inline PRAGMAs from _connect_and_init — single source of
  truth in apply_database_pragmas().
- Documented config keys with examples in function docstring.

* fix(pr): remove remnant local PRAGMAs from PR branch

* test(session): guard config-gated performance PRAGMAs across all connection types

E2E guard for the salvaged PR #71755: database.cache_size/mmap_size/
temp_store from config.yaml must reach the writer connection, the
read-only cross-profile attach, and the WAL per-thread reader — and a
default install (no database: keys) must keep byte-identical SQLite
defaults on every connection type. Also covers integer-coercion
rejection of garbage values for the three new keys.

cache_size uses -16000 (not the doc example -2000) because -2000 is
SQLite's compiled-in default and would not discriminate a regression.

* fix(desktop): flush queued deltas on window focus

* perf(desktop): stop scroll and status loops in busy sessions

* perf(desktop): pause hidden-pane timers in agents view, cron sidebar, and floating pet

Partial pick of the surviving renderer hunks from #75395 (perf commit
6502e441d plus fixup 3fbbc9c1d): gate the 500ms subagent now-ticker and
the cron sidebar 1s ticker/run-poll on usePaneVisible, and skip the
legacy floating-pet poll while the document is hidden. Dropped hunks
(electron/main.ts, vitest.setup.ts/config) intentionally excluded.

* style(desktop): restore alphabetical import order in agents/index.tsx

* refactor(desktop): shared pulse beat + fully-gated cron peek (simplify folds)

Two findings from the simplify pass on the final trio diff:

- status-pulse: one pause controller + one aligned period timer shared by all StatusPulse instances (ref-counted), instead of N x (document/window/bridge listeners + unsynchronized 5s wakes) — a sidebar can show dozens of pulsing dots. Pause still cancels in-flight animations so the compositor sleeps immediately.
- cron-jobs-section: the runs-peek effect created its interval even while the pane was hidden (callback no-oped but the timer still woke the renderer every 8s/60s per expanded job). Early-return when hidden — visibility is already in the dep array, so becoming visible restarts load + timer.

* fix(lint): import sort + eslint-disable for timer-handle ref clear in effect

CI-caught: cron-jobs-section had an extra blank line between sorted imports; use-message-stream's visibility-flush effect assigns flushHandleRef.current=null inside a useEffect (legitimate timer-clear, not an atom mirror) — eslint-disable-next-line per the rule's documented convention.

* fix(state): narrow FTS UPDATE triggers with AFTER UPDATE OF + migration

Retarget #73639 onto the SessionDB mixin split (hermes_state_common /
hermes_state_schema). Fresh installs create UPDATE OF content/tool_*
triggers; existing broad AFTER UPDATE triggers are inspected and
replaced under schema init without an FTS rebuild (WHEN clauses already
guarded content correctness; OF skips non-content status writes that
saturated disk I/O on large state.db).

Tests: tests/test_fts_update_of_narrowing.py (4)

* fix(state): fail closed on CJK trigger migration

* fix(state): quarantine CJK when ensure soft-fails after OF migration

_ensure_fts_cjk_schema never raises on OperationalError; post-condition
after dropping messages_fts_cjk_update now requires a narrowed UPDATE
trigger or durable fts_cjk_stale + unavailable. Covers the production
soft-fail path the raise-only handler missed.

* refactor(state): drop unreachable regex guard in trigger migration

Simplify-pass fold: to_drop names come from the literal update_names\nallowlist via IN binding, so the [A-Za-z0-9_]+ fullmatch could never\nfail — and if it somehow did, its `continue` would miscount (the\nskipped trigger stayed in len(to_drop)/the log while CREATE TRIGGER\nIF NOT EXISTS silently kept the broad variant). Delete the guard and\nits function-local re import; keep the invariant as a comment.

* fix(security): reject always-blocked OpenViking endpoints

## Summary
- Normalize OpenViking endpoints through `is_always_blocked_url` and fall back to the default local endpoint when poisoned.
- Keep intentional loopback / LAN self-host working.
- Add focused unit tests.

## Salvage / credit
Memory-provider endpoint floor sibling of RetainDB/Supermemory always-blocked hardening (avoids over-broad #4984-style private-IP bans).

(cherry picked from commit 8fa607d0aedb8c5fca398d7f112b1b25ade54fa2)

* fix(openviking): fail closed on blocked endpoints

(cherry picked from commit 389a90b81c9c2c89810f2fa7461f8faa9a5c9578)

* fix(openviking): don't spawn a second server onto a live port

`_start_local_openviking_server()` spawned `openviking-server`
unconditionally. Both callers — `initialize()` and the runtime
unreachable handler — reach it from a health probe, and that probe can
time out client-side while the server is up and serving. The spawned
process then loses the data-directory lock and exits immediately with
`DataDirectoryLocked`; because the probe keeps timing out, the cycle
repeats every cooldown window (~5 min observed).

The existing 30s `_failed_refresh` cooldown paces the loop but cannot
stop it, since it expires while the underlying condition persists.

Probe the target host:port before spawning and treat an occupied port as
already-started. This guards both call sites at their single convergence
point. The probe deliberately tests only that a listener owns the port —
enough to know a second server would lose the lock — and says nothing
about that listener's health.

The parse/probe now precedes the PATH lookup, so a reachable server is
reported as running even when `openviking-server` is not on PATH.

Fixes #74846

(cherry picked from commit b49427d85fd6628eb4a7fe099e5c390c5c4cc935)

* fix(openviking): drop stale "disabled for this Hermes run" warnings

The provider used to disable OpenViking permanently when the server was
unreachable. That was fixed: `_ensure_client()` now reconnects lazily,
with a 30s cooldown gate in `_ensure_client_locked`.

Only one of the seven user-facing warnings was updated to match. The
other six still told the user memory was "disabled for this Hermes run",
which is no longer true — every one of those paths is retried on the next
access. A user who reads the old message has no reason to retry, which is
very likely how #5721 ("never recovers") came to be filed against
behaviour that already recovers.

All six sites were traced to confirm none is terminal for the run: the
`initialize()`-time and waiter-thread failures never arm `_failed_refresh`
(only line 2439 does), so they retry on the very next access with no
cooldown at all.

The replacement wording deliberately omits the "(after cooldown)"
parenthetical used at the already-correct site — that detail is only
accurate where `_failed_refresh` was just armed. The neutral phrasing is
true at all six.

Also promotes two clause separators to periods to avoid "…; …disabled;"
collisions.

(cherry picked from commit 8346403a4b97af503d26b0f7905ff513828d821e)

* fix(openviking): re-arm the commit guard after in-place compression

`_committed_session_ids` is a permanent per-sid latch, and
`_session_needs_commit` checks it before the turn counter by design — a
racing sync_turn can re-increment `_turn_count` after commit+reset, so
the guard must win to stop a double-commit.

That is correct for a session being left behind. It is wrong for one
that keeps its id. `compress_context()` commits before rewriting the
transcript in both modes, and with `compression.in_place: true` (the
default) `on_session_switch` receives the same id and does not rotate.
The latch then rejects every later commit for a still-live session — the
next compression, /new, normal session end, startup recovery — so every
post-compression turn is silently never extracted.

Rotation mode is unaffected because a fresh child id is minted and
starts clean, which is what confirms the latch's intent was only ever to
dedupe the departing id.

Clear the latch when compression completes without rotation. Turns
arriving after that point are genuinely new, and this is a defined
moment rather than a race. The rotation path is untouched, so the old
id stays latched and its _finalize_session_async still dedupes against
the compression commit.

Fixes #74695

(cherry picked from commit d1e5c3dc33ef0d43d021662674e1a7cd5e43eecd)

* test(openviking): cover the compression lifecycle, not a hand-set latch

Review feedback: the previous test called _mark_session_committed
directly, so it verified the guard's behavior but not the wiring that
sets it — a future break in the commit_memory_session -> same-id
compression-boundary path would not be caught.

Add a lifecycle regression that drives the real sequence: on_session_end
commits through the actual path, on_session_switch(same id,
reason="compression") crosses the boundary, sync_turn records a genuinely
new turn, and a second on_session_end must produce a second commit POST.

Without the fix it fails showing exactly one commit call, which is the
reported data loss: every turn after the first compression is dropped.
The rotation and /undo tests stay as scope guards.

(cherry picked from commit 0ca5a330630a30b105cbbc32e8a23f2c5ffe0eab)

* fix(memory): read non-secret provider config from config.yaml for OpenViking and RetainDB

OpenViking is_available() only consulted env vars and use_ovcli_config, so an
endpoint saved to config.yaml (e.g. by the Dashboard) reported needs_config;
_resolve_connection_settings() likewise never folded config.yaml's non-secret
fields into its chain. RetainDB initialize() read base_url/project from the
environment only, ignoring the values the Dashboard writes to config.yaml.

Both now resolve non-secret fields as env -> (ovcli ->) config.yaml -> default;
secrets still come from the environment. Adds regression tests for both.

Fixes #68209

(cherry picked from commit dca57915b97b5705b30927a062e1d0f2f23d3841)

* fix(openviking): read recall settings from config.yaml first, env vars as fallback

_recall_config() previously read all settings (recall_limit, score_threshold,
recall_resources, etc.) exclusively from environment variables. This forced
users to store behavioural configuration in .env, violating the Hermes
convention that .env is for secrets only.

The infrastructure to load config.yaml -> memory.openviking was already in
place via _load_hermes_openviking_config(), but _recall_config() never
called it.

Fix: call _load_hermes_openviking_config() and pass its values as the
default parameter to _env_int/_env_float/_env_bool. Env vars still override
config.yaml values, preserving backward compatibility.

Closes #62540

(cherry picked from commit 6aadf1256835745e0302aa3d3b5ae0660b368637)

* test(openviking): cover config.yaml recall settings with temp-HERMES_HOME tests

Add three tests to TestOpenVikingConfigSchema:

1. test_recall_config_reads_from_config_yaml — writes memory.openviking
   settings in config.yaml and verifies _recall_config() consumes them.

2. test_recall_config_env_overrides_config_yaml — writes both config.yaml
   and OPENVIKING_RECALL_* env vars, verifies env takes precedence.

3. test_recall_config_partial_config_yaml — partially populated config.yaml
   falls back to defaults for omitted keys.

All 46 openviking_plugin tests pass (43 existing + 3 new).

(cherry picked from commit b8d7834caf06c6912004333c270faa248eaed4cd)

* fix(openviking): integrate reliability and configuration hardening

* chore(contributors): map OpenViking source authors

* test(retaindb): guard scoped secret config resolution

* fix(openviking): verify servers before sending credentials

* fix(openviking): catch endpoint errors in setup validation functions

Review follow-up for salvaged PR #76782. Three setup-wizard
validation functions called _normalize_openviking_url outside their
try/except blocks. Since _normalize_openviking_url now raises
_OpenVikingEndpointError for blocked or malformed endpoints, an
invalid endpoint would crash the wizard instead of returning a
friendly (False, message) tuple.

- _validate_openviking_auth: move _normalize_openviking_url inside try
- _validate_openviking_root_access: same
- _validate_openviking_setup_values: catch _OpenVikingEndpointError explicitly
- Remove dead ternary in _normalize_openviking_url safety check (candidate
  always has http/https scheme by that point)
- Replace redundant float('-inf') < x < float('inf') with math.isfinite()
  in _setting_float; drop the redundant infinity check from _setting_int
  (is_integer() already rejects inf/nan)

* fix(state): deduplicate session system prompts

* chore: map cicav legacy noreply email

* fix(tui): avoid writable Kanban opens on empty polls

* fix(context): dedupe subdirectory hints by content digest and skip backup/vendor dirs

SubdirectoryHintTracker re-injected identical context files whenever the same
AGENTS.md was reachable through more than one path. Symlinked shared
workspaces, hardlinks, and timestamped backup copies all alias a single file,
so a normal session could ship the same 8KB of instructions two or three
times. Nothing deduped it and nothing excluded directories that only ever
hold copies.

Two changes:

* Track a sha256 of every injected hint body. Repeat content is skipped, and
  the working directory's own context file is seeded at construction so the
  copy prompt_builder already loaded at startup is never sent again.
* Skip directories that hold copies rather than authoritative context
  (backups, node_modules, venv, site-packages, .git, .Trash, vendor, caches).
  Screening is relative to working_dir, so a project that legitimately lives
  under vendor/ keeps discovering its own subdirectory hints.

Measured on a real session that touched a symlinked shared workspace:
3 injections / ~24,000 chars before, 1 injection / 8,112 chars after.

14 new tests cover symlink aliasing, byte-identical copies, working-dir
seeding, distinct content still being injected, each excluded directory name,
excluded ancestors, and the working-dir-inside-excluded-name case.

* perf(state): batch the turn flush into one SQLite transaction

Re-derivation of #23254 (@devsart95) on today's flush loop. The turn
flush in _flush_messages_to_session_db wrote one BEGIN IMMEDIATE
transaction per message row; a typical agent turn (user + assistant +
tool results) paid 3-8 transactions -- and, off WAL (the default on
macOS while the WAL-reset guard is active), 3-8 fsyncs -- per turn.

Adds SessionDB.append_messages_batch: same row shape as append_message
(shared _prepare_message_row serializer + _MESSAGE_INSERT_SQL column
list, so the two writers cannot drift), same compression-lock and
compression-closed guards, one aggregated session-counter UPDATE, one
transaction for the whole batch. Row serialization stays outside the
write lock.

The flush loop now collects the turn's new rows and writes them in one
call. All-or-nothing pairs exactly with the persisted-marker stamping:
on failure no rows landed and no markers were stamped, so the next
flush re-writes the whole tail (same recovery contract as before,
minus the partial-prefix case that could double-count).

Measured (same harness, 5-message turn, journal_mode=DELETE,
synchronous=FULL): 2.32ms -> 0.83ms median per turn flush (64% faster,
5 fsyncs -> 1). On WAL the win is smaller but the atomicity fix holds.

* perf(tui-gateway): batch branch-seed history copies (whole-bug-class)

Sibling sites of the per-message flush pattern: both branch-seed
paths (session.branch in methods_session.py and the lazy seed persist
in server.py) copied the parent history row-by-row -- one transaction
per row, and a branch seed can be hundreds of rows. Route both through
SessionDB.append_messages_batch. The server.py path also gains real
atomicity: _branch_seed_persisted assumed every row landed, which the
per-row loop could not guarantee.

* test(run-agent): update flush-path fakes and assertions for batched writes

The flush now goes through append_messages_batch; MagicMock-based
assertions and barrier fakes that hooked append_message observed
nothing (the flush's try/except swallowed the AttributeError). Assert
on the batch payload instead.

* refactor(state): fold simplify findings — reuse _insert_message_rows, share guards, chunk seeds

Simplify-pass folds on the #23254 salvage:

- REUSE (HIGH): append_messages_batch now delegates row serialization to
  the pre-existing _insert_message_rows helper (already shared by
  replace_messages / archive_and_compact / portability import) instead
  of adding a third serialization path (_prepare_message_row +
  _MESSAGE_INSERT_SQL are gone). One row-writer for every multi-row
  path; the row-ID return was consumed by no production caller, so the
  batch returns the inserted count.

- QUALITY (HIGH): the compression-lock + compression-closed admission
  guards are extracted into _check_transcript_write_guards, shared by
  append_message and append_messages_batch (previously duplicated 23
  lines that had already needed targeted fixes, #74478). The role-gated
  reasoning filtering is no longer duplicated in run_agent.py — it
  lives at its one site inside _insert_message_rows.

- EFFICIENCY (MEDIUM, measured): unbounded seed copies hold one BEGIN
  IMMEDIATE for seconds (10k rows ~= 2.4s; FTS triggers dominate) and
  monopolize the in-process write lock. append_messages_batch grows a
  chunk_rows param; all seed/copy call sites use chunk_rows=500. Same
  recovery semantics as the old per-row loops, bounded lock holds.

- REUSE (MEDIUM): the two remaining per-row branch-copy loops found by
  the pass (gateway/slash_commands.py /branch, hermes_cli
  cli_commands_mixin.py branch) are converted to chunked batches too
  (AsyncSessionDB's generic to_thread forwarder covers the async site).

Turn-flush benchmark unchanged after the refactor: 2.43 -> 0.87 ms
median per 5-message flush (64% faster).

* fix(tests): update two more append_message.call_args assertions to append_messages_batch

CI-caught: test_verification_stop_caching and test_tui_gateway_server::test_native_vision_turn_persists_a_renderable_image_ref both assert on append_message.call_args, but the flush loop now calls append_messages_batch. Same class of test-fake fallout fixed in 5 other files — these two were missed.

* perf(tui): memoize useSessionLifecycle return (idea from #38491)

Re-derivation of #38491 by @stremtec onto current main (the original is
10,119 commits behind; the hook moved into ui-tui/src/app/). The hook
returned a fresh object literal every render, defeating memoization in
useMainApp's consumers; useMemo over the (all-useCallback-stable)
handles makes the return referentially stable.

Dep array covers ALL nine returned handles incl. trimTail (the
re-derivation initially omitted it - stale-closure class).

* ci: retry uv python install

* fix(state): route session-resume reads through the WAL read-only connection

get_messages_as_conversation, get_resume_conversations, and
get_ancestor_display_prefix still took self._lock — the same global
choke point the read-path split (WAL per-thread read-only connections)
was meant to remove from every recall/browse read. These three are the
hottest reads in the file: every session resume across the gateway,
CLI, and ACP adapter goes through one of them, so a resume racing a
burst of concurrent-session writer flushes still convoys behind them
exactly like the fixed paths used to.

_session_lineage_root_to_tip (the lineage walk shared by all three,
plus get_conversation_root) had its own independent self._lock use and
needed the same conversion — without it the outer functions still
blocked on the very first line.

Verified empirically: a reader thread calling all three functions
while another thread holds self._lock blocked for the writer's full
hold duration before the fix, and returned immediately after (SQLite
3.50.4 in this dev venv falls back to journal_mode=DELETE per the
WAL-reset-bug guard, so the requires_wal-marked regression test is
exercised via a local WAL-forced script instead; it still runs and
passes on any runtime where WAL is actually active).

* chore: add contributor email mapping for ArcherQAQ

* fix(model_metadata): rewrite localhost->IPv4 for the remaining local probe sites

fetch_endpoint_model_metadata's generic (non-LM-Studio) /models fetch and
its llama.cpp /v1/props context-length follow-up built request URLs
straight from the unrewritten candidate, unlike every other local-probe
site. Both retained the multi-second dual-stack IPv6 connect penalty
that _localhost_to_ipv4() exists to skip (measured on macOS: localhost
32.9ms vs 127.0.0.1 0.1ms on a dead port; ~2s on Windows). normalized
stays the cache key so caching behavior is unchanged; only the outbound
request target is rewritten.

Re-derived from PR #61528 onto current main (original no longer applied
cleanly).

* fix(model_metadata): guard _localhost_to_ipv4 against non-string urls

CI slice 3/7 failures: run_conversation tests pass MagicMock base_urls
through the metadata probe path; re.sub raised TypeError where the old
code let non-strings flow through. Preserve that contract.

* perf(cold-start): mitigate ~14s GIL stall during backend init (#60800)

Three fixes for the Desktop/TUI cold-start stall where the event loop
is blocked for ~14s between HERMES_BACKEND_READY and the first
prompt (#60800):

1. copilot_auth: skip  subprocess fallback when any
   Copilot env var is explicitly set (even if invalid). The user
   expressed token intent via env var; silently substituting a CLI
   token is surprising and the subprocess adds up to 5s on Windows.

2. tui_gateway/ws: run resolve_skin() via asyncio.to_thread so config
   loading + skin engine init do not block the WS read loop during
   the cold-start RPC burst.

3. web_server: extend _warm_gateway_module to pre-import the heavy
   module chains (auth, copilot_auth, runtime_provider, skin_engine,
   inventory, model_switch) that the first WS connection + RPC burst
   would otherwise import on the loop thread. These trigger .pyc
   compilation and Defender scans on Windows (15-30s per the existing
   comment) and were not covered by the original gateway-only warm.

Tests: 5 new tests in test_cold_start_gil_stall.py + 2 new tests in
test_copilot_auth.py. All 36 copilot_auth tests + 16 ws/web_server
tests pass.

* test: harden cold-start regression tests + debug-log the env-var skip

Review folds on the #60807 salvage:
- resolve_skin tests are behavioral (thread-ident probe + ready-frame
  wiring check) instead of pure source inspection, per the #72720
  pattern; a source ass…
vashkartik added a commit to vashkartik/hermes-agent that referenced this pull request Aug 4, 2026
* perf(state): batch compression-tip row fetch in list_sessions_rich

list_sessions_rich()'s compression-root projection called
_get_session_rich_row() once per root — a separate single-row query per
compression root on every session-list render. Resolve every tip id
first, then fetch all tip rows in one WHERE id IN (...) query via the
new _get_session_rich_rows_batch().

_get_session_rich_row() is now a thin wrapper over the batch method, so
the enriched SELECT (preview + last_active) lives in exactly one place —
future column changes (e.g. #42196's include_system_prompt) only touch
one query.

get_compression_tip()'s chain walk is untouched; it's a genuine
per-session graph walk with branch/delegate-exclusion and race handling,
and batching it safely is out of scope here.

Co-Authored-By: Claude Fable 5 <noreply@anthropic.com>

* test(state): guard compact_rows threading through batched tip-row fetch

Adds two regression tests for the #59077 batch: (1) _get_session_rich_rows_batch(compact_rows=True) uses the schema-derived compact projection (no system_prompt, git_branch/git_repo_root kept); (2) list_sessions_rich(compact_rows=True) threads compact_rows through the compression-tip projection call site. Mutation-checked: hardcoding compact_rows=False at the call site fails test 2.

* refactor(state): chunk the batched tip-row IN clause at 900 ids

Simplify-pass fold: SQLITE_MAX_VARIABLE_NUMBER is 999 on pre-3.32\nbuilds (which the repo still supports — the trigram-availability\nmachinery exists for exactly that class), and limit=10000\nlist_sessions_rich callers exist in web_server. Chunk inside the\nbatch helper — the single choke point — so no call site can overflow.

* fix(yuanbao): clear _processing_msg_ids/_processing_msg_texts after each message

_dispatch_inbound_event() writes session_key → msg_id/raw_text into
_processing_msg_ids and _processing_msg_texts so RecallGuardMiddleware
can find and interrupt the currently-processing message.  These entries
were never removed after a message finished processing, causing both
dicts to grow unboundedly — one persistent entry per unique session key
for the lifetime of the bot.

Fix: clear both entries in the _process_message_background() finally
block, after super() returns.  The guard compares the stored msg_id
against event.message_id before popping: a concurrent pending message
may have already overwritten the entry in _dispatch_inbound_event while
we were running, in which case the drain task owns it and we must not
clear it.  When msg_id is absent (nothing was written at dispatch time)
the pop is a safe no-op.

Note: _msg_content_cache already bounds itself to 200 entries at the
same write site; _processing_msg_ids and _processing_msg_texts had no
such bound.

* fix(yuanbao): evict stale entries from _member_cache on TTL expiry

_build_msg_body_with_mentions() checks the TTL of each _member_cache
entry and returns an empty member list when the entry is stale, but
never removes the entry from the dict.  Over time every group_code the
bot has ever queried accumulates a permanent entry, retaining the full
member list (potentially thousands of records per group) until
disconnect().

Fix: delete the stale entry at the point it is detected as expired.
The next call to get_group_member_list_raw() for the same group will
repopulate the cache with fresh data as before.

Symmetric with the existing TTL pattern in MessageDeduplicator, which
evicts on access.

* fix(yuanbao): pop tracking entries only for truthy matching msg_id + regression tests

Follow-up on the salvaged pair: the original guard's `not msg_id` arm let an
id-less internal/synthetic event erase a tracking entry a concurrently-queued
id-bearing message's drain task still needs for recall matching (id-less
events never write entries in _dispatch_inbound_event, so they must never
pop). Tests cover: normal cleanup, id-less non-erasure, overwritten-entry
ownership handoff, TTL eviction + fresh-entry survival.

* chore: add frizikk to AUTHOR_MAP

* perf(zai): parallelize endpoint detection probes

Z.AI has separate billing for general vs coding plans and global vs
China endpoints. On startup, detect_zai_endpoint() probes up to 4
endpoints sequentially with 8s timeout each, taking 8-9 seconds when
the first endpoints return non-200 (rate limited) before a working one
is found.

Replace the sequential loop with concurrent.futures.ThreadPoolExecutor
to probe all 4 endpoints in parallel. Results are returned in
ZAI_ENDPOINTS priority order so the preference chain is preserved.

Benchmark on macOS M4 Max, Python 3.11, Hermes v0.8.0:
  Before: 8.8s (sequential: global=0.9s/429, cn=1.6s/429,
           coding-global=4.3s/200, coding-cn=2.0s/200)
  After:  ~4.5s (single round-trip, bounded by slowest endpoint)

Signed-off-by: Merlin <merlin@merlin.me>

* test(zai): cover parallel-probe contracts + restore candidate-model loop

Rebase fold: the original PR predates ZAI_ENDPOINTS growing per-endpoint
probe_models lists; the parallel worker now preserves that candidate-model
fallback loop (was: scalar model). Tests (both mutation-checked):
- candidate-model fallback within one endpoint worker
- ZAI_ENDPOINTS priority order wins over completion order
- all-fail returns None

* perf(zai): early-exit when the highest-priority endpoint wins (simplify finding)

The as_completed drain + `with` join made the parallel version WORSE than
sequential main in the common case (first endpoint succeeds fast, others
slow/unreachable): main returned at first success, the parallel version
waited for every straggler. Now: after each completion, walk endpoints in
priority order and return as soon as a success is unbeatable (all
higher-priority probes already finished); pool uses shutdown(wait=False) so
losers drain in the background. Mutation-checked: removing the early exit
makes the new timing test fail (8.2s vs <1.5s).

* Bound MiniMax OAuth error responses

* fix(minimax-oauth): read streamed error bodies inside the client context + real-transport tests

Follow-ups on the salvaged bounded-read fix:
- refresh flow: the non-200 branch reads a STREAMED body, which fails
  (ReadError/StreamClosed) once the httpx.Client context has exited —
  moved inside the context. Repro + regression test use a real socket
  server (MockTransport buffers in memory and cannot catch this).
- truncation guard: >limit bodies end with ...[truncated] (mutation-checked
  against the is_stream_consumed fallback).
- test mocks now model the streamed-read surface (is_stream_consumed,
  iter_bytes, client.send) so non-200 paths exercise the real bounded read.

* chore: map xaydinoktay@gmail.com to aydnOktay

* chore(contributors): map four B2 salvage author emails (#77641)

unixwzrd.register@mac.com -> unixwzrd (#74679); dai.suzuki.829@gmail.com -> hariNEzuMI928 (#75395); lexharddrive69@gmail.com -> hdd69 (#38470); coder@trevhome.local -> trevornk (#76282). Needed for the B2 desktop-renderer salvage attributions.

* perf(session-search): project fields before enrichment

* test(session-search): guard projected enrichment

* fix: skip memory prefetch on trivial user prompts (greetings)

Salvage of PR #25350 (commits 88ffede2d + 2b848a0b2 + 3136dc63a, squashed
and ported): the run_agent.py prefetch site the PR gated has since moved
into agent/turn_context.py's build_turn_context(), so the trivial-query
gate lands there instead.

- Gate the per-turn memory_manager.prefetch_all() on a trivial-prompt
  check so greetings/acknowledgements ('hi!', 'thanks', 'ok') no longer
  block the turn on provider network round-trips or inject stale context.
- Extend honcho's _TRIVIAL_PROMPT_RE with greetings and a trailing
  punctuation class so 'hey!' / 'hello.' classify as trivial.
- Add honcho classifier tests for greeting forms.

* chore: add ayushere to AUTHOR_MAP

* refactor(memory): single shared trivial-prompt classifier + gate tests

Rebase fold on the salvaged gate:
- is_trivial_prompt/TRIVIAL_PROMPT_RE move to agent/memory_provider (the
  ABC both the core gate and providers already import) — one source of
  truth; honcho's _TRIVIAL_PROMPT_RE now aliases it, turn_context and the
  queue_prefetch_all warm path (a sibling site main grew after the PR's
  base) both use it
- tests: gate tests at the prefetch call site (mutation-checked), shared
  classifier tests incl. prefix-collision guards (k8s/yolo/note/supper),
  and honcho dialectic-machinery tests re-driven with a substantive prompt
  ("hello" became trivial by design — those tests exercise thread cadence,
  not the classifier)

* refactor(honcho): delegate _is_trivial_prompt wholly to the shared classifier

Simplify-pass finding: sharing only the REGEX left the wrapper logic
(empty/strip/slash checks) duplicated, half-defeating the no-drift goal.
The classmethod now calls agent/memory_provider.is_trivial_prompt directly;
_TRIVIAL_PROMPT_RE stays as a class attr for backward compatibility with
any external referents.

* fix(desktop): measure adaptive stream flush through the deferred commit frame

scheduleDeltaFlush's adaptive floor is driven by lastFlushCostRef, but
runFlush only timed flushQueuedDeltas(), the synchronous store write.
While a session streams, syncSessionStateToView defers the $messages
publish (React commit + Streamdown re-parse) to its own rAF, so the
measured cost stayed near zero and the floor collapsed to the fixed
33ms path no matter how expensive the real commit was.

runFlush now records the write cost as a fallback, then extends the
measurement through a rAF registered after the view-sync one: it runs
in the same frame right after the deferred commit, and the rAF
timestamp marks frame start so only in-frame work is counted, not the
vsync wait. A stale callback from before a newer flush is ignored, and
a hidden renderer that never fires rAF keeps the write-cost fallback.

* fix(desktop): dedupe optimistic user turns for all wire references, not only images

* test(desktop): cover wire reference normalization edges

* fix(desktop): sort reference-kinds import per lint gate

* fix(desktop): full-jitter backoff on gateway WS reconnect loops

All three desktop reconnect loops (primary gateway boot, secondary
multi-profile gateway pool, plugin event socket) used bare exponential
backoff with no jitter. After a gateway restart every disconnected
client redials on the exact same schedule, so the reconnect attempts
land in lockstep instead of spreading out -- a burst that can starve
the gateway's file descriptors while it's still coming back up.

Add reconnect-backoff.ts implementing AWS-style full-jitter backoff
(random delay in [0, min(cap, base * 2^attempt))) and wire it into all
three call sites in place of their local Math.min/2**attempt math.
Manual reconnect paths already reset the attempt counter and bypass
the timer entirely -- unchanged.

* fix(desktop): escalate gateway reconnect on elapsed time, not attempt count

With the full-jitter backoff (300ms base) six attempts can elapse in ~9s,
so the old RECONNECT_ESCALATE_AFTER=6 attempt threshold raised the
recoverable boot error during a brief post-boot blip — breaking the
'a remote that drops post-boot keeps looping with NO boot.error' contract.
Escalate after RECONNECT_ESCALATE_AFTER_MS (45s, matching the old
deterministic 1->15s ladder's calibration) elapsed since the first failed
reconnect of the episode. Reset on clean open, manual/wake reconnect, and
soft switch, preserving the reset-on-success path.

* chore(contributors): map vittoria3103.123@gmail.com -> VittoriaLanzo (#77665)

Needed for the #62082 curator toolset-pin salvage attribution.

* fix(desktop): un-break the .btn-arc rule — '*/' inside a CSS comment ended it early

The comment above .btn-arc contained 'bg-*/', whose */ terminated the comment mid-sentence, leaving 'text-* variant utilities. */ .btn-arc {' as an invalid prelude — CSS error recovery can drop the whole .btn-arc rule. Reword so no */ appears inside the comment.

Extracted from #59352 by @rerdi92 (the rest of that PR — a month-stale icons.ts rewrite and a chunk-size warning-ceiling bump — is superseded/masking and was not salvaged).

* perf(desktop): stop idle chat re-renders — memo ChatView, stable tile props, gated adapter re-sync

Re-derive of PR #38470 on today's main (its target file desktop-controller.tsx no longer exists after the contrib/ refactor; the three surviving ideas are applied at their new homes):

- incremental-external-store-runtime: the dep-less setAdapter effect ran every render; gate on [runtime, store] — behavior-preserving because __internal_setAdapter early-exits on identical store.
- ChatView is now memo()d, and session-tile hoists its inline arrow props to useCallbacks/module constants so the memo actually holds.
- Render-count regression test (mocked Thread) proves an unrelated parent re-render no longer re-renders the chat shell.

Credit: idea and original implementation by @hdd69 in #38470.

* perf(curator): trim dead tool-schema from the LLM review fork

The curator LLM review loop (_run_llm_review) built its AIAgent without
enabled_toolsets, so it advertised the full default catalog (~30 tools plus the
context_engine lcm_* family) on every call. The fork uses only four tools, fixed
by its own system prompt, with no dispatch path to the rest, so ~26 tool schemas
shipped on every request as dead weight: ~7K input tokens per call on a loop that
makes 50-100 calls per consolidation pass.

Restrict the fork to enabled_toolsets=["skills", "terminal"], the same tools the
prompt already names. Behavior-neutral: the prompt held the model to these tools
and nothing routed calls to the others. Mirrors the background_review fork
(background_review.py:788-794). Call-site only; AIAgent already forwards the kwarg.

Adds test_review_fork_restricts_toolsets_to_skills_and_terminal (captures the
constructor kwarg) and test_review_fork_toolset_surface_is_skills_plus_terminal
(pins the resolved surface).

* fix(desktop): keep a mid-turn reply on screen when its session is reopened

Switching sessions while a turn streams (or right as it completes) could
leave the assistant reply missing until restart. Resume merges stored
history with the gateway's `inflight` projection, whose assistant row is
text-only and often an empty `assistant-stream-${sessionId}` shell; both
reconcile paths then dropped the local pending row that held the only copy
of the streamed text, reasoning and tool calls.

A shared pair of guards replaces the ad-hoc comparisons at all three sites.
`localPendingSupersedes` accepts the cached row only when it is the same
reply further along — an empty shell it has content for, or text it strictly
extends — so a longer unrelated row can no longer hijack an ordinal or reuse
a stream id, and a retained `inflight.error` snapshot is never mistaken for
an empty shell. `withAuthoritativeTurnState` then takes content from the
renderer while liveness, row id and reactions stay the backend's call, so a
settled shell cannot leave a finished reply spinning.

Co-authored-by: arimu1 <19286898+arimu1@users.noreply.github.com>

* fix(desktop): stop a finished reply rendering twice after history catches up

When a turn's reply commits under its own id, the settled local
`assistant-stream-*` row shifts one assistant ordinal earlier, so ordinal
pairing finds nothing at its slot and re-appends it — the same answer twice.

Drop a settled stream row only when the authoritative transcript already
carries that exact text. Keying `isPendingAssistant` on the explicit pending
flag alone would also have fixed this, but it discards the sibling case in
the same report: a reply that finished locally before the gateway committed
it, where the local row is the only copy that exists.

Co-authored-by: Dolverin <59100064+Dolverin@users.noreply.github.com>

* chore(contributors): map two B3 salvage author emails (#77685)

abdulsalamalotaibi86@gmail.com -> carbongotfound (#74025); soundbrokaz@kakao.com -> JeremyDev87 (#72813).

* refactor(desktop): hoist the reference-line matcher; drop dead textWithoutImageRefs

Follow-up to #77653: textWithoutReferenceLines built a fresh /g RegExp per call and hand-managed lastIndex — but it runs on both sides of every message comparison in the reconcile loops. An anchored non-global regex has no shared-lastIndex hazard and can be hoisted to module scope. Also removes textWithoutImageRefs, whose last production consumer #77653 replaced (kept IMAGE_REF_LINE_RE for extractImageRefs), and retargets its now-stale comment.

* fix(desktop): do not sandwich structured mid-turn rows with inflight dump

Skip pure-text inflight.assistant projections when the transcript already
has reasoning/tool-call structure, and only overlay journal answer text
on strict extension.

Fixes #76444

* fix(desktop): scope inflight dump suppression to the live turn tail

Only skip/graft structure for the current live assistant (stream id,
pending, or after the latest user), not completed historical tool rows.
Require live-tail identity for same-turn structure carry. Align journal
overlay with strict answer-text extension.

Addresses review + CI on #76744.

* fix(desktop): require structure-bearing row for live-tail same-turn carry

Structure-only same-turn carry used (live(previous) || live(message)), so a
new live text-only assistant at a compression-rewritten ordinal could inherit
reasoning/tool parts from an unrelated historical structured row.

Require the structure-bearing cached row itself to be live-tail (pending /
assistant-stream-* / interim). Add regressions for non-extending live dump
carry and the compression graft rejection.

Addresses salvage path on #76744 / #76444.

* refactor(desktop): one live-tail vocabulary for transcript reconciliation

Two fixes landed overlapping helpers on the same statement: the mid-turn
reply guard grew `isLiveProjectionRow` / `hasStreamedContent`, while the
inflight-dump guard grew `isLiveTailRow` / `hasStructuralParts`. Two
definitions of "is this row live" and "does it carry content" in one
function is how the next change silently reshapes one of them.

Collapse to a single module-level pair. `isLiveTailRow` now covers pending,
stream ids, inflight projections and sealed interim rows, so the reply guard
also stops treating an interim row as committed history; `hasStreamedContent`
is defined in terms of `hasStructuralParts`. Both text-extension checks route
through `isStrictAnswerTextExtension` rather than a bare `startsWith`.

Also hoists the live-tail lookup out of an inline IIFE and fixes the lint
warnings it carried.

Co-authored-by: 686f6c61 <github@00b.tech>

* fix(dashboard): cache plugins hub payload and avoid auth probes

* test(dashboard): cover install-hook invalidation of plugins hub cache

* fix(dashboard): warm cold check_fn verdicts with a background probe

On dashboard-only sessions nothing else executes check_fn warmers (they
live only in the tool-schema build), so the hub's read-only cache lookup
would report auth_required=False forever. On a cache miss, schedule a
deduplicated daemon-thread probe off the request path; the short hub TTL
surfaces the verdict on the next fetch.

* fix(desktop): cancel the pending commit-cost measurement rAF

Follow-up to #77652: each runFlush registered a fresh requestAnimationFrame and never cancelled it. Chromium parks rAF callbacks for hidden renderers, so a long hidden stream at the 33ms floor accumulates thousands of parked closures that all fire in the first frame on refocus (all but one no-oping through the stale-frame guard). Track the pending handle, cancel it before requesting a new one (only the newest flush's measurement matters), and cancel on unmount.

* perf(dashboard): skip full InsightsEngine on /api/analytics/usage (#18511)

* perf(dashboard): keep tools in focused analytics usage (#18511)

* refactor(insights): drop consumer-less get_skill_breakdown alias (simplify-pass)

The 2-line alias had zero production consumers (web_server calls
get_usage_breakdown directly). Tests rewired onto the real API; the
contracts they pin are unchanged. Stale test docstring fixed.

* fix(web): clamp dashboard pagination and analytics-days params (#39200 + #74778 salvage)

Re-derivation of aydnOktay's twin clamp PRs onto current main (the
session-list endpoints moved into web_routers/; the analytics endpoints
gained asyncio.to_thread wrappers since the originals):

- limit le=100 on /api/sessions, /api/sessions/search and the
  /api/profiles/sessions fan-out (one unbounded request could drag every
  session row + correlated-subquery preview work out of SQLite, times
  every profile's state.db on the fan-out).
- days ge=1 le=365 on /api/analytics/usage + /api/analytics/models
  (huge or non-positive values force full-history InsightsEngine work or
  inverted windows; the UI only offers 7/30/90 presets).

FastAPI Query bounds reject at the validation layer (422). 8 new tests;
both clamp classes mutation-checked (clamp removed -> its tests fail).

* fix(clamps): raise profile fan-out limit to le=500 (simplify-pass finding)

le=100 would 422 real desktop callers: sessions-settings fetches
archived at limit=200, the command palette lists at 200, and the
electron remote-merge over-fetches limit+offset (exceeds 100 at
offset>=81, and its .catch(()=>null) silently drops remote sessions).
Clamp must sit above real client maxima. New test pins limit=200 w/
offset.

* fix(web): avoid blocking provider validation

* perf(plugins): seed plugin routes from sessionStorage cache for instant render

- Plugin manifests are now cached in sessionStorage on fetch.
- On refresh, plugin routes are registered synchronously from cache, preventing unwanted redirects to /sessions.
- Removes the !pluginsLoading guard from the catch-all route in App.tsx, as plugin routes are now always available on first render.
- Background fetch always updates the cache and routes, so new/removed plugins are reflected after reload.
- Resolves the race condition where plugin pages would redirect to /sessions on hard refresh.

* fix(plugins): validate cached manifests are an array

* test(plugins): export cache helpers and add focused fallback/refresh tests

* fix(plugins): keep loading gate when cached manifests include a /chat override

The sessionStorage seed set loading=false whenever any cache existed, which
defeats App.tsx's load-bearing pluginsLoading gate: with a cached manifest
that declares tab.override === "/chat", the persistent ChatPage host must
NOT mount before plugins resolve, or it spawns a PTY and gets yanked when
the override plugin takes over the route.

Seed loading=false from the cache only when no cached manifest overrides
/chat (canSeedLoadedFromCache); manifests are still seeded either way so
plugin routes register synchronously on refresh. Adds focused tests for
the gate, including the /chat-override case.

* perf(dashboard): serve hashed /assets bundles with immutable cache headers

Every hashed bundle chunk under /assets/ was served with no caching
directives, so each dashboard load re-fetched (or at best revalidated)
every JS/CSS chunk. Those filenames carry a Vite content hash — the
bytes behind a given URL can never change; a rebuild mints new
filenames referenced by a freshly served index.html.

Mark them Cache-Control: public, max-age=31536000, immutable:
- the /assets StaticFiles mount, via a subclass that stamps the header
  on 200s only (404s stay uncached — a rebuild can create the file),
- serve_css, preserving its X-Forwarded-Prefix url() rewrites for
  /fonts/, /fonts-terminal/, /ds-assets/, /assets/.

index.html keeps no-store, no-cache, must-revalidate — it is the
mutable entry point that binds users to the current hashes.

The original PR also added hand-rolled per-request gzip compression of
asset responses; that part is deliberately dropped. This server is a
localhost-default dashboard backend: compressing every response on the
CPU to save loopback bandwidth is a pessimization, and callers that
front it with a real proxy already get compression there.

Salvaged from PR #28543 (idea by @sea-monsters; gzip groups dropped as
described above).

Co-Authored-By: Claude Fable 5 <noreply@anthropic.com>

* feat: reset-aware primary restore — stay on fallback until the rate-limit window resets

restore_primary_runtime retries the primary every turn once the 60s
transient cooldown clears. For subscription-window limits (Claude
Pro/Max 5h windows, Codex weekly caps) the reset is hours or days away,
so every retry is a guaranteed failure costing two provider switches
and two prompt-cache invalidations per turn.

Add CredentialPool.next_available_at() (earliest reset across exhausted
entries; None when available now or no reset info) and gate the restore
on it: skip while the primary's pool says nobody can serve, restore on
the first turn after the reset elapses. Fail-open: any gate error or
missing reset info falls through to the existing per-turn retry, so
recovery can never be later than today. Cross-provider fallbacks
consult the PRIMARY's pool (not the attached fallback pool), reusing
the loaded pool for the existing rebind to keep auth reads at one per
restore.

* fix(credential_pool): run next_available_at under the pool lock

Review fold on the #67642 salvage: next_available_at() called
_available_entries() — which prunes DEAD entries, syncs tokens, and
persists — and iterated self._entries with no lock, racing concurrent
select()/rotation exactly as has_available()'s comment warns. Wrap the
method body in self._lock and pin it with a non-blocking-acquire probe
test.

* fix(credential_pool): defer single-use-token refresh outside threading lock

select() and acquire_lease() held self._lock during the entire
_available_entries() loop, which for openai-codex and xai-oauth providers
includes a cross-process file lock (_auth_store_lock) plus OAuth token
refresh HTTP POST.  The lock timeout can exceed 20 seconds, blocking all
credential pool consumers across every gateway thread and subagent.

Collect single-use-token refresh entries under the lock, then execute the
refreshes outside it.  On success the refreshed entry is merged back into
the pool and re-selected.  Non-single-use providers (anthropic, nous)
continue refreshing inside the lock since their refresh is a simple HTTP
POST with no cross-process coordination.

* fix(credential_pool): serialize deferred-refresh pool mutations

Review folds on the #71775 salvage (dossier findings 1+2):

- self._lock becomes an RLock and the mutation primitives
  (_replace_entry, _persist) are now self-locking, so the deferred
  single-use-token refresh path — which deliberately runs its
  cross-process flock + OAuth network I/O OUTSIDE the pool lock —
  still serializes its pool mutations against concurrent
  select()/rotation. In-lock callers re-acquire reentrantly.
- Dropped _refresh_pending_entries' redundant second _replace_entry:
  _refresh_entry already merges the refreshed entry internally.

Adds tests/agent/test_credential_pool_deferred_refresh.py pinning both
invariants: select() must NOT hold the lock during the refresh window
(the PR's whole point), and the post-refresh mutations MUST contend on
the lock (blocking-thread probe).

* `hermes sessions optimize-storage` aborts with

```
Error: optimization failed: no such table: messages_fts_trigram
No data was lost. Re-run to resume.
```

on any install where the trigram FTS index is legitimately absent. The failure is
deterministic — re-running can never make progress, because the crash happens at the same
point every time — so the database is permanently stuck on the legacy high-footprint FTS
layout with no supported way forward.

Observed on a 5.4 GB production `state.db`. After the fix the same database optimized
successfully and shrank to 3.3 GB.

The trigram index is absent whenever the runtime cannot maintain it. On a SQLite build
without the `trigram` tokenizer, `_ensure_fts_schema()` returns `False`, so `__init__`
leaves `self._trigram_available = False` and no `messages_fts_trigram` table on disk. This
is a **supported degraded runtime**, not damage — CJK/substring search falls back to
`LIKE` and everything else works normally. `_is_fts5_unavailable_error()` and
`_warn_trigram_unavailable()` exist specifically to make this path graceful.

Two code paths write the boundary sweep for the deferred FTS rebuild, and only one of them
respects that flag:

| Function | Trigram `INSERT` guarded? |
|---|---|
| `fts_rebuild_step()` | ✅ `if include_trigram:` where `include_trigram = self._trigram_available` |
| `_fts_rebuild_finish()` | ❌ unconditional |

`_fts_rebuild_finish()` runs the boundary sweep at the *end* of the backfill. Its
unguarded `INSERT INTO messages_fts_trigram …` raises `OperationalError`, which propagates
out of `optimize_fts_storage()` and aborts the entire optimization — *after* the backfill
has already completed. Hence the characteristic output showing 100% progress immediately
before the error:

```
Rebuilding index: 100% (909,671/909,671)
Error: optimization failed: no such table: messages_fts_trigram
```

There is a second, quieter consequence. The teardown phase that reclaims the demoted
`fts_v22_trash_*` shadow tables runs *after* the backfill phase in
`optimize_fts_storage()`. Because the crash happens before teardown is ever reached, those
tables are never emptied or dropped — so the space the migration was supposed to reclaim
stays allocated indefinitely, and the leftover trash tables look (misleadingly) like
evidence of a half-finished migration.

Build a populated v23 database, set the deferred-rebuild markers, then reopen it on a
runtime where `_ensure_fts_schema('messages_fts_trigram', …)` returns `False` (exactly
what a SQLite build without the trigram tokenizer produces) and call
`optimize_fts_storage()`:

```
[precondition] trigram absent, _trigram_available=False, rebuild pending  ✓

RED  ✗ optimize_fts_storage raised OperationalError: no such table: messages_fts_trigram
```

With this patch applied, unchanged harness:

```
     optimize_fts_storage returned {'ok': True, 'vacuumed': None}
GREEN ✓ optimize ok; markers cleared; base FTS 'zebra' -> 200 hits
```

Full harness and transcripts in `TEST-EVIDENCE.md`.

Gate the sweep on `self._trigram_available`, exactly as `fts_rebuild_step()` already does:

```python
include_trigram = self._trigram_available

def _do(conn):
    ...
    if include_trigram:
        conn.execute("INSERT INTO messages_fts_trigram(...) ...")
```

The base `messages_fts` sweep and the marker cleanup are untouched, so the rebuild still
finalizes correctly and the index remains complete for every row it is responsible for.
The fix does not disable or weaken search to dodge the error — the regression tests assert
that base FTS still returns results afterwards.

`TestFtsRebuildFinishWithoutTrigram` in `tests/test_hermes_state.py`:

- `test_rebuild_finish_skips_trigram_when_unavailable` — drives `_fts_rebuild_finish()`
  directly on a trigram-less runtime; asserts it completes, clears both rebuild markers,
  and leaves base FTS searchable.
- `test_optimize_fts_storage_succeeds_without_trigram` — end-to-end through the public
  `optimize_fts_storage()` entry point; asserts `ok=True`, markers cleared, search intact.

Both use the existing `_NoTrigramConnection` helper already in the file. Both fail on
`main` with `no such table: messages_fts_trigram` and pass with this patch.

`tests/test_hermes_state.py` passes in full (463 tests → 465 with these two). `ruff` clean.

This PR is the crash only.

A companion PR narrows `_db_opens_cleanly()` so that
`hermes sessions repair --check-only` stops reporting a write-broken FTS schema as
healthy — the gap that makes this class of problem hard to diagnose in the first place.
The two are independent and can land in either order.

* perf(tools): shrink lazy tool catalog overhead

* refactor(tool-search): drop dead fallback ladder in _available_source_summary

Simplify-pass finding: _listing_group_label already falls back to 'other' for empty source names, and _classify_source guarantees source_name=='' only when source=='other' — both  legs were dead by construction. Aligns the summary path's grouping with the listing path.

* fix(credential_pool): unpack the tuple in next_available_at's gate

Cross-PR interaction fix: #77714 (salvage of #71775) changed
_available_entries to return (available, pending_refresh) while #77631
(salvage of #67642) added next_available_at() which still truthiness-
tests the bare return. A non-empty tuple is always truthy — even
([], []) — so the reset-aware gate silently returned None ('no wait
info') for every exhausted pool, disabling the feature #77631 shipped.
Unpack the tuple and test the available list.

Also adapts the lock-probe test for the RLock introduced by #77714
(same-thread non-blocking acquire always succeeds on an RLock; probe
from a helper thread instead).

* chore: map copii.list@gmail.com to stremtec

* chore(contributors): map marzukia@users.noreply.github.com -> marzukia (#77774)

Needed for the #37117 salvage (#77696 CI failure).

* chore: map bot@bkstock.dev to BKStock

* perf(session): route SQLite PRAGMAs through central apply_database_pragmas

Addresses review from @teknium1 on PR #71755:

- Extended apply_database_pragmas() to handle cache_size, mmap_size,
  and temp_store from config.yaml (alongside existing wal_autocheckpoint
  and journal_size_limit). No hardcoded defaults — all values are
  opt-in via config.yaml, avoiding policy conflicts with other PRs.
- Applied to ALL connection types: writer (_connect_and_init),
  read_only cross-profile attach, and WAL per-thread readers
  (_get_read_conn). Previously PRAGMAs only ran on the writer path.
- Removed inline PRAGMAs from _connect_and_init — single source of
  truth in apply_database_pragmas().
- Documented config keys with examples in function docstring.

* fix(pr): remove remnant local PRAGMAs from PR branch

* test(session): guard config-gated performance PRAGMAs across all connection types

E2E guard for the salvaged PR #71755: database.cache_size/mmap_size/
temp_store from config.yaml must reach the writer connection, the
read-only cross-profile attach, and the WAL per-thread reader — and a
default install (no database: keys) must keep byte-identical SQLite
defaults on every connection type. Also covers integer-coercion
rejection of garbage values for the three new keys.

cache_size uses -16000 (not the doc example -2000) because -2000 is
SQLite's compiled-in default and would not discriminate a regression.

* fix(desktop): flush queued deltas on window focus

* perf(desktop): stop scroll and status loops in busy sessions

* perf(desktop): pause hidden-pane timers in agents view, cron sidebar, and floating pet

Partial pick of the surviving renderer hunks from #75395 (perf commit
6502e441d plus fixup 3fbbc9c1d): gate the 500ms subagent now-ticker and
the cron sidebar 1s ticker/run-poll on usePaneVisible, and skip the
legacy floating-pet poll while the document is hidden. Dropped hunks
(electron/main.ts, vitest.setup.ts/config) intentionally excluded.

* style(desktop): restore alphabetical import order in agents/index.tsx

* refactor(desktop): shared pulse beat + fully-gated cron peek (simplify folds)

Two findings from the simplify pass on the final trio diff:

- status-pulse: one pause controller + one aligned period timer shared by all StatusPulse instances (ref-counted), instead of N x (document/window/bridge listeners + unsynchronized 5s wakes) — a sidebar can show dozens of pulsing dots. Pause still cancels in-flight animations so the compositor sleeps immediately.
- cron-jobs-section: the runs-peek effect created its interval even while the pane was hidden (callback no-oped but the timer still woke the renderer every 8s/60s per expanded job). Early-return when hidden — visibility is already in the dep array, so becoming visible restarts load + timer.

* fix(lint): import sort + eslint-disable for timer-handle ref clear in effect

CI-caught: cron-jobs-section had an extra blank line between sorted imports; use-message-stream's visibility-flush effect assigns flushHandleRef.current=null inside a useEffect (legitimate timer-clear, not an atom mirror) — eslint-disable-next-line per the rule's documented convention.

* fix(state): narrow FTS UPDATE triggers with AFTER UPDATE OF + migration

Retarget #73639 onto the SessionDB mixin split (hermes_state_common /
hermes_state_schema). Fresh installs create UPDATE OF content/tool_*
triggers; existing broad AFTER UPDATE triggers are inspected and
replaced under schema init without an FTS rebuild (WHEN clauses already
guarded content correctness; OF skips non-content status writes that
saturated disk I/O on large state.db).

Tests: tests/test_fts_update_of_narrowing.py (4)

* fix(state): fail closed on CJK trigger migration

* fix(state): quarantine CJK when ensure soft-fails after OF migration

_ensure_fts_cjk_schema never raises on OperationalError; post-condition
after dropping messages_fts_cjk_update now requires a narrowed UPDATE
trigger or durable fts_cjk_stale + unavailable. Covers the production
soft-fail path the raise-only handler missed.

* refactor(state): drop unreachable regex guard in trigger migration

Simplify-pass fold: to_drop names come from the literal update_names\nallowlist via IN binding, so the [A-Za-z0-9_]+ fullmatch could never\nfail — and if it somehow did, its `continue` would miscount (the\nskipped trigger stayed in len(to_drop)/the log while CREATE TRIGGER\nIF NOT EXISTS silently kept the broad variant). Delete the guard and\nits function-local re import; keep the invariant as a comment.

* fix(security): reject always-blocked OpenViking endpoints

## Summary
- Normalize OpenViking endpoints through `is_always_blocked_url` and fall back to the default local endpoint when poisoned.
- Keep intentional loopback / LAN self-host working.
- Add focused unit tests.

## Salvage / credit
Memory-provider endpoint floor sibling of RetainDB/Supermemory always-blocked hardening (avoids over-broad #4984-style private-IP bans).

(cherry picked from commit 8fa607d0aedb8c5fca398d7f112b1b25ade54fa2)

* fix(openviking): fail closed on blocked endpoints

(cherry picked from commit 389a90b81c9c2c89810f2fa7461f8faa9a5c9578)

* fix(openviking): don't spawn a second server onto a live port

`_start_local_openviking_server()` spawned `openviking-server`
unconditionally. Both callers — `initialize()` and the runtime
unreachable handler — reach it from a health probe, and that probe can
time out client-side while the server is up and serving. The spawned
process then loses the data-directory lock and exits immediately with
`DataDirectoryLocked`; because the probe keeps timing out, the cycle
repeats every cooldown window (~5 min observed).

The existing 30s `_failed_refresh` cooldown paces the loop but cannot
stop it, since it expires while the underlying condition persists.

Probe the target host:port before spawning and treat an occupied port as
already-started. This guards both call sites at their single convergence
point. The probe deliberately tests only that a listener owns the port —
enough to know a second server would lose the lock — and says nothing
about that listener's health.

The parse/probe now precedes the PATH lookup, so a reachable server is
reported as running even when `openviking-server` is not on PATH.

Fixes #74846

(cherry picked from commit b49427d85fd6628eb4a7fe099e5c390c5c4cc935)

* fix(openviking): drop stale "disabled for this Hermes run" warnings

The provider used to disable OpenViking permanently when the server was
unreachable. That was fixed: `_ensure_client()` now reconnects lazily,
with a 30s cooldown gate in `_ensure_client_locked`.

Only one of the seven user-facing warnings was updated to match. The
other six still told the user memory was "disabled for this Hermes run",
which is no longer true — every one of those paths is retried on the next
access. A user who reads the old message has no reason to retry, which is
very likely how #5721 ("never recovers") came to be filed against
behaviour that already recovers.

All six sites were traced to confirm none is terminal for the run: the
`initialize()`-time and waiter-thread failures never arm `_failed_refresh`
(only line 2439 does), so they retry on the very next access with no
cooldown at all.

The replacement wording deliberately omits the "(after cooldown)"
parenthetical used at the already-correct site — that detail is only
accurate where `_failed_refresh` was just armed. The neutral phrasing is
true at all six.

Also promotes two clause separators to periods to avoid "…; …disabled;"
collisions.

(cherry picked from commit 8346403a4b97af503d26b0f7905ff513828d821e)

* fix(openviking): re-arm the commit guard after in-place compression

`_committed_session_ids` is a permanent per-sid latch, and
`_session_needs_commit` checks it before the turn counter by design — a
racing sync_turn can re-increment `_turn_count` after commit+reset, so
the guard must win to stop a double-commit.

That is correct for a session being left behind. It is wrong for one
that keeps its id. `compress_context()` commits before rewriting the
transcript in both modes, and with `compression.in_place: true` (the
default) `on_session_switch` receives the same id and does not rotate.
The latch then rejects every later commit for a still-live session — the
next compression, /new, normal session end, startup recovery — so every
post-compression turn is silently never extracted.

Rotation mode is unaffected because a fresh child id is minted and
starts clean, which is what confirms the latch's intent was only ever to
dedupe the departing id.

Clear the latch when compression completes without rotation. Turns
arriving after that point are genuinely new, and this is a defined
moment rather than a race. The rotation path is untouched, so the old
id stays latched and its _finalize_session_async still dedupes against
the compression commit.

Fixes #74695

(cherry picked from commit d1e5c3dc33ef0d43d021662674e1a7cd5e43eecd)

* test(openviking): cover the compression lifecycle, not a hand-set latch

Review feedback: the previous test called _mark_session_committed
directly, so it verified the guard's behavior but not the wiring that
sets it — a future break in the commit_memory_session -> same-id
compression-boundary path would not be caught.

Add a lifecycle regression that drives the real sequence: on_session_end
commits through the actual path, on_session_switch(same id,
reason="compression") crosses the boundary, sync_turn records a genuinely
new turn, and a second on_session_end must produce a second commit POST.

Without the fix it fails showing exactly one commit call, which is the
reported data loss: every turn after the first compression is dropped.
The rotation and /undo tests stay as scope guards.

(cherry picked from commit 0ca5a330630a30b105cbbc32e8a23f2c5ffe0eab)

* fix(memory): read non-secret provider config from config.yaml for OpenViking and RetainDB

OpenViking is_available() only consulted env vars and use_ovcli_config, so an
endpoint saved to config.yaml (e.g. by the Dashboard) reported needs_config;
_resolve_connection_settings() likewise never folded config.yaml's non-secret
fields into its chain. RetainDB initialize() read base_url/project from the
environment only, ignoring the values the Dashboard writes to config.yaml.

Both now resolve non-secret fields as env -> (ovcli ->) config.yaml -> default;
secrets still come from the environment. Adds regression tests for both.

Fixes #68209

(cherry picked from commit dca57915b97b5705b30927a062e1d0f2f23d3841)

* fix(openviking): read recall settings from config.yaml first, env vars as fallback

_recall_config() previously read all settings (recall_limit, score_threshold,
recall_resources, etc.) exclusively from environment variables. This forced
users to store behavioural configuration in .env, violating the Hermes
convention that .env is for secrets only.

The infrastructure to load config.yaml -> memory.openviking was already in
place via _load_hermes_openviking_config(), but _recall_config() never
called it.

Fix: call _load_hermes_openviking_config() and pass its values as the
default parameter to _env_int/_env_float/_env_bool. Env vars still override
config.yaml values, preserving backward compatibility.

Closes #62540

(cherry picked from commit 6aadf1256835745e0302aa3d3b5ae0660b368637)

* test(openviking): cover config.yaml recall settings with temp-HERMES_HOME tests

Add three tests to TestOpenVikingConfigSchema:

1. test_recall_config_reads_from_config_yaml — writes memory.openviking
   settings in config.yaml and verifies _recall_config() consumes them.

2. test_recall_config_env_overrides_config_yaml — writes both config.yaml
   and OPENVIKING_RECALL_* env vars, verifies env takes precedence.

3. test_recall_config_partial_config_yaml — partially populated config.yaml
   falls back to defaults for omitted keys.

All 46 openviking_plugin tests pass (43 existing + 3 new).

(cherry picked from commit b8d7834caf06c6912004333c270faa248eaed4cd)

* fix(openviking): integrate reliability and configuration hardening

* chore(contributors): map OpenViking source authors

* test(retaindb): guard scoped secret config resolution

* fix(openviking): verify servers before sending credentials

* fix(openviking): catch endpoint errors in setup validation functions

Review follow-up for salvaged PR #76782. Three setup-wizard
validation functions called _normalize_openviking_url outside their
try/except blocks. Since _normalize_openviking_url now raises
_OpenVikingEndpointError for blocked or malformed endpoints, an
invalid endpoint would crash the wizard instead of returning a
friendly (False, message) tuple.

- _validate_openviking_auth: move _normalize_openviking_url inside try
- _validate_openviking_root_access: same
- _validate_openviking_setup_values: catch _OpenVikingEndpointError explicitly
- Remove dead ternary in _normalize_openviking_url safety check (candidate
  always has http/https scheme by that point)
- Replace redundant float('-inf') < x < float('inf') with math.isfinite()
  in _setting_float; drop the redundant infinity check from _setting_int
  (is_integer() already rejects inf/nan)

* fix(state): deduplicate session system prompts

* chore: map cicav legacy noreply email

* fix(tui): avoid writable Kanban opens on empty polls

* fix(context): dedupe subdirectory hints by content digest and skip backup/vendor dirs

SubdirectoryHintTracker re-injected identical context files whenever the same
AGENTS.md was reachable through more than one path. Symlinked shared
workspaces, hardlinks, and timestamped backup copies all alias a single file,
so a normal session could ship the same 8KB of instructions two or three
times. Nothing deduped it and nothing excluded directories that only ever
hold copies.

Two changes:

* Track a sha256 of every injected hint body. Repeat content is skipped, and
  the working directory's own context file is seeded at construction so the
  copy prompt_builder already loaded at startup is never sent again.
* Skip directories that hold copies rather than authoritative context
  (backups, node_modules, venv, site-packages, .git, .Trash, vendor, caches).
  Screening is relative to working_dir, so a project that legitimately lives
  under vendor/ keeps discovering its own subdirectory hints.

Measured on a real session that touched a symlinked shared workspace:
3 injections / ~24,000 chars before, 1 injection / 8,112 chars after.

14 new tests cover symlink aliasing, byte-identical copies, working-dir
seeding, distinct content still being injected, each excluded directory name,
excluded ancestors, and the working-dir-inside-excluded-name case.

* perf(state): batch the turn flush into one SQLite transaction

Re-derivation of #23254 (@devsart95) on today's flush loop. The turn
flush in _flush_messages_to_session_db wrote one BEGIN IMMEDIATE
transaction per message row; a typical agent turn (user + assistant +
tool results) paid 3-8 transactions -- and, off WAL (the default on
macOS while the WAL-reset guard is active), 3-8 fsyncs -- per turn.

Adds SessionDB.append_messages_batch: same row shape as append_message
(shared _prepare_message_row serializer + _MESSAGE_INSERT_SQL column
list, so the two writers cannot drift), same compression-lock and
compression-closed guards, one aggregated session-counter UPDATE, one
transaction for the whole batch. Row serialization stays outside the
write lock.

The flush loop now collects the turn's new rows and writes them in one
call. All-or-nothing pairs exactly with the persisted-marker stamping:
on failure no rows landed and no markers were stamped, so the next
flush re-writes the whole tail (same recovery contract as before,
minus the partial-prefix case that could double-count).

Measured (same harness, 5-message turn, journal_mode=DELETE,
synchronous=FULL): 2.32ms -> 0.83ms median per turn flush (64% faster,
5 fsyncs -> 1). On WAL the win is smaller but the atomicity fix holds.

* perf(tui-gateway): batch branch-seed history copies (whole-bug-class)

Sibling sites of the per-message flush pattern: both branch-seed
paths (session.branch in methods_session.py and the lazy seed persist
in server.py) copied the parent history row-by-row -- one transaction
per row, and a branch seed can be hundreds of rows. Route both through
SessionDB.append_messages_batch. The server.py path also gains real
atomicity: _branch_seed_persisted assumed every row landed, which the
per-row loop could not guarantee.

* test(run-agent): update flush-path fakes and assertions for batched writes

The flush now goes through append_messages_batch; MagicMock-based
assertions and barrier fakes that hooked append_message observed
nothing (the flush's try/except swallowed the AttributeError). Assert
on the batch payload instead.

* refactor(state): fold simplify findings — reuse _insert_message_rows, share guards, chunk seeds

Simplify-pass folds on the #23254 salvage:

- REUSE (HIGH): append_messages_batch now delegates row serialization to
  the pre-existing _insert_message_rows helper (already shared by
  replace_messages / archive_and_compact / portability import) instead
  of adding a third serialization path (_prepare_message_row +
  _MESSAGE_INSERT_SQL are gone). One row-writer for every multi-row
  path; the row-ID return was consumed by no production caller, so the
  batch returns the inserted count.

- QUALITY (HIGH): the compression-lock + compression-closed admission
  guards are extracted into _check_transcript_write_guards, shared by
  append_message and append_messages_batch (previously duplicated 23
  lines that had already needed targeted fixes, #74478). The role-gated
  reasoning filtering is no longer duplicated in run_agent.py — it
  lives at its one site inside _insert_message_rows.

- EFFICIENCY (MEDIUM, measured): unbounded seed copies hold one BEGIN
  IMMEDIATE for seconds (10k rows ~= 2.4s; FTS triggers dominate) and
  monopolize the in-process write lock. append_messages_batch grows a
  chunk_rows param; all seed/copy call sites use chunk_rows=500. Same
  recovery semantics as the old per-row loops, bounded lock holds.

- REUSE (MEDIUM): the two remaining per-row branch-copy loops found by
  the pass (gateway/slash_commands.py /branch, hermes_cli
  cli_commands_mixin.py branch) are converted to chunked batches too
  (AsyncSessionDB's generic to_thread forwarder covers the async site).

Turn-flush benchmark unchanged after the refactor: 2.43 -> 0.87 ms
median per 5-message flush (64% faster).

* fix(tests): update two more append_message.call_args assertions to append_messages_batch

CI-caught: test_verification_stop_caching and test_tui_gateway_server::test_native_vision_turn_persists_a_renderable_image_ref both assert on append_message.call_args, but the flush loop now calls append_messages_batch. Same class of test-fake fallout fixed in 5 other files — these two were missed.

* perf(tui): memoize useSessionLifecycle return (idea from #38491)

Re-derivation of #38491 by @stremtec onto current main (the original is
10,119 commits behind; the hook moved into ui-tui/src/app/). The hook
returned a fresh object literal every render, defeating memoization in
useMainApp's consumers; useMemo over the (all-useCallback-stable)
handles makes the return referentially stable.

Dep array covers ALL nine returned handles incl. trimTail (the
re-derivation initially omitted it - stale-closure class).

* ci: retry uv python install

* fix(state): route session-resume reads through the WAL read-only connection

get_messages_as_conversation, get_resume_conversations, and
get_ancestor_display_prefix still took self._lock — the same global
choke point the read-path split (WAL per-thread read-only connections)
was meant to remove from every recall/browse read. These three are the
hottest reads in the file: every session resume across the gateway,
CLI, and ACP adapter goes through one of them, so a resume racing a
burst of concurrent-session writer flushes still convoys behind them
exactly like the fixed paths used to.

_session_lineage_root_to_tip (the lineage walk shared by all three,
plus get_conversation_root) had its own independent self._lock use and
needed the same conversion — without it the outer functions still
blocked on the very first line.

Verified empirically: a reader thread calling all three functions
while another thread holds self._lock blocked for the writer's full
hold duration before the fix, and returned immediately after (SQLite
3.50.4 in this dev venv falls back to journal_mode=DELETE per the
WAL-reset-bug guard, so the requires_wal-marked regression test is
exercised via a local WAL-forced script instead; it still runs and
passes on any runtime where WAL is actually active).

* chore: add contributor email mapping for ArcherQAQ

* fix(model_metadata): rewrite localhost->IPv4 for the remaining local probe sites

fetch_endpoint_model_metadata's generic (non-LM-Studio) /models fetch and
its llama.cpp /v1/props context-length follow-up built request URLs
straight from the unrewritten candidate, unlike every other local-probe
site. Both retained the multi-second dual-stack IPv6 connect penalty
that _localhost_to_ipv4() exists to skip (measured on macOS: localhost
32.9ms vs 127.0.0.1 0.1ms on a dead port; ~2s on Windows). normalized
stays the cache key so caching behavior is unchanged; only the outbound
request target is rewritten.

Re-derived from PR #61528 onto current main (original no longer applied
cleanly).

* fix(model_metadata): guard _localhost_to_ipv4 against non-string urls

CI slice 3/7 failures: run_conversation tests pass MagicMock base_urls
through the metadata probe path; re.sub raised TypeError where the old
code let non-strings flow through. Preserve that contract.

* perf(cold-start): mitigate ~14s GIL stall during backend init (#60800)

Three fixes for the Desktop/TUI cold-start stall where the event loop
is blocked for ~14s between HERMES_BACKEND_READY and the first
prompt (#60800):

1. copilot_auth: skip  subprocess fallback when any
   Copilot env var is explicitly set (even if invalid). The user
   expressed token intent via env var; silently substituting a CLI
   token is surprising and the subprocess adds up to 5s on Windows.

2. tui_gateway/ws: run resolve_skin() via asyncio.to_thread so config
   loading + skin engine init do not block the WS read loop during
   the cold-start RPC burst.

3. web_server: extend _warm_gateway_module to pre-import the heavy
   module chains (auth, copilot_auth, runtime_provider, skin_engine,
   inventory, model_switch) that the first WS connection + RPC burst
   would otherwise import on the loop thread. These trigger .pyc
   compilation and Defender scans on Windows (15-30s per the existing
   comment) and were not covered by the original gateway-only warm.

Tests: 5 new tests in test_cold_start_gil_stall.py + 2 new tests in
test_copilot_auth.py. All 36 copilot_auth tests + 16 ws/web_server
tests pass.

* test: harden cold-start regression tests + debug-log the env-var skip

Review folds on the #60807 salvage:
- resolve_skin tests are behavioral (thread-ident probe + ready-frame
  wiring check) instead of pure source inspection, per the #72720
  pattern; a source assertion remains as belt-and-braces.
- The warm-list test does REAL imports and checks sys.modules —
  _warm_gateway_module swallows ImportError by design, so the PR's
  tracking-stub test would pass even with a typo'd module name.
- resolve_copilot_token logs a debug line when the env-var
  short-circuit skips the gh-CLI fallback (behavioral change made
  observable).

* perf(gateway): per-platform skip_context_files to cut agent build latency

Salvage of #26860 (hunk 2, ported \u2014 the PR's base predates the current
gateway layout by ~11.9K commits). Messaging platforms can set
gateway.platforms.<key>.skip_context_files: true to skip the
filesystem-heavy context-file discovery (SOUL.md, AGENTS.md,
.cursorrules walks) during AIAgent construction \u2014 10-100x slower
stat()/walk costs on Windows made this a real per-turn tax. Soul
identity is still loaded (single small file), so the persona survives.

The flag participates in _agent_config_signature so toggling it
rebuilds the cached agent instead of silently reusing a prompt built
under the other setting (prompt-cache correctness).

The PR's hunk 1 (mtime-caching the per-turn dotenv reload) was dropped:
df51ad797 mtime-cached load_config/read_raw_config and c2eda92fd
removed the per-turn deepcopies, capturing most of that win; the
function has since gained a multiplex early-return and managed-scope
overlay that the original whole-function skip would have bypassed.

* fix(relay): route Discord tool-progress into the auto-thread, not the parent channel (#77830)

When a Discord channel message initiates a relay auto-thread, the thread does
not exist at ingest (source.thread_id is None) — the connector creates it on
its FIRST send and auto-threads any outbound carrying the reply anchor. The
final reply carries that anchor, so it lands in the thread. But the
tool-progress / status bubbles (the "Searching the web for..." updates and the
streaming preamble) were sent with _progress_metadata=None and
_progress_reply_to=None: _resolve_progress_thread_id returns None for Discord
(only slack/mattermost get a synthetic thread), so the progress send had no
anchor and the connector posted it FLAT in the parent channel. Result: the
search-status updates leaked outside the thread while the answer threaded
(staging repro 2026-08-02).

The connector now stamps prospective_thread_id on the inbound (the anchor
message id == the id of the thread it will create). Reuse it: when a
relay-delivered Discord channel-initiate carries prospective_thread_id and has
no real thread yet, carry the reply anchor (event_message_id) on both the
progress metadata (reply_to_message_id) and the progress reply_to, so the
connector routes the progress bubble into the SAME auto-thread as the final
reply. Applied to both the tool-progress path (_progress_metadata /
_progress_reply_to) and the status/interim callback path
(_status_thread_metadata). Events already arriving in a real thread, DMs, and
non-relay sources are untouched (guarded on delivered_via_upstream_relay +
prospective_thread_id + not thread_id).

Tests: two new cases in test_run_progress_topics.py — a relay Discord
channel-initiate asserts every progress send carries the anchor (reply_to +
metadata.reply_to_message_id + non_conversational), and an event already in a
real thread asserts the synthetic-anchor path does NOT engage. Full gateway
progress + relay + session suites green (228 passed).

* fix(agent): stop re-probing endpoints that blackhole TCP connects

Salvage of #71282 (Fixes #71281): a routable-but-dead endpoint (corp
LAN address while off-VPN) blackholes TCP SYNs, so every probe in the
model-metadata waterfall waits out its full connect timeout — 20+
seconds of stall per startup across detect_local_server_type,
fetch_endpoint_model_metadata, and the per-model probes.

A module-level blackhole cache keyed on host:port is populated when
any probe observes a ConnectTimeout (httpx or requests; read timeouts
deliberately excluded — an accepted connection is not a blackhole) and
consulted at the top of each guarded function. 30s TTL: long enough to
collapse one startup burst, short enough that VPN recovery is picked
up without a restart. Guard ordering: blackhole check -> disk L2 ->
HTTP waterfall, and a blackholed leg aborts the remaining legs instead
of letting each stall in turn.

Squash of the PR's two real commits (the branch's merge commits made
it un-rebase-merge-able; content verified identical via merge-tree).

* chore: release v0.20.0 (2026.8.3)

The Herald Release — voice (streaming TTS, barge-in, wake words), A2A v1.0,
outbound webhooks, grounded citations, desktop platform wave. ~3,650 commits,
~1,400 PRs, ~1,200 issues closed, 650+ contributors since v0.19.0.

Also: contributor audit additions (18 email mappings, bot-filter widening).

* chore: add contributor email mapping for Ahmett101

* perf(moa): cache resolved preset + per-slot runtime to cut cold-start latency (#66793)

* fix(discord): leave voice channels before cancelling the bot task

`DiscordAdapter.disconnect()` cancelled the bot task before tearing down voice
clients. `leave_voice_channel()` ends in `await vc.disconnect()`, and discord.py
sends a voice state update over the main gateway websocket and then waits for the
voice socket to close. The bot task is the loop running that gateway connection,
so cancelling it first left the handshake with no transport: it could never
complete and blocked until the caller's shutdown timeout fired.

The effect was a fixed ~5s penalty on every shutdown with a voice connection
open, ending in "discord disconnect timed out after 5.0s - forcing continue",
with the voice disconnect abandoned rather than completed.

Measured on a live gateway with a voice connection open in both cases:

  before: timed out after 5.0s, all adapters disconnected at +5.29s
  after:  discord disconnected (0.12s), all adapters disconnected at +0.46s

Moving the voice-cleanup loop above `_cancel_bot_task()` preserves the
zombie-client protection its comment describes: the bot task is still cancelled
before `client.close()`, just after voice teardown rather than before it. Voice
teardown is the one step that still requires a live gateway.

Adds a regression test asserting the ordering. It fails on the previous ordering
at index 1 with `cancel_bot_task != leave_voice_channel:111`.

Fixes #76044

* feat(image): parallelize image_generate batches

* fix(file-sync): serialize concurrent sync cycles

* fix(tool-executor): unpack 5-tuple runnable_calls in _max_workers_for_tool_batch

* fix: exponential backoff for rate-limit fallback cooldown

Replace the fixed 60-second cooldown with exponential backoff:
30min → 1h → 2h → 4h cap.

The counter is reset by restore_primary_runtime on successful
primary-provider recovery, so the backoff is strictly for
consecutive failures within a single degradation window.

Closes #29702

* fix(backoff): keep 60s first-hit cooldown, escalate only on consecutive rate-limits

Review follow-up on the #30223 salvage: the original changed the base
cooldown from 60s to 1800s, benching the primary for 30 minutes on the
FIRST 429 (30x regression in primary-restore latency) and breaking the
existing test_rate_limit_exhaustion_keeps_60s_cooldown contract.

Keep upstream's 60s base and escalate per consecutive rate-limit:
60s -> 2m -> 4m -> 8m -> ... capped at 4h. Counter still resets on
successful primary restore (cicae's mechanism, unchanged).

New tests: escalation doubling, 14400s cap, reset-on-restore.
Existing 60s contract test passes UNCHANGED. Mutation-checked:
escalation disabled -> 2 fail; reset disabled -> 1 fails.

* fix(catalog): wire api_key auth headers for http MCP servers

When an optional-mcps manifest declares transport.type=http with
auth.type=api_key, install_entry() prompts for the key and saves it to
.env, but _build_server_config() only handled the oauth case — the
api_key case produced a bare url entry with no headers, so every
request to the server was unauthenticated (-> 401).

Reuse _bearer_auth_headers(entry.name) from mcp_config.py so the
catalog path emits the same 'Authorization: Bearer ${MCP_..._API_KEY}'
template as the manual 'hermes mcp add --url' path.

Salvaged from #70782 (production hunk applied clean; tests re-anchored
onto current main). Credit: JonthanaHanh.

* perf(compressor): release allocator pages after successful compaction

A successful compaction frees the largest allocation a long session ever
drops (the compressed-away message dicts), but Python's arena allocator
keeps those pages in the heap — RSS retains the pre-compaction
high-water mark until exit. #76905's trim_memory lifecycle covers the
gateway/TUI housekeeping loops but not the CLI compression path.

Call trim_memory(reason='post-…
github-actions Bot added a commit to dizhaky/hermes-agent that referenced this pull request Aug 5, 2026
* chore: map bot@bkstock.dev to BKStock

* perf(session): route SQLite PRAGMAs through central apply_database_pragmas

Addresses review from @teknium1 on PR #71755:

- Extended apply_database_pragmas() to handle cache_size, mmap_size,
  and temp_store from config.yaml (alongside existing wal_autocheckpoint
  and journal_size_limit). No hardcoded defaults — all values are
  opt-in via config.yaml, avoiding policy conflicts with other PRs.
- Applied to ALL connection types: writer (_connect_and_init),
  read_only cross-profile attach, and WAL per-thread readers
  (_get_read_conn). Previously PRAGMAs only ran on the writer path.
- Removed inline PRAGMAs from _connect_and_init — single source of
  truth in apply_database_pragmas().
- Documented config keys with examples in function docstring.

* fix(pr): remove remnant local PRAGMAs from PR branch

* test(session): guard config-gated performance PRAGMAs across all connection types

E2E guard for the salvaged PR #71755: database.cache_size/mmap_size/
temp_store from config.yaml must reach the writer connection, the
read-only cross-profile attach, and the WAL per-thread reader — and a
default install (no database: keys) must keep byte-identical SQLite
defaults on every connection type. Also covers integer-coercion
rejection of garbage values for the three new keys.

cache_size uses -16000 (not the doc example -2000) because -2000 is
SQLite's compiled-in default and would not discriminate a regression.

* fix(desktop): flush queued deltas on window focus

* perf(desktop): stop scroll and status loops in busy sessions

* perf(desktop): pause hidden-pane timers in agents view, cron sidebar, and floating pet

Partial pick of the surviving renderer hunks from #75395 (perf commit
6502e441d plus fixup 3fbbc9c1d): gate the 500ms subagent now-ticker and
the cron sidebar 1s ticker/run-poll on usePaneVisible, and skip the
legacy floating-pet poll while the document is hidden. Dropped hunks
(electron/main.ts, vitest.setup.ts/config) intentionally excluded.

* style(desktop): restore alphabetical import order in agents/index.tsx

* refactor(desktop): shared pulse beat + fully-gated cron peek (simplify folds)

Two findings from the simplify pass on the final trio diff:

- status-pulse: one pause controller + one aligned period timer shared by all StatusPulse instances (ref-counted), instead of N x (document/window/bridge listeners + unsynchronized 5s wakes) — a sidebar can show dozens of pulsing dots. Pause still cancels in-flight animations so the compositor sleeps immediately.
- cron-jobs-section: the runs-peek effect created its interval even while the pane was hidden (callback no-oped but the timer still woke the renderer every 8s/60s per expanded job). Early-return when hidden — visibility is already in the dep array, so becoming visible restarts load + timer.

* fix(lint): import sort + eslint-disable for timer-handle ref clear in effect

CI-caught: cron-jobs-section had an extra blank line between sorted imports; use-message-stream's visibility-flush effect assigns flushHandleRef.current=null inside a useEffect (legitimate timer-clear, not an atom mirror) — eslint-disable-next-line per the rule's documented convention.

* fix(state): narrow FTS UPDATE triggers with AFTER UPDATE OF + migration

Retarget #73639 onto the SessionDB mixin split (hermes_state_common /
hermes_state_schema). Fresh installs create UPDATE OF content/tool_*
triggers; existing broad AFTER UPDATE triggers are inspected and
replaced under schema init without an FTS rebuild (WHEN clauses already
guarded content correctness; OF skips non-content status writes that
saturated disk I/O on large state.db).

Tests: tests/test_fts_update_of_narrowing.py (4)

* fix(state): fail closed on CJK trigger migration

* fix(state): quarantine CJK when ensure soft-fails after OF migration

_ensure_fts_cjk_schema never raises on OperationalError; post-condition
after dropping messages_fts_cjk_update now requires a narrowed UPDATE
trigger or durable fts_cjk_stale + unavailable. Covers the production
soft-fail path the raise-only handler missed.

* refactor(state): drop unreachable regex guard in trigger migration

Simplify-pass fold: to_drop names come from the literal update_names\nallowlist via IN binding, so the [A-Za-z0-9_]+ fullmatch could never\nfail — and if it somehow did, its `continue` would miscount (the\nskipped trigger stayed in len(to_drop)/the log while CREATE TRIGGER\nIF NOT EXISTS silently kept the broad variant). Delete the guard and\nits function-local re import; keep the invariant as a comment.

* fix(security): reject always-blocked OpenViking endpoints

## Summary
- Normalize OpenViking endpoints through `is_always_blocked_url` and fall back to the default local endpoint when poisoned.
- Keep intentional loopback / LAN self-host working.
- Add focused unit tests.

## Salvage / credit
Memory-provider endpoint floor sibling of RetainDB/Supermemory always-blocked hardening (avoids over-broad #4984-style private-IP bans).

(cherry picked from commit 8fa607d0aedb8c5fca398d7f112b1b25ade54fa2)

* fix(openviking): fail closed on blocked endpoints

(cherry picked from commit 389a90b81c9c2c89810f2fa7461f8faa9a5c9578)

* fix(openviking): don't spawn a second server onto a live port

`_start_local_openviking_server()` spawned `openviking-server`
unconditionally. Both callers — `initialize()` and the runtime
unreachable handler — reach it from a health probe, and that probe can
time out client-side while the server is up and serving. The spawned
process then loses the data-directory lock and exits immediately with
`DataDirectoryLocked`; because the probe keeps timing out, the cycle
repeats every cooldown window (~5 min observed).

The existing 30s `_failed_refresh` cooldown paces the loop but cannot
stop it, since it expires while the underlying condition persists.

Probe the target host:port before spawning and treat an occupied port as
already-started. This guards both call sites at their single convergence
point. The probe deliberately tests only that a listener owns the port —
enough to know a second server would lose the lock — and says nothing
about that listener's health.

The parse/probe now precedes the PATH lookup, so a reachable server is
reported as running even when `openviking-server` is not on PATH.

Fixes #74846

(cherry picked from commit b49427d85fd6628eb4a7fe099e5c390c5c4cc935)

* fix(openviking): drop stale "disabled for this Hermes run" warnings

The provider used to disable OpenViking permanently when the server was
unreachable. That was fixed: `_ensure_client()` now reconnects lazily,
with a 30s cooldown gate in `_ensure_client_locked`.

Only one of the seven user-facing warnings was updated to match. The
other six still told the user memory was "disabled for this Hermes run",
which is no longer true — every one of those paths is retried on the next
access. A user who reads the old message has no reason to retry, which is
very likely how #5721 ("never recovers") came to be filed against
behaviour that already recovers.

All six sites were traced to confirm none is terminal for the run: the
`initialize()`-time and waiter-thread failures never arm `_failed_refresh`
(only line 2439 does), so they retry on the very next access with no
cooldown at all.

The replacement wording deliberately omits the "(after cooldown)"
parenthetical used at the already-correct site — that detail is only
accurate where `_failed_refresh` was just armed. The neutral phrasing is
true at all six.

Also promotes two clause separators to periods to avoid "…; …disabled;"
collisions.

(cherry picked from commit 8346403a4b97af503d26b0f7905ff513828d821e)

* fix(openviking): re-arm the commit guard after in-place compression

`_committed_session_ids` is a permanent per-sid latch, and
`_session_needs_commit` checks it before the turn counter by design — a
racing sync_turn can re-increment `_turn_count` after commit+reset, so
the guard must win to stop a double-commit.

That is correct for a session being left behind. It is wrong for one
that keeps its id. `compress_context()` commits before rewriting the
transcript in both modes, and with `compression.in_place: true` (the
default) `on_session_switch` receives the same id and does not rotate.
The latch then rejects every later commit for a still-live session — the
next compression, /new, normal session end, startup recovery — so every
post-compression turn is silently never extracted.

Rotation mode is unaffected because a fresh child id is minted and
starts clean, which is what confirms the latch's intent was only ever to
dedupe the departing id.

Clear the latch when compression completes without rotation. Turns
arriving after that point are genuinely new, and this is a defined
moment rather than a race. The rotation path is untouched, so the old
id stays latched and its _finalize_session_async still dedupes against
the compression commit.

Fixes #74695

(cherry picked from commit d1e5c3dc33ef0d43d021662674e1a7cd5e43eecd)

* test(openviking): cover the compression lifecycle, not a hand-set latch

Review feedback: the previous test called _mark_session_committed
directly, so it verified the guard's behavior but not the wiring that
sets it — a future break in the commit_memory_session -> same-id
compression-boundary path would not be caught.

Add a lifecycle regression that drives the real sequence: on_session_end
commits through the actual path, on_session_switch(same id,
reason="compression") crosses the boundary, sync_turn records a genuinely
new turn, and a second on_session_end must produce a second commit POST.

Without the fix it fails showing exactly one commit call, which is the
reported data loss: every turn after the first compression is dropped.
The rotation and /undo tests stay as scope guards.

(cherry picked from commit 0ca5a330630a30b105cbbc32e8a23f2c5ffe0eab)

* fix(memory): read non-secret provider config from config.yaml for OpenViking and RetainDB

OpenViking is_available() only consulted env vars and use_ovcli_config, so an
endpoint saved to config.yaml (e.g. by the Dashboard) reported needs_config;
_resolve_connection_settings() likewise never folded config.yaml's non-secret
fields into its chain. RetainDB initialize() read base_url/project from the
environment only, ignoring the values the Dashboard writes to config.yaml.

Both now resolve non-secret fields as env -> (ovcli ->) config.yaml -> default;
secrets still come from the environment. Adds regression tests for both.

Fixes #68209

(cherry picked from commit dca57915b97b5705b30927a062e1d0f2f23d3841)

* fix(openviking): read recall settings from config.yaml first, env vars as fallback

_recall_config() previously read all settings (recall_limit, score_threshold,
recall_resources, etc.) exclusively from environment variables. This forced
users to store behavioural configuration in .env, violating the Hermes
convention that .env is for secrets only.

The infrastructure to load config.yaml -> memory.openviking was already in
place via _load_hermes_openviking_config(), but _recall_config() never
called it.

Fix: call _load_hermes_openviking_config() and pass its values as the
default parameter to _env_int/_env_float/_env_bool. Env vars still override
config.yaml values, preserving backward compatibility.

Closes #62540

(cherry picked from commit 6aadf1256835745e0302aa3d3b5ae0660b368637)

* test(openviking): cover config.yaml recall settings with temp-HERMES_HOME tests

Add three tests to TestOpenVikingConfigSchema:

1. test_recall_config_reads_from_config_yaml — writes memory.openviking
   settings in config.yaml and verifies _recall_config() consumes them.

2. test_recall_config_env_overrides_config_yaml — writes both config.yaml
   and OPENVIKING_RECALL_* env vars, verifies env takes precedence.

3. test_recall_config_partial_config_yaml — partially populated config.yaml
   falls back to defaults for omitted keys.

All 46 openviking_plugin tests pass (43 existing + 3 new).

(cherry picked from commit b8d7834caf06c6912004333c270faa248eaed4cd)

* fix(openviking): integrate reliability and configuration hardening

* chore(contributors): map OpenViking source authors

* test(retaindb): guard scoped secret config resolution

* fix(openviking): verify servers before sending credentials

* fix(openviking): catch endpoint errors in setup validation functions

Review follow-up for salvaged PR #76782. Three setup-wizard
validation functions called _normalize_openviking_url outside their
try/except blocks. Since _normalize_openviking_url now raises
_OpenVikingEndpointError for blocked or malformed endpoints, an
invalid endpoint would crash the wizard instead of returning a
friendly (False, message) tuple.

- _validate_openviking_auth: move _normalize_openviking_url inside try
- _validate_openviking_root_access: same
- _validate_openviking_setup_values: catch _OpenVikingEndpointError explicitly
- Remove dead ternary in _normalize_openviking_url safety check (candidate
  always has http/https scheme by that point)
- Replace redundant float('-inf') < x < float('inf') with math.isfinite()
  in _setting_float; drop the redundant infinity check from _setting_int
  (is_integer() already rejects inf/nan)

* fix(state): deduplicate session system prompts

* chore: map cicav legacy noreply email

* fix(tui): avoid writable Kanban opens on empty polls

* fix(context): dedupe subdirectory hints by content digest and skip backup/vendor dirs

SubdirectoryHintTracker re-injected identical context files whenever the same
AGENTS.md was reachable through more than one path. Symlinked shared
workspaces, hardlinks, and timestamped backup copies all alias a single file,
so a normal session could ship the same 8KB of instructions two or three
times. Nothing deduped it and nothing excluded directories that only ever
hold copies.

Two changes:

* Track a sha256 of every injected hint body. Repeat content is skipped, and
  the working directory's own context file is seeded at construction so the
  copy prompt_builder already loaded at startup is never sent again.
* Skip directories that hold copies rather than authoritative context
  (backups, node_modules, venv, site-packages, .git, .Trash, vendor, caches).
  Screening is relative to working_dir, so a project that legitimately lives
  under vendor/ keeps discovering its own subdirectory hints.

Measured on a real session that touched a symlinked shared workspace:
3 injections / ~24,000 chars before, 1 injection / 8,112 chars after.

14 new tests cover symlink aliasing, byte-identical copies, working-dir
seeding, distinct content still being injected, each excluded directory name,
excluded ancestors, and the working-dir-inside-excluded-name case.

* perf(state): batch the turn flush into one SQLite transaction

Re-derivation of #23254 (@devsart95) on today's flush loop. The turn
flush in _flush_messages_to_session_db wrote one BEGIN IMMEDIATE
transaction per message row; a typical agent turn (user + assistant +
tool results) paid 3-8 transactions -- and, off WAL (the default on
macOS while the WAL-reset guard is active), 3-8 fsyncs -- per turn.

Adds SessionDB.append_messages_batch: same row shape as append_message
(shared _prepare_message_row serializer + _MESSAGE_INSERT_SQL column
list, so the two writers cannot drift), same compression-lock and
compression-closed guards, one aggregated session-counter UPDATE, one
transaction for the whole batch. Row serialization stays outside the
write lock.

The flush loop now collects the turn's new rows and writes them in one
call. All-or-nothing pairs exactly with the persisted-marker stamping:
on failure no rows landed and no markers were stamped, so the next
flush re-writes the whole tail (same recovery contract as before,
minus the partial-prefix case that could double-count).

Measured (same harness, 5-message turn, journal_mode=DELETE,
synchronous=FULL): 2.32ms -> 0.83ms median per turn flush (64% faster,
5 fsyncs -> 1). On WAL the win is smaller but the atomicity fix holds.

* perf(tui-gateway): batch branch-seed history copies (whole-bug-class)

Sibling sites of the per-message flush pattern: both branch-seed
paths (session.branch in methods_session.py and the lazy seed persist
in server.py) copied the parent history row-by-row -- one transaction
per row, and a branch seed can be hundreds of rows. Route both through
SessionDB.append_messages_batch. The server.py path also gains real
atomicity: _branch_seed_persisted assumed every row landed, which the
per-row loop could not guarantee.

* test(run-agent): update flush-path fakes and assertions for batched writes

The flush now goes through append_messages_batch; MagicMock-based
assertions and barrier fakes that hooked append_message observed
nothing (the flush's try/except swallowed the AttributeError). Assert
on the batch payload instead.

* refactor(state): fold simplify findings — reuse _insert_message_rows, share guards, chunk seeds

Simplify-pass folds on the #23254 salvage:

- REUSE (HIGH): append_messages_batch now delegates row serialization to
  the pre-existing _insert_message_rows helper (already shared by
  replace_messages / archive_and_compact / portability import) instead
  of adding a third serialization path (_prepare_message_row +
  _MESSAGE_INSERT_SQL are gone). One row-writer for every multi-row
  path; the row-ID return was consumed by no production caller, so the
  batch returns the inserted count.

- QUALITY (HIGH): the compression-lock + compression-closed admission
  guards are extracted into _check_transcript_write_guards, shared by
  append_message and append_messages_batch (previously duplicated 23
  lines that had already needed targeted fixes, #74478). The role-gated
  reasoning filtering is no longer duplicated in run_agent.py — it
  lives at its one site inside _insert_message_rows.

- EFFICIENCY (MEDIUM, measured): unbounded seed copies hold one BEGIN
  IMMEDIATE for seconds (10k rows ~= 2.4s; FTS triggers dominate) and
  monopolize the in-process write lock. append_messages_batch grows a
  chunk_rows param; all seed/copy call sites use chunk_rows=500. Same
  recovery semantics as the old per-row loops, bounded lock holds.

- REUSE (MEDIUM): the two remaining per-row branch-copy loops found by
  the pass (gateway/slash_commands.py /branch, hermes_cli
  cli_commands_mixin.py branch) are converted to chunked batches too
  (AsyncSessionDB's generic to_thread forwarder covers the async site).

Turn-flush benchmark unchanged after the refactor: 2.43 -> 0.87 ms
median per 5-message flush (64% faster).

* fix(tests): update two more append_message.call_args assertions to append_messages_batch

CI-caught: test_verification_stop_caching and test_tui_gateway_server::test_native_vision_turn_persists_a_renderable_image_ref both assert on append_message.call_args, but the flush loop now calls append_messages_batch. Same class of test-fake fallout fixed in 5 other files — these two were missed.

* perf(tui): memoize useSessionLifecycle return (idea from #38491)

Re-derivation of #38491 by @stremtec onto current main (the original is
10,119 commits behind; the hook moved into ui-tui/src/app/). The hook
returned a fresh object literal every render, defeating memoization in
useMainApp's consumers; useMemo over the (all-useCallback-stable)
handles makes the return referentially stable.

Dep array covers ALL nine returned handles incl. trimTail (the
re-derivation initially omitted it - stale-closure class).

* ci: retry uv python install

* fix(state): route session-resume reads through the WAL read-only connection

get_messages_as_conversation, get_resume_conversations, and
get_ancestor_display_prefix still took self._lock — the same global
choke point the read-path split (WAL per-thread read-only connections)
was meant to remove from every recall/browse read. These three are the
hottest reads in the file: every session resume across the gateway,
CLI, and ACP adapter goes through one of them, so a resume racing a
burst of concurrent-session writer flushes still convoys behind them
exactly like the fixed paths used to.

_session_lineage_root_to_tip (the lineage walk shared by all three,
plus get_conversation_root) had its own independent self._lock use and
needed the same conversion — without it the outer functions still
blocked on the very first line.

Verified empirically: a reader thread calling all three functions
while another thread holds self._lock blocked for the writer's full
hold duration before the fix, and returned immediately after (SQLite
3.50.4 in this dev venv falls back to journal_mode=DELETE per the
WAL-reset-bug guard, so the requires_wal-marked regression test is
exercised via a local WAL-forced script instead; it still runs and
passes on any runtime where WAL is actually active).

* chore: add contributor email mapping for ArcherQAQ

* fix(model_metadata): rewrite localhost->IPv4 for the remaining local probe sites

fetch_endpoint_model_metadata's generic (non-LM-Studio) /models fetch and
its llama.cpp /v1/props context-length follow-up built request URLs
straight from the unrewritten candidate, unlike every other local-probe
site. Both retained the multi-second dual-stack IPv6 connect penalty
that _localhost_to_ipv4() exists to skip (measured on macOS: localhost
32.9ms vs 127.0.0.1 0.1ms on a dead port; ~2s on Windows). normalized
stays the cache key so caching behavior is unchanged; only the outbound
request target is rewritten.

Re-derived from PR #61528 onto current main (original no longer applied
cleanly).

* fix(model_metadata): guard _localhost_to_ipv4 against non-string urls

CI slice 3/7 failures: run_conversation tests pass MagicMock base_urls
through the metadata probe path; re.sub raised TypeError where the old
code let non-strings flow through. Preserve that contract.

* perf(cold-start): mitigate ~14s GIL stall during backend init (#60800)

Three fixes for the Desktop/TUI cold-start stall where the event loop
is blocked for ~14s between HERMES_BACKEND_READY and the first
prompt (#60800):

1. copilot_auth: skip  subprocess fallback when any
   Copilot env var is explicitly set (even if invalid). The user
   expressed token intent via env var; silently substituting a CLI
   token is surprising and the subprocess adds up to 5s on Windows.

2. tui_gateway/ws: run resolve_skin() via asyncio.to_thread so config
   loading + skin engine init do not block the WS read loop during
   the cold-start RPC burst.

3. web_server: extend _warm_gateway_module to pre-import the heavy
   module chains (auth, copilot_auth, runtime_provider, skin_engine,
   inventory, model_switch) that the first WS connection + RPC burst
   would otherwise import on the loop thread. These trigger .pyc
   compilation and Defender scans on Windows (15-30s per the existing
   comment) and were not covered by the original gateway-only warm.

Tests: 5 new tests in test_cold_start_gil_stall.py + 2 new tests in
test_copilot_auth.py. All 36 copilot_auth tests + 16 ws/web_server
tests pass.

* test: harden cold-start regression tests + debug-log the env-var skip

Review folds on the #60807 salvage:
- resolve_skin tests are behavioral (thread-ident probe + ready-frame
  wiring check) instead of pure source inspection, per the #72720
  pattern; a source assertion remains as belt-and-braces.
- The warm-list test does REAL imports and checks sys.modules —
  _warm_gateway_module swallows ImportError by design, so the PR's
  tracking-stub test would pass even with a typo'd module name.
- resolve_copilot_token logs a debug line when the env-var
  short-circuit skips the gh-CLI fallback (behavioral change made
  observable).

* perf(gateway): per-platform skip_context_files to cut agent build latency

Salvage of #26860 (hunk 2, ported \u2014 the PR's base predates the current
gateway layout by ~11.9K commits). Messaging platforms can set
gateway.platforms.<key>.skip_context_files: true to skip the
filesystem-heavy context-file discovery (SOUL.md, AGENTS.md,
.cursorrules walks) during AIAgent construction \u2014 10-100x slower
stat()/walk costs on Windows made this a real per-turn tax. Soul
identity is still loaded (single small file), so the persona survives.

The flag participates in _agent_config_signature so toggling it
rebuilds the cached agent instead of silently reusing a prompt built
under the other setting (prompt-cache correctness).

The PR's hunk 1 (mtime-caching the per-turn dotenv reload) was dropped:
df51ad797 mtime-cached load_config/read_raw_config and c2eda92fd
removed the per-turn deepcopies, capturing most of that win; the
function has since gained a multiplex early-return and managed-scope
overlay that the original whole-function skip would have bypassed.

* fix(relay): route Discord tool-progress into the auto-thread, not the parent channel (#77830)

When a Discord channel message initiates a relay auto-thread, the thread does
not exist at ingest (source.thread_id is None) — the connector creates it on
its FIRST send and auto-threads any outbound carrying the reply anchor. The
final reply carries that anchor, so it lands in the thread. But the
tool-progress / status bubbles (the "Searching the web for..." updates and the
streaming preamble) were sent with _progress_metadata=None and
_progress_reply_to=None: _resolve_progress_thread_id returns None for Discord
(only slack/mattermost get a synthetic thread), so the progress send had no
anchor and the connector posted it FLAT in the parent channel. Result: the
search-status updates leaked outside the thread while the answer threaded
(staging repro 2026-08-02).

The connector now stamps prospective_thread_id on the inbound (the anchor
message id == the id of the thread it will create). Reuse it: when a
relay-delivered Discord channel-initiate carries prospective_thread_id and has
no real thread yet, carry the reply anchor (event_message_id) on both the
progress metadata (reply_to_message_id) and the progress reply_to, so the
connector routes the progress bubble into the SAME auto-thread as the final
reply. Applied to both the tool-progress path (_progress_metadata /
_progress_reply_to) and the status/interim callback path
(_status_thread_metadata). Events already arriving in a real thread, DMs, and
non-relay sources are untouched (guarded on delivered_via_upstream_relay +
prospective_thread_id + not thread_id).

Tests: two new cases in test_run_progress_topics.py — a relay Discord
channel-initiate asserts every progress send carries the anchor (reply_to +
metadata.reply_to_message_id + non_conversational), and an event already in a
real thread asserts the synthetic-anchor path does NOT engage. Full gateway
progress + relay + session suites green (228 passed).

* fix(agent): stop re-probing endpoints that blackhole TCP connects

Salvage of #71282 (Fixes #71281): a routable-but-dead endpoint (corp
LAN address while off-VPN) blackholes TCP SYNs, so every probe in the
model-metadata waterfall waits out its full connect timeout — 20+
seconds of stall per startup across detect_local_server_type,
fetch_endpoint_model_metadata, and the per-model probes.

A module-level blackhole cache keyed on host:port is populated when
any probe observes a ConnectTimeout (httpx or requests; read timeouts
deliberately excluded — an accepted connection is not a blackhole) and
consulted at the top of each guarded function. 30s TTL: long enough to
collapse one startup burst, short enough that VPN recovery is picked
up without a restart. Guard ordering: blackhole check -> disk L2 ->
HTTP waterfall, and a blackholed leg aborts the remaining legs instead
of letting each stall in turn.

Squash of the PR's two real commits (the branch's merge commits made
it un-rebase-merge-able; content verified identical via merge-tree).

* chore: release v0.20.0 (2026.8.3)

The Herald Release — voice (streaming TTS, barge-in, wake words), A2A v1.0,
outbound webhooks, grounded citations, desktop platform wave. ~3,650 commits,
~1,400 PRs, ~1,200 issues closed, 650+ contributors since v0.19.0.

Also: contributor audit additions (18 email mappings, bot-filter widening).

* chore: add contributor email mapping for Ahmett101

* perf(moa): cache resolved preset + per-slot runtime to cut cold-start latency (#66793)

* fix(discord): leave voice channels before cancelling the bot task

`DiscordAdapter.disconnect()` cancelled the bot task before tearing down voice
clients. `leave_voice_channel()` ends in `await vc.disconnect()`, and discord.py
sends a voice state update over the main gateway websocket and then waits for the
voice socket to close. The bot task is the loop running that gateway connection,
so cancelling it first left the handshake with no transport: it could never
complete and blocked until the caller's shutdown timeout fired.

The effect was a fixed ~5s penalty on every shutdown with a voice connection
open, ending in "discord disconnect timed out after 5.0s - forcing continue",
with the voice disconnect abandoned rather than completed.

Measured on a live gateway with a voice connection open in both cases:

  before: timed out after 5.0s, all adapters disconnected at +5.29s
  after:  discord disconnected (0.12s), all adapters disconnected at +0.46s

Moving the voice-cleanup loop above `_cancel_bot_task()` preserves the
zombie-client protection its comment describes: the bot task is still cancelled
before `client.close()`, just after voice teardown rather than before it. Voice
teardown is the one step that still requires a live gateway.

Adds a regression test asserting the ordering. It fails on the previous ordering
at index 1 with `cancel_bot_task != leave_voice_channel:111`.

Fixes #76044

* feat(image): parallelize image_generate batches

* fix(file-sync): serialize concurrent sync cycles

* fix(tool-executor): unpack 5-tuple runnable_calls in _max_workers_for_tool_batch

* fix: exponential backoff for rate-limit fallback cooldown

Replace the fixed 60-second cooldown with exponential backoff:
30min → 1h → 2h → 4h cap.

The counter is reset by restore_primary_runtime on successful
primary-provider recovery, so the backoff is strictly for
consecutive failures within a single degradation window.

Closes #29702

* fix(backoff): keep 60s first-hit cooldown, escalate only on consecutive rate-limits

Review follow-up on the #30223 salvage: the original changed the base
cooldown from 60s to 1800s, benching the primary for 30 minutes on the
FIRST 429 (30x regression in primary-restore latency) and breaking the
existing test_rate_limit_exhaustion_keeps_60s_cooldown contract.

Keep upstream's 60s base and escalate per consecutive rate-limit:
60s -> 2m -> 4m -> 8m -> ... capped at 4h. Counter still resets on
successful primary restore (cicae's mechanism, unchanged).

New tests: escalation doubling, 14400s cap, reset-on-restore.
Existing 60s contract test passes UNCHANGED. Mutation-checked:
escalation disabled -> 2 fail; reset disabled -> 1 fails.

* fix(catalog): wire api_key auth headers for http MCP servers

When an optional-mcps manifest declares transport.type=http with
auth.type=api_key, install_entry() prompts for the key and saves it to
.env, but _build_server_config() only handled the oauth case — the
api_key case produced a bare url entry with no headers, so every
request to the server was unauthenticated (-> 401).

Reuse _bearer_auth_headers(entry.name) from mcp_config.py so the
catalog path emits the same 'Authorization: Bearer ${MCP_..._API_KEY}'
template as the manual 'hermes mcp add --url' path.

Salvaged from #70782 (production hunk applied clean; tests re-anchored
onto current main). Credit: JonthanaHanh.

* perf(compressor): release allocator pages after successful compaction

A successful compaction frees the largest allocation a long session ever
drops (the compressed-away message dicts), but Python's arena allocator
keeps those pages in the heap — RSS retains the pre-compaction
high-water mark until exit. #76905's trim_memory lifecycle covers the
gateway/TUI housekeeping loops but not the CLI compression path.

Call trim_memory(reason='post-compression') at the compression-success
point in ContextCompressor.compress(), following the house pattern
(lazy import in try, debug-level log on failure). The helper is
glibc-gated, config-gated and rate-limited, so it is a safe no-op on
other platforms and cannot fail compression.

Re-expresses the intent of #70782 (JonthanaHanh), which reached for a
bare gc.collect(); trim_memory is the house mechanism and already
wraps a collect.

* fix(catalog): validate http+api_key manifests declare the header's env key

Simplify-pass follow-up on the #70782 salvage: _bearer_auth_headers
hard-emits ${MCP_<NAME>_API_KEY} but install_entry only persists
auth.env-declared vars — a manifest naming its key differently (the
shipped n8n style) would install cleanly yet send a literal-placeholder
header at connect time (silent 401, the #37792 bug class). Enforce the
naming contract at parse time. Also pins the secret-stays-in-.env
property in the install test (raw config.yaml carries the template,
never the secret). Mutation-checked: validation disabled -> guard test
fails.

* fix(agent): cap auxiliary LLM concurrency per task

* fix: thread extra_headers through the call_llm split

The PR's concurrency wrapper splits call_llm into a semaphore-guarded
entry + _call_llm_impl; main added extra_headers to call_llm's
signature after the PR's base, so the split has to forward it too
(dropped silently otherwise — Azure Foundry and custom-endpoint
callers set it).

* fix(nix): tie devShell's HERMES_PYTHON to the venv actually on PATH

`nix/devShell.nix` collected `devShellHook` by scanning every package:

    nonNpmHooks = map (p: p.passthru.devShellHook or "") packages;

But `minimal` and `messaging` are `.override` variants of `default`, so
each carries its own `devShellHook` exporting its own HERMES_PYTHON. The
scan therefore concatenated three conflicting exports and forced Nix to
evaluate and realise three separate uv2nix editable venvs on every
`nix develop`.

`attrValues` is alphabetical, so the last hook won (`minimal`) while
`python`/VIRTUAL_ENV came from `default`'s devDeps:

    HERMES_PYTHON        = ...dimim2... (minimal — no optional deps)
    python / VIRTUAL_ENV = ...85r28... (full)

Inside the shell `$HERMES_PYTHON -c "import anthropic"` failed while
`python -c "import anthropic"` succeeded. Worse, `scripts/run_tests.sh`
prefers HERMES_PYTHON, so the suite ran against the minimal venv. Its
guard did not catch this: it only checks that HERMES_PYTHON has pytest,
and minimal's venv does (pytest is in the `dev` group), so the wrong
interpreter was silently accepted.

Tying the hook to `packages.default` — the same package whose `devDeps`
are installed — keeps HERMES_PYTHON, `python`, and VIRTUAL_ENV pointing
at one venv by construction.

    editable venvs referenced   3      -> 1
    their combined closure      421 MB -> 140 MB
    test failures               85     -> 32

The venv mismatch was masking 53 failures; e.g. test_web_tools_config.py
goes 2-failed -> 38-passed. Full suite is now 25369 passed / 32 failed,
and those 32 reproduce identically on a pristine HEAD worktree with no
nix/ changes under the same interpreter (mostly NixOS artifacts — tests
spawning bare `python3` in a scrubbed env exit 127).

* fix(system_prompt): move skills index to the volatile band

The skills index is runtime-mutable: the agent adds and patches skills mid-session, so it is not byte-stable. Keeping it in the stable band breaks that band prefix-cache contract, because every skill edit changes the stable band and invalidates the entire cached prefix in front of it. Move it to the front of the volatile band so the stable scaffold (identity, tool guidance, model guidance) stays cacheable across skill edits.

* docs(system_prompt): fix stale reconstruct_static_prefix docstring example

Simplify-pass finding: the safety note still cited 'skills edited' as a stable-tier input whose change mismatches the rebuilt prefix — after this PR a skill edit changes only the volatile tail (that's the point). Swap the example for genuinely stable-tier inputs.

* fix(prompt_size): search volatile tier for skills block after the stable->volatile move

CI-caught: compute_prompt_breakdown still looked for <available_skills> in the stable tier, but #37117 moved it to volatile. Search volatile first, fall back to stable for older sessions.

* chore: add contributor email mapping for zabih-sudo

* chore: add contributor email mapping for HAOWANG116

* fix(backup): serialize and atomically publish snapshots

* chore: add contributor email mapping for ElSnacko

* fix: prefer explicit anthropic api key

Cherry-picked from PR #58560 by @itsflownium, adapted to current main
(_getenv instead of os.getenv). Moves ANTHROPIC_API_KEY check ahead of
Claude Code credential file and credential_pool auto-discovery so an
explicitly configured key is never shadowed by auto-discovered OAuth.

Fixes #58546

* docs: document /personality none|default|neutral reset across personality docs

The reset keywords have existed in both CLI and gateway handlers since
June but were undocumented — users couldn't find how to cancel a
personality overlay. Adds a 'Resetting to the default' section to the
personality feature page and mentions the reset in the CLI guide,
slash-command reference (both tables), and messaging command table.

* fix(relay): avoid concurrent turn scope corruption

Signed-off-by: Bryan Bednarski <bbednarski@nvidia.com>

* fix(relay): preserve legacy turn shims

Signed-off-by: Bryan Bednarski <bbednarski@nvidia.com>

* fix(relay): gate skipped turn metrics

Signed-off-by: Bryan Bednarski <bbednarski@nvidia.com>

* test(relay): enforce LIFO in overlap regression

Signed-off-by: Bryan Bednarski <bbednarski@nvidia.com>

* fix(relay): preserve skipped turn context

Signed-off-by: Bryan Bednarski <bbednarski@nvidia.com>

* fix comment about relay workaround

* feat(models): add qwen3.8-max to Nous portal + OpenRouter catalogs, replacing qwen3.7-max

Qwen3.8 Max is live on both OpenRouter and the Nous portal
(qwen/qwen3.8-max, 1M context, 131K max output). Per the
newest-max-replaces-last-max convention, it takes qwen3.7-max's slot
in both curated lists.

- hermes_cli/models.py: OPENROUTER_MODELS + _PROVIDER_MODELS[nous]
  swap qwen/qwen3.7-max -> qwen/qwen3.8-max
- agent/model_metadata.py: DEFAULT_CONTEXT_LENGTHS entry for
  qwen3.8-max at 1,000,000 (verified against OpenRouter live
  metadata and Nous /v1/models 2026-08-03)
- tests/test_empty_model_fallback.py: swap incidental catalog fixture
  to the surviving slug
- website/static/api/model-catalog.json: regenerated

Pricing snapshot skipped: both routes bill via official_models_api
(live pricing), verified with resolve_billing_route. Reasoning
timeout floor already covered by the qwen3 prefix (180s).

* test: swap context-switch-guard fixture off qwen3.8-max-preview

test_custom_provider_context_avoids_false_shrink_warning used
qwen3.8-max-preview as a slug that deliberately falls through to the
generic 'qwen' 131K catalog match. The new qwen3.8-max
DEFAULT_CONTEXT_LENGTHS entry (1M) now substring-matches the preview
slug too, so the no-custom-providers branch stopped warning. Swap the
fixture to qwen3.9-max-preview, which still hits the generic fallback
— the test's intent (custom_providers threading) is unchanged.

* fix(agent): keep context_length pin for named custom providers

Empty model.base_url plus a runtime custom-provider URL was treated as a
route mismatch, so gateway session-reset banners dropped model.context_length
and fell back to the Qwen family default (131K) while /status still showed
the configured 262K pin.

* test(gateway): cover named-custom context pin on session-info banner

* fix(model_metadata): read llama.cpp context from meta.n_ctx + accept sole model

* chore: add contributor email mapping for johnrazmus

* fix(gateway): keep event loop alive during /compress and Relay drain

Offload manual /compress temporary-agent cleanup through the existing
bounded off-loop helper so a slow agent.close() cannot freeze the
gateway event loop, heartbeat, or platform polling.

Guarantee Relay transport teardown even when the runner cancels
adapter.disconnect() during go_idle: shielded finally, 2s drain-path
idle ACK budget under the 5s outer disconnect budget, and bounded
supervisor/reader/ws.close awaits.

Original commits:
- fix(gateway): offload manual /compress cleanup from the event loop
- fix(gateway): tear down Relay transport even if go_idle is cancelled
- fix(gateway): keep Relay disconnect budgets inside the runner window

By @Dannyzen (PR #78027), salvaged onto current main.

* fix(gateway): bound go_dormant ws.close with teardown timeout

Sibling site to the disconnect() fix: go_dormant() still did an
unbounded await self._ws.close(), the exact same pattern bounded in
disconnect(). go_dormant runs on the scale-to-zero suspend path (Fly
autostop), which also has timeout constraints. Apply the same 1s
wait_for treatment using _TEARDOWN_AWAIT_TIMEOUT_S.

Found during review of PR #78027.

* fix: close the Codex app-server session on agent teardown

Salvage of #65260's b7d7cfd0e (ported — the PR's close() predates ~4K
commits of teardown-step churn, so the hunk is re-anchored after step
6b rather than cherry-picked).

agent/codex_runtime.py already drops _codex_session on turn crash and
on retirement, but AIAgent.close() — the hard teardown for /new,
/reset, and session expiry — had no owner for it, so the app-server
child process survived until interpreter exit. Long-lived gateways
accumulate one leaked subprocess per ended Codex session.

The attribute is cleared BEFORE close() so a concurrent reader can't
observe a half-closed session and a raising close() can't strand a
stale reference (tested).

Tests extend the author's original lifecycle test with the
raising-close and no-codex-session cases.

* fix(agent): discard bare tool-call marker before fallback/persistence (#78148)

Local tool-call templates can emit a bare bracketed token (e.g. "[memory]")
as assistant content alongside a function call. The loop treated that
protocol scaffolding as visible content: it got cached as the post-tool
fallback, and when the next turn came back empty, the marker was replayed
as the final response and written into the persisted transcript. Later
context compaction preserved that history, letting the model repeat the
marker in subsequent turns.

Detect content that is only a bracketed marker (`[name]`) when the
response also carries tool_calls, and drop it before it can be cached
or persisted. Scoped narrowly: only fires alongside tool_calls, so a
genuine final response of "[memory]" without a tool call is unaffected.

* fix(agent): repair sessions already contaminated with stale tool-call markers (#78148)

The conversation_loop fix (previous commit) stops new "[memory]"-style
bare tool-call markers from being cached/persisted, but sessions written
before that fix can still carry rows where a bare marker was saved as
the assistant's "final response".

Add a load-on-read repair pass in hermes_state.py, mirroring the existing
_strip_background_review_harness defense-in-depth: on session restore,
any assistant row whose content is only a bracketed marker (e.g.
"[memory]", "[skill_manage]") AND that carries tool_calls has its content
blanked before the history re-enters the model's context. The tool call
and its result are left untouched so provider tool_call/tool_result
pairing stays intact. Sessions with no affected rows pass through the
normal path unchanged.

* feat(cli): add sessions clean-markers to permanently purge stale tool-call markers (#78148)

The load-on-read repair (_strip_stale_tool_call_markers) fixes affected
sessions in memory on every resume, but never touches the DB — long-lived
sessions re-scan and re-repair the same rows on every load, and the
contaminated bytes stay in state.db (and any backup/cache snapshot of it)
indefinitely.

Add SessionDB.purge_stale_tool_call_markers(dry_run=False): a one-time,
idempotent UPDATE that permanently blanks the content column on affected
rows. Only content is touched — tool_calls and every other column are
left untouched, so provider tool_call/tool_result pairing survives.
dry_run reads through the no-lock read path and never writes.

Wire it up as `hermes sessions clean-markers [--dry-run]`, mirroring the
existing optimize/repair subcommands. Verified end-to-end against a real
temp state.db: dry-run reports the row without writing, the real run
clears it and preserves tool_calls, and a second run is a no-op.

* fix(cli): back up state.db before clean-markers writes by default

purge_stale_tool_call_markers ran a permanent, irreversible UPDATE with
no backup — inconsistent with repair_state_db_schema's backup-by-default
convention for destructive state.db operations elsewhere in this file.

Take a full snapshot via VACUUM INTO (safe against a live connection,
unlike the raw-copy _backup_db_file used for malformed-schema repair)
before the write, timestamped beside state.db. Skipped when dry_run or
when there's nothing to change. Add --no-backup to `hermes sessions
clean-markers`, mirroring `sessions repair`.

Verified end-to-end: the CLI run against a real temp state.db produces
the backup file before printing the cleared-row count.

* refactor: dedup stale-marker regex — use compiled _STALE_MARKER_RE in conversation_loop

The bracketed-marker regex was inlined in conversation_loop.py as
re.fullmatch(r"\[...", ...) while hermes_state.py defines the same
pattern as _STALE_TOOL_CALL_MARKER_RE. Both must agree on what counts
as a stale marker — a drift here means the runtime guard silently
disagrees with the load-on-read repair and CLI purge in hermes_state.

Consolidate onto a single compiled constant (_STALE_MARKER_RE) at
module level in conversation_loop.py, with a comment noting it must
mirror _STALE_TOOL_CALL_MARKER_RE in hermes_state.py. A direct import
from hermes_state was tried first but caused a regression: hermes_state
initializes DEFAULT_DB_PATH = get_hermes_home() / 'state.db' at module
import time, which breaks tests that monkeypatch get_hermes_home() to
return a str (test_slash_worker_accepts_profile_home).

Follow-up to PR #78175 (@JoaoMarcos44).

* fix(conversation_loop): compress messages on output-cap retry path (#55546)

The output-cap retry loop reduced max_tokens by 64 tokens per attempt but
never called _compress_context(), so the compressor never fired. Input
growth (~65 tokens/attempt) canceled the savings, leaving the session
stuck at 200,001 tokens — 1 over the 200,000 ceiling.

The fix adds compression to the output-cap retry path. The compressor
drops the middle window, freeing ~50% of tokens. If compression makes
>=5% savings, the session continues; otherwise vision payloads are
stripped or the session ends with compression_exhausted=True.

Also adds CHANGELOG.md entry and bug fix report.

* fix(conversation_loop): prune dead vision-strip fallback; harden output-cap retry tests

* chore: drop CHANGELOG.md and docs/reports/ — not shipped with salvage PRs

* perf: reuse request_input_estimate instead of recomputing estimate_request_tokens_rough

The output-cap error handler already computes request_input_estimate at
line 4722 via estimate_request_tokens_rough(api_messages, tools=...).
The new compression block ~50 lines below was calling the same function
with the same inputs again. Reuse the existing local.

* chore: AUTHOR_MAP — add BobClawblaw for PR #77870 salvage

Bare noreply email (no NNN+ prefix) needs explicit mapping.

* chore(ci): rerun checks

Signed-off-by: Bryan Bednarski <bbednarski@nvidia.com>

* perf(gateway): prewarm /model picker cache on TUI startup

The classic CLI run() loop calls prewarm_picker_cache_async() during the
idle window after the banner is shown, so the first /model open hits a warm
provider-models disk cache and renders in ~100ms. The stdio TUI entry point
never did this, so the first /model open in a TUI session blocked on serial
/v1/models fetches for every authenticated provider.

Mirror the CLI behaviour: kick off the same off-thread prewarm right after
gateway.ready is emitted (banner shown, user about to type). Fire-and-forget,
guarded once-per-process, fully exception-isolated so a slow or offline
provider can never affect TUI startup.

* test(tui): pin picker-cache prewarm wiring in entry.main()

teknium's review gap on #72021: the helper's worker/once-guard was
covered, but nothing asserted the stdio TUI entry point actually
invokes prewarm_picker_cache_async() — or that it does so in the right
place. Add a focused entrypoint test that runs the real entry.main()
with stubbed collaborators (same monkeypatch-module-attrs harness as
test_tui_entry_mcp_owner.py), spies on the helper in
hermes_cli.model_switch (the lazy-import source), and asserts:

- prewarm fires exactly once, strictly AFTER the gateway.ready write
- startup stays non-blocking: main() reaches the stdin loop and
  returns on EOF
- a prewarm failure is swallowed (fire-and-forget) without breaking
  startup

Mutation-checked: deleting the prewarm hunk from entry.py fails both
tests.

* perf(cli): check local auth.json/config before slow provider registry sweep

_has_any_provider_configured() probed every api_key provider (gh subprocess
for copilot alone takes 5s; full sweep ~18s) before consulting auth.json and
config.yaml, which are instant local reads. Desktop setup.status calls
blocked past the UI's timeout, causing the connect/disconnect boot loop.
Reorder so cheap local checks run first. Same semantics, ~35x faster here.

* test(cli): regression tests pinning auth-first ordering skips registry sweep

Teknium's review on #63457: existing tests pin the final boolean but not
that the slow PROVIDER_REGISTRY sweep is skipped. Add three tests that
booby-trap hermes_cli.auth.get_auth_status and verify
_has_any_provider_configured() short-circuits on:
- config.yaml model.provider
- config.yaml base_url/api_key (custom endpoint shape)
- auth.json active_provider (sweep-only call-pattern guard)

Mutation-checked: reverting the reorder makes all three fail.

* perf(desktop): keep spinner frames out of React commits

Advance the existing animated status glyph through its DOM text node instead of React state, and pause its timer for hidden panes or inactive windows. Cover frame advancement, zero update-phase commits, and timer suspension with behavior tests.

* test(desktop): cover minimized/hidden window-state + visibilitychange pause for GlyphSpinner

Regression coverage requested in review of #74357: mock
window.hermesDesktop.onWindowStateChanged (pattern from
persistent.test.tsx) and assert minimized/hidden clears the spinner
interval while restore resumes it; also cover document.visibilityState
hidden/visible via visibilitychange.

* fmt(js): `npm run fix` on merge (#78271)

Co-authored-by: github-actions[bot] <github-actions[bot]@users.noreply.github.com>

* perf(gateway): replace SSE poll loop with call_soon_threadsafe-fed asyncio.Queue

_write_sse_chat_completion and _write_sse_responses bridged their
stream_delta_callback queue into the event loop via
`await loop.run_in_executor(None, lambda: stream_q.get(timeout=0.5))`
in a while-True poll — a thread-pool round trip on every 0.5s tick even
when idle, plus up to 500ms of tail latency between a delta landing in
the queue and it reaching the SSE response.

Add ThreadSafeAsyncQueue (asyncio.Queue + a put_threadsafe() that wraps
call_soon_threadsafe), used by both streaming producer closures
(_on_delta, tool start/complete callbacks — all invoked from the worker
thread running run_conversation via loop.run_in_executor). Consumers
now do a plain `await asyncio.wait_for(stream_q.get(), timeout=0.5)` —
woken immediately when a delta arrives, no executor hop, no poll
interval.

Updated tests/gateway/test_sse_agent_cancel.py's 7 call sites to
construct ThreadSafeAsyncQueue inside the running loop (required, since
it captures asyncio.get_running_loop() at construction) instead of a
bare queue.Queue() at test-method scope.

* refactor(gateway): extract _sse_frame() helper, dedup 5 inline SSE encode call sites

_write_sse_chat_completion had five near-identical
f"data: {json.dumps(...)}\n\n".encode() (and one event-tagged variant)
scattered across its role/content/finish/error chunk writes. Pure
extract-method, no behavior change: encoding is byte-identical for every
call site touched.

Left the pre-serialized-string writers elsewhere (_write_event's
json.dumps(..., ensure_ascii=False) path, the /v1/runs SSE writer) alone
— routing them through this helper's plain json.dumps(data) would
silently change their unicode-escaping behavior, which is out of scope
for a pure dedup.

* refactor(gateway): route all three SSE writers through _sse_frame()

_extend _sse_frame with an explicit ensure_ascii param (default True,
byte-identical to a bare json.dumps) and route the two sibling writers
through it: _write_sse_responses._write_event and the /v1/runs event
stream. This completes the dedup PR #65009 — previously only the five
_write_sse_chat_completion sites used the helper, leaving the other two
writers on inline json.dumps with no shared shape.

No behavior change: every writer's emitted bytes are unchanged (verified
byte-for-byte, including non-ASCII payloads where the default
ensure_ascii=True matches the original inline encoders). The ensure_ascii
option is exposed so a future writer can opt into raw non-ASCII bytes
without fractalizing the format again.

Adds tests/gateway/test_sse_frame.py asserting the byte-contract
invariant between _sse_frame and the historical inline encoders.

* refactor(gateway): route session event stream through _sse_frame (ensure_ascii=False)

The session event stream (api_server.py:~2236) was the one genuinely
unicode-distinct SSE writer — json.dumps(payload, ensure_ascii=False) +
.encode('utf-8'). Every other writer uses plain json.dumps. Route it
through _sse_frame(..., ensure_ascii=False) so _sse_frame is now the single
source of truth for ALL SSE frame serialization in the module (chat-
completion, responses._write_event, /v1/runs, and the session stream).

Byte-identical for non-ASCII payloads: verified against the historical
inline encoder (raw bytes preserved). The ensure_ascii=False path is now
exercised by test_sse_frame_ensure_ascii_false_reproduces_session_event_stream.

* test: add cross-thread put_threadsafe + long-reasoning tail tests

Addresses teknium1 sweeper review (2026-07-30) requiring coverage of:

1. ThreadSafeAsyncQueue.put_threadsafe() off-loop boundary: a real
   daemon thread pushes into the queue from outside the owning event loop
   while the consumer awaits get(), mirroring the run_conversation
   worker-thread producer path. Includes a 20-concurrent-thread
   no-drop regression.

2. Long-reasoning bound stability for thinkingPreview: 100k-char input
   plus empty/collapsed cases must not crash and must retain the visible
   tail marker inside the bounded 24k clean window.

* fix: reconstruct fused test after conflict resolution

The conflict-marker strip fused test_agent_task_raises with the body of
test_failed_result_dict — restore both as separate tests (content from
the PR head, verified verbatim).

* perf(tui): bound reasoning-clean input to the displayed tail

CI caught a split defect in this salvage: the PR's long-reasoning tail
test was kept but its production hunk was dropped as 'cosmetics'. It
isn't — cleanThinkingText runs several full-string regex passes and
reasoning grows on every streamed token, so re-cleaning the whole
accumulated string per chunk is O(n) per token / O(n^2) per stream.
Only the tail is displayed (boundedLiveRenderText caps it downstream),
so bound the input to 1.5x LIVE_RENDER_MAX_CHARS first.

Restores the one text.ts hunk from cd99e65fc (author preserved); the
italic-thinking display change and profile script from that commit
remain out of scope.

* test: exercise the production _loop_ref path in put_threadsafe tests

Gate finding (/simplify-code pass): both cross-thread tests passed
loop=loop explicitly, but no production caller does — all six
(_on_delta, _on_tool_*) rely on the queue resolving its own
_loop_ref in __init__. The kwarg made the tests vacuous: a broken
_loop_ref still passed them.

Dropping the kwarg exercises the real path. Verified by mutation:
with self._loop_ref = asyncio.new_event_loop() (wrong loop), both
tests now FAIL; they passed before this change.

* fix(test): feed the SSE writers an asyncio queue, not queue.Queue

CI caught a missed caller-shape update. Both PRODUCTION callers of
_write_sse_chat_completion / _write_sse_responses were converted to
ThreadSafeAsyncQueue, but two pre-existing tests in
tests/gateway/test_api_server.py construct the writer's queue
themselves and still passed a stdlib queue.Queue.

The consumer now does 'await asyncio.wait_for(stream_q.get(), ...)',
which on a queue.Queue blocks the thread forever:
test_stream_cancelled_persists_incomplete_snapshot hung until
pytest-timeout killed it (CI reported the whole file as 'no tests
ran (timeout before collection)'). The sibling disconnect test only
survived because it pre-fills before the first await.

tests/gateway/test_api_server.py: 99 passed (was 1 failed + a 60s
hang); with the SSE/api_server suites: 147 passed.

* fix(credential-pool): re-select in acquire_lease after a deferred refresh

select() re-selects once deferred single-use-token refreshes complete;
acquire_lease() performed the refresh but returned its pre-refresh
answer. Since _acquire_lease_under_lock returns early exactly when a
refresh is pending (if not available: return None, pending_refresh),
a pool whose entries all needed a refresh always returned None — the
caller failed an answerable request right after the refresh succeeded.

Retry once, only when the first pass was empty and a refresh ran.

Post-merge gate-sweep finding on the #71775 salvage (#77714).

* fix(credential-pool): lock the quarantine read-modify-write of _entries

#71775 moved deferred single-use-token refreshes outside the pool lock
(correct — they hold a cross-process flock plus network I/O). But
_refresh_entry_impl's three terminal-auth-failure quarantine paths do a
bare read-modify-write of self._entries. Those used to run with the
caller holding self._lock; on the deferred path they run unlocked, so a
concurrent mutation between the read and the write is silently lost.

Wrap all three in 'with self._lock' (an RLock, so locked callers
re-enter safely) and correct the _refresh_pending_entries docstring,
which claimed the mutations were already self-locking.

Post-merge gate-sweep finding on the #71775 salvage (#77714).
Sibling to the acquire_lease re-select fix.

* perf(file-ops): eliminate redundant subprocess calls in write_file and V4A patch path

write_file currently spawns up to 6 subprocesses per call:
  1. mkdir -p (separate call before atomic write)
  2. cat (to read pre-content for lint/BOM/line-ending detection)
  3. _atomic_write (mktemp + write + mv — the essential one)
  4. wc -c (to measure bytes written)
  5. _check_lint_delta (post-write lint — also essential)
  6. LSP snapshot (also essential)

This PR removes three of them without changing any observable behavior:

1. Fold mkdir -p into _atomic_write shell script (−1 subprocess/write)
   The atomic write script already runs a single shell; adding mkdir -p
   to it costs zero extra processes.

2. Add optional pre_content parameter to write_file (−1 subprocess/patch)
   patch_replace and V4A _apply_update already read the file for fuzzy
   matching. Passing that content as pre_content skips the redundant cat
   inside write_file. Fully backward-compatible: callers that don't pass
   pre_content still read from disk as before.

3. Replace wc -c with len(content.encode('utf-8')) (−1 subprocess/write)
   We already have the content in memory; encoding it to get the byte count
   is equivalent to wc -c for UTF-8 text.

4. Remove redundant _check_lint loop in apply_v4a_operations (−N subprocesses/V4A)
   write_file already runs _check_lint_delta internally. The old code ran a
   bare _check_lint(f) loop over all modified files — a re-read + re-lint
   without post_content context. Now lint results propagate from write_file
   via a four-tuple return, zeroing out the extra subprocesses.

Net effect:
  - write_file: 6 → 3 subprocesses per call (new files)
  - patch_replace: 6 → 5 subprocesses per call (pre_content skips cat)
  - V4A multi-file patches: saves 1 subprocess per modified file
  - A typical 4-file V4A patch drops from ~28 to ~16 subprocess calls

* fix(file-ops): decouple BOM detection from pre_content, add V4A backward compat

Bug 1 (UTF-8 BOM loss on V4A UPDATE):
_file_has_bom() trusted pre_content for BOM detection, but the most
common pre_content provider — read_file_raw() — deliberately strips
BOMs so the agent never sees U+FEFF glyphs.  Passing BOM-stripped
content through pre_content caused a false-negative: the method
returned False and write_file() silently removed the marker on rewrite.

Fix: _file_has_bom() now always probes the first 3 bytes on disk
(head -c 3), ignoring pre_content for BOM purposes.  pre_content is
still used by two other consumers — line-ending detection and lint/LSP
delta computation — neither of which is affected by BOM stripping.

Bug 2 (backward compatibility):
_apply_update() called write_file(path, content, pre_content=...) as a
keyword argument.  Duck-typed file_ops implementations that only
implement the two-argument write_file(path, content) contract would
raise TypeError.

Fix: wrap the call in try/except TypeError, falling back to the
two-argument form when the keyword is not accepted.

Also declare tomli in pyproject.toml (pre-existing conditional import
for pre-3.11 Python, caught by the pre-commit dep scan after staging
file_operations.py).

Tests:
Add TestV4ABomRoundTrip with two cases:
  - UPDATE on BOM-bearing file preserves the marker
  - UPDATE on plain file does not inject a BOM

Addresses teknium1 review on PR #55661.

* fix(file-ops): surrogatepass in bytes_written encode (review finding)

Content that flowed through a surrogateescape decode (backend output via
patch_replace) can carry lone surrogates; a strict encode raises
UnicodeEncodeError where the old wc -c path could not. Mirrors the
existing sha256 verification encode.

* refactor(file-ops): fold simplify-pass findings

- write_file: encode content once, share bytes between bytes_written and
  the sha256 verification (drops a second full-content encode per write)
- patch_parser: replace the except-TypeError retry around
  write_file(pre_content=...) with signature-based feature detection so a
  TypeError raised inside a capable implementation propagates instead of
  triggering a duplicate write; tests for both duck-typing contracts
- tests: real-ops V4A BOM round-trip + _file_has_bom disk-probe guard
  (the teknium1-review regression previously only covered by a fake)
- comment: document dirs_created's long-standing "parent ensured" meaning

* chore: update uv.lock for tomli dependency (rebase fix)

* chore: remove dead tomli dependency declaration

requires-python is >=3.11 so tomllib is always in stdlib; the
tomli fallback branch in _lint_toml_inproc was unreachable. Removes
the dependency from pyproject.toml + uv.lock and deletes the dead
try/except ImportError fallback in the code.

* fix(telegram+sqlite): resolve polling conflict loop + misleading WAL warning

#75017: Telegram polling conflict retry used drop_pending_updates=False,
starting a new getUpdates session that immediately got 409'd by the
previous still-expiring session — creating the very conflict it was
trying to recover from. Switch to drop_pending_updates=True so Telegram
terminates stale sessions. Also add a recovery-generation guard so the
first transient getUpdates success after a retry doesn't reset the
conflict counter back to 0 (defense-in-depth from PR #75096).

#75153: The WAL-reset warning always said 'hermes update can repair'
even for git/pip/system Python installs where it can't. Now uses
detect_install_method() + recommended_update_command_for_method() to
give a context-appropriate hint (hermes update for git, docker pull for
docker, nix message for nix, generic install hint as fallback).

* chore: map jun@junho.co to junhohong

* fix(dashboard): reload loopback tabs after stale session-token closes

Loopback dashboard tabs now share one one-shot stale-token recovery path across REST 401s, the PTY socket, the structured event socket, and the shared JSON-RPC gateway wrapper. The shared client exposes only an optional close-event interception hook; the dashboard remains responsible for deciding that loopback 4401 means reload.

Constraint: Current main delegates the web gateway to apps/shared JsonRpcGatewayClient, and #54022 review requires a shared-client-compatible close-code hook plus direct ChatSidebar event-socket coverage.
Rejected: Restore the dashboard's old direct WebSocket implementation | stale against the sh…
dizhaky added a commit to dizhaky/hermes-agent that referenced this pull request Aug 5, 2026
…85) (#148)

* fix(lint): import sort + eslint-disable for timer-handle ref clear in effect

CI-caught: cron-jobs-section had an extra blank line between sorted imports; use-message-stream's visibility-flush effect assigns flushHandleRef.current=null inside a useEffect (legitimate timer-clear, not an atom mirror) — eslint-disable-next-line per the rule's documented convention.

* fix(state): narrow FTS UPDATE triggers with AFTER UPDATE OF + migration

Retarget #73639 onto the SessionDB mixin split (hermes_state_common /
hermes_state_schema). Fresh installs create UPDATE OF content/tool_*
triggers; existing broad AFTER UPDATE triggers are inspected and
replaced under schema init without an FTS rebuild (WHEN clauses already
guarded content correctness; OF skips non-content status writes that
saturated disk I/O on large state.db).

Tests: tests/test_fts_update_of_narrowing.py (4)

* fix(state): fail closed on CJK trigger migration

* fix(state): quarantine CJK when ensure soft-fails after OF migration

_ensure_fts_cjk_schema never raises on OperationalError; post-condition
after dropping messages_fts_cjk_update now requires a narrowed UPDATE
trigger or durable fts_cjk_stale + unavailable. Covers the production
soft-fail path the raise-only handler missed.

* refactor(state): drop unreachable regex guard in trigger migration

Simplify-pass fold: to_drop names come from the literal update_names\nallowlist via IN binding, so the [A-Za-z0-9_]+ fullmatch could never\nfail — and if it somehow did, its `continue` would miscount (the\nskipped trigger stayed in len(to_drop)/the log while CREATE TRIGGER\nIF NOT EXISTS silently kept the broad variant). Delete the guard and\nits function-local re import; keep the invariant as a comment.

* fix(security): reject always-blocked OpenViking endpoints

## Summary
- Normalize OpenViking endpoints through `is_always_blocked_url` and fall back to the default local endpoint when poisoned.
- Keep intentional loopback / LAN self-host working.
- Add focused unit tests.

## Salvage / credit
Memory-provider endpoint floor sibling of RetainDB/Supermemory always-blocked hardening (avoids over-broad #4984-style private-IP bans).

(cherry picked from commit 8fa607d0aedb8c5fca398d7f112b1b25ade54fa2)

* fix(openviking): fail closed on blocked endpoints

(cherry picked from commit 389a90b81c9c2c89810f2fa7461f8faa9a5c9578)

* fix(openviking): don't spawn a second server onto a live port

`_start_local_openviking_server()` spawned `openviking-server`
unconditionally. Both callers — `initialize()` and the runtime
unreachable handler — reach it from a health probe, and that probe can
time out client-side while the server is up and serving. The spawned
process then loses the data-directory lock and exits immediately with
`DataDirectoryLocked`; because the probe keeps timing out, the cycle
repeats every cooldown window (~5 min observed).

The existing 30s `_failed_refresh` cooldown paces the loop but cannot
stop it, since it expires while the underlying condition persists.

Probe the target host:port before spawning and treat an occupied port as
already-started. This guards both call sites at their single convergence
point. The probe deliberately tests only that a listener owns the port —
enough to know a second server would lose the lock — and says nothing
about that listener's health.

The parse/probe now precedes the PATH lookup, so a reachable server is
reported as running even when `openviking-server` is not on PATH.

Fixes #74846

(cherry picked from commit b49427d85fd6628eb4a7fe099e5c390c5c4cc935)

* fix(openviking): drop stale "disabled for this Hermes run" warnings

The provider used to disable OpenViking permanently when the server was
unreachable. That was fixed: `_ensure_client()` now reconnects lazily,
with a 30s cooldown gate in `_ensure_client_locked`.

Only one of the seven user-facing warnings was updated to match. The
other six still told the user memory was "disabled for this Hermes run",
which is no longer true — every one of those paths is retried on the next
access. A user who reads the old message has no reason to retry, which is
very likely how #5721 ("never recovers") came to be filed against
behaviour that already recovers.

All six sites were traced to confirm none is terminal for the run: the
`initialize()`-time and waiter-thread failures never arm `_failed_refresh`
(only line 2439 does), so they retry on the very next access with no
cooldown at all.

The replacement wording deliberately omits the "(after cooldown)"
parenthetical used at the already-correct site — that detail is only
accurate where `_failed_refresh` was just armed. The neutral phrasing is
true at all six.

Also promotes two clause separators to periods to avoid "…; …disabled;"
collisions.

(cherry picked from commit 8346403a4b97af503d26b0f7905ff513828d821e)

* fix(openviking): re-arm the commit guard after in-place compression

`_committed_session_ids` is a permanent per-sid latch, and
`_session_needs_commit` checks it before the turn counter by design — a
racing sync_turn can re-increment `_turn_count` after commit+reset, so
the guard must win to stop a double-commit.

That is correct for a session being left behind. It is wrong for one
that keeps its id. `compress_context()` commits before rewriting the
transcript in both modes, and with `compression.in_place: true` (the
default) `on_session_switch` receives the same id and does not rotate.
The latch then rejects every later commit for a still-live session — the
next compression, /new, normal session end, startup recovery — so every
post-compression turn is silently never extracted.

Rotation mode is unaffected because a fresh child id is minted and
starts clean, which is what confirms the latch's intent was only ever to
dedupe the departing id.

Clear the latch when compression completes without rotation. Turns
arriving after that point are genuinely new, and this is a defined
moment rather than a race. The rotation path is untouched, so the old
id stays latched and its _finalize_session_async still dedupes against
the compression commit.

Fixes #74695

(cherry picked from commit d1e5c3dc33ef0d43d021662674e1a7cd5e43eecd)

* test(openviking): cover the compression lifecycle, not a hand-set latch

Review feedback: the previous test called _mark_session_committed
directly, so it verified the guard's behavior but not the wiring that
sets it — a future break in the commit_memory_session -> same-id
compression-boundary path would not be caught.

Add a lifecycle regression that drives the real sequence: on_session_end
commits through the actual path, on_session_switch(same id,
reason="compression") crosses the boundary, sync_turn records a genuinely
new turn, and a second on_session_end must produce a second commit POST.

Without the fix it fails showing exactly one commit call, which is the
reported data loss: every turn after the first compression is dropped.
The rotation and /undo tests stay as scope guards.

(cherry picked from commit 0ca5a330630a30b105cbbc32e8a23f2c5ffe0eab)

* fix(memory): read non-secret provider config from config.yaml for OpenViking and RetainDB

OpenViking is_available() only consulted env vars and use_ovcli_config, so an
endpoint saved to config.yaml (e.g. by the Dashboard) reported needs_config;
_resolve_connection_settings() likewise never folded config.yaml's non-secret
fields into its chain. RetainDB initialize() read base_url/project from the
environment only, ignoring the values the Dashboard writes to config.yaml.

Both now resolve non-secret fields as env -> (ovcli ->) config.yaml -> default;
secrets still come from the environment. Adds regression tests for both.

Fixes #68209

(cherry picked from commit dca57915b97b5705b30927a062e1d0f2f23d3841)

* fix(openviking): read recall settings from config.yaml first, env vars as fallback

_recall_config() previously read all settings (recall_limit, score_threshold,
recall_resources, etc.) exclusively from environment variables. This forced
users to store behavioural configuration in .env, violating the Hermes
convention that .env is for secrets only.

The infrastructure to load config.yaml -> memory.openviking was already in
place via _load_hermes_openviking_config(), but _recall_config() never
called it.

Fix: call _load_hermes_openviking_config() and pass its values as the
default parameter to _env_int/_env_float/_env_bool. Env vars still override
config.yaml values, preserving backward compatibility.

Closes #62540

(cherry picked from commit 6aadf1256835745e0302aa3d3b5ae0660b368637)

* test(openviking): cover config.yaml recall settings with temp-HERMES_HOME tests

Add three tests to TestOpenVikingConfigSchema:

1. test_recall_config_reads_from_config_yaml — writes memory.openviking
   settings in config.yaml and verifies _recall_config() consumes them.

2. test_recall_config_env_overrides_config_yaml — writes both config.yaml
   and OPENVIKING_RECALL_* env vars, verifies env takes precedence.

3. test_recall_config_partial_config_yaml — partially populated config.yaml
   falls back to defaults for omitted keys.

All 46 openviking_plugin tests pass (43 existing + 3 new).

(cherry picked from commit b8d7834caf06c6912004333c270faa248eaed4cd)

* fix(openviking): integrate reliability and configuration hardening

* chore(contributors): map OpenViking source authors

* test(retaindb): guard scoped secret config resolution

* fix(openviking): verify servers before sending credentials

* fix(openviking): catch endpoint errors in setup validation functions

Review follow-up for salvaged PR #76782. Three setup-wizard
validation functions called _normalize_openviking_url outside their
try/except blocks. Since _normalize_openviking_url now raises
_OpenVikingEndpointError for blocked or malformed endpoints, an
invalid endpoint would crash the wizard instead of returning a
friendly (False, message) tuple.

- _validate_openviking_auth: move _normalize_openviking_url inside try
- _validate_openviking_root_access: same
- _validate_openviking_setup_values: catch _OpenVikingEndpointError explicitly
- Remove dead ternary in _normalize_openviking_url safety check (candidate
  always has http/https scheme by that point)
- Replace redundant float('-inf') < x < float('inf') with math.isfinite()
  in _setting_float; drop the redundant infinity check from _setting_int
  (is_integer() already rejects inf/nan)

* fix(state): deduplicate session system prompts

* chore: map cicav legacy noreply email

* fix(tui): avoid writable Kanban opens on empty polls

* fix(context): dedupe subdirectory hints by content digest and skip backup/vendor dirs

SubdirectoryHintTracker re-injected identical context files whenever the same
AGENTS.md was reachable through more than one path. Symlinked shared
workspaces, hardlinks, and timestamped backup copies all alias a single file,
so a normal session could ship the same 8KB of instructions two or three
times. Nothing deduped it and nothing excluded directories that only ever
hold copies.

Two changes:

* Track a sha256 of every injected hint body. Repeat content is skipped, and
  the working directory's own context file is seeded at construction so the
  copy prompt_builder already loaded at startup is never sent again.
* Skip directories that hold copies rather than authoritative context
  (backups, node_modules, venv, site-packages, .git, .Trash, vendor, caches).
  Screening is relative to working_dir, so a project that legitimately lives
  under vendor/ keeps discovering its own subdirectory hints.

Measured on a real session that touched a symlinked shared workspace:
3 injections / ~24,000 chars before, 1 injection / 8,112 chars after.

14 new tests cover symlink aliasing, byte-identical copies, working-dir
seeding, distinct content still being injected, each excluded directory name,
excluded ancestors, and the working-dir-inside-excluded-name case.

* perf(state): batch the turn flush into one SQLite transaction

Re-derivation of #23254 (@devsart95) on today's flush loop. The turn
flush in _flush_messages_to_session_db wrote one BEGIN IMMEDIATE
transaction per message row; a typical agent turn (user + assistant +
tool results) paid 3-8 transactions -- and, off WAL (the default on
macOS while the WAL-reset guard is active), 3-8 fsyncs -- per turn.

Adds SessionDB.append_messages_batch: same row shape as append_message
(shared _prepare_message_row serializer + _MESSAGE_INSERT_SQL column
list, so the two writers cannot drift), same compression-lock and
compression-closed guards, one aggregated session-counter UPDATE, one
transaction for the whole batch. Row serialization stays outside the
write lock.

The flush loop now collects the turn's new rows and writes them in one
call. All-or-nothing pairs exactly with the persisted-marker stamping:
on failure no rows landed and no markers were stamped, so the next
flush re-writes the whole tail (same recovery contract as before,
minus the partial-prefix case that could double-count).

Measured (same harness, 5-message turn, journal_mode=DELETE,
synchronous=FULL): 2.32ms -> 0.83ms median per turn flush (64% faster,
5 fsyncs -> 1). On WAL the win is smaller but the atomicity fix holds.

* perf(tui-gateway): batch branch-seed history copies (whole-bug-class)

Sibling sites of the per-message flush pattern: both branch-seed
paths (session.branch in methods_session.py and the lazy seed persist
in server.py) copied the parent history row-by-row -- one transaction
per row, and a branch seed can be hundreds of rows. Route both through
SessionDB.append_messages_batch. The server.py path also gains real
atomicity: _branch_seed_persisted assumed every row landed, which the
per-row loop could not guarantee.

* test(run-agent): update flush-path fakes and assertions for batched writes

The flush now goes through append_messages_batch; MagicMock-based
assertions and barrier fakes that hooked append_message observed
nothing (the flush's try/except swallowed the AttributeError). Assert
on the batch payload instead.

* refactor(state): fold simplify findings — reuse _insert_message_rows, share guards, chunk seeds

Simplify-pass folds on the #23254 salvage:

- REUSE (HIGH): append_messages_batch now delegates row serialization to
  the pre-existing _insert_message_rows helper (already shared by
  replace_messages / archive_and_compact / portability import) instead
  of adding a third serialization path (_prepare_message_row +
  _MESSAGE_INSERT_SQL are gone). One row-writer for every multi-row
  path; the row-ID return was consumed by no production caller, so the
  batch returns the inserted count.

- QUALITY (HIGH): the compression-lock + compression-closed admission
  guards are extracted into _check_transcript_write_guards, shared by
  append_message and append_messages_batch (previously duplicated 23
  lines that had already needed targeted fixes, #74478). The role-gated
  reasoning filtering is no longer duplicated in run_agent.py — it
  lives at its one site inside _insert_message_rows.

- EFFICIENCY (MEDIUM, measured): unbounded seed copies hold one BEGIN
  IMMEDIATE for seconds (10k rows ~= 2.4s; FTS triggers dominate) and
  monopolize the in-process write lock. append_messages_batch grows a
  chunk_rows param; all seed/copy call sites use chunk_rows=500. Same
  recovery semantics as the old per-row loops, bounded lock holds.

- REUSE (MEDIUM): the two remaining per-row branch-copy loops found by
  the pass (gateway/slash_commands.py /branch, hermes_cli
  cli_commands_mixin.py branch) are converted to chunked batches too
  (AsyncSessionDB's generic to_thread forwarder covers the async site).

Turn-flush benchmark unchanged after the refactor: 2.43 -> 0.87 ms
median per 5-message flush (64% faster).

* fix(tests): update two more append_message.call_args assertions to append_messages_batch

CI-caught: test_verification_stop_caching and test_tui_gateway_server::test_native_vision_turn_persists_a_renderable_image_ref both assert on append_message.call_args, but the flush loop now calls append_messages_batch. Same class of test-fake fallout fixed in 5 other files — these two were missed.

* perf(tui): memoize useSessionLifecycle return (idea from #38491)

Re-derivation of #38491 by @stremtec onto current main (the original is
10,119 commits behind; the hook moved into ui-tui/src/app/). The hook
returned a fresh object literal every render, defeating memoization in
useMainApp's consumers; useMemo over the (all-useCallback-stable)
handles makes the return referentially stable.

Dep array covers ALL nine returned handles incl. trimTail (the
re-derivation initially omitted it - stale-closure class).

* ci: retry uv python install

* fix(state): route session-resume reads through the WAL read-only connection

get_messages_as_conversation, get_resume_conversations, and
get_ancestor_display_prefix still took self._lock — the same global
choke point the read-path split (WAL per-thread read-only connections)
was meant to remove from every recall/browse read. These three are the
hottest reads in the file: every session resume across the gateway,
CLI, and ACP adapter goes through one of them, so a resume racing a
burst of concurrent-session writer flushes still convoys behind them
exactly like the fixed paths used to.

_session_lineage_root_to_tip (the lineage walk shared by all three,
plus get_conversation_root) had its own independent self._lock use and
needed the same conversion — without it the outer functions still
blocked on the very first line.

Verified empirically: a reader thread calling all three functions
while another thread holds self._lock blocked for the writer's full
hold duration before the fix, and returned immediately after (SQLite
3.50.4 in this dev venv falls back to journal_mode=DELETE per the
WAL-reset-bug guard, so the requires_wal-marked regression test is
exercised via a local WAL-forced script instead; it still runs and
passes on any runtime where WAL is actually active).

* chore: add contributor email mapping for ArcherQAQ

* fix(model_metadata): rewrite localhost->IPv4 for the remaining local probe sites

fetch_endpoint_model_metadata's generic (non-LM-Studio) /models fetch and
its llama.cpp /v1/props context-length follow-up built request URLs
straight from the unrewritten candidate, unlike every other local-probe
site. Both retained the multi-second dual-stack IPv6 connect penalty
that _localhost_to_ipv4() exists to skip (measured on macOS: localhost
32.9ms vs 127.0.0.1 0.1ms on a dead port; ~2s on Windows). normalized
stays the cache key so caching behavior is unchanged; only the outbound
request target is rewritten.

Re-derived from PR #61528 onto current main (original no longer applied
cleanly).

* fix(model_metadata): guard _localhost_to_ipv4 against non-string urls

CI slice 3/7 failures: run_conversation tests pass MagicMock base_urls
through the metadata probe path; re.sub raised TypeError where the old
code let non-strings flow through. Preserve that contract.

* perf(cold-start): mitigate ~14s GIL stall during backend init (#60800)

Three fixes for the Desktop/TUI cold-start stall where the event loop
is blocked for ~14s between HERMES_BACKEND_READY and the first
prompt (#60800):

1. copilot_auth: skip  subprocess fallback when any
   Copilot env var is explicitly set (even if invalid). The user
   expressed token intent via env var; silently substituting a CLI
   token is surprising and the subprocess adds up to 5s on Windows.

2. tui_gateway/ws: run resolve_skin() via asyncio.to_thread so config
   loading + skin engine init do not block the WS read loop during
   the cold-start RPC burst.

3. web_server: extend _warm_gateway_module to pre-import the heavy
   module chains (auth, copilot_auth, runtime_provider, skin_engine,
   inventory, model_switch) that the first WS connection + RPC burst
   would otherwise import on the loop thread. These trigger .pyc
   compilation and Defender scans on Windows (15-30s per the existing
   comment) and were not covered by the original gateway-only warm.

Tests: 5 new tests in test_cold_start_gil_stall.py + 2 new tests in
test_copilot_auth.py. All 36 copilot_auth tests + 16 ws/web_server
tests pass.

* test: harden cold-start regression tests + debug-log the env-var skip

Review folds on the #60807 salvage:
- resolve_skin tests are behavioral (thread-ident probe + ready-frame
  wiring check) instead of pure source inspection, per the #72720
  pattern; a source assertion remains as belt-and-braces.
- The warm-list test does REAL imports and checks sys.modules —
  _warm_gateway_module swallows ImportError by design, so the PR's
  tracking-stub test would pass even with a typo'd module name.
- resolve_copilot_token logs a debug line when the env-var
  short-circuit skips the gh-CLI fallback (behavioral change made
  observable).

* perf(gateway): per-platform skip_context_files to cut agent build latency

Salvage of #26860 (hunk 2, ported \u2014 the PR's base predates the current
gateway layout by ~11.9K commits). Messaging platforms can set
gateway.platforms.<key>.skip_context_files: true to skip the
filesystem-heavy context-file discovery (SOUL.md, AGENTS.md,
.cursorrules walks) during AIAgent construction \u2014 10-100x slower
stat()/walk costs on Windows made this a real per-turn tax. Soul
identity is still loaded (single small file), so the persona survives.

The flag participates in _agent_config_signature so toggling it
rebuilds the cached agent instead of silently reusing a prompt built
under the other setting (prompt-cache correctness).

The PR's hunk 1 (mtime-caching the per-turn dotenv reload) was dropped:
df51ad797 mtime-cached load_config/read_raw_config and c2eda92fd
removed the per-turn deepcopies, capturing most of that win; the
function has since gained a multiplex early-return and managed-scope
overlay that the original whole-function skip would have bypassed.

* fix(relay): route Discord tool-progress into the auto-thread, not the parent channel (#77830)

When a Discord channel message initiates a relay auto-thread, the thread does
not exist at ingest (source.thread_id is None) — the connector creates it on
its FIRST send and auto-threads any outbound carrying the reply anchor. The
final reply carries that anchor, so it lands in the thread. But the
tool-progress / status bubbles (the "Searching the web for..." updates and the
streaming preamble) were sent with _progress_metadata=None and
_progress_reply_to=None: _resolve_progress_thread_id returns None for Discord
(only slack/mattermost get a synthetic thread), so the progress send had no
anchor and the connector posted it FLAT in the parent channel. Result: the
search-status updates leaked outside the thread while the answer threaded
(staging repro 2026-08-02).

The connector now stamps prospective_thread_id on the inbound (the anchor
message id == the id of the thread it will create). Reuse it: when a
relay-delivered Discord channel-initiate carries prospective_thread_id and has
no real thread yet, carry the reply anchor (event_message_id) on both the
progress metadata (reply_to_message_id) and the progress reply_to, so the
connector routes the progress bubble into the SAME auto-thread as the final
reply. Applied to both the tool-progress path (_progress_metadata /
_progress_reply_to) and the status/interim callback path
(_status_thread_metadata). Events already arriving in a real thread, DMs, and
non-relay sources are untouched (guarded on delivered_via_upstream_relay +
prospective_thread_id + not thread_id).

Tests: two new cases in test_run_progress_topics.py — a relay Discord
channel-initiate asserts every progress send carries the anchor (reply_to +
metadata.reply_to_message_id + non_conversational), and an event already in a
real thread asserts the synthetic-anchor path does NOT engage. Full gateway
progress + relay + session suites green (228 passed).

* fix(agent): stop re-probing endpoints that blackhole TCP connects

Salvage of #71282 (Fixes #71281): a routable-but-dead endpoint (corp
LAN address while off-VPN) blackholes TCP SYNs, so every probe in the
model-metadata waterfall waits out its full connect timeout — 20+
seconds of stall per startup across detect_local_server_type,
fetch_endpoint_model_metadata, and the per-model probes.

A module-level blackhole cache keyed on host:port is populated when
any probe observes a ConnectTimeout (httpx or requests; read timeouts
deliberately excluded — an accepted connection is not a blackhole) and
consulted at the top of each guarded function. 30s TTL: long enough to
collapse one startup burst, short enough that VPN recovery is picked
up without a restart. Guard ordering: blackhole check -> disk L2 ->
HTTP waterfall, and a blackholed leg aborts the remaining legs instead
of letting each stall in turn.

Squash of the PR's two real commits (the branch's merge commits made
it un-rebase-merge-able; content verified identical via merge-tree).

* chore: release v0.20.0 (2026.8.3)

The Herald Release — voice (streaming TTS, barge-in, wake words), A2A v1.0,
outbound webhooks, grounded citations, desktop platform wave. ~3,650 commits,
~1,400 PRs, ~1,200 issues closed, 650+ contributors since v0.19.0.

Also: contributor audit additions (18 email mappings, bot-filter widening).

* chore: add contributor email mapping for Ahmett101

* perf(moa): cache resolved preset + per-slot runtime to cut cold-start latency (#66793)

* fix(discord): leave voice channels before cancelling the bot task

`DiscordAdapter.disconnect()` cancelled the bot task before tearing down voice
clients. `leave_voice_channel()` ends in `await vc.disconnect()`, and discord.py
sends a voice state update over the main gateway websocket and then waits for the
voice socket to close. The bot task is the loop running that gateway connection,
so cancelling it first left the handshake with no transport: it could never
complete and blocked until the caller's shutdown timeout fired.

The effect was a fixed ~5s penalty on every shutdown with a voice connection
open, ending in "discord disconnect timed out after 5.0s - forcing continue",
with the voice disconnect abandoned rather than completed.

Measured on a live gateway with a voice connection open in both cases:

  before: timed out after 5.0s, all adapters disconnected at +5.29s
  after:  discord disconnected (0.12s), all adapters disconnected at +0.46s

Moving the voice-cleanup loop above `_cancel_bot_task()` preserves the
zombie-client protection its comment describes: the bot task is still cancelled
before `client.close()`, just after voice teardown rather than before it. Voice
teardown is the one step that still requires a live gateway.

Adds a regression test asserting the ordering. It fails on the previous ordering
at index 1 with `cancel_bot_task != leave_voice_channel:111`.

Fixes #76044

* feat(image): parallelize image_generate batches

* fix(file-sync): serialize concurrent sync cycles

* fix(tool-executor): unpack 5-tuple runnable_calls in _max_workers_for_tool_batch

* fix: exponential backoff for rate-limit fallback cooldown

Replace the fixed 60-second cooldown with exponential backoff:
30min → 1h → 2h → 4h cap.

The counter is reset by restore_primary_runtime on successful
primary-provider recovery, so the backoff is strictly for
consecutive failures within a single degradation window.

Closes #29702

* fix(backoff): keep 60s first-hit cooldown, escalate only on consecutive rate-limits

Review follow-up on the #30223 salvage: the original changed the base
cooldown from 60s to 1800s, benching the primary for 30 minutes on the
FIRST 429 (30x regression in primary-restore latency) and breaking the
existing test_rate_limit_exhaustion_keeps_60s_cooldown contract.

Keep upstream's 60s base and escalate per consecutive rate-limit:
60s -> 2m -> 4m -> 8m -> ... capped at 4h. Counter still resets on
successful primary restore (cicae's mechanism, unchanged).

New tests: escalation doubling, 14400s cap, reset-on-restore.
Existing 60s contract test passes UNCHANGED. Mutation-checked:
escalation disabled -> 2 fail; reset disabled -> 1 fails.

* fix(catalog): wire api_key auth headers for http MCP servers

When an optional-mcps manifest declares transport.type=http with
auth.type=api_key, install_entry() prompts for the key and saves it to
.env, but _build_server_config() only handled the oauth case — the
api_key case produced a bare url entry with no headers, so every
request to the server was unauthenticated (-> 401).

Reuse _bearer_auth_headers(entry.name) from mcp_config.py so the
catalog path emits the same 'Authorization: Bearer ${MCP_..._API_KEY}'
template as the manual 'hermes mcp add --url' path.

Salvaged from #70782 (production hunk applied clean; tests re-anchored
onto current main). Credit: JonthanaHanh.

* perf(compressor): release allocator pages after successful compaction

A successful compaction frees the largest allocation a long session ever
drops (the compressed-away message dicts), but Python's arena allocator
keeps those pages in the heap — RSS retains the pre-compaction
high-water mark until exit. #76905's trim_memory lifecycle covers the
gateway/TUI housekeeping loops but not the CLI compression path.

Call trim_memory(reason='post-compression') at the compression-success
point in ContextCompressor.compress(), following the house pattern
(lazy import in try, debug-level log on failure). The helper is
glibc-gated, config-gated and rate-limited, so it is a safe no-op on
other platforms and cannot fail compression.

Re-expresses the intent of #70782 (JonthanaHanh), which reached for a
bare gc.collect(); trim_memory is the house mechanism and already
wraps a collect.

* fix(catalog): validate http+api_key manifests declare the header's env key

Simplify-pass follow-up on the #70782 salvage: _bearer_auth_headers
hard-emits ${MCP_<NAME>_API_KEY} but install_entry only persists
auth.env-declared vars — a manifest naming its key differently (the
shipped n8n style) would install cleanly yet send a literal-placeholder
header at connect time (silent 401, the #37792 bug class). Enforce the
naming contract at parse time. Also pins the secret-stays-in-.env
property in the install test (raw config.yaml carries the template,
never the secret). Mutation-checked: validation disabled -> guard test
fails.

* fix(agent): cap auxiliary LLM concurrency per task

* fix: thread extra_headers through the call_llm split

The PR's concurrency wrapper splits call_llm into a semaphore-guarded
entry + _call_llm_impl; main added extra_headers to call_llm's
signature after the PR's base, so the split has to forward it too
(dropped silently otherwise — Azure Foundry and custom-endpoint
callers set it).

* fix(nix): tie devShell's HERMES_PYTHON to the venv actually on PATH

`nix/devShell.nix` collected `devShellHook` by scanning every package:

    nonNpmHooks = map (p: p.passthru.devShellHook or "") packages;

But `minimal` and `messaging` are `.override` variants of `default`, so
each carries its own `devShellHook` exporting its own HERMES_PYTHON. The
scan therefore concatenated three conflicting exports and forced Nix to
evaluate and realise three separate uv2nix editable venvs on every
`nix develop`.

`attrValues` is alphabetical, so the last hook won (`minimal`) while
`python`/VIRTUAL_ENV came from `default`'s devDeps:

    HERMES_PYTHON        = ...dimim2... (minimal — no optional deps)
    python / VIRTUAL_ENV = ...85r28... (full)

Inside the shell `$HERMES_PYTHON -c "import anthropic"` failed while
`python -c "import anthropic"` succeeded. Worse, `scripts/run_tests.sh`
prefers HERMES_PYTHON, so the suite ran against the minimal venv. Its
guard did not catch this: it only checks that HERMES_PYTHON has pytest,
and minimal's venv does (pytest is in the `dev` group), so the wrong
interpreter was silently accepted.

Tying the hook to `packages.default` — the same package whose `devDeps`
are installed — keeps HERMES_PYTHON, `python`, and VIRTUAL_ENV pointing
at one venv by construction.

    editable venvs referenced   3      -> 1
    their combined closure      421 MB -> 140 MB
    test failures               85     -> 32

The venv mismatch was masking 53 failures; e.g. test_web_tools_config.py
goes 2-failed -> 38-passed. Full suite is now 25369 passed / 32 failed,
and those 32 reproduce identically on a pristine HEAD worktree with no
nix/ changes under the same interpreter (mostly NixOS artifacts — tests
spawning bare `python3` in a scrubbed env exit 127).

* fix(system_prompt): move skills index to the volatile band

The skills index is runtime-mutable: the agent adds and patches skills mid-session, so it is not byte-stable. Keeping it in the stable band breaks that band prefix-cache contract, because every skill edit changes the stable band and invalidates the entire cached prefix in front of it. Move it to the front of the volatile band so the stable scaffold (identity, tool guidance, model guidance) stays cacheable across skill edits.

* docs(system_prompt): fix stale reconstruct_static_prefix docstring example

Simplify-pass finding: the safety note still cited 'skills edited' as a stable-tier input whose change mismatches the rebuilt prefix — after this PR a skill edit changes only the volatile tail (that's the point). Swap the example for genuinely stable-tier inputs.

* fix(prompt_size): search volatile tier for skills block after the stable->volatile move

CI-caught: compute_prompt_breakdown still looked for <available_skills> in the stable tier, but #37117 moved it to volatile. Search volatile first, fall back to stable for older sessions.

* chore: add contributor email mapping for zabih-sudo

* chore: add contributor email mapping for HAOWANG116

* fix(backup): serialize and atomically publish snapshots

* chore: add contributor email mapping for ElSnacko

* fix: prefer explicit anthropic api key

Cherry-picked from PR #58560 by @itsflownium, adapted to current main
(_getenv instead of os.getenv). Moves ANTHROPIC_API_KEY check ahead of
Claude Code credential file and credential_pool auto-discovery so an
explicitly configured key is never shadowed by auto-discovered OAuth.

Fixes #58546

* docs: document /personality none|default|neutral reset across personality docs

The reset keywords have existed in both CLI and gateway handlers since
June but were undocumented — users couldn't find how to cancel a
personality overlay. Adds a 'Resetting to the default' section to the
personality feature page and mentions the reset in the CLI guide,
slash-command reference (both tables), and messaging command table.

* fix(relay): avoid concurrent turn scope corruption

Signed-off-by: Bryan Bednarski <bbednarski@nvidia.com>

* fix(relay): preserve legacy turn shims

Signed-off-by: Bryan Bednarski <bbednarski@nvidia.com>

* fix(relay): gate skipped turn metrics

Signed-off-by: Bryan Bednarski <bbednarski@nvidia.com>

* test(relay): enforce LIFO in overlap regression

Signed-off-by: Bryan Bednarski <bbednarski@nvidia.com>

* fix(relay): preserve skipped turn context

Signed-off-by: Bryan Bednarski <bbednarski@nvidia.com>

* fix comment about relay workaround

* feat(models): add qwen3.8-max to Nous portal + OpenRouter catalogs, replacing qwen3.7-max

Qwen3.8 Max is live on both OpenRouter and the Nous portal
(qwen/qwen3.8-max, 1M context, 131K max output). Per the
newest-max-replaces-last-max convention, it takes qwen3.7-max's slot
in both curated lists.

- hermes_cli/models.py: OPENROUTER_MODELS + _PROVIDER_MODELS[nous]
  swap qwen/qwen3.7-max -> qwen/qwen3.8-max
- agent/model_metadata.py: DEFAULT_CONTEXT_LENGTHS entry for
  qwen3.8-max at 1,000,000 (verified against OpenRouter live
  metadata and Nous /v1/models 2026-08-03)
- tests/test_empty_model_fallback.py: swap incidental catalog fixture
  to the surviving slug
- website/static/api/model-catalog.json: regenerated

Pricing snapshot skipped: both routes bill via official_models_api
(live pricing), verified with resolve_billing_route. Reasoning
timeout floor already covered by the qwen3 prefix (180s).

* test: swap context-switch-guard fixture off qwen3.8-max-preview

test_custom_provider_context_avoids_false_shrink_warning used
qwen3.8-max-preview as a slug that deliberately falls through to the
generic 'qwen' 131K catalog match. The new qwen3.8-max
DEFAULT_CONTEXT_LENGTHS entry (1M) now substring-matches the preview
slug too, so the no-custom-providers branch stopped warning. Swap the
fixture to qwen3.9-max-preview, which still hits the generic fallback
— the test's intent (custom_providers threading) is unchanged.

* fix(agent): keep context_length pin for named custom providers

Empty model.base_url plus a runtime custom-provider URL was treated as a
route mismatch, so gateway session-reset banners dropped model.context_length
and fell back to the Qwen family default (131K) while /status still showed
the configured 262K pin.

* test(gateway): cover named-custom context pin on session-info banner

* fix(model_metadata): read llama.cpp context from meta.n_ctx + accept sole model

* chore: add contributor email mapping for johnrazmus

* fix(gateway): keep event loop alive during /compress and Relay drain

Offload manual /compress temporary-agent cleanup through the existing
bounded off-loop helper so a slow agent.close() cannot freeze the
gateway event loop, heartbeat, or platform polling.

Guarantee Relay transport teardown even when the runner cancels
adapter.disconnect() during go_idle: shielded finally, 2s drain-path
idle ACK budget under the 5s outer disconnect budget, and bounded
supervisor/reader/ws.close awaits.

Original commits:
- fix(gateway): offload manual /compress cleanup from the event loop
- fix(gateway): tear down Relay transport even if go_idle is cancelled
- fix(gateway): keep Relay disconnect budgets inside the runner window

By @Dannyzen (PR #78027), salvaged onto current main.

* fix(gateway): bound go_dormant ws.close with teardown timeout

Sibling site to the disconnect() fix: go_dormant() still did an
unbounded await self._ws.close(), the exact same pattern bounded in
disconnect(). go_dormant runs on the scale-to-zero suspend path (Fly
autostop), which also has timeout constraints. Apply the same 1s
wait_for treatment using _TEARDOWN_AWAIT_TIMEOUT_S.

Found during review of PR #78027.

* fix: close the Codex app-server session on agent teardown

Salvage of #65260's b7d7cfd0e (ported — the PR's close() predates ~4K
commits of teardown-step churn, so the hunk is re-anchored after step
6b rather than cherry-picked).

agent/codex_runtime.py already drops _codex_session on turn crash and
on retirement, but AIAgent.close() — the hard teardown for /new,
/reset, and session expiry — had no owner for it, so the app-server
child process survived until interpreter exit. Long-lived gateways
accumulate one leaked subprocess per ended Codex session.

The attribute is cleared BEFORE close() so a concurrent reader can't
observe a half-closed session and a raising close() can't strand a
stale reference (tested).

Tests extend the author's original lifecycle test with the
raising-close and no-codex-session cases.

* fix(agent): discard bare tool-call marker before fallback/persistence (#78148)

Local tool-call templates can emit a bare bracketed token (e.g. "[memory]")
as assistant content alongside a function call. The loop treated that
protocol scaffolding as visible content: it got cached as the post-tool
fallback, and when the next turn came back empty, the marker was replayed
as the final response and written into the persisted transcript. Later
context compaction preserved that history, letting the model repeat the
marker in subsequent turns.

Detect content that is only a bracketed marker (`[name]`) when the
response also carries tool_calls, and drop it before it can be cached
or persisted. Scoped narrowly: only fires alongside tool_calls, so a
genuine final response of "[memory]" without a tool call is unaffected.

* fix(agent): repair sessions already contaminated with stale tool-call markers (#78148)

The conversation_loop fix (previous commit) stops new "[memory]"-style
bare tool-call markers from being cached/persisted, but sessions written
before that fix can still carry rows where a bare marker was saved as
the assistant's "final response".

Add a load-on-read repair pass in hermes_state.py, mirroring the existing
_strip_background_review_harness defense-in-depth: on session restore,
any assistant row whose content is only a bracketed marker (e.g.
"[memory]", "[skill_manage]") AND that carries tool_calls has its content
blanked before the history re-enters the model's context. The tool call
and its result are left untouched so provider tool_call/tool_result
pairing stays intact. Sessions with no affected rows pass through the
normal path unchanged.

* feat(cli): add sessions clean-markers to permanently purge stale tool-call markers (#78148)

The load-on-read repair (_strip_stale_tool_call_markers) fixes affected
sessions in memory on every resume, but never touches the DB — long-lived
sessions re-scan and re-repair the same rows on every load, and the
contaminated bytes stay in state.db (and any backup/cache snapshot of it)
indefinitely.

Add SessionDB.purge_stale_tool_call_markers(dry_run=False): a one-time,
idempotent UPDATE that permanently blanks the content column on affected
rows. Only content is touched — tool_calls and every other column are
left untouched, so provider tool_call/tool_result pairing survives.
dry_run reads through the no-lock read path and never writes.

Wire it up as `hermes sessions clean-markers [--dry-run]`, mirroring the
existing optimize/repair subcommands. Verified end-to-end against a real
temp state.db: dry-run reports the row without writing, the real run
clears it and preserves tool_calls, and a second run is a no-op.

* fix(cli): back up state.db before clean-markers writes by default

purge_stale_tool_call_markers ran a permanent, irreversible UPDATE with
no backup — inconsistent with repair_state_db_schema's backup-by-default
convention for destructive state.db operations elsewhere in this file.

Take a full snapshot via VACUUM INTO (safe against a live connection,
unlike the raw-copy _backup_db_file used for malformed-schema repair)
before the write, timestamped beside state.db. Skipped when dry_run or
when there's nothing to change. Add --no-backup to `hermes sessions
clean-markers`, mirroring `sessions repair`.

Verified end-to-end: the CLI run against a real temp state.db produces
the backup file before printing the cleared-row count.

* refactor: dedup stale-marker regex — use compiled _STALE_MARKER_RE in conversation_loop

The bracketed-marker regex was inlined in conversation_loop.py as
re.fullmatch(r"\[...", ...) while hermes_state.py defines the same
pattern as _STALE_TOOL_CALL_MARKER_RE. Both must agree on what counts
as a stale marker — a drift here means the runtime guard silently
disagrees with the load-on-read repair and CLI purge in hermes_state.

Consolidate onto a single compiled constant (_STALE_MARKER_RE) at
module level in conversation_loop.py, with a comment noting it must
mirror _STALE_TOOL_CALL_MARKER_RE in hermes_state.py. A direct import
from hermes_state was tried first but caused a regression: hermes_state
initializes DEFAULT_DB_PATH = get_hermes_home() / 'state.db' at module
import time, which breaks tests that monkeypatch get_hermes_home() to
return a str (test_slash_worker_accepts_profile_home).

Follow-up to PR #78175 (@JoaoMarcos44).

* fix(conversation_loop): compress messages on output-cap retry path (#55546)

The output-cap retry loop reduced max_tokens by 64 tokens per attempt but
never called _compress_context(), so the compressor never fired. Input
growth (~65 tokens/attempt) canceled the savings, leaving the session
stuck at 200,001 tokens — 1 over the 200,000 ceiling.

The fix adds compression to the output-cap retry path. The compressor
drops the middle window, freeing ~50% of tokens. If compression makes
>=5% savings, the session continues; otherwise vision payloads are
stripped or the session ends with compression_exhausted=True.

Also adds CHANGELOG.md entry and bug fix report.

* fix(conversation_loop): prune dead vision-strip fallback; harden output-cap retry tests

* chore: drop CHANGELOG.md and docs/reports/ — not shipped with salvage PRs

* perf: reuse request_input_estimate instead of recomputing estimate_request_tokens_rough

The output-cap error handler already computes request_input_estimate at
line 4722 via estimate_request_tokens_rough(api_messages, tools=...).
The new compression block ~50 lines below was calling the same function
with the same inputs again. Reuse the existing local.

* chore: AUTHOR_MAP — add BobClawblaw for PR #77870 salvage

Bare noreply email (no NNN+ prefix) needs explicit mapping.

* chore(ci): rerun checks

Signed-off-by: Bryan Bednarski <bbednarski@nvidia.com>

* perf(gateway): prewarm /model picker cache on TUI startup

The classic CLI run() loop calls prewarm_picker_cache_async() during the
idle window after the banner is shown, so the first /model open hits a warm
provider-models disk cache and renders in ~100ms. The stdio TUI entry point
never did this, so the first /model open in a TUI session blocked on serial
/v1/models fetches for every authenticated provider.

Mirror the CLI behaviour: kick off the same off-thread prewarm right after
gateway.ready is emitted (banner shown, user about to type). Fire-and-forget,
guarded once-per-process, fully exception-isolated so a slow or offline
provider can never affect TUI startup.

* test(tui): pin picker-cache prewarm wiring in entry.main()

teknium's review gap on #72021: the helper's worker/once-guard was
covered, but nothing asserted the stdio TUI entry point actually
invokes prewarm_picker_cache_async() — or that it does so in the right
place. Add a focused entrypoint test that runs the real entry.main()
with stubbed collaborators (same monkeypatch-module-attrs harness as
test_tui_entry_mcp_owner.py), spies on the helper in
hermes_cli.model_switch (the lazy-import source), and asserts:

- prewarm fires exactly once, strictly AFTER the gateway.ready write
- startup stays non-blocking: main() reaches the stdin loop and
  returns on EOF
- a prewarm failure is swallowed (fire-and-forget) without breaking
  startup

Mutation-checked: deleting the prewarm hunk from entry.py fails both
tests.

* perf(cli): check local auth.json/config before slow provider registry sweep

_has_any_provider_configured() probed every api_key provider (gh subprocess
for copilot alone takes 5s; full sweep ~18s) before consulting auth.json and
config.yaml, which are instant local reads. Desktop setup.status calls
blocked past the UI's timeout, causing the connect/disconnect boot loop.
Reorder so cheap local checks run first. Same semantics, ~35x faster here.

* test(cli): regression tests pinning auth-first ordering skips registry sweep

Teknium's review on #63457: existing tests pin the final boolean but not
that the slow PROVIDER_REGISTRY sweep is skipped. Add three tests that
booby-trap hermes_cli.auth.get_auth_status and verify
_has_any_provider_configured() short-circuits on:
- config.yaml model.provider
- config.yaml base_url/api_key (custom endpoint shape)
- auth.json active_provider (sweep-only call-pattern guard)

Mutation-checked: reverting the reorder makes all three fail.

* perf(desktop): keep spinner frames out of React commits

Advance the existing animated status glyph through its DOM text node instead of React state, and pause its timer for hidden panes or inactive windows. Cover frame advancement, zero update-phase commits, and timer suspension with behavior tests.

* test(desktop): cover minimized/hidden window-state + visibilitychange pause for GlyphSpinner

Regression coverage requested in review of #74357: mock
window.hermesDesktop.onWindowStateChanged (pattern from
persistent.test.tsx) and assert minimized/hidden clears the spinner
interval while restore resumes it; also cover document.visibilityState
hidden/visible via visibilitychange.

* fmt(js): `npm run fix` on merge (#78271)

Co-authored-by: github-actions[bot] <github-actions[bot]@users.noreply.github.com>

* perf(gateway): replace SSE poll loop with call_soon_threadsafe-fed asyncio.Queue

_write_sse_chat_completion and _write_sse_responses bridged their
stream_delta_callback queue into the event loop via
`await loop.run_in_executor(None, lambda: stream_q.get(timeout=0.5))`
in a while-True poll — a thread-pool round trip on every 0.5s tick even
when idle, plus up to 500ms of tail latency between a delta landing in
the queue and it reaching the SSE response.

Add ThreadSafeAsyncQueue (asyncio.Queue + a put_threadsafe() that wraps
call_soon_threadsafe), used by both streaming producer closures
(_on_delta, tool start/complete callbacks — all invoked from the worker
thread running run_conversation via loop.run_in_executor). Consumers
now do a plain `await asyncio.wait_for(stream_q.get(), timeout=0.5)` —
woken immediately when a delta arrives, no executor hop, no poll
interval.

Updated tests/gateway/test_sse_agent_cancel.py's 7 call sites to
construct ThreadSafeAsyncQueue inside the running loop (required, since
it captures asyncio.get_running_loop() at construction) instead of a
bare queue.Queue() at test-method scope.

* refactor(gateway): extract _sse_frame() helper, dedup 5 inline SSE encode call sites

_write_sse_chat_completion had five near-identical
f"data: {json.dumps(...)}\n\n".encode() (and one event-tagged variant)
scattered across its role/content/finish/error chunk writes. Pure
extract-method, no behavior change: encoding is byte-identical for every
call site touched.

Left the pre-serialized-string writers elsewhere (_write_event's
json.dumps(..., ensure_ascii=False) path, the /v1/runs SSE writer) alone
— routing them through this helper's plain json.dumps(data) would
silently change their unicode-escaping behavior, which is out of scope
for a pure dedup.

* refactor(gateway): route all three SSE writers through _sse_frame()

_extend _sse_frame with an explicit ensure_ascii param (default True,
byte-identical to a bare json.dumps) and route the two sibling writers
through it: _write_sse_responses._write_event and the /v1/runs event
stream. This completes the dedup PR #65009 — previously only the five
_write_sse_chat_completion sites used the helper, leaving the other two
writers on inline json.dumps with no shared shape.

No behavior change: every writer's emitted bytes are unchanged (verified
byte-for-byte, including non-ASCII payloads where the default
ensure_ascii=True matches the original inline encoders). The ensure_ascii
option is exposed so a future writer can opt into raw non-ASCII bytes
without fractalizing the format again.

Adds tests/gateway/test_sse_frame.py asserting the byte-contract
invariant between _sse_frame and the historical inline encoders.

* refactor(gateway): route session event stream through _sse_frame (ensure_ascii=False)

The session event stream (api_server.py:~2236) was the one genuinely
unicode-distinct SSE writer — json.dumps(payload, ensure_ascii=False) +
.encode('utf-8'). Every other writer uses plain json.dumps. Route it
through _sse_frame(..., ensure_ascii=False) so _sse_frame is now the single
source of truth for ALL SSE frame serialization in the module (chat-
completion, responses._write_event, /v1/runs, and the session stream).

Byte-identical for non-ASCII payloads: verified against the historical
inline encoder (raw bytes preserved). The ensure_ascii=False path is now
exercised by test_sse_frame_ensure_ascii_false_reproduces_session_event_stream.

* test: add cross-thread put_threadsafe + long-reasoning tail tests

Addresses teknium1 sweeper review (2026-07-30) requiring coverage of:

1. ThreadSafeAsyncQueue.put_threadsafe() off-loop boundary: a real
   daemon thread pushes into the queue from outside the owning event loop
   while the consumer awaits get(), mirroring the run_conversation
   worker-thread producer path. Includes a 20-concurrent-thread
   no-drop regression.

2. Long-reasoning bound stability for thinkingPreview: 100k-char input
   plus empty/collapsed cases must not crash and must retain the visible
   tail marker inside the bounded 24k clean window.

* fix: reconstruct fused test after conflict resolution

The conflict-marker strip fused test_agent_task_raises with the body of
test_failed_result_dict — restore both as separate tests (content from
the PR head, verified verbatim).

* perf(tui): bound reasoning-clean input to the displayed tail

CI caught a split defect in this salvage: the PR's long-reasoning tail
test was kept but its production hunk was dropped as 'cosmetics'. It
isn't — cleanThinkingText runs several full-string regex passes and
reasoning grows on every streamed token, so re-cleaning the whole
accumulated string per chunk is O(n) per token / O(n^2) per stream.
Only the tail is displayed (boundedLiveRenderText caps it downstream),
so bound the input to 1.5x LIVE_RENDER_MAX_CHARS first.

Restores the one text.ts hunk from cd99e65fc (author preserved); the
italic-thinking display change and profile script from that commit
remain out of scope.

* test: exercise the production _loop_ref path in put_threadsafe tests

Gate finding (/simplify-code pass): both cross-thread tests passed
loop=loop explicitly, but no production caller does — all six
(_on_delta, _on_tool_*) rely on the queue resolving its own
_loop_ref in __init__. The kwarg made the tests vacuous: a broken
_loop_ref still passed them.

Dropping the kwarg exercises the real path. Verified by mutation:
with self._loop_ref = asyncio.new_event_loop() (wrong loop), both
tests now FAIL; they passed before this change.

* fix(test): feed the SSE writers an asyncio queue, not queue.Queue

CI caught a missed caller-shape update. Both PRODUCTION callers of
_write_sse_chat_completion / _write_sse_responses were converted to
ThreadSafeAsyncQueue, but two pre-existing tests in
tests/gateway/test_api_server.py construct the writer's queue
themselves and still passed a stdlib queue.Queue.

The consumer now does 'await asyncio.wait_for(stream_q.get(), ...)',
which on a queue.Queue blocks the thread forever:
test_stream_cancelled_persists_incomplete_snapshot hung until
pytest-timeout killed it (CI reported the whole file as 'no tests
ran (timeout before collection)'). The sibling disconnect test only
survived because it pre-fills before the first await.

tests/gateway/test_api_server.py: 99 passed (was 1 failed + a 60s
hang); with the SSE/api_server suites: 147 passed.

* fix(credential-pool): re-select in acquire_lease after a deferred refresh

select() re-selects once deferred single-use-token refreshes complete;
acquire_lease() performed the refresh but returned its pre-refresh
answer. Since _acquire_lease_under_lock returns early exactly when a
refresh is pending (if not available: return None, pending_refresh),
a pool whose entries all needed a refresh always returned None — the
caller failed an answerable request right after the refresh succeeded.

Retry once, only when the first pass was empty and a refresh ran.

Post-merge gate-sweep finding on the #71775 salvage (#77714).

* fix(credential-pool): lock the quarantine read-modify-write of _entries

#71775 moved deferred single-use-token refreshes outside the pool lock
(correct — they hold a cross-process flock plus network I/O). But
_refresh_entry_impl's three terminal-auth-failure quarantine paths do a
bare read-modify-write of self._entries. Those used to run with the
caller holding self._lock; on the deferred path they run unlocked, so a
concurrent mutation between the read and the write is silently lost.

Wrap all three in 'with self._lock' (an RLock, so locked callers
re-enter safely) and correct the _refresh_pending_entries docstring,
which claimed the mutations were already self-locking.

Post-merge gate-sweep finding on the #71775 salvage (#77714).
Sibling to the acquire_lease re-select fix.

* perf(file-ops): eliminate redundant subprocess calls in write_file and V4A patch path

write_file currently spawns up to 6 subprocesses per call:
  1. mkdir -p (separate call before atomic write)
  2. cat (to read pre-content for lint/BOM/line-ending detection)
  3. _atomic_write (mktemp + write + mv — the essential one)
  4. wc -c (to measure bytes written)
  5. _check_lint_delta (post-write lint — also essential)
  6. LSP snapshot (also essential)

This PR removes three of them without changing any observable behavior:

1. Fold mkdir -p into _atomic_write shell script (−1 subprocess/write)
   The atomic write script already runs a single shell; adding mkdir -p
   to it costs zero extra processes.

2. Add optional pre_content parameter to write_file (−1 subprocess/patch)
   patch_replace and V4A _apply_update already read the file for fuzzy
   matching. Passing that content as pre_content skips the redundant cat
   inside write_file. Fully backward-compatible: callers that don't pass
   pre_content still read from disk as before.

3. Replace wc -c with len(content.encode('utf-8')) (−1 subprocess/write)
   We already have the content in memory; encoding it to get the byte count
   is equivalent to wc -c for UTF-8 text.

4. Remove redundant _check_lint loop in apply_v4a_operations (−N subprocesses/V4A)
   write_file already runs _check_lint_delta internally. The old code ran a
   bare _check_lint(f) loop over all modified files — a re-read + re-lint
   without post_content context. Now lint results propagate from write_file
   via a four-tuple return, zeroing out the extra subprocesses.

Net effect:
  - write_file: 6 → 3 subprocesses per call (new files)
  - patch_replace: 6 → 5 subprocesses per call (pre_content skips cat)
  - V4A multi-file patches: saves 1 subprocess per modified file
  - A typical 4-file V4A patch drops from ~28 to ~16 subprocess calls

* fix(file-ops): decouple BOM detection from pre_content, add V4A backward compat

Bug 1 (UTF-8 BOM loss on V4A UPDATE):
_file_has_bom() trusted pre_content for BOM detection, but the most
common pre_content provider — read_file_raw() — deliberately strips
BOMs so the agent never sees U+FEFF glyphs.  Passing BOM-stripped
content through pre_content caused a false-negative: the method
returned False and write_file() silently removed the marker on rewrite.

Fix: _file_has_bom() now always probes the first 3 bytes on disk
(head -c 3), ignoring pre_content for BOM purposes.  pre_content is
still used by two other consumers — line-ending detection and lint/LSP
delta computation — neither of which is affected by BOM stripping.

Bug 2 (backward compatibility):
_apply_update() called write_file(path, content, pre_content=...) as a
keyword argument.  Duck-typed file_ops implementations that only
implement the two-argument write_file(path, content) contract would
raise TypeError.

Fix: wrap the call in try/except TypeError, falling back to the
two-argument form when the keyword is not accepted.

Also declare tomli in pyproject.toml (pre-existing conditional import
for pre-3.11 Python, caught by the pre-commit dep scan after staging
file_operations.py).

Tests:
Add TestV4ABomRoundTrip with two cases:
  - UPDATE on BOM-bearing file preserves the marker
  - UPDATE on plain file does not inject a BOM

Addresses teknium1 review on PR #55661.

* fix(file-ops): surrogatepass in bytes_written encode (review finding)

Content that flowed through a surrogateescape decode (backend output via
patch_replace) can carry lone surrogates; a strict encode raises
UnicodeEncodeError where the old wc -c path could not. Mirrors the
existing sha256 verification encode.

* refactor(file-ops): fold simplify-pass findings

- write_file: encode content once, share bytes between bytes_written and
  the sha256 verification (drops a second full-content encode per write)
- patch_parser: replace the except-TypeError retry around
  write_file(pre_content=...) with signature-based feature detection so a
  TypeError raised inside a capable implementation propagates instead of
  triggering a duplicate write; tests for both duck-typing contracts
- tests: real-ops V4A BOM round-trip + _file_has_bom disk-probe guard
  (the teknium1-review regression previously only covered by a fake)
- comment: document dirs_created's long-standing "parent ensured" meaning

* chore: update uv.lock for tomli dependency (rebase fix)

* chore: remove dead tomli dependency declaration

requires-python is >=3.11 so tomllib is always in stdlib; the
tomli fallback branch in _lint_toml_inproc was unreachable. Removes
the dependency from pyproject.toml + uv.lock and deletes the dead
try/except ImportError fallback in the code.

* fix(telegram+sqlite): resolve polling conflict loop + misleading WAL warning

#75017: Telegram polling conflict retry used drop_pending_updates=False,
starting a new getUpdates session that immediately got 409'd by the
previous still-expiring session — creating the very conflict it was
trying to recover from. Switch to drop_pending_updates=True so Telegram
terminates stale sessions. Also add a recovery-generation guard so the
first transient getUpdates success after a retry doesn't reset the
conflict counter back to 0 (defense-in-depth from PR #75096).

#75153: The WAL-reset warning always said 'hermes update can repair'
even for git/pip/system Python installs where it can't. Now uses
detect_install_method() + recommended_update_command_for_method() to
give a context-appropriate hint (hermes update for git, docker pull for
docker, nix message for nix, generic install hint as fallback).

* chore: map jun@junho.co to junhohong

* fix(dashboard): reload loopback tabs after stale session-token closes

Loopback dashboard tabs now share one one-shot stale-token recovery path across REST 401s, the PTY socket, the structured event socket, and the shared JSON-RPC gateway wrapper. The shared client exposes only an optional close-event interception hook; the dashboard remains responsible for deciding that loopback 4401 means reload.

Constraint: Current main delegates the web gateway to apps/shared JsonRpcGatewayClient, and #54022 review requires a shared-client-compatible close-code hook plus direct ChatSidebar event-socket coverage.
Rejected: Restore the dashboard's old direct WebSocket implementation | stale against the shared JSON-RPC client and would duplicate transport behavior.
Confidence: high
Scope-risk: moderate
Directive: Keep stale-token policy dashboard-specific; the shared JSON-RPC client should expose close events without learning dashboard auth semantics.
Tested: npm --workspace web test (21 files, 106 tests); focused stale-token tests (5 files, 14 tests); npm --workspace web run typecheck; npm --workspace @hermes/shared run lint; npm --workspace @hermes/shared run typecheck; focused web eslint; git diff --check.
Not-tested: Manual browser smoke test across a real dashboard restart.

* fix: update ChatPage test import for react-router v7

react-router v7 exports MemoryRouter from 'react-router', not
'react-router-dom'. The test was written when the repo still imported
from 'react-router-dom' (4000+ commits ago).

* fix: wire HermesConsoleModal WS into stale-token reload guard

Sibling site missed by PR #54022 — /api/console WebSocket in
HermesConsoleModal.tsx has the same buildWsUrl → stale-token → 4401
close path as the PTY and events WebSockets. Without this guard,
opening the console after a dashboard restart shows 'Console closed
(4401). auth: token_mismatch' with no recovery.

* fix(xai): honor configured web search backend on Responses path

When Grok runs on xAI Responses, only swap to native server-side
web_search when the active/configured backend is xai. For Firecrawl
and other Hermes providers, keep client dispatch under a renamed wire
tool so Grok cannot hijack web_search and ignore user config.

* test(xai): cover Firecrawl vs native web_search on Responses

Lock in backend preference, wire-name aliasing, and normalize mapping
so configured non-xai search providers stay on the Hermes client path.
Also init conflict-recovery generation on the telegram bare-adapter
helper so CI polling progress tests do not AttributeError.

* refactor(xai): simplify _xai_prefers_native_web_search to use registry

Drop the manual web.search_backend / web.backend config-reading block
that duplicated _read_config_key in web_search_registry.py. The function
now delegates directly to get_active_search_provider() (which reads the
same config keys via the registry's canonical resolver) and falls back
to _get_search_backend() only when the registry has no providers loaded.

Also updates the TestXaiWebSearchBackendPreference tests to monkeypatch
the registry instead of load_config_readonly, and adds two new tests for
the legacy fallback path (no provider registered -> _get_search_backend).

* fix(relay): gate skipped task completion

Signed-off-by: Bryan Bednarski <bbednarski@nvidia.com>

* fix(model-switch): treat models dict as metadata, not allowlist

hermes model saves custom_providers models: {default: {context_length}} for
local Ollama. That dict shape was treated as an explicit catalog, so no-key
end…
nepenth pushed a commit to nepenth/hermes-agent that referenced this pull request Aug 7, 2026
…#38491)

Re-derivation of NousResearch#38491 by @stremtec onto current main (the original is
10,119 commits behind; the hook moved into ui-tui/src/app/). The hook
returned a fresh object literal every render, defeating memoization in
useMainApp's consumers; useMemo over the (all-useCallback-stable)
handles makes the return referentially stable.

Dep array covers ALL nine returned handles incl. trimTail (the
re-derivation initially omitted it - stale-closure class).
Eynzof added a commit to Eynzof/Hermes-CN-Core that referenced this pull request Aug 7, 2026
* test(dashboard): cover install-hook invalidation of plugins hub cache

* fix(dashboard): warm cold check_fn verdicts with a background probe

On dashboard-only sessions nothing else executes check_fn warmers (they
live only in the tool-schema build), so the hub's read-only cache lookup
would report auth_required=False forever. On a cache miss, schedule a
deduplicated daemon-thread probe off the request path; the short hub TTL
surfaces the verdict on the next fetch.

* fix(desktop): cancel the pending commit-cost measurement rAF

Follow-up to #77652: each runFlush registered a fresh requestAnimationFrame and never cancelled it. Chromium parks rAF callbacks for hidden renderers, so a long hidden stream at the 33ms floor accumulates thousands of parked closures that all fire in the first frame on refocus (all but one no-oping through the stale-frame guard). Track the pending handle, cancel it before requesting a new one (only the newest flush's measurement matters), and cancel on unmount.

* perf(dashboard): skip full InsightsEngine on /api/analytics/usage (#18511)

* perf(dashboard): keep tools in focused analytics usage (#18511)

* refactor(insights): drop consumer-less get_skill_breakdown alias (simplify-pass)

The 2-line alias had zero production consumers (web_server calls
get_usage_breakdown directly). Tests rewired onto the real API; the
contracts they pin are unchanged. Stale test docstring fixed.

* fix(web): clamp dashboard pagination and analytics-days params (#39200 + #74778 salvage)

Re-derivation of aydnOktay's twin clamp PRs onto current main (the
session-list endpoints moved into web_routers/; the analytics endpoints
gained asyncio.to_thread wrappers since the originals):

- limit le=100 on /api/sessions, /api/sessions/search and the
  /api/profiles/sessions fan-out (one unbounded request could drag every
  session row + correlated-subquery preview work out of SQLite, times
  every profile's state.db on the fan-out).
- days ge=1 le=365 on /api/analytics/usage + /api/analytics/models
  (huge or non-positive values force full-history InsightsEngine work or
  inverted windows; the UI only offers 7/30/90 presets).

FastAPI Query bounds reject at the validation layer (422). 8 new tests;
both clamp classes mutation-checked (clamp removed -> its tests fail).

* fix(clamps): raise profile fan-out limit to le=500 (simplify-pass finding)

le=100 would 422 real desktop callers: sessions-settings fetches
archived at limit=200, the command palette lists at 200, and the
electron remote-merge over-fetches limit+offset (exceeds 100 at
offset>=81, and its .catch(()=>null) silently drops remote sessions).
Clamp must sit above real client maxima. New test pins limit=200 w/
offset.

* fix(web): avoid blocking provider validation

* perf(plugins): seed plugin routes from sessionStorage cache for instant render

- Plugin manifests are now cached in sessionStorage on fetch.
- On refresh, plugin routes are registered synchronously from cache, preventing unwanted redirects to /sessions.
- Removes the !pluginsLoading guard from the catch-all route in App.tsx, as plugin routes are now always available on first render.
- Background fetch always updates the cache and routes, so new/removed plugins are reflected after reload.
- Resolves the race condition where plugin pages would redirect to /sessions on hard refresh.

* fix(plugins): validate cached manifests are an array

* test(plugins): export cache helpers and add focused fallback/refresh tests

* fix(plugins): keep loading gate when cached manifests include a /chat override

The sessionStorage seed set loading=false whenever any cache existed, which
defeats App.tsx's load-bearing pluginsLoading gate: with a cached manifest
that declares tab.override === "/chat", the persistent ChatPage host must
NOT mount before plugins resolve, or it spawns a PTY and gets yanked when
the override plugin takes over the route.

Seed loading=false from the cache only when no cached manifest overrides
/chat (canSeedLoadedFromCache); manifests are still seeded either way so
plugin routes register synchronously on refresh. Adds focused tests for
the gate, including the /chat-override case.

* perf(dashboard): serve hashed /assets bundles with immutable cache headers

Every hashed bundle chunk under /assets/ was served with no caching
directives, so each dashboard load re-fetched (or at best revalidated)
every JS/CSS chunk. Those filenames carry a Vite content hash — the
bytes behind a given URL can never change; a rebuild mints new
filenames referenced by a freshly served index.html.

Mark them Cache-Control: public, max-age=31536000, immutable:
- the /assets StaticFiles mount, via a subclass that stamps the header
  on 200s only (404s stay uncached — a rebuild can create the file),
- serve_css, preserving its X-Forwarded-Prefix url() rewrites for
  /fonts/, /fonts-terminal/, /ds-assets/, /assets/.

index.html keeps no-store, no-cache, must-revalidate — it is the
mutable entry point that binds users to the current hashes.

The original PR also added hand-rolled per-request gzip compression of
asset responses; that part is deliberately dropped. This server is a
localhost-default dashboard backend: compressing every response on the
CPU to save loopback bandwidth is a pessimization, and callers that
front it with a real proxy already get compression there.

Salvaged from PR #28543 (idea by @sea-monsters; gzip groups dropped as
described above).

Co-Authored-By: Claude Fable 5 <noreply@anthropic.com>

* feat: reset-aware primary restore — stay on fallback until the rate-limit window resets

restore_primary_runtime retries the primary every turn once the 60s
transient cooldown clears. For subscription-window limits (Claude
Pro/Max 5h windows, Codex weekly caps) the reset is hours or days away,
so every retry is a guaranteed failure costing two provider switches
and two prompt-cache invalidations per turn.

Add CredentialPool.next_available_at() (earliest reset across exhausted
entries; None when available now or no reset info) and gate the restore
on it: skip while the primary's pool says nobody can serve, restore on
the first turn after the reset elapses. Fail-open: any gate error or
missing reset info falls through to the existing per-turn retry, so
recovery can never be later than today. Cross-provider fallbacks
consult the PRIMARY's pool (not the attached fallback pool), reusing
the loaded pool for the existing rebind to keep auth reads at one per
restore.

* fix(credential_pool): run next_available_at under the pool lock

Review fold on the #67642 salvage: next_available_at() called
_available_entries() — which prunes DEAD entries, syncs tokens, and
persists — and iterated self._entries with no lock, racing concurrent
select()/rotation exactly as has_available()'s comment warns. Wrap the
method body in self._lock and pin it with a non-blocking-acquire probe
test.

* fix(credential_pool): defer single-use-token refresh outside threading lock

select() and acquire_lease() held self._lock during the entire
_available_entries() loop, which for openai-codex and xai-oauth providers
includes a cross-process file lock (_auth_store_lock) plus OAuth token
refresh HTTP POST.  The lock timeout can exceed 20 seconds, blocking all
credential pool consumers across every gateway thread and subagent.

Collect single-use-token refresh entries under the lock, then execute the
refreshes outside it.  On success the refreshed entry is merged back into
the pool and re-selected.  Non-single-use providers (anthropic, nous)
continue refreshing inside the lock since their refresh is a simple HTTP
POST with no cross-process coordination.

* fix(credential_pool): serialize deferred-refresh pool mutations

Review folds on the #71775 salvage (dossier findings 1+2):

- self._lock becomes an RLock and the mutation primitives
  (_replace_entry, _persist) are now self-locking, so the deferred
  single-use-token refresh path — which deliberately runs its
  cross-process flock + OAuth network I/O OUTSIDE the pool lock —
  still serializes its pool mutations against concurrent
  select()/rotation. In-lock callers re-acquire reentrantly.
- Dropped _refresh_pending_entries' redundant second _replace_entry:
  _refresh_entry already merges the refreshed entry internally.

Adds tests/agent/test_credential_pool_deferred_refresh.py pinning both
invariants: select() must NOT hold the lock during the refresh window
(the PR's whole point), and the post-refresh mutations MUST contend on
the lock (blocking-thread probe).

* `hermes sessions optimize-storage` aborts with

```
Error: optimization failed: no such table: messages_fts_trigram
No data was lost. Re-run to resume.
```

on any install where the trigram FTS index is legitimately absent. The failure is
deterministic — re-running can never make progress, because the crash happens at the same
point every time — so the database is permanently stuck on the legacy high-footprint FTS
layout with no supported way forward.

Observed on a 5.4 GB production `state.db`. After the fix the same database optimized
successfully and shrank to 3.3 GB.

The trigram index is absent whenever the runtime cannot maintain it. On a SQLite build
without the `trigram` tokenizer, `_ensure_fts_schema()` returns `False`, so `__init__`
leaves `self._trigram_available = False` and no `messages_fts_trigram` table on disk. This
is a **supported degraded runtime**, not damage — CJK/substring search falls back to
`LIKE` and everything else works normally. `_is_fts5_unavailable_error()` and
`_warn_trigram_unavailable()` exist specifically to make this path graceful.

Two code paths write the boundary sweep for the deferred FTS rebuild, and only one of them
respects that flag:

| Function | Trigram `INSERT` guarded? |
|---|---|
| `fts_rebuild_step()` | ✅ `if include_trigram:` where `include_trigram = self._trigram_available` |
| `_fts_rebuild_finish()` | ❌ unconditional |

`_fts_rebuild_finish()` runs the boundary sweep at the *end* of the backfill. Its
unguarded `INSERT INTO messages_fts_trigram …` raises `OperationalError`, which propagates
out of `optimize_fts_storage()` and aborts the entire optimization — *after* the backfill
has already completed. Hence the characteristic output showing 100% progress immediately
before the error:

```
Rebuilding index: 100% (909,671/909,671)
Error: optimization failed: no such table: messages_fts_trigram
```

There is a second, quieter consequence. The teardown phase that reclaims the demoted
`fts_v22_trash_*` shadow tables runs *after* the backfill phase in
`optimize_fts_storage()`. Because the crash happens before teardown is ever reached, those
tables are never emptied or dropped — so the space the migration was supposed to reclaim
stays allocated indefinitely, and the leftover trash tables look (misleadingly) like
evidence of a half-finished migration.

Build a populated v23 database, set the deferred-rebuild markers, then reopen it on a
runtime where `_ensure_fts_schema('messages_fts_trigram', …)` returns `False` (exactly
what a SQLite build without the trigram tokenizer produces) and call
`optimize_fts_storage()`:

```
[precondition] trigram absent, _trigram_available=False, rebuild pending  ✓

RED  ✗ optimize_fts_storage raised OperationalError: no such table: messages_fts_trigram
```

With this patch applied, unchanged harness:

```
     optimize_fts_storage returned {'ok': True, 'vacuumed': None}
GREEN ✓ optimize ok; markers cleared; base FTS 'zebra' -> 200 hits
```

Full harness and transcripts in `TEST-EVIDENCE.md`.

Gate the sweep on `self._trigram_available`, exactly as `fts_rebuild_step()` already does:

```python
include_trigram = self._trigram_available

def _do(conn):
    ...
    if include_trigram:
        conn.execute("INSERT INTO messages_fts_trigram(...) ...")
```

The base `messages_fts` sweep and the marker cleanup are untouched, so the rebuild still
finalizes correctly and the index remains complete for every row it is responsible for.
The fix does not disable or weaken search to dodge the error — the regression tests assert
that base FTS still returns results afterwards.

`TestFtsRebuildFinishWithoutTrigram` in `tests/test_hermes_state.py`:

- `test_rebuild_finish_skips_trigram_when_unavailable` — drives `_fts_rebuild_finish()`
  directly on a trigram-less runtime; asserts it completes, clears both rebuild markers,
  and leaves base FTS searchable.
- `test_optimize_fts_storage_succeeds_without_trigram` — end-to-end through the public
  `optimize_fts_storage()` entry point; asserts `ok=True`, markers cleared, search intact.

Both use the existing `_NoTrigramConnection` helper already in the file. Both fail on
`main` with `no such table: messages_fts_trigram` and pass with this patch.

`tests/test_hermes_state.py` passes in full (463 tests → 465 with these two). `ruff` clean.

This PR is the crash only.

A companion PR narrows `_db_opens_cleanly()` so that
`hermes sessions repair --check-only` stops reporting a write-broken FTS schema as
healthy — the gap that makes this class of problem hard to diagnose in the first place.
The two are independent and can land in either order.

* perf(tools): shrink lazy tool catalog overhead

* refactor(tool-search): drop dead fallback ladder in _available_source_summary

Simplify-pass finding: _listing_group_label already falls back to 'other' for empty source names, and _classify_source guarantees source_name=='' only when source=='other' — both  legs were dead by construction. Aligns the summary path's grouping with the listing path.

* fix(credential_pool): unpack the tuple in next_available_at's gate

Cross-PR interaction fix: #77714 (salvage of #71775) changed
_available_entries to return (available, pending_refresh) while #77631
(salvage of #67642) added next_available_at() which still truthiness-
tests the bare return. A non-empty tuple is always truthy — even
([], []) — so the reset-aware gate silently returned None ('no wait
info') for every exhausted pool, disabling the feature #77631 shipped.
Unpack the tuple and test the available list.

Also adapts the lock-probe test for the RLock introduced by #77714
(same-thread non-blocking acquire always succeeds on an RLock; probe
from a helper thread instead).

* chore: map copii.list@gmail.com to stremtec

* chore(contributors): map marzukia@users.noreply.github.com -> marzukia (#77774)

Needed for the #37117 salvage (#77696 CI failure).

* chore: map bot@bkstock.dev to BKStock

* perf(session): route SQLite PRAGMAs through central apply_database_pragmas

Addresses review from @teknium1 on PR #71755:

- Extended apply_database_pragmas() to handle cache_size, mmap_size,
  and temp_store from config.yaml (alongside existing wal_autocheckpoint
  and journal_size_limit). No hardcoded defaults — all values are
  opt-in via config.yaml, avoiding policy conflicts with other PRs.
- Applied to ALL connection types: writer (_connect_and_init),
  read_only cross-profile attach, and WAL per-thread readers
  (_get_read_conn). Previously PRAGMAs only ran on the writer path.
- Removed inline PRAGMAs from _connect_and_init — single source of
  truth in apply_database_pragmas().
- Documented config keys with examples in function docstring.

* fix(pr): remove remnant local PRAGMAs from PR branch

* test(session): guard config-gated performance PRAGMAs across all connection types

E2E guard for the salvaged PR #71755: database.cache_size/mmap_size/
temp_store from config.yaml must reach the writer connection, the
read-only cross-profile attach, and the WAL per-thread reader — and a
default install (no database: keys) must keep byte-identical SQLite
defaults on every connection type. Also covers integer-coercion
rejection of garbage values for the three new keys.

cache_size uses -16000 (not the doc example -2000) because -2000 is
SQLite's compiled-in default and would not discriminate a regression.

* fix(desktop): flush queued deltas on window focus

* perf(desktop): stop scroll and status loops in busy sessions

* perf(desktop): pause hidden-pane timers in agents view, cron sidebar, and floating pet

Partial pick of the surviving renderer hunks from #75395 (perf commit
6502e441d plus fixup 3fbbc9c1d): gate the 500ms subagent now-ticker and
the cron sidebar 1s ticker/run-poll on usePaneVisible, and skip the
legacy floating-pet poll while the document is hidden. Dropped hunks
(electron/main.ts, vitest.setup.ts/config) intentionally excluded.

* style(desktop): restore alphabetical import order in agents/index.tsx

* refactor(desktop): shared pulse beat + fully-gated cron peek (simplify folds)

Two findings from the simplify pass on the final trio diff:

- status-pulse: one pause controller + one aligned period timer shared by all StatusPulse instances (ref-counted), instead of N x (document/window/bridge listeners + unsynchronized 5s wakes) — a sidebar can show dozens of pulsing dots. Pause still cancels in-flight animations so the compositor sleeps immediately.
- cron-jobs-section: the runs-peek effect created its interval even while the pane was hidden (callback no-oped but the timer still woke the renderer every 8s/60s per expanded job). Early-return when hidden — visibility is already in the dep array, so becoming visible restarts load + timer.

* fix(lint): import sort + eslint-disable for timer-handle ref clear in effect

CI-caught: cron-jobs-section had an extra blank line between sorted imports; use-message-stream's visibility-flush effect assigns flushHandleRef.current=null inside a useEffect (legitimate timer-clear, not an atom mirror) — eslint-disable-next-line per the rule's documented convention.

* fix(state): narrow FTS UPDATE triggers with AFTER UPDATE OF + migration

Retarget #73639 onto the SessionDB mixin split (hermes_state_common /
hermes_state_schema). Fresh installs create UPDATE OF content/tool_*
triggers; existing broad AFTER UPDATE triggers are inspected and
replaced under schema init without an FTS rebuild (WHEN clauses already
guarded content correctness; OF skips non-content status writes that
saturated disk I/O on large state.db).

Tests: tests/test_fts_update_of_narrowing.py (4)

* fix(state): fail closed on CJK trigger migration

* fix(state): quarantine CJK when ensure soft-fails after OF migration

_ensure_fts_cjk_schema never raises on OperationalError; post-condition
after dropping messages_fts_cjk_update now requires a narrowed UPDATE
trigger or durable fts_cjk_stale + unavailable. Covers the production
soft-fail path the raise-only handler missed.

* refactor(state): drop unreachable regex guard in trigger migration

Simplify-pass fold: to_drop names come from the literal update_names\nallowlist via IN binding, so the [A-Za-z0-9_]+ fullmatch could never\nfail — and if it somehow did, its `continue` would miscount (the\nskipped trigger stayed in len(to_drop)/the log while CREATE TRIGGER\nIF NOT EXISTS silently kept the broad variant). Delete the guard and\nits function-local re import; keep the invariant as a comment.

* fix(security): reject always-blocked OpenViking endpoints

## Summary
- Normalize OpenViking endpoints through `is_always_blocked_url` and fall back to the default local endpoint when poisoned.
- Keep intentional loopback / LAN self-host working.
- Add focused unit tests.

## Salvage / credit
Memory-provider endpoint floor sibling of RetainDB/Supermemory always-blocked hardening (avoids over-broad #4984-style private-IP bans).

(cherry picked from commit 8fa607d0aedb8c5fca398d7f112b1b25ade54fa2)

* fix(openviking): fail closed on blocked endpoints

(cherry picked from commit 389a90b81c9c2c89810f2fa7461f8faa9a5c9578)

* fix(openviking): don't spawn a second server onto a live port

`_start_local_openviking_server()` spawned `openviking-server`
unconditionally. Both callers — `initialize()` and the runtime
unreachable handler — reach it from a health probe, and that probe can
time out client-side while the server is up and serving. The spawned
process then loses the data-directory lock and exits immediately with
`DataDirectoryLocked`; because the probe keeps timing out, the cycle
repeats every cooldown window (~5 min observed).

The existing 30s `_failed_refresh` cooldown paces the loop but cannot
stop it, since it expires while the underlying condition persists.

Probe the target host:port before spawning and treat an occupied port as
already-started. This guards both call sites at their single convergence
point. The probe deliberately tests only that a listener owns the port —
enough to know a second server would lose the lock — and says nothing
about that listener's health.

The parse/probe now precedes the PATH lookup, so a reachable server is
reported as running even when `openviking-server` is not on PATH.

Fixes #74846

(cherry picked from commit b49427d85fd6628eb4a7fe099e5c390c5c4cc935)

* fix(openviking): drop stale "disabled for this Hermes run" warnings

The provider used to disable OpenViking permanently when the server was
unreachable. That was fixed: `_ensure_client()` now reconnects lazily,
with a 30s cooldown gate in `_ensure_client_locked`.

Only one of the seven user-facing warnings was updated to match. The
other six still told the user memory was "disabled for this Hermes run",
which is no longer true — every one of those paths is retried on the next
access. A user who reads the old message has no reason to retry, which is
very likely how #5721 ("never recovers") came to be filed against
behaviour that already recovers.

All six sites were traced to confirm none is terminal for the run: the
`initialize()`-time and waiter-thread failures never arm `_failed_refresh`
(only line 2439 does), so they retry on the very next access with no
cooldown at all.

The replacement wording deliberately omits the "(after cooldown)"
parenthetical used at the already-correct site — that detail is only
accurate where `_failed_refresh` was just armed. The neutral phrasing is
true at all six.

Also promotes two clause separators to periods to avoid "…; …disabled;"
collisions.

(cherry picked from commit 8346403a4b97af503d26b0f7905ff513828d821e)

* fix(openviking): re-arm the commit guard after in-place compression

`_committed_session_ids` is a permanent per-sid latch, and
`_session_needs_commit` checks it before the turn counter by design — a
racing sync_turn can re-increment `_turn_count` after commit+reset, so
the guard must win to stop a double-commit.

That is correct for a session being left behind. It is wrong for one
that keeps its id. `compress_context()` commits before rewriting the
transcript in both modes, and with `compression.in_place: true` (the
default) `on_session_switch` receives the same id and does not rotate.
The latch then rejects every later commit for a still-live session — the
next compression, /new, normal session end, startup recovery — so every
post-compression turn is silently never extracted.

Rotation mode is unaffected because a fresh child id is minted and
starts clean, which is what confirms the latch's intent was only ever to
dedupe the departing id.

Clear the latch when compression completes without rotation. Turns
arriving after that point are genuinely new, and this is a defined
moment rather than a race. The rotation path is untouched, so the old
id stays latched and its _finalize_session_async still dedupes against
the compression commit.

Fixes #74695

(cherry picked from commit d1e5c3dc33ef0d43d021662674e1a7cd5e43eecd)

* test(openviking): cover the compression lifecycle, not a hand-set latch

Review feedback: the previous test called _mark_session_committed
directly, so it verified the guard's behavior but not the wiring that
sets it — a future break in the commit_memory_session -> same-id
compression-boundary path would not be caught.

Add a lifecycle regression that drives the real sequence: on_session_end
commits through the actual path, on_session_switch(same id,
reason="compression") crosses the boundary, sync_turn records a genuinely
new turn, and a second on_session_end must produce a second commit POST.

Without the fix it fails showing exactly one commit call, which is the
reported data loss: every turn after the first compression is dropped.
The rotation and /undo tests stay as scope guards.

(cherry picked from commit 0ca5a330630a30b105cbbc32e8a23f2c5ffe0eab)

* fix(memory): read non-secret provider config from config.yaml for OpenViking and RetainDB

OpenViking is_available() only consulted env vars and use_ovcli_config, so an
endpoint saved to config.yaml (e.g. by the Dashboard) reported needs_config;
_resolve_connection_settings() likewise never folded config.yaml's non-secret
fields into its chain. RetainDB initialize() read base_url/project from the
environment only, ignoring the values the Dashboard writes to config.yaml.

Both now resolve non-secret fields as env -> (ovcli ->) config.yaml -> default;
secrets still come from the environment. Adds regression tests for both.

Fixes #68209

(cherry picked from commit dca57915b97b5705b30927a062e1d0f2f23d3841)

* fix(openviking): read recall settings from config.yaml first, env vars as fallback

_recall_config() previously read all settings (recall_limit, score_threshold,
recall_resources, etc.) exclusively from environment variables. This forced
users to store behavioural configuration in .env, violating the Hermes
convention that .env is for secrets only.

The infrastructure to load config.yaml -> memory.openviking was already in
place via _load_hermes_openviking_config(), but _recall_config() never
called it.

Fix: call _load_hermes_openviking_config() and pass its values as the
default parameter to _env_int/_env_float/_env_bool. Env vars still override
config.yaml values, preserving backward compatibility.

Closes #62540

(cherry picked from commit 6aadf1256835745e0302aa3d3b5ae0660b368637)

* test(openviking): cover config.yaml recall settings with temp-HERMES_HOME tests

Add three tests to TestOpenVikingConfigSchema:

1. test_recall_config_reads_from_config_yaml — writes memory.openviking
   settings in config.yaml and verifies _recall_config() consumes them.

2. test_recall_config_env_overrides_config_yaml — writes both config.yaml
   and OPENVIKING_RECALL_* env vars, verifies env takes precedence.

3. test_recall_config_partial_config_yaml — partially populated config.yaml
   falls back to defaults for omitted keys.

All 46 openviking_plugin tests pass (43 existing + 3 new).

(cherry picked from commit b8d7834caf06c6912004333c270faa248eaed4cd)

* fix(openviking): integrate reliability and configuration hardening

* chore(contributors): map OpenViking source authors

* test(retaindb): guard scoped secret config resolution

* fix(openviking): verify servers before sending credentials

* fix(openviking): catch endpoint errors in setup validation functions

Review follow-up for salvaged PR #76782. Three setup-wizard
validation functions called _normalize_openviking_url outside their
try/except blocks. Since _normalize_openviking_url now raises
_OpenVikingEndpointError for blocked or malformed endpoints, an
invalid endpoint would crash the wizard instead of returning a
friendly (False, message) tuple.

- _validate_openviking_auth: move _normalize_openviking_url inside try
- _validate_openviking_root_access: same
- _validate_openviking_setup_values: catch _OpenVikingEndpointError explicitly
- Remove dead ternary in _normalize_openviking_url safety check (candidate
  always has http/https scheme by that point)
- Replace redundant float('-inf') < x < float('inf') with math.isfinite()
  in _setting_float; drop the redundant infinity check from _setting_int
  (is_integer() already rejects inf/nan)

* fix(state): deduplicate session system prompts

* chore: map cicav legacy noreply email

* fix(tui): avoid writable Kanban opens on empty polls

* fix(context): dedupe subdirectory hints by content digest and skip backup/vendor dirs

SubdirectoryHintTracker re-injected identical context files whenever the same
AGENTS.md was reachable through more than one path. Symlinked shared
workspaces, hardlinks, and timestamped backup copies all alias a single file,
so a normal session could ship the same 8KB of instructions two or three
times. Nothing deduped it and nothing excluded directories that only ever
hold copies.

Two changes:

* Track a sha256 of every injected hint body. Repeat content is skipped, and
  the working directory's own context file is seeded at construction so the
  copy prompt_builder already loaded at startup is never sent again.
* Skip directories that hold copies rather than authoritative context
  (backups, node_modules, venv, site-packages, .git, .Trash, vendor, caches).
  Screening is relative to working_dir, so a project that legitimately lives
  under vendor/ keeps discovering its own subdirectory hints.

Measured on a real session that touched a symlinked shared workspace:
3 injections / ~24,000 chars before, 1 injection / 8,112 chars after.

14 new tests cover symlink aliasing, byte-identical copies, working-dir
seeding, distinct content still being injected, each excluded directory name,
excluded ancestors, and the working-dir-inside-excluded-name case.

* perf(state): batch the turn flush into one SQLite transaction

Re-derivation of #23254 (@devsart95) on today's flush loop. The turn
flush in _flush_messages_to_session_db wrote one BEGIN IMMEDIATE
transaction per message row; a typical agent turn (user + assistant +
tool results) paid 3-8 transactions -- and, off WAL (the default on
macOS while the WAL-reset guard is active), 3-8 fsyncs -- per turn.

Adds SessionDB.append_messages_batch: same row shape as append_message
(shared _prepare_message_row serializer + _MESSAGE_INSERT_SQL column
list, so the two writers cannot drift), same compression-lock and
compression-closed guards, one aggregated session-counter UPDATE, one
transaction for the whole batch. Row serialization stays outside the
write lock.

The flush loop now collects the turn's new rows and writes them in one
call. All-or-nothing pairs exactly with the persisted-marker stamping:
on failure no rows landed and no markers were stamped, so the next
flush re-writes the whole tail (same recovery contract as before,
minus the partial-prefix case that could double-count).

Measured (same harness, 5-message turn, journal_mode=DELETE,
synchronous=FULL): 2.32ms -> 0.83ms median per turn flush (64% faster,
5 fsyncs -> 1). On WAL the win is smaller but the atomicity fix holds.

* perf(tui-gateway): batch branch-seed history copies (whole-bug-class)

Sibling sites of the per-message flush pattern: both branch-seed
paths (session.branch in methods_session.py and the lazy seed persist
in server.py) copied the parent history row-by-row -- one transaction
per row, and a branch seed can be hundreds of rows. Route both through
SessionDB.append_messages_batch. The server.py path also gains real
atomicity: _branch_seed_persisted assumed every row landed, which the
per-row loop could not guarantee.

* test(run-agent): update flush-path fakes and assertions for batched writes

The flush now goes through append_messages_batch; MagicMock-based
assertions and barrier fakes that hooked append_message observed
nothing (the flush's try/except swallowed the AttributeError). Assert
on the batch payload instead.

* refactor(state): fold simplify findings — reuse _insert_message_rows, share guards, chunk seeds

Simplify-pass folds on the #23254 salvage:

- REUSE (HIGH): append_messages_batch now delegates row serialization to
  the pre-existing _insert_message_rows helper (already shared by
  replace_messages / archive_and_compact / portability import) instead
  of adding a third serialization path (_prepare_message_row +
  _MESSAGE_INSERT_SQL are gone). One row-writer for every multi-row
  path; the row-ID return was consumed by no production caller, so the
  batch returns the inserted count.

- QUALITY (HIGH): the compression-lock + compression-closed admission
  guards are extracted into _check_transcript_write_guards, shared by
  append_message and append_messages_batch (previously duplicated 23
  lines that had already needed targeted fixes, #74478). The role-gated
  reasoning filtering is no longer duplicated in run_agent.py — it
  lives at its one site inside _insert_message_rows.

- EFFICIENCY (MEDIUM, measured): unbounded seed copies hold one BEGIN
  IMMEDIATE for seconds (10k rows ~= 2.4s; FTS triggers dominate) and
  monopolize the in-process write lock. append_messages_batch grows a
  chunk_rows param; all seed/copy call sites use chunk_rows=500. Same
  recovery semantics as the old per-row loops, bounded lock holds.

- REUSE (MEDIUM): the two remaining per-row branch-copy loops found by
  the pass (gateway/slash_commands.py /branch, hermes_cli
  cli_commands_mixin.py branch) are converted to chunked batches too
  (AsyncSessionDB's generic to_thread forwarder covers the async site).

Turn-flush benchmark unchanged after the refactor: 2.43 -> 0.87 ms
median per 5-message flush (64% faster).

* fix(tests): update two more append_message.call_args assertions to append_messages_batch

CI-caught: test_verification_stop_caching and test_tui_gateway_server::test_native_vision_turn_persists_a_renderable_image_ref both assert on append_message.call_args, but the flush loop now calls append_messages_batch. Same class of test-fake fallout fixed in 5 other files — these two were missed.

* perf(tui): memoize useSessionLifecycle return (idea from #38491)

Re-derivation of #38491 by @stremtec onto current main (the original is
10,119 commits behind; the hook moved into ui-tui/src/app/). The hook
returned a fresh object literal every render, defeating memoization in
useMainApp's consumers; useMemo over the (all-useCallback-stable)
handles makes the return referentially stable.

Dep array covers ALL nine returned handles incl. trimTail (the
re-derivation initially omitted it - stale-closure class).

* ci: retry uv python install

* fix(state): route session-resume reads through the WAL read-only connection

get_messages_as_conversation, get_resume_conversations, and
get_ancestor_display_prefix still took self._lock — the same global
choke point the read-path split (WAL per-thread read-only connections)
was meant to remove from every recall/browse read. These three are the
hottest reads in the file: every session resume across the gateway,
CLI, and ACP adapter goes through one of them, so a resume racing a
burst of concurrent-session writer flushes still convoys behind them
exactly like the fixed paths used to.

_session_lineage_root_to_tip (the lineage walk shared by all three,
plus get_conversation_root) had its own independent self._lock use and
needed the same conversion — without it the outer functions still
blocked on the very first line.

Verified empirically: a reader thread calling all three functions
while another thread holds self._lock blocked for the writer's full
hold duration before the fix, and returned immediately after (SQLite
3.50.4 in this dev venv falls back to journal_mode=DELETE per the
WAL-reset-bug guard, so the requires_wal-marked regression test is
exercised via a local WAL-forced script instead; it still runs and
passes on any runtime where WAL is actually active).

* chore: add contributor email mapping for ArcherQAQ

* fix(model_metadata): rewrite localhost->IPv4 for the remaining local probe sites

fetch_endpoint_model_metadata's generic (non-LM-Studio) /models fetch and
its llama.cpp /v1/props context-length follow-up built request URLs
straight from the unrewritten candidate, unlike every other local-probe
site. Both retained the multi-second dual-stack IPv6 connect penalty
that _localhost_to_ipv4() exists to skip (measured on macOS: localhost
32.9ms vs 127.0.0.1 0.1ms on a dead port; ~2s on Windows). normalized
stays the cache key so caching behavior is unchanged; only the outbound
request target is rewritten.

Re-derived from PR #61528 onto current main (original no longer applied
cleanly).

* fix(model_metadata): guard _localhost_to_ipv4 against non-string urls

CI slice 3/7 failures: run_conversation tests pass MagicMock base_urls
through the metadata probe path; re.sub raised TypeError where the old
code let non-strings flow through. Preserve that contract.

* perf(cold-start): mitigate ~14s GIL stall during backend init (#60800)

Three fixes for the Desktop/TUI cold-start stall where the event loop
is blocked for ~14s between HERMES_BACKEND_READY and the first
prompt (#60800):

1. copilot_auth: skip  subprocess fallback when any
   Copilot env var is explicitly set (even if invalid). The user
   expressed token intent via env var; silently substituting a CLI
   token is surprising and the subprocess adds up to 5s on Windows.

2. tui_gateway/ws: run resolve_skin() via asyncio.to_thread so config
   loading + skin engine init do not block the WS read loop during
   the cold-start RPC burst.

3. web_server: extend _warm_gateway_module to pre-import the heavy
   module chains (auth, copilot_auth, runtime_provider, skin_engine,
   inventory, model_switch) that the first WS connection + RPC burst
   would otherwise import on the loop thread. These trigger .pyc
   compilation and Defender scans on Windows (15-30s per the existing
   comment) and were not covered by the original gateway-only warm.

Tests: 5 new tests in test_cold_start_gil_stall.py + 2 new tests in
test_copilot_auth.py. All 36 copilot_auth tests + 16 ws/web_server
tests pass.

* test: harden cold-start regression tests + debug-log the env-var skip

Review folds on the #60807 salvage:
- resolve_skin tests are behavioral (thread-ident probe + ready-frame
  wiring check) instead of pure source inspection, per the #72720
  pattern; a source assertion remains as belt-and-braces.
- The warm-list test does REAL imports and checks sys.modules —
  _warm_gateway_module swallows ImportError by design, so the PR's
  tracking-stub test would pass even with a typo'd module name.
- resolve_copilot_token logs a debug line when the env-var
  short-circuit skips the gh-CLI fallback (behavioral change made
  observable).

* perf(gateway): per-platform skip_context_files to cut agent build latency

Salvage of #26860 (hunk 2, ported \u2014 the PR's base predates the current
gateway layout by ~11.9K commits). Messaging platforms can set
gateway.platforms.<key>.skip_context_files: true to skip the
filesystem-heavy context-file discovery (SOUL.md, AGENTS.md,
.cursorrules walks) during AIAgent construction \u2014 10-100x slower
stat()/walk costs on Windows made this a real per-turn tax. Soul
identity is still loaded (single small file), so the persona survives.

The flag participates in _agent_config_signature so toggling it
rebuilds the cached agent instead of silently reusing a prompt built
under the other setting (prompt-cache correctness).

The PR's hunk 1 (mtime-caching the per-turn dotenv reload) was dropped:
df51ad797 mtime-cached load_config/read_raw_config and c2eda92fd
removed the per-turn deepcopies, capturing most of that win; the
function has since gained a multiplex early-return and managed-scope
overlay that the original whole-function skip would have bypassed.

* fix(relay): route Discord tool-progress into the auto-thread, not the parent channel (#77830)

When a Discord channel message initiates a relay auto-thread, the thread does
not exist at ingest (source.thread_id is None) — the connector creates it on
its FIRST send and auto-threads any outbound carrying the reply anchor. The
final reply carries that anchor, so it lands in the thread. But the
tool-progress / status bubbles (the "Searching the web for..." updates and the
streaming preamble) were sent with _progress_metadata=None and
_progress_reply_to=None: _resolve_progress_thread_id returns None for Discord
(only slack/mattermost get a synthetic thread), so the progress send had no
anchor and the connector posted it FLAT in the parent channel. Result: the
search-status updates leaked outside the thread while the answer threaded
(staging repro 2026-08-02).

The connector now stamps prospective_thread_id on the inbound (the anchor
message id == the id of the thread it will create). Reuse it: when a
relay-delivered Discord channel-initiate carries prospective_thread_id and has
no real thread yet, carry the reply anchor (event_message_id) on both the
progress metadata (reply_to_message_id) and the progress reply_to, so the
connector routes the progress bubble into the SAME auto-thread as the final
reply. Applied to both the tool-progress path (_progress_metadata /
_progress_reply_to) and the status/interim callback path
(_status_thread_metadata). Events already arriving in a real thread, DMs, and
non-relay sources are untouched (guarded on delivered_via_upstream_relay +
prospective_thread_id + not thread_id).

Tests: two new cases in test_run_progress_topics.py — a relay Discord
channel-initiate asserts every progress send carries the anchor (reply_to +
metadata.reply_to_message_id + non_conversational), and an event already in a
real thread asserts the synthetic-anchor path does NOT engage. Full gateway
progress + relay + session suites green (228 passed).

* fix(agent): stop re-probing endpoints that blackhole TCP connects

Salvage of #71282 (Fixes #71281): a routable-but-dead endpoint (corp
LAN address while off-VPN) blackholes TCP SYNs, so every probe in the
model-metadata waterfall waits out its full connect timeout — 20+
seconds of stall per startup across detect_local_server_type,
fetch_endpoint_model_metadata, and the per-model probes.

A module-level blackhole cache keyed on host:port is populated when
any probe observes a ConnectTimeout (httpx or requests; read timeouts
deliberately excluded — an accepted connection is not a blackhole) and
consulted at the top of each guarded function. 30s TTL: long enough to
collapse one startup burst, short enough that VPN recovery is picked
up without a restart. Guard ordering: blackhole check -> disk L2 ->
HTTP waterfall, and a blackholed leg aborts the remaining legs instead
of letting each stall in turn.

Squash of the PR's two real commits (the branch's merge commits made
it un-rebase-merge-able; content verified identical via merge-tree).

* chore: release v0.20.0 (2026.8.3)

The Herald Release — voice (streaming TTS, barge-in, wake words), A2A v1.0,
outbound webhooks, grounded citations, desktop platform wave. ~3,650 commits,
~1,400 PRs, ~1,200 issues closed, 650+ contributors since v0.19.0.

Also: contributor audit additions (18 email mappings, bot-filter widening).

* chore: add contributor email mapping for Ahmett101

* perf(moa): cache resolved preset + per-slot runtime to cut cold-start latency (#66793)

* fix(discord): leave voice channels before cancelling the bot task

`DiscordAdapter.disconnect()` cancelled the bot task before tearing down voice
clients. `leave_voice_channel()` ends in `await vc.disconnect()`, and discord.py
sends a voice state update over the main gateway websocket and then waits for the
voice socket to close. The bot task is the loop running that gateway connection,
so cancelling it first left the handshake with no transport: it could never
complete and blocked until the caller's shutdown timeout fired.

The effect was a fixed ~5s penalty on every shutdown with a voice connection
open, ending in "discord disconnect timed out after 5.0s - forcing continue",
with the voice disconnect abandoned rather than completed.

Measured on a live gateway with a voice connection open in both cases:

  before: timed out after 5.0s, all adapters disconnected at +5.29s
  after:  discord disconnected (0.12s), all adapters disconnected at +0.46s

Moving the voice-cleanup loop above `_cancel_bot_task()` preserves the
zombie-client protection its comment describes: the bot task is still cancelled
before `client.close()`, just after voice teardown rather than before it. Voice
teardown is the one step that still requires a live gateway.

Adds a regression test asserting the ordering. It fails on the previous ordering
at index 1 with `cancel_bot_task != leave_voice_channel:111`.

Fixes #76044

* feat(image): parallelize image_generate batches

* fix(file-sync): serialize concurrent sync cycles

* fix(tool-executor): unpack 5-tuple runnable_calls in _max_workers_for_tool_batch

* fix: exponential backoff for rate-limit fallback cooldown

Replace the fixed 60-second cooldown with exponential backoff:
30min → 1h → 2h → 4h cap.

The counter is reset by restore_primary_runtime on successful
primary-provider recovery, so the backoff is strictly for
consecutive failures within a single degradation window.

Closes #29702

* fix(backoff): keep 60s first-hit cooldown, escalate only on consecutive rate-limits

Review follow-up on the #30223 salvage: the original changed the base
cooldown from 60s to 1800s, benching the primary for 30 minutes on the
FIRST 429 (30x regression in primary-restore latency) and breaking the
existing test_rate_limit_exhaustion_keeps_60s_cooldown contract.

Keep upstream's 60s base and escalate per consecutive rate-limit:
60s -> 2m -> 4m -> 8m -> ... capped at 4h. Counter still resets on
successful primary restore (cicae's mechanism, unchanged).

New tests: escalation doubling, 14400s cap, reset-on-restore.
Existing 60s contract test passes UNCHANGED. Mutation-checked:
escalation disabled -> 2 fail; reset disabled -> 1 fails.

* fix(catalog): wire api_key auth headers for http MCP servers

When an optional-mcps manifest declares transport.type=http with
auth.type=api_key, install_entry() prompts for the key and saves it to
.env, but _build_server_config() only handled the oauth case — the
api_key case produced a bare url entry with no headers, so every
request to the server was unauthenticated (-> 401).

Reuse _bearer_auth_headers(entry.name) from mcp_config.py so the
catalog path emits the same 'Authorization: Bearer ${MCP_..._API_KEY}'
template as the manual 'hermes mcp add --url' path.

Salvaged from #70782 (production hunk applied clean; tests re-anchored
onto current main). Credit: JonthanaHanh.

* perf(compressor): release allocator pages after successful compaction

A successful compaction frees the largest allocation a long session ever
drops (the compressed-away message dicts), but Python's arena allocator
keeps those pages in the heap — RSS retains the pre-compaction
high-water mark until exit. #76905's trim_memory lifecycle covers the
gateway/TUI housekeeping loops but not the CLI compression path.

Call trim_memory(reason='post-compression') at the compression-success
point in ContextCompressor.compress(), following the house pattern
(lazy import in try, debug-level log on failure). The helper is
glibc-gated, config-gated and rate-limited, so it is a safe no-op on
other platforms and cannot fail compression.

Re-expresses the intent of #70782 (JonthanaHanh), which reached for a
bare gc.collect(); trim_memory is the house mechanism and already
wraps a collect.

* fix(catalog): validate http+api_key manifests declare the header's env key

Simplify-pass follow-up on the #70782 salvage: _bearer_auth_headers
hard-emits ${MCP_<NAME>_API_KEY} but install_entry only persists
auth.env-declared vars — a manifest naming its key differently (the
shipped n8n style) would install cleanly yet send a literal-placeholder
header at connect time (silent 401, the #37792 bug class). Enforce the
naming contract at parse time. Also pins the secret-stays-in-.env
property in the install test (raw config.yaml carries the template,
never the secret). Mutation-checked: validation disabled -> guard test
fails.

* fix(agent): cap auxiliary LLM concurrency per task

* fix: thread extra_headers through the call_llm split

The PR's concurrency wrapper splits call_llm into a semaphore-guarded
entry + _call_llm_impl; main added extra_headers to call_llm's
signature after the PR's base, so the split has to forward it too
(dropped silently otherwise — Azure Foundry and custom-endpoint
callers set it).

* fix(nix): tie devShell's HERMES_PYTHON to the venv actually on PATH

`nix/devShell.nix` collected `devShellHook` by scanning every package:

    nonNpmHooks = map (p: p.passthru.devShellHook or "") packages;

But `minimal` and `messaging` are `.override` variants of `default`, so
each carries its own `devShellHook` exporting its own HERMES_PYTHON. The
scan therefore concatenated three conflicting exports and forced Nix to
evaluate and realise three separate uv2nix editable venvs on every
`nix develop`.

`attrValues` is alphabetical, so the last hook won (`minimal`) while
`python`/VIRTUAL_ENV came from `default`'s devDeps:

    HERMES_PYTHON        = ...dimim2... (minimal — no optional deps)
    python / VIRTUAL_ENV = ...85r28... (full)

Inside the shell `$HERMES_PYTHON -c "import anthropic"` failed while
`python -c "import anthropic"` succeeded. Worse, `scripts/run_tests.sh`
prefers HERMES_PYTHON, so the suite ran against the minimal venv. Its
guard did not catch this: it only checks that HERMES_PYTHON has pytest,
and minimal's venv does (pytest is in the `dev` group), so the wrong
interpreter was silently accepted.

Tying the hook to `packages.default` — the same package whose `devDeps`
are installed — keeps HERMES_PYTHON, `python`, and VIRTUAL_ENV pointing
at one venv by construction.

    editable venvs referenced   3      -> 1
    their combined closure      421 MB -> 140 MB
    test failures               85     -> 32

The venv mismatch was masking 53 failures; e.g. test_web_tools_config.py
goes 2-failed -> 38-passed. Full suite is now 25369 passed / 32 failed,
and those 32 reproduce identically on a pristine HEAD worktree with no
nix/ changes under the same interpreter (mostly NixOS artifacts — tests
spawning bare `python3` in a scrubbed env exit 127).

* fix(system_prompt): move skills index to the volatile band

The skills index is runtime-mutable: the agent adds and patches skills mid-session, so it is not byte-stable. Keeping it in the stable band breaks that band prefix-cache contract, because every skill edit changes the stable band and invalidates the entire cached prefix in front of it. Move it to the front of the volatile band so the stable scaffold (identity, tool guidance, model guidance) stays cacheable across skill edits.

* docs(system_prompt): fix stale reconstruct_static_prefix docstring example

Simplify-pass finding: the safety note still cited 'skills edited' as a stable-tier input whose change mismatches the rebuilt prefix — after this PR a skill edit changes only the volatile tail (that's the point). Swap the example for genuinely stable-tier inputs.

* fix(prompt_size): search volatile tier for skills block after the stable->volatile move

CI-caught: compute_prompt_breakdown still looked for <available_skills> in the stable tier, but #37117 moved it to volatile. Search volatile first, fall back to stable for older sessions.

* chore: add contributor email mapping for zabih-sudo

* chore: add contributor email mapping for HAOWANG116

* fix(backup): serialize and atomically publish snapshots

* chore: add contributor email mapping for ElSnacko

* fix: prefer explicit anthropic api key

Cherry-picked from PR #58560 by @itsflownium, adapted to current main
(_getenv instead of os.getenv). Moves ANTHROPIC_API_KEY check ahead of
Claude Code credential file and credential_pool auto-discovery so an
explicitly configured key is never shadowed by auto-discovered OAuth.

Fixes #58546

* docs: document /personality none|default|neutral reset across personality docs

The reset keywords have existed in both CLI and gateway handlers since
June but were undocumented — users couldn't find how to cancel a
personality overlay. Adds a 'Resetting to the default' section to the
personality feature page and mentions the reset in the CLI guide,
slash-command reference (both tables), and messaging command table.

* fix(relay): avoid concurrent turn scope corruption

Signed-off-by: Bryan Bednarski <bbednarski@nvidia.com>

* fix(relay): preserve legacy turn shims

Signed-off-by: Bryan Bednarski <bbednarski@nvidia.com>

* fix(relay): gate skipped turn metrics

Signed-off-by: Bryan Bednarski <bbednarski@nvidia.com>

* test(relay): enforce LIFO in overlap regression

Signed-off-by: Bryan Bednarski <bbednarski@nvidia.com>

* fix(relay): preserve skipped turn context

Signed-off-by: Bryan Bednarski <bbednarski@nvidia.com>

* fix comment about relay workaround

* feat(models): add qwen3.8-max to Nous portal + OpenRouter catalogs, replacing qwen3.7-max

Qwen3.8 Max is live on both OpenRouter and the Nous portal
(qwen/qwen3.8-max, 1M context, 131K max output). Per the
newest-max-replaces-last-max convention, it takes qwen3.7-max's slot
in both curated lists.

- hermes_cli/models.py: OPENROUTER_MODELS + _PROVIDER_MODELS[nous]
  swap qwen/qwen3.7-max -> qwen/qwen3.8-max
- agent/model_metadata.py: DEFAULT_CONTEXT_LENGTHS entry for
  qwen3.8-max at 1,000,000 (verified against OpenRouter live
  metadata and Nous /v1/models 2026-08-03)
- tests/test_empty_model_fallback.py: swap incidental catalog fixture
  to the surviving slug
- website/static/api/model-catalog.json: regenerated

Pricing snapshot skipped: both routes bill via official_models_api
(live pricing), verified with resolve_billing_route. Reasoning
timeout floor already covered by the qwen3 prefix (180s).

* test: swap context-switch-guard fixture off qwen3.8-max-preview

test_custom_provider_context_avoids_false_shrink_warning used
qwen3.8-max-preview as a slug that deliberately falls through to the
generic 'qwen' 131K catalog match. The new qwen3.8-max
DEFAULT_CONTEXT_LENGTHS entry (1M) now substring-matches the preview
slug too, so the no-custom-providers branch stopped warning. Swap the
fixture to qwen3.9-max-preview, which still hits the generic fallback
— the test's intent (custom_providers threading) is unchanged.

* fix(agent): keep context_length pin for named custom providers

Empty model.base_url plus a runtime custom-provider URL was treated as a
route mismatch, so gateway session-reset banners dropped model.context_length
and fell back to the Qwen family default (131K) while /status still showed
the configured 262K pin.

* test(gateway): cover named-custom context pin on session-info banner

* fix(model_metadata): read llama.cpp context from meta.n_ctx + accept sole model

* chore: add contributor email mapping for johnrazmus

* fix(gateway): keep event loop alive during /compress and Relay drain

Offload manual /compress temporary-agent cleanup through the existing
bounded off-loop helper so a slow agent.close() cannot freeze the
gateway event loop, heartbeat, or platform polling.

Guarantee Relay transport teardown even when the runner cancels
adapter.disconnect() during go_idle: shielded finally, 2s drain-path
idle ACK budget under the 5s outer disconnect budget, and bounded
supervisor/reader/ws.close awaits.

Original commits:
- fix(gateway): offload manual /compress cleanup from the event loop
- fix(gateway): tear down Relay transport even if go_idle is cancelled
- fix(gateway): keep Relay disconnect budgets inside the runner window

By @Dannyzen (PR #78027), salvaged onto current main.

* fix(gateway): bound go_dormant ws.close with teardown timeout

Sibling site to the disconnect() fix: go_dormant() still did an
unbounded await self._ws.close(), the exact same pattern bounded in
disconnect(). go_dormant runs on the scale-to-zero suspend path (Fly
autostop), which also has timeout constraints. Apply the same 1s
wait_for treatment using _TEARDOWN_AWAIT_TIMEOUT_S.

Found during review of PR #78027.

* fix: close the Codex app-server session on agent teardown

Salvage of #65260's b7d7cfd0e (ported — the PR's close() predates ~4K
commits of teardown-step churn, so the hunk is re-anchored after step
6b rather than cherry-picked).

agent/codex_runtime.py already drops _codex_session on turn crash and
on retirement, but AIAgent.close() — the hard teardown for /new,
/reset, and session expiry — had no owner for it, so the app-server
child process survived until interpreter exit. Long-lived gateways
accumulate one leaked subprocess per ended Codex session.

The attribute is cleared BEFORE close() so a concurrent reader can't
observe a half-closed session and a raising close() can't strand a
stale reference (tested).

Tests extend the author's original lifecycle test with the
raising-close and no-codex-session cases.

* fix(agent): discard bare tool-call marker before fallback/persistence (#78148)

Local tool-call templates can emit a bare bracketed token (e.g. "[memory]")
as assistant content alongside a function call. The loop treated that
protocol scaffolding as visible content: it got cached as the post-tool
fallback, and when the next turn came back empty, the marker was replayed
as the final response and written into the persisted transcript. Later
context compaction preserved that history, letting the model repeat the
marker in subsequent turns.

Detect content that is only a bracketed marker (`[name]`) when the
response also carries tool_calls, and drop it before it can be cached
or persisted. Scoped narrowly: only fires alongside tool_calls, so a
genuine final response of "[memory]" without a tool call is unaffected.

* fix(agent): repair sessions already contaminated with stale tool-call markers (#78148)

The conversation_loop fix (previous commit) stops new "[memory]"-style
bare tool-call markers from being cached/persisted, but sessions written
before that fix can still carry rows where a bare marker was saved as
the assistant's "final response".

Add a load-on-read repair pass in hermes_state.py, mirroring the existing
_strip_background_review_harness defense-in-depth: on session restore,
any assistant row whose content is only a bracketed marker (e.g.
"[memory]", "[skill_manage]") AND that carries tool_calls has its content
blanked before the history re-enters the model's context. The tool call
and its result are left untouched so provider tool_call/tool_result
pairing stays intact. Sessions with no affected rows pass through the
normal path unchanged.

* feat(cli): add sessions clean-markers to permanently purge stale tool-call markers (#78148)

The load-on-read repair (_strip_stale_tool_call_markers) fixes affected
sessions in memory on every resume, but never touches the DB — long-lived
sessions re-scan and re-repair the same rows on every load, and the
contaminated bytes stay in state.db (and any backup/cache snapshot of it)
indefinitely.

Add SessionDB.purge_stale_tool_call_markers(dry_run=False): a one-time,
idempotent UPDATE that permanently blanks the content column on affected
rows. Only content is touched — tool_calls and every other column are
left untouched, so provider tool_call/tool_result pairing survives.
dry_run reads through the no-lock read path and never writes.

Wire it up as `hermes sessions clean-markers [--dry-run]`, mirroring the
existing optimize/repair subcommands. Verified end-to-end against a real
temp state.db: dry-run reports the row without writing, the real run
clears it and preserves tool_calls, and a second run is a no-op.

* fix(cli): back up state.db before clean-markers writes by default

purge_stale_tool_call_markers ran a permanent, irreversible UPDATE with
no backup — inconsistent with repair_state_db_schema's backup-by-default
convention for destructive state.db operations elsewhere in this file.

Take a full snapshot via VACUUM INTO (safe against a live connection,
unlike the raw-copy _backup_db_file used for malformed-schema repair)
before the write, timestamped beside state.db. Skipped when dry_run or
when there's nothing to change. Add --no-backup to `hermes sessions
clean-markers`, mirroring `sessions repair`.

Verified end-to-end: the CLI run against a real temp state.db produces
the backup file before printing the cleared-row count.

* refactor: dedup stale-marker regex — use compiled _STALE_MARKER_RE in conversation_loop

The bracketed-marker regex was inlined in conversation_loop.py as
re.fullmatch(r"\[...", ...) while hermes_state.py defines the same
pattern as _STALE_TOOL_CALL_MARKER_RE. Both must agree on what counts
as a stale marker — a drift here means the runtime guard silently
disagrees with the load-on-read repair and CLI purge in hermes_state.

Consolidate onto a single compiled constant (_STALE_MARKER_RE) at
module level in conversation_loop.py, with a comment noting it must
mirror _STALE_TOOL_CALL_MARKER_RE in hermes_state.py. A direct import
from hermes_state was tried first but caused a regression: hermes_state
initializes DEFAULT_DB_PATH = get_hermes_home() / 'state.db' at module
import time, which breaks tests that monkeypatch get_hermes_home() to
return a str (test_slash_worker_accepts_profile_home).

Follow-up to PR #78175 (@JoaoMarcos44).

* fix(conversation_loop): compress messages on output-cap retry path (#55546)

The output-cap retry loop reduced max_tokens by 64 tokens per attempt but
never called _compress_context(), so the compressor never fired. Input
growth (~65 tokens/attempt) canceled the savings, leaving the session
stuck at 200,001 tokens — 1 over the 200,000 ceiling.

The fix adds compression to the output-cap retry path. The compressor
drops the middle window, freeing ~50% of tokens. If compression makes
>=5% savings, the session continues; otherwise vision payloads are
stripped or the session ends with compression_exhausted=True.

Also adds CHANGELOG.md entry and bug fix report.

* fix(conversation_loop): prune dead vision-strip fallback; harden output-cap retry tests

* chore: drop CHANGELOG.md and docs/reports/ — not shipped with salvage PRs

* perf: reuse request_input_estimate instead of recomputing estimate_request_tokens_rough

The output-cap error handler already computes request_input_estimate at
line 4722 via estimate_request_tokens_rough(api_messages, tools=...).
The new compression block ~50 lines below was calling the same function
with the same inputs again. Reuse the existing local.

* chore: AUTHOR_MAP — add BobClawblaw for PR #77870 salvage

Bare noreply email (no NNN+ prefix) needs explicit mapping.

* chore(ci): rerun checks

Signed-off-by: Bryan Bednarski <bbednarski@nvidia.com>

* perf(gateway): prewarm /model picker cache on TUI startup

The classic CLI run() loop calls prewarm_picker_cache_async() during the
idle window after the banner is shown, so the first /model open hits a warm
provider-models disk cache and renders in ~100ms. The stdio TUI entry point
never did this, so the first /model open in a TUI session blocked on serial
/v1/models fetches for every authenticated provider.

Mirror the CLI behaviour: kick off the same off-thread prewarm right after
gateway.ready is emitted (banner shown, user about to type). Fire-and-forget,
guarded once-per-process, fully exception-isolated so a slow or offline
provider can never affect TUI startup.

* test(tui): pin picker-cache prewarm wiring in entry.main()

teknium's review gap on #72021: the helper's worker/once-guard was
covered, but nothing asserted the stdio TUI entry point actually
invokes prewarm_picker_cache_async() — or that it does so in the right
place. Add a focused entrypoint test that runs the real entry.main()
with stubbed collaborators (same monkeypatch-module-attrs harness as
test_tui_entry_mcp_owner.py), spies on the helper in
hermes_cli.model_switch (the lazy-import source), and asserts:

- prewarm fires exactly once, strictly AFTER the gateway.ready write
- startup stays non-blocking: main() reaches the stdin loop and
  returns on EOF
- a prewarm failure is swallowed (fire-and-forget) without breaking
  startup

Mutation-checked: deleting the prewarm hunk from entry.py fails both
tests.

* perf(cli): check local auth.json/config before slow provider registry sweep

_has_any_provider_configured() probed every api_key provider (gh subprocess
for copilot alone takes 5s; full sweep ~18s) before consulting auth.json and
config.yaml, which are instant local reads. Desktop setup.status calls
blocked past the UI's timeout, causing the connect/disconnect boot loop.
Reorder so cheap local checks run first. Same semantics, ~35x faster here.

* test(cli): regression tests pinning auth-first ordering skips registry sweep

Teknium's review on #63457: existing tests pin the final boolean but not
that the slow PROVIDER_REGISTRY sweep is skipped. Add three tests that
booby-trap hermes_cli.auth.get_auth_status and verify
_has_any_provider_configured() short-circuits on:
- config.yaml model.provider
- config.yaml base_url/api_key (custom endpoint shape)
- auth.json active_provider (sweep-only call-pattern guard)

Mutation-checked: reverting the reorder makes all three fail.

* perf(desktop): keep spinner frames out of React commits

Advance the existing animated status glyph through its DOM text node instead of React state, and pause its timer for hidden panes or inactive windows. Cover frame advancement, zero update-phase commits, and timer suspension with behavior tests.

* test(desktop): cover minimized/hidden window-state + visibilitychange pause for GlyphSpinner

Regression coverage requested in review of #74357: mock
window.hermesDesktop.onWindowStateChanged (pattern from
persistent.test.tsx) and assert minimize…
randlee pushed a commit to randlee/hermes-agent that referenced this pull request Aug 11, 2026
…#38491)

Re-derivation of NousResearch#38491 by @stremtec onto current main (the original is
10,119 commits behind; the hook moved into ui-tui/src/app/). The hook
returned a fresh object literal every render, defeating memoization in
useMainApp's consumers; useMemo over the (all-useCallback-stable)
handles makes the return referentially stable.

Dep array covers ALL nine returned handles incl. trimTail (the
re-derivation initially omitted it - stale-closure class).
33hodl pushed a commit to 33hodl/hermes-agent that referenced this pull request Aug 12, 2026
…#38491)

Re-derivation of NousResearch#38491 by @stremtec onto current main (the original is
10,119 commits behind; the hook moved into ui-tui/src/app/). The hook
returned a fresh object literal every render, defeating memoization in
useMainApp's consumers; useMemo over the (all-useCallback-stable)
handles makes the return referentially stable.

Dep array covers ALL nine returned handles incl. trimTail (the
re-derivation initially omitted it - stale-closure class).
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

area/streaming Streaming responses: gateway delivery, provider wire comp/tui Terminal UI (ui-tui/ + tui_gateway/) P3 Low — cosmetic, nice to have sweeper:blast-contained Sweeper blast radius: contained — one narrow path / opt-in / few users type/perf Performance improvement or optimization

Projects

None yet

Development

Successfully merging this pull request may close these issues.

4 participants