fix(state): self-heal FTS corruption on the SessionDB write path (cron/CLI writers) by teknium1 · Pull Request #66296 · NousResearch/hermes-agent

teknium1 · 2026-07-17T12:28:35Z

Summary

Cron and CLI SessionDB writers now self-heal FTS corruption instead of hard-failing every append: _execute_write detects the corrupt-index error class, runs a one-shot in-place rebuild via the existing rebuild_fts(), and retries the failed write.

Scope reworked after the parallel #65637 salvage landed (53d3588 + a9cc17f, credit @MaartenDMT + @kshitijk4poor): the gateway session store now has its own transcript retry queue + warning logging, so this PR's original gateway half was dropped. What main still lacked: writers that call SessionDB directly (cron, CLI) — a corrupt FTS index hard-failed their appends until the next process restart triggered the offline repair. This PR closes that gap at the SessionDB layer.

Root cause (E2E-reproduced): a corrupted messages_fts_data shadow table makes every message INSERT raise through the FTS sync triggers while canonical messages rows stay intact.

Changes

hermes_state.py: _execute_write catches the FTS-corruption class — both the generic database disk image is malformed and newer SQLite's fts5: corrupt structure record variant — and delegates to rebuild_fts() (lock-safe: the failed txn is rolled back and self._lock released before the except path runs). One-shot per instance; lock/busy jitter-retry untouched.
tests/state/test_fts_runtime_rebuild.py: 6 tests — error-class classification across both SQLite message variants, self-heal lands the append, search works post-heal, one-shot guard doesn't loop, unrelated errors propagate, lock-retry unchanged.

Validation

	Result
E2E corruption repro (isolated HERMES_HOME, real SQLite)	append fails → self-heals, row persists, FTS search works
New tests	6/6 pass
tests/state + tests/hermes_state + gateway session tests	197/197 pass

Infographic

alt-glitch · 2026-07-17T12:49:46Z

This was generated by AI during triage.

Related to #66256: both cover FTS-corruption transcript writes, but this is a narrow SessionDB write-path self-heal while #66256 also implements ordered gateway retry and replacement cleanup.

tonydwb

Code Review Summary

Verdict: Approved

Looks Good

Transcript append failures now tracked via _transcript_append_failures counter and logged at WARNING level instead of DEBUG
This makes persistent write failures visible to operators instead of silently accumulating
Good comment explaining why DEBUG was insufficient (amnesia issues from disk lagging behind memory)
No security concerns

Reviewed by Hermes Agent

Complements the #65637 salvage (53d3588 + a9cc17f): the gateway session store now retries transcript appends through its own queue, but cron and CLI writers call SessionDB directly — a corrupt FTS index still hard-failed their appends until the next process restart triggered the offline repair. _execute_write now detects the FTS-corruption error class (both the generic 'database disk image is malformed' and newer SQLite's 'fts5: corrupt structure record' variant), performs a one-shot in-place rebuild by delegating to the existing rebuild_fts(), and retries the failed write. One-shot per instance so an unrecoverable database cannot loop; lock/busy jitter-retry path untouched. E2E-verified: corrupted messages_fts_data rejects appends; with this fix the same append self-heals, persists, and FTS search works again.

…in-20260717T161903Z * origin/main: (726 commits) test(compression): expect complete runtime tuple fix(auxiliary): scope runtime state to each turn chore(release): map auxiliary runtime contributors fix(tui): route images with the live switched model fix(compression): reset failure cooldown on runtime switch fix(auxiliary): sync runtime after fallback restoration fix(auxiliary): isolate runtime cache by live context fix(state): self-heal FTS corruption on the SessionDB write path (NousResearch#66296) fmt(js): `npm run fix` on merge (NousResearch#66348) feat(kanban): modal create-task dialog, editable board project directory, comment workflow hint (NousResearch#66333) fix(terminal): fall back when the configured cwd is unenterable, not just missing (NousResearch#66306) test(tui): accept repair_alternation in the top-level server test doubles too fix(tui): heal alternation at the remaining live-replay resume sites test(delegate): assert copilot probe with assert_any_call to de-flake under slicing test(tui_gateway): accept repair_alternation in resume-path DB doubles fix(state): heal alternation at the ACP / CLI-resume / TUI-resume restore sites too feat(codex): webSearch bubbles + bare hermes-tools names in app-server bridge fix(codex): forward drained notifications to on_event during approval roundtrips fix(compression): affirm tool use stays active in the compaction handoff prefix (NousResearch#66291) fix(codex): claim the stream-writer token on the codex_responses path too ... # Conflicts: # cron/scheduler.py

…sResearch#66296) Complements the NousResearch#65637 salvage (53d3588 + a9cc17f): the gateway session store now retries transcript appends through its own queue, but cron and CLI writers call SessionDB directly — a corrupt FTS index still hard-failed their appends until the next process restart triggered the offline repair. _execute_write now detects the FTS-corruption error class (both the generic 'database disk image is malformed' and newer SQLite's 'fts5: corrupt structure record' variant), performs a one-shot in-place rebuild by delegating to the existing rebuild_fts(), and retries the failed write. One-shot per instance so an unrecoverable database cannot loop; lock/busy jitter-retry path untouched. E2E-verified: corrupted messages_fts_data rejects appends; with this fix the same append self-heals, persists, and FTS search works again.

Complements #66296 (self-heal on the write path): search_messages()'s main FTS5 MATCH query caught only sqlite3.OperationalError (a query-syntax error → return empty). A corrupt FTS index raises the malformed / "fts5: corrupt structure record" class, which is a sqlite3.DatabaseError — the parent of OperationalError, so it was NOT caught and propagated straight out of search_messages, crashing session/history search. The write path now rebuilds and retries on that class, but a read-only session (cron/CLI history search, or a search issued before any write) never triggers a write, so its search stayed broken until the next process restart ran the offline repair. Catch the DatabaseError corruption class on the search MATCH read too and route it through the existing one-shot _try_runtime_fts_rebuild(), then retry the query. The catch is moved outside `with self._lock` so rebuild_fts() can re-acquire the lock (mirrors _execute_write). The one-shot guard is shared with the write path, so a single instance never loops on a genuinely unrecoverable index. OperationalError syntax handling is unchanged (caught first). Adds a regression test: with a corrupted messages_fts and no post-corruption write, search_messages() rebuilds in place and returns the match; without the fix it raises DatabaseError.

The trigram MATCH branch in search_messages() had the same OperationalError-only catch that #66420 fixed on the main FTS5 branch: a corrupt messages_fts_trigram shadow table raises the malformed / 'fts5: corrupt structure record' class (sqlite3.DatabaseError, parent of OperationalError), which propagated straight out of search_messages and crashed CJK session/history search for read-only sessions. Route that class through the shared one-shot _try_runtime_fts_rebuild() and retry the trigram query (catch moved outside self._lock so rebuild_fts() can re-acquire it, mirroring the main branch). If the rebuild is refused (guard consumed / FTS disabled / different error) or the retry fails, fall through to the existing LIKE substring fallback — which reads only the canonical messages table — instead of raising, so CJK search degrades gracefully rather than crashing. Adds two regression tests: trigram search self-heals in place after shadow-table corruption (answers from the rebuilt trigram index, not the LIKE fallback), and degrades to LIKE without raising when the one-shot rebuild was already consumed. Follow-up to #66420; refs #66296 #66724

…ma v23) (#65798) * fix(desktop): refresh repo status on session switch with unchanged cwd (#68208) fix(desktop): refresh repo status on session switch with unchanged cwd * fix(checkpoints): honor gateway config and task cwd (#68195) * fix(gateway): wire checkpoint config into agents * fix(checkpoints): resolve gateway file paths by task cwd * ci: live-updating PR review comment with structured job statuses Replace the static comment-pending + comment-results two-job pattern with a live-updating comment system that polls the GitHub Actions API every 15s, re-assembles the review comment from whatever results are available, and upserts it via the  marker. The comment updates in real time as each job finishes — no waiting for the full pipeline. Every CI job that wants to appear in the review comment emits a review_status output — a JSON array of objects, each with a source and a results array: [ { "source": "review-label-gate", "results": [ {"kind": "action_required", "title": "...", "summary": "...", "how_to_fix": "..."}, {"kind": "info", "title": "...", "summary": "..."} ] }, { "source": "ci timing", "results": [ {"kind": "warning", "title": "CI timings", "summary": "...", "detail": "...", "link": "..."} ] } ] One job can emit multiple results of different kinds. The source field is used to exclude the corresponding job from the synthesized error list (case-insensitive, hyphen-normalized matching against GitHub Actions job display names). | job | source | kind (on failure) | section | |----------------------------|--------------------------|---------------------------|----------------------| | review-labels | review label gate | action_required / info | Action required | | lockfile-diff | lockfile-diff | action_required | Action required | | ci-timings | ci timing | warning / info | Warnings | | supply-chain scan | supply chain | error / (none) | Job failures | | supply-chain dep-bounds | supply chain | action_required / (none) | Action required | | osv-scanner | osv scan | warning / (none) | Warnings | | uv-lockfile-check | uv.lock check | action_required / (none) | Action required | | history-check | unrelated histories | action_required | Action required | | contributor-check | contributor attribution | action_required | Action required | Jobs that find nothing emit [] (empty array) — no noise info items. A single comment-live job polls the GitHub Actions API every 15s, classifies jobs into (completed, pending), assembles the comment, and upserts it. Merges review_status outputs from all needs jobs via toJSON(needs.*.outputs.review_status), and downloads the ci-timings artifact when it becomes available. Shows commit SHA + message below the header. The assembler has ZERO job-specific knowledge. It just: 1. collect_from_statuses() — flattens all nested status objects into ReviewItems 2. collect_failed_jobs() — synthesizes errors for failed jobs with no declared status 3. _attach_job_urls() — fills in per-job log links for ALL items 4. render_comment() — groups by severity, renders with group headers Each item shows links inline next to the title: View report (job-emitted URL) and View job (auto-attached logs link). Each info item is its own collapsible <details> block. # ૮ >ﻌ< ა ci review running on abc1234 — commit message first line ## ❌ Job failures ### {title} · [View job](url) {summary} ## ⚠️ Action required ### {title} · [View job](url) {summary} **How to fix:** {how_to_fix} ## ⚠️ Warnings ### {title} · [View report](url) · [View job](url) {summary} {detail} <details><summary>{title}</summary> {content} </details> Still running 3 jobs: ci-timings, docker - test_assemble_review_comment.py (48 tests): collect_from_statuses, collect_failed_jobs with exclude_sources, _attach_job_urls, render_comment (group headers, inline links, commit info, per-item details, pending footer), assemble integration - test_live_comment.py (16 tests): classify_jobs pure function - test_timings_report.py (10 tests): generate_review_status nested format - test_lockfile_diff.py (6 tests) - test_classify_changes.py (32 tests, pre-existing) * ci: migrate AUTOFIX_BOT_PAT to GitHub App token Replace the long-lived fine-grained PAT (AUTOFIX_BOT_PAT) with short-lived (1-hour) installation access tokens minted via a new get-app-token composite action wrapping actions/create-github-app-token@v3.2.0. The PAT was used in 13 spots across 8 workflow files for gh CLI / GitHub API calls. The per-repo GITHUB_TOKEN (1,000 req/hr) was getting rate-limited when multiple workflows fire concurrently (deploy-site, skills-index, ci-timings, supply-chain-audit, js-autofix). App installation tokens get 5,000 req/hr per installation and are scoped to the App's permissions, not a user account. New composite action: .github/actions/get-app-token/ - Wraps actions/create-github-app-token@bcd2ba49 (v3.2.0, SHA-pinned) - Reads APP_ID + APP_PRIVATE_KEY repo secrets - Outputs a 1hr installation token via steps.app-token.outputs.token Requires two new repo secrets (set after creating the GitHub App): - APP_ID: the App's numeric ID - APP_PRIVATE_KEY: the PEM private key App installation permissions needed: contents: write (js-autofix push, pypi release upload) pull-requests: write (js-autofix PR create/merge, supply-chain comment) issues: write (skills-index-freshness issue creation) actions: write (skills-index workflow trigger) workflows: write (skills-index triggers deploy-site.yml) The AUTOFIX_BOT_PAT secret can be deleted once CI passes on this PR. The comment in js-autofix.yml noting that PAT pushes trigger downstream workflows is updated — App tokens have the same property (they are not GITHUB_TOKEN), so the concurrency-cancel loop logic is unchanged. * style(desktop): satisfy merged eslint/prettier config The SSH modules predate the stricter lint config that landed on main (curly, no-empty, perfectionist sorting, prettier). Mechanical lint:fix + fmt pass, empty catch blocks filled with the codebase's void-0 convention, and inline no-control-regex disables on the three deliberate control-char patterns (same pattern as lib/ansi.ts). * fix(ci): pass App secrets as inputs to composite action Composite actions cannot access the secrets context — the runner's template engine rejects secrets.* references at load time with 'Unrecognized named-value: secrets'. Move APP_ID and APP_PRIVATE_KEY from direct secrets.* references inside the composite action to inputs passed by each calling workflow. The fallback logic (GITHUB_TOKEN when APP_ID is empty, for fork PRs) stays in the composite action's check step. * fix(ci): add detect to all-checks-pass needs so its failure blocks merge If detect fails, all downstream sub-workflows get SKIPPED (they have needs: detect). all-checks-pass used if: always() and only checked the sub-workflows — which all showed as 'skipped' (= success) — so it passed even though the root cause (detect) failed. This made the PR mergeable despite a broken CI pipeline. Add detect to all-checks-pass needs so its failure propagates to the gate job and blocks the merge. * fix(desktop): bump skills test timeout to fix cold-start flake (#68235) Test 1 in skills/index.test.tsx pays the full cold-start cost (jsdom env init + module transform + the @/hermes/@/store/profile import graph), which pushed past vitest's 5000ms default under load — caught at 8871ms on one run, 6.6s pure test time on another. Tests 2-4 are ~30-130ms each because all that setup is already cached, so only test 1 was at risk of timing out. Bump the describe-level timeout to 15s. Verified with 10 consecutive runs, 4 of which took 5.5-6.6s of test time and would have hard-failed under the old 5s default. * feat(desktop): open multiple full app windows (electron) Add createInstanceWindow() — a full-chrome peer of the primary that renders the complete app (sidebar, routing, its own draft) against the shared backend, so several GUI windows can run at once. Mirrors the primary's window options + chatWindowWebPreferences (backgroundThrottling stays off so a streamed answer never stalls when blurred) but never overwrites the mainWindow global and doesn't respawn the backend — the renderer's getConnection() joins the running one. New windows cascade off their source via the pure, tested instanceWindowBounds(). Exposed via the hermes:window:openInstance IPC and a "New Window" File menu item. Per-window fullscreen state now targets the window itself, and titlebar/native-theme repaints reach every open chat window instead of only the primary. Retires the now-orphaned compact new-session pop-out (its only caller was ⌘⇧N, repointed in the follow-up commit): drops createNewSessionWindow, the hermes:window:openNewSession handler, and the newSession/new=1 URL flag. * feat(desktop): wire New Window to ⌘⇧N + command palette Repoint session.newWindow (⌘⇧N) from the compact new-session pop-out to openNewWindow(), which opens a full peer instance via the new openWindow bridge, and add a "New Window" entry to the ⌘K palette (shown with its hotkey hint, gated on canOpenNewWindow()). Relabel the action "New window". Drops the retired openNewSessionWindow bridge and the vestigial isNewSessionWindow()/new=1 flag; renames the shared opener helper. * fix(desktop): de-dupe cross-window cues so peers don't spam With multiple full windows, each renderer independently reacts to the same backend event, so one-shot cues fired N times: OS notifications (the per-renderer throttle can't see other windows), the turn-end sound (playCompletionSound runs on every message.complete, ungated by focus), and auto-spoken replies (double voice when a chat is open in two windows). Add a single race-free owner in the main process (electron/event-dedupe.ts): main handles IPC serially, so the first window to claim a key within a short window wins and peers stay quiet. Notifications collapse at the hermes:notify choke point; the sound and spoken replies claim via a new hermes:ambient:claim IPC (keyed by session / reply id). Off Electron the claim falls back to "emit", preserving single-window behavior. The sound's mute check runs before the claim so a muted window can't win the cue and silence an audible peer. * refactor(desktop): tidy the cross-window deduper Drop the unused DEDUPE_WINDOW_MS export and rename its interval so "window" isn't overloaded against BrowserWindow in a multi-window feature (windowMs → intervalMs). DRY the completion-sound play path. No behavior change. * nix: add cage to devDeps * fix(desktop): avoid false remote gateway reauthentication (#68250) * fix(desktop): avoid false remote gateway reauthentication Co-authored-by: Rod-fernandez <rodrigo@nxtlevelsaas.com> Co-authored-by: David Andrews (LexGenius.ai) <david@lexgenius.ai> * fix(desktop): harden remote revalidation state --------- Co-authored-by: Rod-fernandez <rodrigo@nxtlevelsaas.com> Co-authored-by: David Andrews (LexGenius.ai) <david@lexgenius.ai> * fix(desktop): keep composer draft across compression tip rotation (#68079) * fix(desktop): keep composer draft across compression tip rotation Auto-compression swaps the live stored session id while the user may still be typing. Scope the composer/queue key on the lineage root and migrate any tip-keyed draft/queue entries onto that durable key when the tip rotates so the in-progress prompt does not vanish when the response lands. * test(desktop): cover draft survival across compression tip rotation Add regression coverage for migrateSessionDraft, lineage-scoped composer keys, and the rotation path that previously wiped an in-progress draft. * fmt(js): `npm run fix` on merge (#68305) Co-authored-by: github-actions[bot] <github-actions[bot]@users.noreply.github.com> * fix(desktop): Stop parks the queue instead of firing the next queued prompt Interrupting a busy turn with the Stop button (or Esc) settles the session to idle, and the edge-independent auto-drain immediately submits the head of the composer queue. The user pressed Stop to halt the agent, but it looks like Stop skipped the current turn and kept going — and the queued text is hard to find, since its only surface is the collapsed 'N queued' pill above the composer. The old userInterruptedRef latch (a23728dcc) fixed this but was removed in #40221 because it also suppressed the drain that send-now-while-busy depends on. This reintroduces the halt with source awareness instead of a blanket latch: - Explicit halts (Stop button, composer Esc, chat-focus Esc, the streaming message's hover Stop, runtime cancel) park the session's queue before interrupting. Parked queues are skipped by both auto-drain paths (mounted ChatBar + background drainer). - Interrupts that exist to advance the queue (send-now-while-busy) unpark first, so the settle drain they rely on still flows. - The park lifts on any renewed intent: resume, a manual drain (Enter on empty composer or the per-row send arrow), queueing a new prompt, or emptying the queue. It migrates with entries on a runtime re-key and is deliberately not persisted (a fresh process starts unparked). - The queue panel expands on park, switches to 'N Queued — paused' with a pause icon, and grows a Resume action, so the held prompts are visible instead of reading as vanished. Store contract, hook wiring, and background-drain coverage included; docs updated. * fix(cli,tui): recall real paste content on up-arrow Large pastes collapse to a placeholder in the composer, but input history stored the placeholder — so up-arrow recall showed a truncated reference (CLI) or lost the content entirely (TUI, where the `[[…]]` label has no backing snip after submit). Store the expanded content in history instead: - CLI: `_inline_pastes()` expands `[Pasted text #N -> file]` into the buffer before `reset(append_to_history=True)`; also reused by the external editor (dedup). History nav suppresses re-collapse of recalled content. - TUI: `dispatchSubmission` pushes `expandSnips(pasteSnips)(full)`; idempotent on label-free text so re-submitting a recalled entry stays stable. * fix(cli): suppress CPR on POSIX local TTYs under load Delayed ESC[6n replies leak as ^[[row;colR into the classic CLI on SSH/slow PTYs (#13870) and on local POSIX TTYs under heavy subagent load. Suppress CPR on non-Windows platforms (layout hint only); keep native Windows on prompt_toolkit's default pending native coverage. Wire selection through _select_classic_cli_pt_output. * test(cli): prove local CPR leak and Application CPR-disabled wiring Add a delayed-CPR PTY harness (no SSH) plus selection/Application assertions for POSIX local and Windows preserve-default. Update the gating unit test to the new contract. * refactor: drop platform kwarg, fix PTY test cleanup - Remove redundant platform= test seam from _terminal_may_leak_cpr(); use monkeypatch.setattr(sys, 'platform', ...) consistently in both test files. - Wrap PTY tests in try/finally for fd cleanup on assertion failure. - Guard select.select() in terminal thread against OSError after fd close (fixes PytestUnhandledThreadExceptionWarning). - Trim PR-number reference from test module docstring. * docs(portal): remove retired Nous Chat references * fix(web/ddgs): isolate DuckDuckGo search in a disposable process ThreadPoolExecutor timeouts cannot fire when primp holds the GIL in native code (#68096). Run each search in a child process the parent can terminate/kill, and honor tools.interrupt between polls. * test(web/ddgs): cover GIL-hold timeout, interrupt, and worker reap Regression tests for #68096: native GIL-hold and sleep hooks must time out or interrupt promptly with no orphaned search workers. * fix: sanitize subprocess env for DDGS worker os.environ.copy() passes all Hermes secrets (gateway tokens, API keys, dashboard session tokens) into the DDGS child process. Use _sanitize_subprocess_env() to strip Hermes-managed secrets before spawning the worker. * fix(agent): pass persisted-prefix boundary when rotation flushes on cold resume (#68196) The legacy rotation branch in agent/conversation_compression.py flushes the current turn to the OLD session before ending it (#47202) via _flush_messages_to_session_db(messages) with no conversation_history boundary. On the first turn after a cold Desktop resume, the restored transcript rows live in the message list as plain dicts that have not yet been stamped with _DB_PERSISTED_MARKER — the normal turn flush that stamps them runs after preflight compression. With no boundary, _flush_messages_to_session_db builds an empty history_ids set and treats every restored row as new, durably re-appending the whole transcript to the parent session. Repeated restart/resume + threshold compression keeps growing the parent transcript. Pass messages[:_persist_user_message_idx] (the already-durable prefix that turn_context anchors before preflight runs, guarded for int/bounds) as conversation_history so the flush skips the persisted rows by identity and writes only the current turn's new messages. Adds a regression test that pre-populates SQLite, cold-loads the transcript, appends one current user row, and forces rotating compression: it fails before this change (parent grows to 5 rows) and passes after (parent holds the two originals plus the single new turn). * fix(desktop): prevent contentEditable composer input from visually collapsing to near-zero height Fix #68095 The composer input box (contentEditable div) randomly shrank to a tiny/pixelated size when typing character-by-character (paste worked fine). Root cause: during per-keystroke input, the normalizeComposerEditorDom cleanup could briefly leave the contentEditable with zero child nodes, and without intrinsic content the browser collapsed it visually despite the CSS min-height. Two-pronged fix: 1. Add min-h-[1.625rem] bracket syntax alongside the CSS variable min-height to ensure the minimum height is enforced even if the CSS variable resolution is delayed or overridden by browser defaults. 2. In normalizeComposerEditorDom, ensure the contentEditable always has at least one <br> child when empty, giving it intrinsic height that the browser cannot collapse. This is a belt-and-suspenders approach with the CSS min-height. Closes #68095 * fix(agent): circuit-break AttributeError from commit-splice and detect code skew Fix #68178 The git-install auto-updater rewrites source while the desktop backend is live. Because agent/conversation_loop.py is imported lazily on the first API call, a process can end up running two different commits spliced together — one commit's AIAgent against another commit's conversation_loop. When the interface differs, every turn fails permanently with an AttributeError, and the loop retries indefinitely, burning provider API calls (576 failures, 149 wasted API calls observed). Three-prong fix: 1. Circuit-break AttributeError on agent objects: the outer-loop error classifier now detects AttributeError targeting agent/run_agent modules and breaks immediately instead of continuing the retry loop. 2. Code skew detection for desktop/serve backend: run_agent.py now snapshots the checkout revision at import time and exposes a cheap per-iteration check that the conversation loop uses to refuse new work with a clear 'restart required' message before the lazy import can crash. 3. Informative error message: when code skew is detected, the user gets a clear explanation of the mismatch (boot revision vs current revision) and actionable guidance to restart the application. * fix(telegram): preserve fatal recovery handoff Release the current polling-recovery task's ownership before invoking the fatal-error handler. The runner bounds adapter cleanup in a child task; disconnect() cancels the tracked polling-recovery task, so retaining the current notifier in _polling_error_task would cancel the fatal callback before the runner can finish its reconnect-queue or shutdown decision. The new _handoff_polling_fatal_error() helper clears _polling_error_task only when it is the current notifier. Other recovery tasks remain tracked and are still cancelled and awaited during teardown. Covers both network retry exhaustion and polling-conflict exhaustion. Replaces the misleading "Restarting gateway" message with "Escalating to gateway recovery". Fixes #68406. * fix(telegram): widen fatal handoff to heartbeat watchdog path The wedged-recovery heartbeat watchdog (line 2526) calls _notify_fatal_error() directly from the heartbeat task. disconnect() cancels _polling_heartbeat_task unconditionally (no current_task guard, unlike _polling_error_task). Same bug class as #68406: the child disconnect cancels the heartbeat parent before the runner can queue reconnect. Widen _handoff_polling_fatal_error() to also clear _polling_heartbeat_task when it is the current task, and route the heartbeat watchdog call site through the handoff helper. Co-authored-by: Imgaojp <6065749+Imgaojp@users.noreply.github.com> * fix(tests): make the live-system-guard canary fail closed tests/test_live_system_guard_self_test.py executes real kill primitives (os.kill(-1, SIGTERM), os.killpg, pkill -f python) and depends entirely on the autouse _live_system_guard fixture in tests/conftest.py to intercept them. That makes the canary fail-OPEN: in any collection context where the file is present but its home conftest is not — a published sdist that ships tests/ but not tests/conftest.py, a tree assembled by copying test*.py (that glob does not match conftest.py), pytest --noconftest, or a foreign rootdir — the primitives fire for real, and os.kill(-1, SIGTERM) SIGTERMs every process the invoking user owns (a full desktop-session kill was reported in the field). Add an autouse fixture that refuses to run any canary test unless the guard is provably active. The one thing the canary can detect about its own safety is that the guard monkeypatches os.kill with a plain Python function, whereas the unguarded primitive is a C builtin — so the probe keys off that. Tests marked @pytest.mark.live_system_guard_bypass still opt out, matching the guard's own bypass contract (e.g. test_bypass_marker_disables_guard). With the guard loaded every canary test behaves exactly as before; without it each test refuses at setup with zero side effects. Fixes #68311 * fix(billing): rename user-facing "terminal billing" copy to Remote Spending (#68355) * fix(billing): rename user-facing "terminal billing" copy to Remote Spending The capability was renamed Remote Spending on the portal (consent CTA: "Allow Remote Spending"; per-terminal states Granted/Stopped), but the terminal, desktop, and docs still said "terminal billing" everywhere. - Feature name: Remote Spending in titles/labels, lowercase mid-sentence. - Step-up action verb is now "allow", matching the portal consent CTA. - Kill-switch-off recovery copy points at the actual control ("a billing admin can turn it on from the portal's Hermes Agent page") instead of the dead-end "manage it on the portal". - Per-terminal revoke copy uses the portal vocabulary ("stopped"). - Wire identifiers (cli_billing_enabled, cli_billing_disabled, ...) are unchanged; copy, comments, docs, and test expectations only. * fix(billing): correct the post-step-up denial diagnosis + finish the desktop rename Adversarial review findings: (1) a repeated insufficient_scope after a successful step-up is a per-terminal authorization failure, but the copy blamed the org kill-switch and pointed at the wrong recovery control — now: "Remote Spending still isn't active for this terminal — the authorization didn't take. Retry, or make this change on the portal." (2) the desktop step-up flow started in Remote Spending vocabulary but finished in "billing management access" — renamed both end states. (3) prettier formatting on the touched files (matches the post-merge fmt bot). * feat(tui): show the plan catalog in /subscription on Free (#68357) * feat(tui): show the plan catalog in /subscription on Free The server returns the tier list even with no subscription, but the overlay hid the picker behind can_change_plan && !isFree, so a Free account got only "Start a subscription" with no idea what the plans cost. Now: - Overview on Free offers "Choose a plan" whenever the catalog has enabled paid tiers. - The picker on Free lists each plan as name · price · monthly credits (no upgrade/downgrade hints — there is nothing to move from), and picking one opens the portal, where starting a subscription actually happens (card capture + checkout live there; the upgrade RPC requires an existing subscription). - Paid-plan behavior (preview → confirm → apply) is unchanged. * refactor(tui): compute the picker row suffix once Review feedback: the isFree fork duplicated the label template and run handler; only the suffix differs. * fix(tui): arm the busy guard before the Free portal handoff Adversarial review: the Free branch returned before setting busyRef, so a double-Enter could open the portal twice; and the picker narrated a handoff that openManageLink already narrates (duplicate on success, contradictory on failure). Guard first, let the helper do the talking. * fix(tui): monthly credits are dollars — label them as such The Free picker showed "1000 credits/mo" for what is $1,000 of monthly credit — render "$1,000 credits/mo" (grouped, dollar-signed). * feat(tui): render the Free-plan catalog inline in the /subscription overview Sid ruling: the upsell belongs where the user already is — no intermediate "Choose a plan" hop. On Free the overview lists each paid plan (name · $/mo · $credits/mo) as a pickable row; picking opens the portal (openManageLink narrates). The generic "Start a subscription" row survives only when the catalog is empty. The picker reverts to its original change-only form (Free never reaches it). * feat(desktop): tier catalog chips on the Subscription row Desktop parity with the TUI inline catalog (Sid ruling): accounts that can act see the plans where they already are — Free gets the upsell list (every chip opens the portal), a subscriber sees all tiers with the current one marked inert. Members and team contexts see no chips. Chips learn an optional url (portal handoff) in the shared row model. * chore(tui): fixture harness mirrors the live tier catalog The dev screenshot fixtures showed invented plans ($50 Super / $99 Ultra, "1,000 credits"); align with the real catalog ($20/$100/$200 with $22/$110/$220 monthly credits) so fixture renders cannot be mistaken for product truth. The overlay itself always reads tiers from the subscription API. * chore: trim narration comments * fmt(js): `npm run fix` on merge (#68462) Co-authored-by: github-actions[bot] <github-actions[bot]@users.noreply.github.com> * fix(relay): attach metadata.user_id on guild replies for egress fallback (#68320) The relay adapter re-attaches an egress discriminator on outbound replies so the connector can resolve the owning tenant. It captured scope_id for scoped (guild) messages and user_id for DMs, but as MUTUALLY EXCLUSIVE: a scoped inbound hit an early return, so the author's user_id was never recorded, and _with_scope only attached user_id when there was no scope_id. Guild replies therefore went out with scope_id only. That's fine while the guild has a provision-time route row. But a MANAGED Discord agent joins guilds dynamically (the shared bot is added to / removed from servers at runtime), and GATEWAY_RELAY_ROUTE_KEYS — the only thing that writes guild route rows — is a self-hosted, static field never stamped for managed agents. So their guild has no route row, the connector's guild-route lookup misses, and with no user_id on the frame there's nothing to fall back to → every guild reply is declined "discord egress declined: target not routed to an onboarded tenant" even though INBOUND resolved the same guild fine (via the author-first SharedSocketRouter.targets() fallback). Fix: capture the authentic author user_id for EVERY inbound (DM and scoped alike) and re-attach it on the outbound reply alongside scope_id. The connector consults it only on a route/scope miss, so carrying both never overrides routing-table resolution. This is the gateway half of the paired gateway-gateway change (makeDiscordTenantOf guild-route-miss author-binding fallback); together they make guild replies resolve the same observed-author way inbound already does. Tests (tests/gateway/relay/test_relay_adapter.py): a guild reply now carries both scope_id AND user_id; a scoped inbound with no author still yields scope_id only (never invents one). Verified fail-without / pass-with. * build: declare pywin32 as a direct win32 dependency hermes_cli/windows_ssh_runtime.py imports win32security/win32file/etc. directly but pywin32 only arrived transitively via concurrent-log-handler -> portalocker. Declare it with a sys_platform gate so the Windows SSH runtime doesn't depend on the logging dep chain. Review follow-up on PR #68130. * fix(desktop): preserve dragging with empty titlebar slots * Revert "fix(agent): circuit-break AttributeError from commit-splice and detect code skew" This reverts commit 3a9b9d65d505646212c4c875bab19b96ae14b2e6. * fix(context): revalidate Codex OAuth context windows * test(context): document Codex cache persistence coverage * fix(context): scope Codex catalogue cache by credential * test(context): cover Codex context rollback * fix(compression): report live-resolved Codex window in the autoraise notice The autoraise banner hardcoded '272K' for the gpt-5.4/5.5/5.6 family, but the Codex /models catalog is authoritative and shifts server-side (gpt-5.6 served 372K during July 9-18, 2026 before OpenAI rolled it back). Pass the compressor's live-resolved context_length through so the notice reports the window the session actually got; the static 272K/128K text remains as the fallback when no resolved value is available. * fix(codex): send ChatGPT-Account-Id on /models probes The Codex backend returns the per-account model catalog only when the ChatGPT-Account-Id header is present. Without it, GET /backend-api/codex/models responds 200 OK with {"models":[]} and the picker silently degrades to the hardcoded fallback list — which is stale or wrong for the active plan (no GPT-5.6 family, wrong context windows). This was the upstream bug behind slow first responses and HTTP 520/120s SSE hangs: Hermes was sending invalid slugs because the probe never saw them in the catalog, and Codex's request builder also depends on the same JWT claim that's now being threaded through both probe paths. Fixes the probe-side paths in hermes_cli/codex_models.py and agent/model_metadata.py by extracting chatgpt_account_id from the OAuth JWT (mirroring the request-side logic already in auxiliary_client.py) and sending it as a header. Verified live: - _fetch_models_from_api now returns the 10-model catalog (gpt-5.6-sol, gpt-5.6-terra, gpt-5.6-luna, gpt-5.5, gpt-5.4, gpt-5.4-mini, gpt-5.3-codex-spark, 3x -pro variants) instead of []. - _fetch_codex_oauth_context_lengths resolves all 8 account models to 272K context (matches direct API probes of the same account). - end-to-end: hermes chat -m gpt-5.6-sol -q 'Reply with one word: pong' returns 'pong' cleanly via the openai-codex route. Same class of bug as PR #64760. * test(codex): cover ChatGPT-Account-Id header on /models probe Add regression tests locking in the new behavior: a JWT carrying a chatgpt_account_id claim causes the probe to send ChatGPT-Account-Id, while a malformed token omits the header instead of crashing. * fix(tools): make the tool-search context gate provider-aware (#68589) _resolve_active_context_length() called get_model_context_length() with the model id alone, so provider-enforced windows (e.g. Codex OAuth's 272K for gpt-5.x vs the direct API's 1.05M) never reached the tool-search activation gate — it sized against generic metadata for the same slug. Resolve the runtime provider for the configured model and pass provider, base_url, and api_key through. If credential resolution fails (offline, no keys), degrade to a provider+base_url-only lookup so the static provider-aware fallbacks still apply; explicit model.context_length keeps short-circuiting as before (#46620). Gap flagged during review of #16735. * feat(skills): bundle docx, xlsx, and pdf office skills; refresh powerpoint (#68595) Non-technical users asking for Word docs, spreadsheets, or PDF work had no bundled skill coverage — docx/xlsx creation required discovering and installing hub skills, and PDF manipulation had no skill at all beyond OCR extraction and nano-pdf edits. - skills/productivity/docx: create (docx-js), edit (unzip -> XML -> zip), tracked changes, comments, validation. Adapted from anthropics/skills. - skills/productivity/xlsx: openpyxl creation/editing, mandatory LibreOffice recalc gate, formula-compatibility rules, financial-model conventions. Points at optional excel-author for finance-grade work. - skills/productivity/pdf: merge/split/rotate/watermark/encrypt, form filling (AcroForm + flat overlay scripts), text/table extraction, reportlab creation, forms.md + reference.md companions. - skills/productivity/powerpoint: synced to current upstream pptx skill — richer pptxgenjs corruption footguns, template workflow, validate.py + validators + thumbnail.py, font-substitution QA guidance; drops the stale pack.py/editing.md/pptxgenjs.md workflow files. - Cross-linked ocr-and-documents, nano-pdf, excel-author via related_skills so each office skill routes to its siblings. - deliverable-mode docs mention the new skills; regenerated per-skill docs pages, catalogs, and sidebar. - tests/skills/test_office_document_skills.py: frontmatter contracts, referenced-script existence, schema-map integrity, cross-link resolution, script compilation. E2E validated: docx create->render->edit->validate, xlsx recalc (SUM + _xlfn.TEXTJOIN evaluate correctly), pdf create->merge->extract, pptx generate->validate->thumbnail. * fix(approval): raise gateway approval timeout to 300s, honest stale-tap UX, offer Always on mixed prompts (#68597) Three related messaging-approval fixes: 1. approvals.timeout default 60 -> 300. PR #63501 collapsed the gateway wait onto the canonical approvals.timeout (previously gateway_timeout=300), silently shrinking messaging approval windows to 60s. Push-notification approvals routinely arrive later than a minute; taps landed after the wait had already failed closed. 2. Stale-tap honesty: adapters resolved the approval AFTER rendering '<checkmark> Approved by <user>' (Telegram/Discord/Slack), or ignored a zero resolve count (WhatsApp Cloud/Feishu). A tap on an expired prompt claimed approval while the command had already been denied. All button paths now resolve first and render 'Approval expired - command was not run' when nothing was waiting. 3. Mixed-warning prompts (dangerous pattern + tirith finding) now offer Always: the persistence layer already permanently allowlists the pattern key and downgrades the tirith key to session scope, but the UI hid Always whenever ANY tirith warning was present. Pure-tirith prompts still withhold Always (content findings are session-max by design), and Smart-DENY overrides remain once-only. * feat(secrets): one-command token rotation + actionable startup errors for all secret sources (#68605) * feat(secrets): one-command token rotation + actionable startup errors for all secret sources When a Bitwarden machine-account token expired, users saw a raw Rust error dump (invalid_client + Location: + backtrace hints) and the only fix was manually editing .env or re-running the whole setup wizard. - New `hermes secrets bitwarden token` / `hermes secrets onepassword token`: paste a new token (masked prompt or flag), the command probes the backend BEFORE persisting — a rejected token changes nothing; a good one is written to .env and the fetch caches are cleared. - New optional SecretSource.remediation(kind, cfg) hook: startup warnings now print a '→ Run `hermes secrets <name> token`…' fix-it line after any fetch error, for bundled AND plugin sources (generic per-ErrorKind defaults in the ABC). - bws stderr is summarized to its cause line (Location:/backtrace noise dropped) and invalid_client/invalid_grant/400 identity rejects are now classified AUTH_FAILED (was INTERNAL) with a plain-English explanation naming the token env var. - op whoami probe accepts a candidate token so rotation validates the NEW credential, not the ambient one. Additive hook with defaults — no SECRET_SOURCE_API_VERSION bump. * docs: fix MDX parse error in secret-source-plugin hook table Escaped backticks around a <name> placeholder made MDX parse it as an unclosed JSX tag, breaking the docs-site build. Use a plain code span instead. * feat(desktop): configure repository discovery (supersedes #67630) (#68642) * feat(desktop): configure repository discovery * fix(config): preserve additive default migration * fix(desktop): stabilize session-actions-menu gateway mock for repo-scan subscribe projects.ts now runs $gateway.subscribe(syncReposScanning) at module load, and nanostores fires the subscriber synchronously. session-actions-menu.test.ts reaches projects.ts transitively via the session store but mocked @/store/gateway without $gateway, crashing the whole desktop vitest suite ("No \ export is defined"). Simply adding $gateway: atom(null) exposed a second issue: the synchronous subscriber calls the mock's activeGateway() during the transitive import, before the module-level const initializes (TDZ). Hoist the mock fns via vi.hoisted() so activeGateway is defined before the hoisted vi.mock factory runs, and add $gateway: atom(null) to the mock. Mirrors the self-contained mock pattern already used in projects.test.ts. Also maps the PR author's commit email for attribution. Supersedes #67630; incorporates review feedback from that PR. Co-authored-by: Rudimar Ronsoni <rudimar@outlook.com> --------- Co-authored-by: Rudimar Ronsoni <rudimar@outlook.com> Co-authored-by: Austin Pickett <austinpickett@users.noreply.github.com> * fix(desktop): ⌘W closes visible file tab when preview selection is stale (#68639) * fix(desktop): make ⌘W close visible file tab on stale preview selection When the live preview target is gone but $rightRailActiveTabId still points at preview, file tabs remain on screen while ⌘W fell through to a workspace no-op. Close the visible file tab instead. * test(desktop): cover ⌘W close for file tabs and ghost preview selection Lock the happy path and the stale-preview regression so ⌘W keeps closing the file tab the rail is actually showing. * fmt(js): `npm run fix` on merge (#68681) Co-authored-by: github-actions[bot] <github-actions[bot]@users.noreply.github.com> * feat(billing): plan chips and rows deep-link their tier (#68666) * fix(desktop): drop the decorative top-up credits bar (#68649) The bar rendered full-or-empty (value 1|0) because top-ups have no denominator — the wire carries only the current balance and the pool is open-ended, so a fill fraction is fiction. Show the amount alone; subscription credits and the monthly cap keep their bars (real denominators). * fix(ci): route critical supply-chain findings through review gate (#68833) Let the scanner report critical findings without failing. The review-label gate owns the action-required status and blocking result, allowing the ci-reviewed label rerun to clear both CI and the PR comment. * fix: `tool_calls` double-encoding on import (#68856) * nix: add `cage` to devShell * test(desktop): add pre-filled sessions support Exports createSandbox, writeMockProviderConfig, writeEnvFile, buildAppEnv, findElectron, and launchDesktop from fixtures.ts so specs can compose their own seeded-backend fixtures without duplicating the sandbox/config/launch logic. * test(desktop): auto-fail e2e tests on error banner Adds a shared test fixture (e2e/test.ts) that wraps @playwright/test's page with an error-banner guard. When any [role="alert"] element (error notification toast) appears in the DOM during a test, the test fails with the error message text. The guard uses: - A MutationObserver (injected via addInitScript) that watches for [role="alert"] elements appearing at any point during the test - A final DOM scan in afterEach for alerts still visible at teardown - Deduplication so the same error text only fires once All existing e2e specs updated to import { test, expect } from './test' instead of '@playwright/test'. No per-spec setup needed — the guard is auto-installed on every page via the extended fixture. This catches issues like the "resume failed" error banner that can appear during session loading — previously the test would pass while an error toast was silently visible on screen. * fix(state): parse tool_calls JSON string before re-serializing _insert_message_rows and append_message both do json.dumps(tool_calls) to serialize the field for SQLite storage. But when tool_calls arrives as a JSON string (from import_sessions / export_session, which store it as TEXT), json.dumps double-encodes it — wrapping the already-serialized string in quotes and escaping the inner quotes. When _rows_to_conversation later does json.loads(row['tool_calls']), the double-encoded string parses back to a plain string (not a list). _history_to_messages then iterates this string character-by-character, calling tc.get('function', {}) on each char — 'str' object has no attribute 'get'. This was a pre-existing bug (on main), but only triggered by the import_sessions path (the live agent always passes tool_calls as a Python list). The e2e error-banner guard caught it via the 'Resume failed' notification toast. Fix: in both append_message and _insert_message_rows, parse tool_calls with json.loads first if it's a string, then re-serialize. * fix(desktop): exempt boot-failure from error guard - boot-failure: add allowErrorBanners() beforeEach — these tests deliberately trigger boot errors, so error toasts are expected - test.ts: export allowErrorBanners() opt-out + reset flag in afterEach * feat(status-bar): add /battery toggle for a color-coded battery read-out Add an opt-in battery indicator to the CLI and TUI status bars, shown as the first element and colour-coded by charge (green/yellow/orange/red, or green while charging). Off by default and a no-op on machines without a battery. - agent/battery.py: shared psutil-backed reader with a short TTL cache, category bucketing, and a compact 🔋/⚡ label. Fails open to "unavailable" everywhere. - CLI: /battery [on|off|status] toggle persisted to display.battery, rendered first in every status-bar width tier. - TUI: /battery slash command, config sync, a system.battery RPC polled while enabled, and a pinned first segment in StatusRule. * fix(approval): restore session approval for Tirith-flagged commands Adds an allow_session flag to the gateway approval payload so adapters can render the session tier independently of the permanent tier. Matrix gains a session reaction (🌀) and a reaction legend; pure-tirith prompts now offer once/session/deny instead of collapsing to once/deny. Salvaged from PR #67312, adapted to the allow_permanent semantics that landed in #68597 (Always offered when any dangerous-pattern warning is persistable; pure-tirith prompts stay session-max). * fix(approval): honor allow_session across all button adapters Widen the allow_session tier from Matrix to every adapter the gateway notifies: Telegram, Discord, Slack, Feishu, and Teams gate their Session button on it; WhatsApp Cloud and qqbot accept the kwarg (no session tier in their button sets). Also thread allow_session through the plugin- escalation gate, the execute_code guard payload, and the plain-text fallback so every notify path carries the same capability flags. * test(approval): cover allow_session tiers in Matrix reaction seeding and gateway payload Update the Matrix reaction-seeding contract to the four-reaction default (once/session/always/deny), add tirith-tier (session without always) and no-session-tier cases, and assert allow_session=True in the tirith gateway payload. * fix(desktop): wrap missing sidebar icon-button tooltips (#67500) * fix(desktop): wrap sidebar icon buttons in Tip tooltips Several icon-only buttons in the sidebar (header actions, workspace menu, project menu, session actions, load-more) had aria-label but no visual tooltip on hover. Wrap them in the existing <Tip> component, matching the pattern already used elsewhere (e.g. ProfilePill). No behavioral changes -- purely wraps existing buttons. Adds vitest coverage asserting the Tip wrapper (data-slot=tooltip-trigger) for 6 of 7 files; index.tsx is a 1500+ line top-level page component and was verified manually via screenshots instead. * fix(desktop): satisfy consistent-type-imports lint rule in project-dialog test * test(desktop): update session-row mocks for restored sessionColorById * fix(desktop): compose Tip around the real trigger instead of inside it Tip was being placed as SessionActionsMenu's/PlatformAvatar's DIRECT child, which asChild then cloned instead of the actual button/span. Neither Tip nor PlatformAvatar forwarded the injected onClick/ref, so both silently dropped the wiring: - session-actions-menu.tsx: Tip now wraps DropdownMenuTrigger internally (new ooltip prop) instead of the caller wrapping its children in Tip. - platform-icon.tsx: PlatformAvatar now forwards ref and spreads rest props onto its span so a wrapping Tip's trigger actually attaches. - session-row.tsx: updated call site to use the new tooltip prop. - Added session-actions-menu.test.tsx exercising the real DropdownMenu open behavior end-to-end (no Tip/Dropdown mocks). - session-row.test.tsx no longer mocks PlatformAvatar's behavior; it now exercises the real (fixed) component for the handoff-avatar tooltip. * fix(desktop): compose Tip outside PopoverAnchor in ProjectMenu (#67500) * test(desktop): update session-row test for the tooltip-prop composition (cbbbeb2fd) * fix(desktop): satisfy consistent-type-imports in session-row.test.tsx mocks * chore: retrigger CI * test(desktop): stop mocking PlatformAvatar's behavior (#67500, third pass) The mock was re-introduced by a prior edit that fixed an unrelated lint error, silently undoing the earlier fix where this test started exercising the real (forwardRef) PlatformAvatar. Removed the mock; updated the two handoff-avatar tests to query the real component's rendered span instead of text content, since it renders a brand SVG icon for known platforms rather than the platform name as text. * fmt(js): `npm run fix` on merge (#68867) Co-authored-by: github-actions[bot] <github-actions[bot]@users.noreply.github.com> * fix(gateway): detect stale lock when macOS psutil returns valid start_time for recycled PID On macOS, the lock record's start_time is None (no /proc at creation), but psutil.Process(recycled_pid).create_time() returns a valid float for the unrelated process that now owns the PID. The old condition required both sides to be None before falling back to cmdline checking, so the recycled PID was never detected as stale. Change the fallback condition from AND to OR: when either side's start_time is missing, fall back to cmdline-based gateway detection. Fixes #53763 * fix(gateway): handle PermissionError on stale root-owned lock file When the macOS launchd service runs in a Background session, the gateway process spawns as root and creates a root-owned gateway.lock. On restart as the normal user, open() on that file raises PermissionError, crashing the gateway immediately and entering a launchd crash loop. Catch PermissionError in is_gateway_runtime_lock_active(), remove the stale lock file, and return False so the new process can start cleanly. Fixes #42685 * fix(gateway): guard acquire_gateway_runtime_lock against root-owned lock PermissionError Widen the PermissionError handling from is_gateway_runtime_lock_active (#42689) to the sibling open() in acquire_gateway_runtime_lock: a stale root-owned gateway.lock left by a launchd Background session previously crashed the acquiring process. Unlink the stale file and retry once; if the unlink or retry fails, return False cleanly instead of raising. * fix(gateway): make stale scoped-lock removal atomic via tombstone rename Replace the unlink()+O_EXCL sequence in acquire_scoped_lock with an atomic os.replace() of the stale lock to a <lock>.stale tombstone followed by the existing O_EXCL create. With plain unlink(), two racing starters could both judge the lock stale and the second unlink() would silently delete the first racer's freshly-created lock — both would then 'win'. os.replace() guarantees exactly one racer claims the stale file; the loser gets FileNotFoundError and falls through to O_EXCL, which admits at most one winner. Tombstones are cleaned up immediately; behavior is otherwise identical. * fix(gateway): detect stale gateway_state.json in `gateway status` (TTL + PID liveness) Verified: applies cleanly and the patched module compiles. Tests are described in the PR body (not bundled in this commit). Co-Authored-By: Claude Opus 4.8 <noreply@anthropic.com> * test(gateway): cover stale gateway_state.json detection (TTL + PID liveness) Co-Authored-By: Claude Opus 4.8 <noreply@anthropic.com> * fix(gateway): take over live platform-lock token holders once When --replace misses a cross-HERMES_HOME Telegram token holder, platform connect used to retry forever. Terminate a verified gateway holder once (with the takeover marker) and re-acquire the scoped lock (#65176). Co-authored-by: Cursor <cursoragent@cursor.com> * chore(contributors): map jaretbottoms@gmail.com -> jbbottoms (PR #65178 salvage) * fix(gateway): reap the replaced gateway's orphaned children on POSIX Builds on jbbottoms's #65178 takeover fix (cherry-picked as the previous commit). Windows --replace already tree-kills via taskkill /T, but the POSIX paths signalled only the recorded gateway PID — adapter subprocesses that outlived their parent kept holding scoped token locks and blocked the replacement gateway. - gateway/status.py: _snapshot_gateway_children() captures the old gateway's descendants (psutil, recursive) while it is still alive; reap_gateway_children() SIGTERMs verified orphans after the main PID is confirmed dead, waits bounded, SIGKILLs survivors. Identity-aware (psutil is_running is PID+create-time), skips zombies and children whose ppid still equals the old gateway (parent actually alive), and never raises — best-effort with debug/info logging only. - take_over_scoped_lock_holder() snapshots before terminating and reaps only on a confirmed successful handoff. - gateway/run.py: start_gateway --replace snapshots before SIGTERM and reaps after the old PID is confirmed gone, mirroring taskkill /T. - tests/gateway/test_replace_child_reap.py: reap/skip/never-raise unit coverage plus end-to-end --replace ordering (snapshot → terminate → reap) and the no---replace path never touching the old process. * chore(contributors): map emails for PRs #66906, #66420, #63398 salvage * fix(state): probe FTS5 read path in _db_opens_cleanly so partial index corruption is detected (#66724) `hermes sessions repair --check-only` opens cleanly on state.db files with partial FTS5 index corruption — base tables read fine, the rolled-back write probe from #50502 succeeds, and `PRAGMA integrity_check` returns "ok". But every session_search / /resume title resolution / feature backed by MATCH / snippet / rank queries errors out with `database disk image is malformed` because internal shadow-table segments are bad. The official repair tool then gives false confidence. Add a representative FTS5 read probe against both `messages_fts` and `messages_fts_trigram` (the latter backs title resolution). Empty MATCH strings are accepted by every FTS5 index without requiring populated content, so the probe is safe on a freshly-init'd DB; missing-table / missing-column errors fall through to the existing "not yet a populated DB" branch, matching the write-probe's behaviour. Any other OperationalError is surfaced as the check reason, which sends `hermes sessions repair` to its existing FTS 'rebuild' path (repair_state_db_schema, line 616). Single-file change in hermes_state.py::_db_opens_cleanly. No public API change. No new imports. Fixes #66724. * fix(state): also catch sqlite3.DatabaseError in FTS5 read probe (#66724) The FTS5 read probe in _db_opens_cleanly() only caught sqlite3.OperationalError. But the corruption class #66724 actually wants caught — partial shadow-table damage where MATCH / snippet / rank queries raise DatabaseError("database disk image is malformed") — is a DatabaseError, not OperationalError. Without this catch the probe crashes the caller instead of returning a reason, which is exactly the silent-fail mode the issue describes. Move the try/except inside the for-loop so each FTS table is probed independently (one table corrupted should still surface as a reason), add a separate except clause for DatabaseError that surfaces the same reason format, and use continue instead of pass so the loop still walks both tables when only one is missing on a brand-new DB. Tested by hand: with a corrupted messages_fts_trigram shadow table the function now returns 'fts5 read probe failed on messages_fts_trigram: database disk image is malformed' instead of crashing out. Without this fix it would still crash. * fix(state): preserve degraded-runtime read probe + use canonical FTS5 classifier Two follow-ups on top of f842733 (the FTS5 read probe added in #66906): 1. The original probe query used MATCH '', which FTS5 rejects with 'fts5: syntax error near '. Empty MATCH syntax is not valid FTS5. Switch to MATCH '""' — a quoted empty phrase that parses, scans zero rows, and exercises the same shadow-table read path the search tools use. The probe previously never reached the shadow segments at all on a healthy DB; the read-corruption class was only being detected because the existing write probe happens to fail first on a DatabaseError. 2. The probe's degraded-runtime branch only checked the substrings 'no such table' / 'no such column'. On a SQLite build without the fts5 module, MATCH against a legacy messages_fts table raises 'no such module: fts5' (a different OperationalError class). The substring check would misclassify that as corruption and trigger repair, whose final fallback deletes the messages_fts% schema (#66906 review). Use SessionDB._is_fts5_unavailable_error() — the canonical classifier already used by the degraded-runtime init path — to recognize both 'no such module: fts5' and 'no such tokenizer: trigram' as capability errors. Add tests covering: - Partial shadow-table damage (read-corruption class) - Repair brings reads back online - Healthy degraded DB without fts5 module stays healthy (regression for the misclassification risk) - Healthy degraded DB without trigram tokenizer stays healthy Closes #66906 review feedback Refs #66724 * fix(state): self-heal FTS corruption on the SessionDB search path too Complements #66296 (self-heal on the write path): search_messages()'s main FTS5 MATCH query caught only sqlite3.OperationalError (a query-syntax error → return empty). A corrupt FTS index raises the malformed / "fts5: corrupt structure record" class, which is a sqlite3.DatabaseError — the parent of OperationalError, so it was NOT caught and propagated straight out of search_messages, crashing session/history search. The write path now rebuilds and retries on that class, but a read-only session (cron/CLI history search, or a search issued before any write) never triggers a write, so its search stayed broken until the next process restart ran the offline repair. Catch the DatabaseError corruption class on the search MATCH read too and route it through the existing one-shot _try_runtime_fts_rebuild(), then retry the query. The catch is moved outside `with self._lock` so rebuild_fts() can re-acquire the lock (mirrors _execute_write). The one-shot guard is shared with the write path, so a single instance never loops on a genuinely unrecoverable index. OperationalError syntax handling is unchanged (caught first). Adds a regression test: with a corrupted messages_fts and no post-corruption write, search_messages() rebuilds in place and returns the match; without the fix it raises DatabaseError. * fix(state): extend search-path FTS self-heal to the CJK/trigram branch The trigram MATCH branch in search_messages() had the same OperationalError-only catch that #66420 fixed on the main FTS5 branch: a corrupt messages_fts_trigram shadow table raises the malformed / 'fts5: corrupt structure record' class (sqlite3.DatabaseError, parent of OperationalError), which propagated straight out of search_messages and crashed CJK session/history search for read-only sessions. Route that class through the shared one-shot _try_runtime_fts_rebuild() and retry the trigram query (catch moved outside self._lock so rebuild_fts() can re-acquire it, mirroring the main branch). If the rebuild is refused (guard consumed / FTS disabled / different error) or the retry fails, fall through to the existing LIKE substring fallback — which reads only the canonical messages table — instead of raising, so CJK search degrades gracefully rather than crashing. Adds two regression tests: trigram search self-heals in place after shadow-table corruption (answers from the rebuilt trigram index, not the LIKE fallback), and degrades to LIKE without raising when the one-shot rebuild was already consumed. Follow-up to #66420; refs #66296 #66724 * fix(state): add REINDEX strategy to repair stale B-tree indexes (#63386) When PRAGMA integrity_check reports 'wrong # of entries in index' for B-tree indexes (e.g. idx_sessions_handoff_state), the existing repair strategies (FTS rebuild, sqlite_master dedup, drop-FTS+VACUUM) don't address the mismatch. Add Strategy 0.5: run REINDEX to rewrite the index b-tree from canonical table rows before escalating to more destructive strategies. * test(state): exercise REINDEX repair against a REAL stale B-tree index Replace the mocked test for #63398's REINDEX strategy: the original monkeypatched _db_opens_cleanly to return the corruption string, so the REINDEX pass itself was never exercised against actual index corruption — the test would pass even if REINDEX didn't fix anything. New fixture _corrupt_btree_index() builds genuine on-disk staleness with a writable_schema hack: rewrite the index definition to a partial index (WHERE 0), REINDEX so the b-tree is rebuilt empty, then restore the full definition. integrity_check then reports the real 'wrong # of entries in index idx_messages_session' / 'row N missing from index' class from #63386 — no mocks anywhere. The rewritten test asserts end-to-end with real function calls: - the real _db_opens_cleanly detects the stale index, - repair_state_db_schema repairs it with strategy 'reindex_btree', - post-repair the detector and raw PRAGMA integrity_check both report healthy, and a query forced through the rebuilt index (INDEXED BY) sees every row. Adds a second test asserting the REINDEX strategy is non-destructive (all sessions/messages survive, readable via SessionDB). Follow-up to #63398; refs #63386 * fix(kanban): auto-repair index-only kanban.db corruption via REINDEX _guard_existing_db_is_healthy previously failed closed on ANY integrity_check failure, including the index-scoped class ('wrong # of entries in index <name>' / 'row N missing from index <name>') where the table b-trees are intact and REINDEX rebuilds the damaged indexes losslessly. Boards hit by that class were bricked until manual surgery even though SQLite can fix them in-place. Now, when integrity_check output consists ONLY of index-scoped errors (index name parsed generically from the message — no hardcoded list): 1. quarantine the corrupt bytes FIRST via the existing content- addressed _backup_corrupt_db, 2. under the caller-held cross-process init flock, REINDEX each named index (falling back to bare REINDEX if a parsed name doesn't resolve), 3. re-run integrity_check and proceed only if it comes back clean. Any non-index error class (page corruption, malformed image, freelist damage) — or a REINDEX whose re-check is still dirty — fails closed exactly as before: backup + KanbanDbCorruptError, no silent recreation. Transient OperationalError (locked/busy) still propagates raw with no quarantine. Tests build a real board DB and corrupt a live index via the writable_schema/partial-index REINDEX trick to produce the genuine 'wrong # of entries in index' shape, then assert auto-repair recovers with data intact, page corruption still raises, and a dirty re-check fails closed. * fix(kanban): cap corrupt-backup retention at 10 files per board DB Content-addressed quarantine backups dedupe identical corrupt bytes, but corruption that keeps mutating between failures (partial repairs, further damage across dispatcher retries, multi-profile fleets) mints a new sha-named backup every round — a user accumulated 124 .corrupt.*.bak files with no bound. After each NEW backup is created, prune oldest-by-mtime backups beyond _CORRUPT_BACKUP_RETENTION (module constant, default 10), including the copied -wal/-shm sidecars. The just-created backup is always exempt (copy2 preserves the source mtime, which can be older than existing backups). Pruning is best-effort and never masks the corruption error about to be raised; dedupe of identical corrupt bytes is unchanged. * feat(kanban): periodic WAL checkpoint (TRUNCATE) on the dispatcher tick Kanban connections set wal_autocheckpoint=100, but SQLite's passive autocheckpoint backs off whenever any reader holds an open snapshot — on a busy multi-process board the -wal file can grow without bound between gateway restarts. After each successful dispatch tick, while still holding the board's single-writer dispatch flock, run PRAGMA wal_checkpoint(TRUNCATE) best-effort at a coarse interval (>=5 min since this process last checkpointed that board; module-level per-path monotonic timestamp, so multi-board dispatchers checkpoint each board on its own clock). Success and busy/locked skips are both logged at DEBUG; a failing checkpoint can never fail the tick. * feat(kanban): add `hermes kanban repair` CLI verb Adds kanban_db.repair_db() — a structured, non-raising wrapper around the same narrow repair policy as the connect-time guard: probe with PRAGMA integrity_check under the board's cross-process init flock; quarantine the corrupt bytes FIRST via the content-addressed backup; REINDEX only when every integrity message is index-scoped; re-check; report ok / repaired / corrupt / missing. Locked/busy OperationalError still propagates raw (a locked healthy DB is not corruption and gets no quarantine), and a repair invalidates the per-process healthy-path cache so the next connect() re-probes. The CLI verb reports status human-readably (or --json), exits 0 for ok/repaired/missing and 1 when the DB is still corrupt (non-index corruption stays fail-closed with manual-recovery guidance). It dispatches BEFORE kanban_command's auto-init: init_db() raises KanbanDbCorruptError on a corrupt board, which previously would have made a repair verb unreachable on exactly the boards that need it. CLI tests drive the real argparse surface (build_parser + kanban_command) against real corrupted SQLite fixtures. * fix(packaging): graft web_dist in MANIFEST.in and add sdist regression test Wheels ship hermes_cli/web_dist via pyproject package-data, but the sdist did not: MANIFEST.in had no graft and .gitignore excludes web_dist, so source tarballs installed a dashboard-less package. Graft the directory and add an sdist regression test that builds the tarball and asserts index.html is inside. Salvaged from #29661; the PR's [web]-extra 404-message change was dropped per maintainer review (misleading guidance for source installs). * fix(dashboard): attempt one recovery build when --skip-build finds no dist --skip-build with a missing web_dist/index.html p…

…ma v23) (NousResearch#65798) * fix(desktop): refresh repo status on session switch with unchanged cwd (#68208) fix(desktop): refresh repo status on session switch with unchanged cwd * fix(checkpoints): honor gateway config and task cwd (#68195) * fix(gateway): wire checkpoint config into agents * fix(checkpoints): resolve gateway file paths by task cwd * ci: live-updating PR review comment with structured job statuses Replace the static comment-pending + comment-results two-job pattern with a live-updating comment system that polls the GitHub Actions API every 15s, re-assembles the review comment from whatever results are available, and upserts it via the  marker. The comment updates in real time as each job finishes — no waiting for the full pipeline. Every CI job that wants to appear in the review comment emits a review_status output — a JSON array of objects, each with a source and a results array: [ { "source": "review-label-gate", "results": [ {"kind": "action_required", "title": "...", "summary": "...", "how_to_fix": "..."}, {"kind": "info", "title": "...", "summary": "..."} ] }, { "source": "ci timing", "results": [ {"kind": "warning", "title": "CI timings", "summary": "...", "detail": "...", "link": "..."} ] } ] One job can emit multiple results of different kinds. The source field is used to exclude the corresponding job from the synthesized error list (case-insensitive, hyphen-normalized matching against GitHub Actions job display names). | job | source | kind (on failure) | section | |----------------------------|--------------------------|---------------------------|----------------------| | review-labels | review label gate | action_required / info | Action required | | lockfile-diff | lockfile-diff | action_required | Action required | | ci-timings | ci timing | warning / info | Warnings | | supply-chain scan | supply chain | error / (none) | Job failures | | supply-chain dep-bounds | supply chain | action_required / (none) | Action required | | osv-scanner | osv scan | warning / (none) | Warnings | | uv-lockfile-check | uv.lock check | action_required / (none) | Action required | | history-check | unrelated histories | action_required | Action required | | contributor-check | contributor attribution | action_required | Action required | Jobs that find nothing emit [] (empty array) — no noise info items. A single comment-live job polls the GitHub Actions API every 15s, classifies jobs into (completed, pending), assembles the comment, and upserts it. Merges review_status outputs from all needs jobs via toJSON(needs.*.outputs.review_status), and downloads the ci-timings artifact when it becomes available. Shows commit SHA + message below the header. The assembler has ZERO job-specific knowledge. It just: 1. collect_from_statuses() — flattens all nested status objects into ReviewItems 2. collect_failed_jobs() — synthesizes errors for failed jobs with no declared status 3. _attach_job_urls() — fills in per-job log links for ALL items 4. render_comment() — groups by severity, renders with group headers Each item shows links inline next to the title: View report (job-emitted URL) and View job (auto-attached logs link). Each info item is its own collapsible <details> block. # ૮ >ﻌ< ა ci review running on abc1234 — commit message first line ## ❌ Job failures ### {title} · [View job](url) {summary} ## ⚠️ Action required ### {title} · [View job](url) {summary} **How to fix:** {how_to_fix} ## ⚠️ Warnings ### {title} · [View report](url) · [View job](url) {summary} {detail} <details><summary>{title}</summary> {content} </details> Still running 3 jobs: ci-timings, docker - test_assemble_review_comment.py (48 tests): collect_from_statuses, collect_failed_jobs with exclude_sources, _attach_job_urls, render_comment (group headers, inline links, commit info, per-item details, pending footer), assemble integration - test_live_comment.py (16 tests): classify_jobs pure function - test_timings_report.py (10 tests): generate_review_status nested format - test_lockfile_diff.py (6 tests) - test_classify_changes.py (32 tests, pre-existing) * ci: migrate AUTOFIX_BOT_PAT to GitHub App token Replace the long-lived fine-grained PAT (AUTOFIX_BOT_PAT) with short-lived (1-hour) installation access tokens minted via a new get-app-token composite action wrapping actions/create-github-app-token@v3.2.0. The PAT was used in 13 spots across 8 workflow files for gh CLI / GitHub API calls. The per-repo GITHUB_TOKEN (1,000 req/hr) was getting rate-limited when multiple workflows fire concurrently (deploy-site, skills-index, ci-timings, supply-chain-audit, js-autofix). App installation tokens get 5,000 req/hr per installation and are scoped to the App's permissions, not a user account. New composite action: .github/actions/get-app-token/ - Wraps actions/create-github-app-token@bcd2ba49 (v3.2.0, SHA-pinned) - Reads APP_ID + APP_PRIVATE_KEY repo secrets - Outputs a 1hr installation token via steps.app-token.outputs.token Requires two new repo secrets (set after creating the GitHub App): - APP_ID: the App's numeric ID - APP_PRIVATE_KEY: the PEM private key App installation permissions needed: contents: write (js-autofix push, pypi release upload) pull-requests: write (js-autofix PR create/merge, supply-chain comment) issues: write (skills-index-freshness issue creation) actions: write (skills-index workflow trigger) workflows: write (skills-index triggers deploy-site.yml) The AUTOFIX_BOT_PAT secret can be deleted once CI passes on this PR. The comment in js-autofix.yml noting that PAT pushes trigger downstream workflows is updated — App tokens have the same property (they are not GITHUB_TOKEN), so the concurrency-cancel loop logic is unchanged. * style(desktop): satisfy merged eslint/prettier config The SSH modules predate the stricter lint config that landed on main (curly, no-empty, perfectionist sorting, prettier). Mechanical lint:fix + fmt pass, empty catch blocks filled with the codebase's void-0 convention, and inline no-control-regex disables on the three deliberate control-char patterns (same pattern as lib/ansi.ts). * fix(ci): pass App secrets as inputs to composite action Composite actions cannot access the secrets context — the runner's template engine rejects secrets.* references at load time with 'Unrecognized named-value: secrets'. Move APP_ID and APP_PRIVATE_KEY from direct secrets.* references inside the composite action to inputs passed by each calling workflow. The fallback logic (GITHUB_TOKEN when APP_ID is empty, for fork PRs) stays in the composite action's check step. * fix(ci): add detect to all-checks-pass needs so its failure blocks merge If detect fails, all downstream sub-workflows get SKIPPED (they have needs: detect). all-checks-pass used if: always() and only checked the sub-workflows — which all showed as 'skipped' (= success) — so it passed even though the root cause (detect) failed. This made the PR mergeable despite a broken CI pipeline. Add detect to all-checks-pass needs so its failure propagates to the gate job and blocks the merge. * fix(desktop): bump skills test timeout to fix cold-start flake (#68235) Test 1 in skills/index.test.tsx pays the full cold-start cost (jsdom env init + module transform + the @/hermes/@/store/profile import graph), which pushed past vitest's 5000ms default under load — caught at 8871ms on one run, 6.6s pure test time on another. Tests 2-4 are ~30-130ms each because all that setup is already cached, so only test 1 was at risk of timing out. Bump the describe-level timeout to 15s. Verified with 10 consecutive runs, 4 of which took 5.5-6.6s of test time and would have hard-failed under the old 5s default. * feat(desktop): open multiple full app windows (electron) Add createInstanceWindow() — a full-chrome peer of the primary that renders the complete app (sidebar, routing, its own draft) against the shared backend, so several GUI windows can run at once. Mirrors the primary's window options + chatWindowWebPreferences (backgroundThrottling stays off so a streamed answer never stalls when blurred) but never overwrites the mainWindow global and doesn't respawn the backend — the renderer's getConnection() joins the running one. New windows cascade off their source via the pure, tested instanceWindowBounds(). Exposed via the hermes:window:openInstance IPC and a "New Window" File menu item. Per-window fullscreen state now targets the window itself, and titlebar/native-theme repaints reach every open chat window instead of only the primary. Retires the now-orphaned compact new-session pop-out (its only caller was ⌘⇧N, repointed in the follow-up commit): drops createNewSessionWindow, the hermes:window:openNewSession handler, and the newSession/new=1 URL flag. * feat(desktop): wire New Window to ⌘⇧N + command palette Repoint session.newWindow (⌘⇧N) from the compact new-session pop-out to openNewWindow(), which opens a full peer instance via the new openWindow bridge, and add a "New Window" entry to the ⌘K palette (shown with its hotkey hint, gated on canOpenNewWindow()). Relabel the action "New window". Drops the retired openNewSessionWindow bridge and the vestigial isNewSessionWindow()/new=1 flag; renames the shared opener helper. * fix(desktop): de-dupe cross-window cues so peers don't spam With multiple full windows, each renderer independently reacts to the same backend event, so one-shot cues fired N times: OS notifications (the per-renderer throttle can't see other windows), the turn-end sound (playCompletionSound runs on every message.complete, ungated by focus), and auto-spoken replies (double voice when a chat is open in two windows). Add a single race-free owner in the main process (electron/event-dedupe.ts): main handles IPC serially, so the first window to claim a key within a short window wins and peers stay quiet. Notifications collapse at the hermes:notify choke point; the sound and spoken replies claim via a new hermes:ambient:claim IPC (keyed by session / reply id). Off Electron the claim falls back to "emit", preserving single-window behavior. The sound's mute check runs before the claim so a muted window can't win the cue and silence an audible peer. * refactor(desktop): tidy the cross-window deduper Drop the unused DEDUPE_WINDOW_MS export and rename its interval so "window" isn't overloaded against BrowserWindow in a multi-window feature (windowMs → intervalMs). DRY the completion-sound play path. No behavior change. * nix: add cage to devDeps * fix(desktop): avoid false remote gateway reauthentication (#68250) * fix(desktop): avoid false remote gateway reauthentication Co-authored-by: Rod-fernandez <rodrigo@nxtlevelsaas.com> Co-authored-by: David Andrews (LexGenius.ai) <david@lexgenius.ai> * fix(desktop): harden remote revalidation state --------- Co-authored-by: Rod-fernandez <rodrigo@nxtlevelsaas.com> Co-authored-by: David Andrews (LexGenius.ai) <david@lexgenius.ai> * fix(desktop): keep composer draft across compression tip rotation (#68079) * fix(desktop): keep composer draft across compression tip rotation Auto-compression swaps the live stored session id while the user may still be typing. Scope the composer/queue key on the lineage root and migrate any tip-keyed draft/queue entries onto that durable key when the tip rotates so the in-progress prompt does not vanish when the response lands. * test(desktop): cover draft survival across compression tip rotation Add regression coverage for migrateSessionDraft, lineage-scoped composer keys, and the rotation path that previously wiped an in-progress draft. * fmt(js): `npm run fix` on merge (#68305) Co-authored-by: github-actions[bot] <github-actions[bot]@users.noreply.github.com> * fix(desktop): Stop parks the queue instead of firing the next queued prompt Interrupting a busy turn with the Stop button (or Esc) settles the session to idle, and the edge-independent auto-drain immediately submits the head of the composer queue. The user pressed Stop to halt the agent, but it looks like Stop skipped the current turn and kept going — and the queued text is hard to find, since its only surface is the collapsed 'N queued' pill above the composer. The old userInterruptedRef latch (a23728dcc) fixed this but was removed in #40221 because it also suppressed the drain that send-now-while-busy depends on. This reintroduces the halt with source awareness instead of a blanket latch: - Explicit halts (Stop button, composer Esc, chat-focus Esc, the streaming message's hover Stop, runtime cancel) park the session's queue before interrupting. Parked queues are skipped by both auto-drain paths (mounted ChatBar + background drainer). - Interrupts that exist to advance the queue (send-now-while-busy) unpark first, so the settle drain they rely on still flows. - The park lifts on any renewed intent: resume, a manual drain (Enter on empty composer or the per-row send arrow), queueing a new prompt, or emptying the queue. It migrates with entries on a runtime re-key and is deliberately not persisted (a fresh process starts unparked). - The queue panel expands on park, switches to 'N Queued — paused' with a pause icon, and grows a Resume action, so the held prompts are visible instead of reading as vanished. Store contract, hook wiring, and background-drain coverage included; docs updated. * fix(cli,tui): recall real paste content on up-arrow Large pastes collapse to a placeholder in the composer, but input history stored the placeholder — so up-arrow recall showed a truncated reference (CLI) or lost the content entirely (TUI, where the `[[…]]` label has no backing snip after submit). Store the expanded content in history instead: - CLI: `_inline_pastes()` expands `[Pasted text #N -> file]` into the buffer before `reset(append_to_history=True)`; also reused by the external editor (dedup). History nav suppresses re-collapse of recalled content. - TUI: `dispatchSubmission` pushes `expandSnips(pasteSnips)(full)`; idempotent on label-free text so re-submitting a recalled entry stays stable. * fix(cli): suppress CPR on POSIX local TTYs under load Delayed ESC[6n replies leak as ^[[row;colR into the classic CLI on SSH/slow PTYs (#13870) and on local POSIX TTYs under heavy subagent load. Suppress CPR on non-Windows platforms (layout hint only); keep native Windows on prompt_toolkit's default pending native coverage. Wire selection through _select_classic_cli_pt_output. * test(cli): prove local CPR leak and Application CPR-disabled wiring Add a delayed-CPR PTY harness (no SSH) plus selection/Application assertions for POSIX local and Windows preserve-default. Update the gating unit test to the new contract. * refactor: drop platform kwarg, fix PTY test cleanup - Remove redundant platform= test seam from _terminal_may_leak_cpr(); use monkeypatch.setattr(sys, 'platform', ...) consistently in both test files. - Wrap PTY tests in try/finally for fd cleanup on assertion failure. - Guard select.select() in terminal thread against OSError after fd close (fixes PytestUnhandledThreadExceptionWarning). - Trim PR-number reference from test module docstring. * docs(portal): remove retired Nous Chat references * fix(web/ddgs): isolate DuckDuckGo search in a disposable process ThreadPoolExecutor timeouts cannot fire when primp holds the GIL in native code (#68096). Run each search in a child process the parent can terminate/kill, and honor tools.interrupt between polls. * test(web/ddgs): cover GIL-hold timeout, interrupt, and worker reap Regression tests for #68096: native GIL-hold and sleep hooks must time out or interrupt promptly with no orphaned search workers. * fix: sanitize subprocess env for DDGS worker os.environ.copy() passes all Hermes secrets (gateway tokens, API keys, dashboard session tokens) into the DDGS child process. Use _sanitize_subprocess_env() to strip Hermes-managed secrets before spawning the worker. * fix(agent): pass persisted-prefix boundary when rotation flushes on cold resume (#68196) The legacy rotation branch in agent/conversation_compression.py flushes the current turn to the OLD session before ending it (#47202) via _flush_messages_to_session_db(messages) with no conversation_history boundary. On the first turn after a cold Desktop resume, the restored transcript rows live in the message list as plain dicts that have not yet been stamped with _DB_PERSISTED_MARKER — the normal turn flush that stamps them runs after preflight compression. With no boundary, _flush_messages_to_session_db builds an empty history_ids set and treats every restored row as new, durably re-appending the whole transcript to the parent session. Repeated restart/resume + threshold compression keeps growing the parent transcript. Pass messages[:_persist_user_message_idx] (the already-durable prefix that turn_context anchors before preflight runs, guarded for int/bounds) as conversation_history so the flush skips the persisted rows by identity and writes only the current turn's new messages. Adds a regression test that pre-populates SQLite, cold-loads the transcript, appends one current user row, and forces rotating compression: it fails before this change (parent grows to 5 rows) and passes after (parent holds the two originals plus the single new turn). * fix(desktop): prevent contentEditable composer input from visually collapsing to near-zero height Fix #68095 The composer input box (contentEditable div) randomly shrank to a tiny/pixelated size when typing character-by-character (paste worked fine). Root cause: during per-keystroke input, the normalizeComposerEditorDom cleanup could briefly leave the contentEditable with zero child nodes, and without intrinsic content the browser collapsed it visually despite the CSS min-height. Two-pronged fix: 1. Add min-h-[1.625rem] bracket syntax alongside the CSS variable min-height to ensure the minimum height is enforced even if the CSS variable resolution is delayed or overridden by browser defaults. 2. In normalizeComposerEditorDom, ensure the contentEditable always has at least one <br> child when empty, giving it intrinsic height that the browser cannot collapse. This is a belt-and-suspenders approach with the CSS min-height. Closes #68095 * fix(agent): circuit-break AttributeError from commit-splice and detect code skew Fix #68178 The git-install auto-updater rewrites source while the desktop backend is live. Because agent/conversation_loop.py is imported lazily on the first API call, a process can end up running two different commits spliced together — one commit's AIAgent against another commit's conversation_loop. When the interface differs, every turn fails permanently with an AttributeError, and the loop retries indefinitely, burning provider API calls (576 failures, 149 wasted API calls observed). Three-prong fix: 1. Circuit-break AttributeError on agent objects: the outer-loop error classifier now detects AttributeError targeting agent/run_agent modules and breaks immediately instead of continuing the retry loop. 2. Code skew detection for desktop/serve backend: run_agent.py now snapshots the checkout revision at import time and exposes a cheap per-iteration check that the conversation loop uses to refuse new work with a clear 'restart required' message before the lazy import can crash. 3. Informative error message: when code skew is detected, the user gets a clear explanation of the mismatch (boot revision vs current revision) and actionable guidance to restart the application. * fix(telegram): preserve fatal recovery handoff Release the current polling-recovery task's ownership before invoking the fatal-error handler. The runner bounds adapter cleanup in a child task; disconnect() cancels the tracked polling-recovery task, so retaining the current notifier in _polling_error_task would cancel the fatal callback before the runner can finish its reconnect-queue or shutdown decision. The new _handoff_polling_fatal_error() helper clears _polling_error_task only when it is the current notifier. Other recovery tasks remain tracked and are still cancelled and awaited during teardown. Covers both network retry exhaustion and polling-conflict exhaustion. Replaces the misleading "Restarting gateway" message with "Escalating to gateway recovery". Fixes #68406. * fix(telegram): widen fatal handoff to heartbeat watchdog path The wedged-recovery heartbeat watchdog (line 2526) calls _notify_fatal_error() directly from the heartbeat task. disconnect() cancels _polling_heartbeat_task unconditionally (no current_task guard, unlike _polling_error_task). Same bug class as #68406: the child disconnect cancels the heartbeat parent before the runner can queue reconnect. Widen _handoff_polling_fatal_error() to also clear _polling_heartbeat_task when it is the current task, and route the heartbeat watchdog call site through the handoff helper. Co-authored-by: Imgaojp <6065749+Imgaojp@users.noreply.github.com> * fix(tests): make the live-system-guard canary fail closed tests/test_live_system_guard_self_test.py executes real kill primitives (os.kill(-1, SIGTERM), os.killpg, pkill -f python) and depends entirely on the autouse _live_system_guard fixture in tests/conftest.py to intercept them. That makes the canary fail-OPEN: in any collection context where the file is present but its home conftest is not — a published sdist that ships tests/ but not tests/conftest.py, a tree assembled by copying test*.py (that glob does not match conftest.py), pytest --noconftest, or a foreign rootdir — the primitives fire for real, and os.kill(-1, SIGTERM) SIGTERMs every process the invoking user owns (a full desktop-session kill was reported in the field). Add an autouse fixture that refuses to run any canary test unless the guard is provably active. The one thing the canary can detect about its own safety is that the guard monkeypatches os.kill with a plain Python function, whereas the unguarded primitive is a C builtin — so the probe keys off that. Tests marked @pytest.mark.live_system_guard_bypass still opt out, matching the guard's own bypass contract (e.g. test_bypass_marker_disables_guard). With the guard loaded every canary test behaves exactly as before; without it each test refuses at setup with zero side effects. Fixes #68311 * fix(billing): rename user-facing "terminal billing" copy to Remote Spending (#68355) * fix(billing): rename user-facing "terminal billing" copy to Remote Spending The capability was renamed Remote Spending on the portal (consent CTA: "Allow Remote Spending"; per-terminal states Granted/Stopped), but the terminal, desktop, and docs still said "terminal billing" everywhere. - Feature name: Remote Spending in titles/labels, lowercase mid-sentence. - Step-up action verb is now "allow", matching the portal consent CTA. - Kill-switch-off recovery copy points at the actual control ("a billing admin can turn it on from the portal's Hermes Agent page") instead of the dead-end "manage it on the portal". - Per-terminal revoke copy uses the portal vocabulary ("stopped"). - Wire identifiers (cli_billing_enabled, cli_billing_disabled, ...) are unchanged; copy, comments, docs, and test expectations only. * fix(billing): correct the post-step-up denial diagnosis + finish the desktop rename Adversarial review findings: (1) a repeated insufficient_scope after a successful step-up is a per-terminal authorization failure, but the copy blamed the org kill-switch and pointed at the wrong recovery control — now: "Remote Spending still isn't active for this terminal — the authorization didn't take. Retry, or make this change on the portal." (2) the desktop step-up flow started in Remote Spending vocabulary but finished in "billing management access" — renamed both end states. (3) prettier formatting on the touched files (matches the post-merge fmt bot). * feat(tui): show the plan catalog in /subscription on Free (#68357) * feat(tui): show the plan catalog in /subscription on Free The server returns the tier list even with no subscription, but the overlay hid the picker behind can_change_plan && !isFree, so a Free account got only "Start a subscription" with no idea what the plans cost. Now: - Overview on Free offers "Choose a plan" whenever the catalog has enabled paid tiers. - The picker on Free lists each plan as name · price · monthly credits (no upgrade/downgrade hints — there is nothing to move from), and picking one opens the portal, where starting a subscription actually happens (card capture + checkout live there; the upgrade RPC requires an existing subscription). - Paid-plan behavior (preview → confirm → apply) is unchanged. * refactor(tui): compute the picker row suffix once Review feedback: the isFree fork duplicated the label template and run handler; only the suffix differs. * fix(tui): arm the busy guard before the Free portal handoff Adversarial review: the Free branch returned before setting busyRef, so a double-Enter could open the portal twice; and the picker narrated a handoff that openManageLink already narrates (duplicate on success, contradictory on failure). Guard first, let the helper do the talking. * fix(tui): monthly credits are dollars — label them as such The Free picker showed "1000 credits/mo" for what is $1,000 of monthly credit — render "$1,000 credits/mo" (grouped, dollar-signed). * feat(tui): render the Free-plan catalog inline in the /subscription overview Sid ruling: the upsell belongs where the user already is — no intermediate "Choose a plan" hop. On Free the overview lists each paid plan (name · $/mo · $credits/mo) as a pickable row; picking opens the portal (openManageLink narrates). The generic "Start a subscription" row survives only when the catalog is empty. The picker reverts to its original change-only form (Free never reaches it). * feat(desktop): tier catalog chips on the Subscription row Desktop parity with the TUI inline catalog (Sid ruling): accounts that can act see the plans where they already are — Free gets the upsell list (every chip opens the portal), a subscriber sees all tiers with the current one marked inert. Members and team contexts see no chips. Chips learn an optional url (portal handoff) in the shared row model. * chore(tui): fixture harness mirrors the live tier catalog The dev screenshot fixtures showed invented plans ($50 Super / $99 Ultra, "1,000 credits"); align with the real catalog ($20/$100/$200 with $22/$110/$220 monthly credits) so fixture renders cannot be mistaken for product truth. The overlay itself always reads tiers from the subscription API. * chore: trim narration comments * fmt(js): `npm run fix` on merge (#68462) Co-authored-by: github-actions[bot] <github-actions[bot]@users.noreply.github.com> * fix(relay): attach metadata.user_id on guild replies for egress fallback (#68320) The relay adapter re-attaches an egress discriminator on outbound replies so the connector can resolve the owning tenant. It captured scope_id for scoped (guild) messages and user_id for DMs, but as MUTUALLY EXCLUSIVE: a scoped inbound hit an early return, so the author's user_id was never recorded, and _with_scope only attached user_id when there was no scope_id. Guild replies therefore went out with scope_id only. That's fine while the guild has a provision-time route row. But a MANAGED Discord agent joins guilds dynamically (the shared bot is added to / removed from servers at runtime), and GATEWAY_RELAY_ROUTE_KEYS — the only thing that writes guild route rows — is a self-hosted, static field never stamped for managed agents. So their guild has no route row, the connector's guild-route lookup misses, and with no user_id on the frame there's nothing to fall back to → every guild reply is declined "discord egress declined: target not routed to an onboarded tenant" even though INBOUND resolved the same guild fine (via the author-first SharedSocketRouter.targets() fallback). Fix: capture the authentic author user_id for EVERY inbound (DM and scoped alike) and re-attach it on the outbound reply alongside scope_id. The connector consults it only on a route/scope miss, so carrying both never overrides routing-table resolution. This is the gateway half of the paired gateway-gateway change (makeDiscordTenantOf guild-route-miss author-binding fallback); together they make guild replies resolve the same observed-author way inbound already does. Tests (tests/gateway/relay/test_relay_adapter.py): a guild reply now carries both scope_id AND user_id; a scoped inbound with no author still yields scope_id only (never invents one). Verified fail-without / pass-with. * build: declare pywin32 as a direct win32 dependency hermes_cli/windows_ssh_runtime.py imports win32security/win32file/etc. directly but pywin32 only arrived transitively via concurrent-log-handler -> portalocker. Declare it with a sys_platform gate so the Windows SSH runtime doesn't depend on the logging dep chain. Review follow-up on PR #68130. * fix(desktop): preserve dragging with empty titlebar slots * Revert "fix(agent): circuit-break AttributeError from commit-splice and detect code skew" This reverts commit 3a9b9d65d505646212c4c875bab19b96ae14b2e6. * fix(context): revalidate Codex OAuth context windows * test(context): document Codex cache persistence coverage * fix(context): scope Codex catalogue cache by credential * test(context): cover Codex context rollback * fix(compression): report live-resolved Codex window in the autoraise notice The autoraise banner hardcoded '272K' for the gpt-5.4/5.5/5.6 family, but the Codex /models catalog is authoritative and shifts server-side (gpt-5.6 served 372K during July 9-18, 2026 before OpenAI rolled it back). Pass the compressor's live-resolved context_length through so the notice reports the window the session actually got; the static 272K/128K text remains as the fallback when no resolved value is available. * fix(codex): send ChatGPT-Account-Id on /models probes The Codex backend returns the per-account model catalog only when the ChatGPT-Account-Id header is present. Without it, GET /backend-api/codex/models responds 200 OK with {"models":[]} and the picker silently degrades to the hardcoded fallback list — which is stale or wrong for the active plan (no GPT-5.6 family, wrong context windows). This was the upstream bug behind slow first responses and HTTP 520/120s SSE hangs: Hermes was sending invalid slugs because the probe never saw them in the catalog, and Codex's request builder also depends on the same JWT claim that's now being threaded through both probe paths. Fixes the probe-side paths in hermes_cli/codex_models.py and agent/model_metadata.py by extracting chatgpt_account_id from the OAuth JWT (mirroring the request-side logic already in auxiliary_client.py) and sending it as a header. Verified live: - _fetch_models_from_api now returns the 10-model catalog (gpt-5.6-sol, gpt-5.6-terra, gpt-5.6-luna, gpt-5.5, gpt-5.4, gpt-5.4-mini, gpt-5.3-codex-spark, 3x -pro variants) instead of []. - _fetch_codex_oauth_context_lengths resolves all 8 account models to 272K context (matches direct API probes of the same account). - end-to-end: hermes chat -m gpt-5.6-sol -q 'Reply with one word: pong' returns 'pong' cleanly via the openai-codex route. Same class of bug as PR #64760. * test(codex): cover ChatGPT-Account-Id header on /models probe Add regression tests locking in the new behavior: a JWT carrying a chatgpt_account_id claim causes the probe to send ChatGPT-Account-Id, while a malformed token omits the header instead of crashing. * fix(tools): make the tool-search context gate provider-aware (#68589) _resolve_active_context_length() called get_model_context_length() with the model id alone, so provider-enforced windows (e.g. Codex OAuth's 272K for gpt-5.x vs the direct API's 1.05M) never reached the tool-search activation gate — it sized against generic metadata for the same slug. Resolve the runtime provider for the configured model and pass provider, base_url, and api_key through. If credential resolution fails (offline, no keys), degrade to a provider+base_url-only lookup so the static provider-aware fallbacks still apply; explicit model.context_length keeps short-circuiting as before (#46620). Gap flagged during review of #16735. * feat(skills): bundle docx, xlsx, and pdf office skills; refresh powerpoint (#68595) Non-technical users asking for Word docs, spreadsheets, or PDF work had no bundled skill coverage — docx/xlsx creation required discovering and installing hub skills, and PDF manipulation had no skill at all beyond OCR extraction and nano-pdf edits. - skills/productivity/docx: create (docx-js), edit (unzip -> XML -> zip), tracked changes, comments, validation. Adapted from anthropics/skills. - skills/productivity/xlsx: openpyxl creation/editing, mandatory LibreOffice recalc gate, formula-compatibility rules, financial-model conventions. Points at optional excel-author for finance-grade work. - skills/productivity/pdf: merge/split/rotate/watermark/encrypt, form filling (AcroForm + flat overlay scripts), text/table extraction, reportlab creation, forms.md + reference.md companions. - skills/productivity/powerpoint: synced to current upstream pptx skill — richer pptxgenjs corruption footguns, template workflow, validate.py + validators + thumbnail.py, font-substitution QA guidance; drops the stale pack.py/editing.md/pptxgenjs.md workflow files. - Cross-linked ocr-and-documents, nano-pdf, excel-author via related_skills so each office skill routes to its siblings. - deliverable-mode docs mention the new skills; regenerated per-skill docs pages, catalogs, and sidebar. - tests/skills/test_office_document_skills.py: frontmatter contracts, referenced-script existence, schema-map integrity, cross-link resolution, script compilation. E2E validated: docx create->render->edit->validate, xlsx recalc (SUM + _xlfn.TEXTJOIN evaluate correctly), pdf create->merge->extract, pptx generate->validate->thumbnail. * fix(approval): raise gateway approval timeout to 300s, honest stale-tap UX, offer Always on mixed prompts (#68597) Three related messaging-approval fixes: 1. approvals.timeout default 60 -> 300. PR #63501 collapsed the gateway wait onto the canonical approvals.timeout (previously gateway_timeout=300), silently shrinking messaging approval windows to 60s. Push-notification approvals routinely arrive later than a minute; taps landed after the wait had already failed closed. 2. Stale-tap honesty: adapters resolved the approval AFTER rendering '<checkmark> Approved by <user>' (Telegram/Discord/Slack), or ignored a zero resolve count (WhatsApp Cloud/Feishu). A tap on an expired prompt claimed approval while the command had already been denied. All button paths now resolve first and render 'Approval expired - command was not run' when nothing was waiting. 3. Mixed-warning prompts (dangerous pattern + tirith finding) now offer Always: the persistence layer already permanently allowlists the pattern key and downgrades the tirith key to session scope, but the UI hid Always whenever ANY tirith warning was present. Pure-tirith prompts still withhold Always (content findings are session-max by design), and Smart-DENY overrides remain once-only. * feat(secrets): one-command token rotation + actionable startup errors for all secret sources (#68605) * feat(secrets): one-command token rotation + actionable startup errors for all secret sources When a Bitwarden machine-account token expired, users saw a raw Rust error dump (invalid_client + Location: + backtrace hints) and the only fix was manually editing .env or re-running the whole setup wizard. - New `hermes secrets bitwarden token` / `hermes secrets onepassword token`: paste a new token (masked prompt or flag), the command probes the backend BEFORE persisting — a rejected token changes nothing; a good one is written to .env and the fetch caches are cleared. - New optional SecretSource.remediation(kind, cfg) hook: startup warnings now print a '→ Run `hermes secrets <name> token`…' fix-it line after any fetch error, for bundled AND plugin sources (generic per-ErrorKind defaults in the ABC). - bws stderr is summarized to its cause line (Location:/backtrace noise dropped) and invalid_client/invalid_grant/400 identity rejects are now classified AUTH_FAILED (was INTERNAL) with a plain-English explanation naming the token env var. - op whoami probe accepts a candidate token so rotation validates the NEW credential, not the ambient one. Additive hook with defaults — no SECRET_SOURCE_API_VERSION bump. * docs: fix MDX parse error in secret-source-plugin hook table Escaped backticks around a <name> placeholder made MDX parse it as an unclosed JSX tag, breaking the docs-site build. Use a plain code span instead. * feat(desktop): configure repository discovery (supersedes #67630) (#68642) * feat(desktop): configure repository discovery * fix(config): preserve additive default migration * fix(desktop): stabilize session-actions-menu gateway mock for repo-scan subscribe projects.ts now runs $gateway.subscribe(syncReposScanning) at module load, and nanostores fires the subscriber synchronously. session-actions-menu.test.ts reaches projects.ts transitively via the session store but mocked @/store/gateway without $gateway, crashing the whole desktop vitest suite ("No \ export is defined"). Simply adding $gateway: atom(null) exposed a second issue: the synchronous subscriber calls the mock's activeGateway() during the transitive import, before the module-level const initializes (TDZ). Hoist the mock fns via vi.hoisted() so activeGateway is defined before the hoisted vi.mock factory runs, and add $gateway: atom(null) to the mock. Mirrors the self-contained mock pattern already used in projects.test.ts. Also maps the PR author's commit email for attribution. Supersedes #67630; incorporates review feedback from that PR. Co-authored-by: Rudimar Ronsoni <rudimar@outlook.com> --------- Co-authored-by: Rudimar Ronsoni <rudimar@outlook.com> Co-authored-by: Austin Pickett <austinpickett@users.noreply.github.com> * fix(desktop): ⌘W closes visible file tab when preview selection is stale (#68639) * fix(desktop): make ⌘W close visible file tab on stale preview selection When the live preview target is gone but $rightRailActiveTabId still points at preview, file tabs remain on screen while ⌘W fell through to a workspace no-op. Close the visible file tab instead. * test(desktop): cover ⌘W close for file tabs and ghost preview selection Lock the happy path and the stale-preview regression so ⌘W keeps closing the file tab the rail is actually showing. * fmt(js): `npm run fix` on merge (#68681) Co-authored-by: github-actions[bot] <github-actions[bot]@users.noreply.github.com> * feat(billing): plan chips and rows deep-link their tier (#68666) * fix(desktop): drop the decorative top-up credits bar (#68649) The bar rendered full-or-empty (value 1|0) because top-ups have no denominator — the wire carries only the current balance and the pool is open-ended, so a fill fraction is fiction. Show the amount alone; subscription credits and the monthly cap keep their bars (real denominators). * fix(ci): route critical supply-chain findings through review gate (#68833) Let the scanner report critical findings without failing. The review-label gate owns the action-required status and blocking result, allowing the ci-reviewed label rerun to clear both CI and the PR comment. * fix: `tool_calls` double-encoding on import (#68856) * nix: add `cage` to devShell * test(desktop): add pre-filled sessions support Exports createSandbox, writeMockProviderConfig, writeEnvFile, buildAppEnv, findElectron, and launchDesktop from fixtures.ts so specs can compose their own seeded-backend fixtures without duplicating the sandbox/config/launch logic. * test(desktop): auto-fail e2e tests on error banner Adds a shared test fixture (e2e/test.ts) that wraps @playwright/test's page with an error-banner guard. When any [role="alert"] element (error notification toast) appears in the DOM during a test, the test fails with the error message text. The guard uses: - A MutationObserver (injected via addInitScript) that watches for [role="alert"] elements appearing at any point during the test - A final DOM scan in afterEach for alerts still visible at teardown - Deduplication so the same error text only fires once All existing e2e specs updated to import { test, expect } from './test' instead of '@playwright/test'. No per-spec setup needed — the guard is auto-installed on every page via the extended fixture. This catches issues like the "resume failed" error banner that can appear during session loading — previously the test would pass while an error toast was silently visible on screen. * fix(state): parse tool_calls JSON string before re-serializing _insert_message_rows and append_message both do json.dumps(tool_calls) to serialize the field for SQLite storage. But when tool_calls arrives as a JSON string (from import_sessions / export_session, which store it as TEXT), json.dumps double-encodes it — wrapping the already-serialized string in quotes and escaping the inner quotes. When _rows_to_conversation later does json.loads(row['tool_calls']), the double-encoded string parses back to a plain string (not a list). _history_to_messages then iterates this string character-by-character, calling tc.get('function', {}) on each char — 'str' object has no attribute 'get'. This was a pre-existing bug (on main), but only triggered by the import_sessions path (the live agent always passes tool_calls as a Python list). The e2e error-banner guard caught it via the 'Resume failed' notification toast. Fix: in both append_message and _insert_message_rows, parse tool_calls with json.loads first if it's a string, then re-serialize. * fix(desktop): exempt boot-failure from error guard - boot-failure: add allowErrorBanners() beforeEach — these tests deliberately trigger boot errors, so error toasts are expected - test.ts: export allowErrorBanners() opt-out + reset flag in afterEach * feat(status-bar): add /battery toggle for a color-coded battery read-out Add an opt-in battery indicator to the CLI and TUI status bars, shown as the first element and colour-coded by charge (green/yellow/orange/red, or green while charging). Off by default and a no-op on machines without a battery. - agent/battery.py: shared psutil-backed reader with a short TTL cache, category bucketing, and a compact 🔋/⚡ label. Fails open to "unavailable" everywhere. - CLI: /battery [on|off|status] toggle persisted to display.battery, rendered first in every status-bar width tier. - TUI: /battery slash command, config sync, a system.battery RPC polled while enabled, and a pinned first segment in StatusRule. * fix(approval): restore session approval for Tirith-flagged commands Adds an allow_session flag to the gateway approval payload so adapters can render the session tier independently of the permanent tier. Matrix gains a session reaction (🌀) and a reaction legend; pure-tirith prompts now offer once/session/deny instead of collapsing to once/deny. Salvaged from PR #67312, adapted to the allow_permanent semantics that landed in #68597 (Always offered when any dangerous-pattern warning is persistable; pure-tirith prompts stay session-max). * fix(approval): honor allow_session across all button adapters Widen the allow_session tier from Matrix to every adapter the gateway notifies: Telegram, Discord, Slack, Feishu, and Teams gate their Session button on it; WhatsApp Cloud and qqbot accept the kwarg (no session tier in their button sets). Also thread allow_session through the plugin- escalation gate, the execute_code guard payload, and the plain-text fallback so every notify path carries the same capability flags. * test(approval): cover allow_session tiers in Matrix reaction seeding and gateway payload Update the Matrix reaction-seeding contract to the four-reaction default (once/session/always/deny), add tirith-tier (session without always) and no-session-tier cases, and assert allow_session=True in the tirith gateway payload. * fix(desktop): wrap missing sidebar icon-button tooltips (#67500) * fix(desktop): wrap sidebar icon buttons in Tip tooltips Several icon-only buttons in the sidebar (header actions, workspace menu, project menu, session actions, load-more) had aria-label but no visual tooltip on hover. Wrap them in the existing <Tip> component, matching the pattern already used elsewhere (e.g. ProfilePill). No behavioral changes -- purely wraps existing buttons. Adds vitest coverage asserting the Tip wrapper (data-slot=tooltip-trigger) for 6 of 7 files; index.tsx is a 1500+ line top-level page component and was verified manually via screenshots instead. * fix(desktop): satisfy consistent-type-imports lint rule in project-dialog test * test(desktop): update session-row mocks for restored sessionColorById * fix(desktop): compose Tip around the real trigger instead of inside it Tip was being placed as SessionActionsMenu's/PlatformAvatar's DIRECT child, which asChild then cloned instead of the actual button/span. Neither Tip nor PlatformAvatar forwarded the injected onClick/ref, so both silently dropped the wiring: - session-actions-menu.tsx: Tip now wraps DropdownMenuTrigger internally (new ooltip prop) instead of the caller wrapping its children in Tip. - platform-icon.tsx: PlatformAvatar now forwards ref and spreads rest props onto its span so a wrapping Tip's trigger actually attaches. - session-row.tsx: updated call site to use the new tooltip prop. - Added session-actions-menu.test.tsx exercising the real DropdownMenu open behavior end-to-end (no Tip/Dropdown mocks). - session-row.test.tsx no longer mocks PlatformAvatar's behavior; it now exercises the real (fixed) component for the handoff-avatar tooltip. * fix(desktop): compose Tip outside PopoverAnchor in ProjectMenu (#67500) * test(desktop): update session-row test for the tooltip-prop composition (cbbbeb2fd) * fix(desktop): satisfy consistent-type-imports in session-row.test.tsx mocks * chore: retrigger CI * test(desktop): stop mocking PlatformAvatar's behavior (#67500, third pass) The mock was re-introduced by a prior edit that fixed an unrelated lint error, silently undoing the earlier fix where this test started exercising the real (forwardRef) PlatformAvatar. Removed the mock; updated the two handoff-avatar tests to query the real component's rendered span instead of text content, since it renders a brand SVG icon for known platforms rather than the platform name as text. * fmt(js): `npm run fix` on merge (#68867) Co-authored-by: github-actions[bot] <github-actions[bot]@users.noreply.github.com> * fix(gateway): detect stale lock when macOS psutil returns valid start_time for recycled PID On macOS, the lock record's start_time is None (no /proc at creation), but psutil.Process(recycled_pid).create_time() returns a valid float for the unrelated process that now owns the PID. The old condition required both sides to be None before falling back to cmdline checking, so the recycled PID was never detected as stale. Change the fallback condition from AND to OR: when either side's start_time is missing, fall back to cmdline-based gateway detection. Fixes #53763 * fix(gateway): handle PermissionError on stale root-owned lock file When the macOS launchd service runs in a Background session, the gateway process spawns as root and creates a root-owned gateway.lock. On restart as the normal user, open() on that file raises PermissionError, crashing the gateway immediately and entering a launchd crash loop. Catch PermissionError in is_gateway_runtime_lock_active(), remove the stale lock file, and return False so the new process can start cleanly. Fixes #42685 * fix(gateway): guard acquire_gateway_runtime_lock against root-owned lock PermissionError Widen the PermissionError handling from is_gateway_runtime_lock_active (#42689) to the sibling open() in acquire_gateway_runtime_lock: a stale root-owned gateway.lock left by a launchd Background session previously crashed the acquiring process. Unlink the stale file and retry once; if the unlink or retry fails, return False cleanly instead of raising. * fix(gateway): make stale scoped-lock removal atomic via tombstone rename Replace the unlink()+O_EXCL sequence in acquire_scoped_lock with an atomic os.replace() of the stale lock to a <lock>.stale tombstone followed by the existing O_EXCL create. With plain unlink(), two racing starters could both judge the lock stale and the second unlink() would silently delete the first racer's freshly-created lock — both would then 'win'. os.replace() guarantees exactly one racer claims the stale file; the loser gets FileNotFoundError and falls through to O_EXCL, which admits at most one winner. Tombstones are cleaned up immediately; behavior is otherwise identical. * fix(gateway): detect stale gateway_state.json in `gateway status` (TTL + PID liveness) Verified: applies cleanly and the patched module compiles. Tests are described in the PR body (not bundled in this commit). Co-Authored-By: Claude Opus 4.8 <noreply@anthropic.com> * test(gateway): cover stale gateway_state.json detection (TTL + PID liveness) Co-Authored-By: Claude Opus 4.8 <noreply@anthropic.com> * fix(gateway): take over live platform-lock token holders once When --replace misses a cross-HERMES_HOME Telegram token holder, platform connect used to retry forever. Terminate a verified gateway holder once (with the takeover marker) and re-acquire the scoped lock (#65176). Co-authored-by: Cursor <cursoragent@cursor.com> * chore(contributors): map jaretbottoms@gmail.com -> jbbottoms (PR #65178 salvage) * fix(gateway): reap the replaced gateway's orphaned children on POSIX Builds on jbbottoms's #65178 takeover fix (cherry-picked as the previous commit). Windows --replace already tree-kills via taskkill /T, but the POSIX paths signalled only the recorded gateway PID — adapter subprocesses that outlived their parent kept holding scoped token locks and blocked the replacement gateway. - gateway/status.py: _snapshot_gateway_children() captures the old gateway's descendants (psutil, recursive) while it is still alive; reap_gateway_children() SIGTERMs verified orphans after the main PID is confirmed dead, waits bounded, SIGKILLs survivors. Identity-aware (psutil is_running is PID+create-time), skips zombies and children whose ppid still equals the old gateway (parent actually alive), and never raises — best-effort with debug/info logging only. - take_over_scoped_lock_holder() snapshots before terminating and reaps only on a confirmed successful handoff. - gateway/run.py: start_gateway --replace snapshots before SIGTERM and reaps after the old PID is confirmed gone, mirroring taskkill /T. - tests/gateway/test_replace_child_reap.py: reap/skip/never-raise unit coverage plus end-to-end --replace ordering (snapshot → terminate → reap) and the no---replace path never touching the old process. * chore(contributors): map emails for PRs #66906, #66420, #63398 salvage * fix(state): probe FTS5 read path in _db_opens_cleanly so partial index corruption is detected (#66724) `hermes sessions repair --check-only` opens cleanly on state.db files with partial FTS5 index corruption — base tables read fine, the rolled-back write probe from #50502 succeeds, and `PRAGMA integrity_check` returns "ok". But every session_search / /resume title resolution / feature backed by MATCH / snippet / rank queries errors out with `database disk image is malformed` because internal shadow-table segments are bad. The official repair tool then gives false confidence. Add a representative FTS5 read probe against both `messages_fts` and `messages_fts_trigram` (the latter backs title resolution). Empty MATCH strings are accepted by every FTS5 index without requiring populated content, so the probe is safe on a freshly-init'd DB; missing-table / missing-column errors fall through to the existing "not yet a populated DB" branch, matching the write-probe's behaviour. Any other OperationalError is surfaced as the check reason, which sends `hermes sessions repair` to its existing FTS 'rebuild' path (repair_state_db_schema, line 616). Single-file change in hermes_state.py::_db_opens_cleanly. No public API change. No new imports. Fixes #66724. * fix(state): also catch sqlite3.DatabaseError in FTS5 read probe (#66724) The FTS5 read probe in _db_opens_cleanly() only caught sqlite3.OperationalError. But the corruption class #66724 actually wants caught — partial shadow-table damage where MATCH / snippet / rank queries raise DatabaseError("database disk image is malformed") — is a DatabaseError, not OperationalError. Without this catch the probe crashes the caller instead of returning a reason, which is exactly the silent-fail mode the issue describes. Move the try/except inside the for-loop so each FTS table is probed independently (one table corrupted should still surface as a reason), add a separate except clause for DatabaseError that surfaces the same reason format, and use continue instead of pass so the loop still walks both tables when only one is missing on a brand-new DB. Tested by hand: with a corrupted messages_fts_trigram shadow table the function now returns 'fts5 read probe failed on messages_fts_trigram: database disk image is malformed' instead of crashing out. Without this fix it would still crash. * fix(state): preserve degraded-runtime read probe + use canonical FTS5 classifier Two follow-ups on top of f842733 (the FTS5 read probe added in #66906): 1. The original probe query used MATCH '', which FTS5 rejects with 'fts5: syntax error near '. Empty MATCH syntax is not valid FTS5. Switch to MATCH '""' — a quoted empty phrase that parses, scans zero rows, and exercises the same shadow-table read path the search tools use. The probe previously never reached the shadow segments at all on a healthy DB; the read-corruption class was only being detected because the existing write probe happens to fail first on a DatabaseError. 2. The probe's degraded-runtime branch only checked the substrings 'no such table' / 'no such column'. On a SQLite build without the fts5 module, MATCH against a legacy messages_fts table raises 'no such module: fts5' (a different OperationalError class). The substring check would misclassify that as corruption and trigger repair, whose final fallback deletes the messages_fts% schema (#66906 review). Use SessionDB._is_fts5_unavailable_error() — the canonical classifier already used by the degraded-runtime init path — to recognize both 'no such module: fts5' and 'no such tokenizer: trigram' as capability errors. Add tests covering: - Partial shadow-table damage (read-corruption class) - Repair brings reads back online - Healthy degraded DB without fts5 module stays healthy (regression for the misclassification risk) - Healthy degraded DB without trigram tokenizer stays healthy Closes #66906 review feedback Refs #66724 * fix(state): self-heal FTS corruption on the SessionDB search path too Complements #66296 (self-heal on the write path): search_messages()'s main FTS5 MATCH query caught only sqlite3.OperationalError (a query-syntax error → return empty). A corrupt FTS index raises the malformed / "fts5: corrupt structure record" class, which is a sqlite3.DatabaseError — the parent of OperationalError, so it was NOT caught and propagated straight out of search_messages, crashing session/history search. The write path now rebuilds and retries on that class, but a read-only session (cron/CLI history search, or a search issued before any write) never triggers a write, so its search stayed broken until the next process restart ran the offline repair. Catch the DatabaseError corruption class on the search MATCH read too and route it through the existing one-shot _try_runtime_fts_rebuild(), then retry the query. The catch is moved outside `with self._lock` so rebuild_fts() can re-acquire the lock (mirrors _execute_write). The one-shot guard is shared with the write path, so a single instance never loops on a genuinely unrecoverable index. OperationalError syntax handling is unchanged (caught first). Adds a regression test: with a corrupted messages_fts and no post-corruption write, search_messages() rebuilds in place and returns the match; without the fix it raises DatabaseError. * fix(state): extend search-path FTS self-heal to the CJK/trigram branch The trigram MATCH branch in search_messages() had the same OperationalError-only catch that #66420 fixed on the main FTS5 branch: a corrupt messages_fts_trigram shadow table raises the malformed / 'fts5: corrupt structure record' class (sqlite3.DatabaseError, parent of OperationalError), which propagated straight out of search_messages and crashed CJK session/history search for read-only sessions. Route that class through the shared one-shot _try_runtime_fts_rebuild() and retry the trigram query (catch moved outside self._lock so rebuild_fts() can re-acquire it, mirroring the main branch). If the rebuild is refused (guard consumed / FTS disabled / different error) or the retry fails, fall through to the existing LIKE substring fallback — which reads only the canonical messages table — instead of raising, so CJK search degrades gracefully rather than crashing. Adds two regression tests: trigram search self-heals in place after shadow-table corruption (answers from the rebuilt trigram index, not the LIKE fallback), and degrades to LIKE without raising when the one-shot rebuild was already consumed. Follow-up to #66420; refs #66296 #66724 * fix(state): add REINDEX strategy to repair stale B-tree indexes (#63386) When PRAGMA integrity_check reports 'wrong # of entries in index' for B-tree indexes (e.g. idx_sessions_handoff_state), the existing repair strategies (FTS rebuild, sqlite_master dedup, drop-FTS+VACUUM) don't address the mismatch. Add Strategy 0.5: run REINDEX to rewrite the index b-tree from canonical table rows before escalating to more destructive strategies. * test(state): exercise REINDEX repair against a REAL stale B-tree index Replace the mocked test for #63398's REINDEX strategy: the original monkeypatched _db_opens_cleanly to return the corruption string, so the REINDEX pass itself was never exercised against actual index corruption — the test would pass even if REINDEX didn't fix anything. New fixture _corrupt_btree_index() builds genuine on-disk staleness with a writable_schema hack: rewrite the index definition to a partial index (WHERE 0), REINDEX so the b-tree is rebuilt empty, then restore the full definition. integrity_check then reports the real 'wrong # of entries in index idx_messages_session' / 'row N missing from index' class from #63386 — no mocks anywhere. The rewritten test asserts end-to-end with real function calls: - the real _db_opens_cleanly detects the stale index, - repair_state_db_schema repairs it with strategy 'reindex_btree', - post-repair the detector and raw PRAGMA integrity_check both report healthy, and a query forced through the rebuilt index (INDEXED BY) sees every row. Adds a second test asserting the REINDEX strategy is non-destructive (all sessions/messages survive, readable via SessionDB). Follow-up to #63398; refs #63386 * fix(kanban): auto-repair index-only kanban.db corruption via REINDEX _guard_existing_db_is_healthy previously failed closed on ANY integrity_check failure, including the index-scoped class ('wrong # of entries in index <name>' / 'row N missing from index <name>') where the table b-trees are intact and REINDEX rebuilds the damaged indexes losslessly. Boards hit by that class were bricked until manual surgery even though SQLite can fix them in-place. Now, when integrity_check output consists ONLY of index-scoped errors (index name parsed generically from the message — no hardcoded list): 1. quarantine the corrupt bytes FIRST via the existing content- addressed _backup_corrupt_db, 2. under the caller-held cross-process init flock, REINDEX each named index (falling back to bare REINDEX if a parsed name doesn't resolve), 3. re-run integrity_check and proceed only if it comes back clean. Any non-index error class (page corruption, malformed image, freelist damage) — or a REINDEX whose re-check is still dirty — fails closed exactly as before: backup + KanbanDbCorruptError, no silent recreation. Transient OperationalError (locked/busy) still propagates raw with no quarantine. Tests build a real board DB and corrupt a live index via the writable_schema/partial-index REINDEX trick to produce the genuine 'wrong # of entries in index' shape, then assert auto-repair recovers with data intact, page corruption still raises, and a dirty re-check fails closed. * fix(kanban): cap corrupt-backup retention at 10 files per board DB Content-addressed quarantine backups dedupe identical corrupt bytes, but corruption that keeps mutating between failures (partial repairs, further damage across dispatcher retries, multi-profile fleets) mints a new sha-named backup every round — a user accumulated 124 .corrupt.*.bak files with no bound. After each NEW backup is created, prune oldest-by-mtime backups beyond _CORRUPT_BACKUP_RETENTION (module constant, default 10), including the copied -wal/-shm sidecars. The just-created backup is always exempt (copy2 preserves the source mtime, which can be older than existing backups). Pruning is best-effort and never masks the corruption error about to be raised; dedupe of identical corrupt bytes is unchanged. * feat(kanban): periodic WAL checkpoint (TRUNCATE) on the dispatcher tick Kanban connections set wal_autocheckpoint=100, but SQLite's passive autocheckpoint backs off whenever any reader holds an open snapshot — on a busy multi-process board the -wal file can grow without bound between gateway restarts. After each successful dispatch tick, while still holding the board's single-writer dispatch flock, run PRAGMA wal_checkpoint(TRUNCATE) best-effort at a coarse interval (>=5 min since this process last checkpointed that board; module-level per-path monotonic timestamp, so multi-board dispatchers checkpoint each board on its own clock). Success and busy/locked skips are both logged at DEBUG; a failing checkpoint can never fail the tick. * feat(kanban): add `hermes kanban repair` CLI verb Adds kanban_db.repair_db() — a structured, non-raising wrapper around the same narrow repair policy as the connect-time guard: probe with PRAGMA integrity_check under the board's cross-process init flock; quarantine the corrupt bytes FIRST via the content-addressed backup; REINDEX only when every integrity message is index-scoped; re-check; report ok / repaired / corrupt / missing. Locked/busy OperationalError still propagates raw (a locked healthy DB is not corruption and gets no quarantine), and a repair invalidates the per-process healthy-path cache so the next connect() re-probes. The CLI verb reports status human-readably (or --json), exits 0 for ok/repaired/missing and 1 when the DB is still corrupt (non-index corruption stays fail-closed with manual-recovery guidance). It dispatches BEFORE kanban_command's auto-init: init_db() raises KanbanDbCorruptError on a corrupt board, which previously would have made a repair verb unreachable on exactly the boards that need it. CLI tests drive the real argparse surface (build_parser + kanban_command) against real corrupted SQLite fixtures. * fix(packaging): graft web_dist in MANIFEST.in and add sdist regression test Wheels ship hermes_cli/web_dist via pyproject package-data, but the sdist did not: MANIFEST.in had no graft and .gitignore excludes web_dist, so source tarballs installed a dashboard-less package. Graft the directory and add an sdist regression test that builds the tarball and asserts index.html is inside. Salvaged from #29661; the PR's [web]-extra 404-message change was dropped per maintainer review (misleading guidance for source installs). * fix(dashboard): attempt one recovery build when --skip-build finds no dist --skip-build with a missing web_dist/index.html p…

* feat(cron): add truthful execution ledger * fix(cron): harden execution attempt ledger * test(file-safety): unbreak session-snapshot suite; de-flake fixture to env-var resolution (#66293) Two changes to tests/agent/test_file_safety_session_state.py: 1. Drop the stale monkeypatch on tools.file_tools._get_live_tracking_cwd — the helper was deleted in the cwd-tracking refactor (c80b244b5), and monkeypatch.setattr on a missing attribute raises AttributeError, breaking CI slice 4/8 on main for every PR. The patch was redundant: the test writes an absolute path, so cwd resolution never engages. 2. Make the fixture stale-proof: instead of monkeypatching the private _hermes_home_path/_hermes_root_path helpers (same failure class if they're ever renamed), set HERMES_HOME to <root>/profiles/work and let the real resolution chain (get_hermes_home / get_default_hermes_root's profiles-parent rule) derive both paths. The fixture now references zero private symbols and exercises the production resolution path. * fix(gateway): retry transcript appends Queue failed session DB appends so disk order cannot silently lag memory.\nRebuild corrupt FTS indexes once and surface repeated failures as warnings. * fix: harden transcript append retry — lock, matcher, encapsulation, cap Follow-up fixes for salvaged PR #65637: 1. Clear _dirty_transcripts in rewrite_transcript + rewind_session — stale pending messages were re-inserted after /retry, /undo, /compress. 2. Narrow _is_fts_corruption_error to specific SQLite error strings — bare 'fts' substring matched 'shifts', 'gifts', etc. 3. Move DB write outside _transcript_retry_lock — holding the lock during writes serialized all sessions' transcript appends and blocked during FTS rebuild. Now the lock guards only the pending queue. 4. Push rebuild_fts() into SessionDB — SessionStore was reaching into _conn/_lock private attrs. SessionDB.rebuild_fts() follows the same pattern as optimize_fts(). 5. Cap pending per session at 200 — prevents unbounded memory growth when DB is persistently broken. Oldest messages dropped with warning. Added 4 new tests: dirty-clear on rewrite/rewind, FTS matcher false positives, pending cap enforcement. * fix(agent): execute valid tool calls in mixed batches with invalid names (#66317) Degrading models (observed with gpt-5.6 past ~350K input) emit tool-call batches like 6 valid named calls + 1 blank-name call. Previously the whole turn was voided — every valid call got 'Skipped: another tool call in this turn used an invalid name' — and three such batches tripped the 3-strike stop, killing sessions that were still making progress. Now a mixed batch error-results ONLY the invalid call(s) (terse anti-priming error for blank names per #47967, catalog dump for typos) and dispatches the valid subset for execution. The assistant message keeps every emitted call so provider-side tool_call/result pairing stays intact. The 3-strike counter only advances when a turn contains NO valid call, so a fully-degenerate model still stops while a mostly-coherent one keeps working. Broken JSON args on a never-executing invalid call no longer trigger the whole-turn JSON retry loop. Field evidence: July 2026 debug bundle showed gpt-5.6-sol emitting 6-call batches with one blank-name rider at 559K/384K-token context in two separate sessions; 13 valid tool calls were discarded before the session stopped as partial. * fix(cli): arm exit watchdog on shutdown signal, not at chat startup (#66278) A hermes --tui session whose main thread wedges before app.run() returns never executes the finally that calls _run_cleanup — the only place the exit watchdog was armed — so a dead CLI lingered indefinitely (observed ~47 min at 4% CPU, the #65998 class). Arm the backstop from the SIGTERM/SIGHUP handlers instead (both the interactive and single-query paths), the earliest moment shutdown intent is unambiguous. The signal-armed leash is 2x HERMES_EXIT_WATCHDOG_S so a slow-but-progressing _run_cleanup (which still arms its own tighter timer) is never cut short; the outer timer only wins when cleanup was never reached. Idempotent across repeated signals; never raises from a handler. Deliberately NOT armed at startup: the watchdog thread calls os._exit(0) unconditionally after its sleep, so a startup-armed timer (the #65998 approach) would hard-kill every session that outlives the timeout. Supersedes #65998; thanks @JeffStone69 for the report and root-cause gap analysis. * fix(streaming): fence superseded streams out of the delta sink (single-writer) When the stale-stream detector reconnects past a stream whose socket abort raced (the close never actually stopped the old worker), the superseded stream and the retry's stream both write deltas into the same turn. The persisted transcript is then two coherent responses interleaved token-by-token — de-interleaving the stored text by alternation yields two complete, independent answers to the same prompt, which is a dual-writer race in the harness, not a model/context failure (#65991). The interrupt path already positively cancels before force-closing (#6600), but the stale-kill path relies only on the socket abort, and nothing fenced late chunks from a superseded stream out of the shared delta sink. Enforce a single-writer invariant on the sink itself, guarded by attempt id rather than only socket state: every streaming attempt (chat_completions, anthropic_messages, and bedrock paths) claims a monotonic writer token before it begins consuming its stream. A newer claim supersedes any older one, so the consume loop bails the instant it is superseded and _fire_stream_delta / _fire_reasoning_delta / _record_streamed_assistant_text drop chunks from a stale writer. The token is stored per-thread, so a thread that never claimed (a non-streaming delta caller) is never fenced — the guard can only ever drop a superseded stream, never the single legitimate writer. Discards are counted and logged sparsely so a real provider problem stays visible instead of being silently swallowed. * test(streaming): cover the single-writer invariant for superseded streams Assert that a superseded stream (older writer token, other thread) is fenced from the delta sink, the active writer is never fenced, a non-claiming thread is never treated as a writer, and the real consume loop stops the instant it is superseded — so two streams can never interleave into one turn (#65991). * fix(codex): claim the stream-writer token on the codex_responses path too Widen the #65991 single-writer fence to run_codex_stream: each codex attempt claims the delta sink before consuming events, and the consume loop's interrupt_check now also stops the instant a newer attempt supersedes this one. Parity with the chat_completions / anthropic / bedrock paths from the salvaged fix. Two regression tests: superseded codex stream is fenced mid-stream; sole-writer codex stream delivers unchanged. * fix(compression): affirm tool use stays active in the compaction handoff prefix (#66291) The REFERENCE ONLY framing ('treat as background reference, NOT as active instructions... Do NOT answer questions or fulfill requests') was observed bleeding into general tool-use suppression: a production session went narration-only for 7 consecutive turns immediately after a compression event, describing edits instead of calling tools (#65848 report). Fix is additive: one clause stating the note does not restrict HOW the agent works — tools remain fully active for the active task. Every anti-resumption protection stays intact; the previous prefix generation is frozen into _HISTORICAL_SUMMARY_PREFIXES per the module contract so persisted summaries still get the directive-strip on re-compaction. The #65848 rewrite was not taken: dropping the 'Do NOT answer questions' line and the four-heading discard directive risks re-opening the stale-task-resumption class those clauses exist to prevent (the carveout era regressions #41607/#38364/#42812 documented in this file). Report and root-cause analysis: @yasserbousrih (#65848). * fix(codex): forward drained notifications to on_event during approval roundtrips The approval-drain loop in CodexAppServerSession.run_turn drains up to 8 pending notifications to keep per-turn state current before answering a server-initiated approval request — but never forwarded them to the on_event display hook. Tool bubbles for items drained alongside an approval (e.g. the item/started for the very command awaiting approval) silently disappeared. Mirror the main notification path's on_event invocation in the drain loop. Regression test demonstrates RED→GREEN. Grafted from PR #26541 by @simpolism — the earliest submission of the codex app-server display-bridge family (May 15). Confirmed independently by #64698 and #65412. * feat(codex): webSearch bubbles + bare hermes-tools names in app-server bridge Two more display gaps from #26541 grafted onto the merged bridge: - webSearch: codex's built-in web search now produces a tool.started/ tool.completed bubble pair (query as preview + args). Previously the item type wasn't in _CODEX_TOOL_ITEM_TYPES, so built-in searches showed nothing. - mcp.hermes-tools.* stripping: tools codex invokes through Hermes' own hermes-tools MCP server display as their bare names (web_search, browser_navigate) instead of mcp.hermes-tools.web_search. The inner dispatch subprocess can't fire native progress events, so the codex-level event is the display event — name it the way users know the tool. Credit: both behaviors designed and first implemented by @simpolism in PR #26541 (May 15, earliest of the app-server display-bridge family). * fix(state): heal alternation at the ACP / CLI-resume / TUI-resume restore sites too Follow-up to the restore-boundary alternation heal (#65492): get_messages_ as_conversation grew a repair_alternation flag, wired into gateway load_transcript and the CLI startup resume. Three other LIVE-REPLAY restore sites still loaded the transcript verbatim, so a durable 'user;user' violation there re-fires the pre-request defensive repair on every request for the rest of the session (it only ever mutates the per-request list, never the restored working conversation): - acp_adapter/session.py::SessionManager._restore — the loaded history becomes the resumed ACP (Zed) agent's SessionState.history. - hermes_cli/cli_commands_mixin.py — the /resume slash command sets self.conversation_history from the load (the startup resume was fixed, this mid-session one was missed). - tui_gateway/server.py — the resume handler feeds the load into the deferred session record's working conversation. Pass repair_alternation=True at all three so the wedge is healed once at restore. Inspection/export consumers (trace upload, context guard, api_server history, display_history) keep the verbatim default. Adds an end-to-end regression test driving the ACP _restore path: a seeded user;user session restores to an alternation-clean live history with no user input lost. * test(tui_gateway): accept repair_alternation in resume-path DB doubles The lazy session.resume path now calls db.get_messages_as_conversation(target, repair_alternation=True), but the fake _DB stubs in test_protocol.py still declared the pre-change signature, so the resume raised "unexpected keyword argument 'repair_alternation'" and the three session_resume_lazy tests failed. Mirror the real get_messages_as_conversation signature in the stubs by accepting (and ignoring) repair_alternation. * test(delegate): assert copilot probe with assert_any_call to de-flake under slicing test_build_child_agent_ignores_acp_command_when_binary_missing patches shutil.which globally and asserted the LAST call was which("copilot"). That is order-dependent: an unrelated which("uv") reached later in the same process (which happens under some CI test-slice orderings) becomes the last call, so assert_called_with("copilot") fails even though the copilot binary was probed exactly as intended. Switch to assert_any_call("copilot"), which verifies the actual intent and is robust to unrelated which() calls. The behavioural assertions (provider, acp_command, acp_args) are unchanged. * fix(tui): heal alternation at the remaining live-replay resume sites Sibling-site audit on top of #65672: the interactive TUI resume, the profile-scoped resume, and the /undo history reload also feed LIVE REPLAY (raw_history -> sanitize_replay_history -> working conversation; session['history'] after rewind). Pass repair_alternation=True on the model-fed copies; display_history stays verbatim so inspection/export show what is actually stored. Display-only consumers (session.history RPC, formatted transcript output) intentionally unchanged. * test(tui): accept repair_alternation in the top-level server test doubles too The widened resume sites pass repair_alternation=True; the DB doubles in tests/test_tui_gateway_server.py (separate from tests/tui_gateway/) needed the same signature update as Frowtek's originals. * fix(terminal): fall back when the configured cwd is unenterable, not just missing (#66306) A root-launched CLI session can leak /root into the terminal cwd state a non-root gateway/cron process later resolves (#65583). os.path.isdir('/root') is True for a non-root user — stat only needs search permission on / — so _resolve_safe_cwd returned it and subprocess.Popen(cwd='/root') died with PermissionError: [Errno 13], failing EVERY cron job's terminal/file/search tool on every command until restart. _resolve_safe_cwd now requires X_OK (new _cwd_usable helper) and climbs to the nearest enterable ancestor, logging a WARNING that names the leak class when an existing-but-denied cwd is skipped. Missing-cwd recovery (#17558) behavior unchanged. E2E-verified: LocalEnvironment constructed with an unenterable cwd now runs commands from the fallback directory instead of raising. * perf(desktop): pre-warm profile pool backends on hover intent A cold profile switch pays the full pool-backend spawn — Python boot, port announcement, readiness probe, token adoption — before the profile's gateway can even open. Measured with the new CDP harness (scripts/measure-profile-switch.mjs, same family as profile-session-switch.mjs): click → WS open is ~2.5-2.9s on a cold profile, ~3-3.6s to a settled sidebar; a warm profile settles in ~0.5-0.8s. The pointer entering a profile square telegraphs the switch hundreds of ms before the click lands, so start the spawn then. - store/profile: prewarmProfileBackend(name) — fires the existing hermesDesktop.getConnection IPC, which is idempotent (ensureBackend returns the pooled connectionPromise), so the real switch joins the in-flight spawn instead of starting it. Skips the active gateway profile, throttles per profile (60s) so drive-by hovers can't spam spawn attempts, and swallows failures — error UX belongs to the real switch. No new IPC surface; the pool's existing LRU cap + idle reaper still bound resource use, and the LRU guard never evicts a keepalive-fresh backend for a hover spawn. - sidebar/use-profile-prewarm: pointerenter/pointerleave handlers with a 120ms dwell so sweeping the pointer across the rail or a mixed-profile session list doesn't spawn a backend per element crossed. - Wired at the three switch surfaces: rail ProfileSquare, the condensed ProfileDropdown items (extracted ProfileDropdownItem so each row owns its dwell timer), and SidebarSessionRow (covers cross-profile resumes from the all-profiles view; same-profile rows no-op inside the guard). Measured E2E over CDP: synthetic hover on a cold profile square spawns its backend in the background; the subsequent click settles in ~519ms vs ~3.0-3.6s unhovered — and any hover shorter than the spawn still shaves its dwell off the click's wait. Verification: apps/desktop `npx tsc --noEmit` clean; full `npx vitest run` 212 files / 1777 passed (new prewarm guard/throttle tests in store/profile.test.ts); eslint + prettier clean. * feat(kanban): modal create-task dialog, editable board project directory, comment workflow hint (#66333) Community feedback (@LSanapalli on X): the inline task-creation form is cramped inside a ~280px column with no way to resize; board-level workspace defaults can't be changed after board creation; and users believe they must block a task, comment, then unblock just to talk to a worker. - Create-task dialog: replace the inline column form with a centered modal (reuses hermes-kanban-dialog chrome, 36rem wide) with labeled fields for title, assignee, priority, skills, workspace kind/path, goal mode, and parent task. Same request shape; Enter/Escape behavior preserved; submit disabled until a title is present. - Board settings dialog: new Settings button in the board switcher opens a modal to edit display name, description, and the board-level default project directory (default_workdir). PATCH /boards/:slug now accepts default_workdir (validated absolute existing dir; empty string clears; omitted leaves unchanged) and returns the recomputed default_workspace_kind so task-creation defaults follow immediately. - Comment workflow hint: the task drawer's comment box now explains that comments land on the thread immediately and reach the worker on its next run/kanban_show() — no block/unblock dance needed — with a fuller tooltip for when blocking IS the right tool. - i18n: new keys optional in the kanban namespace with English fallbacks in the bundle (established pattern; avoids churning 17 locale files). - Docs: dashboard section updated for the dialog + Settings button. * fmt(js): `npm run fix` on merge (#66348) Co-authored-by: github-actions[bot] <github-actions[bot]@users.noreply.github.com> * fix(state): self-heal FTS corruption on the SessionDB write path (#66296) Complements the #65637 salvage (53d358838 + a9cc17fd8): the gateway session store now retries transcript appends through its own queue, but cron and CLI writers call SessionDB directly — a corrupt FTS index still hard-failed their appends until the next process restart triggered the offline repair. _execute_write now detects the FTS-corruption error class (both the generic 'database disk image is malformed' and newer SQLite's 'fts5: corrupt structure record' variant), performs a one-shot in-place rebuild by delegating to the existing rebuild_fts(), and retries the failed write. One-shot per instance so an unrecoverable database cannot loop; lock/busy jitter-retry path untouched. E2E-verified: corrupted messages_fts_data rejects appends; with this fix the same append self-heals, persists, and FTS search works again. * fix(auxiliary): isolate runtime cache by live context * fix(auxiliary): sync runtime after fallback restoration * fix(compression): reset failure cooldown on runtime switch * fix(tui): route images with the live switched model * chore(release): map auxiliary runtime contributors * fix(auxiliary): scope runtime state to each turn * test(compression): expect complete runtime tuple * fix(credential-pool): throttle "no available entries" log to stop Windows log-lock storm (contributes to #62698) (#66338) * fix(credential-pool): throttle "no available entries" log to stop Windows log-lock storm Credential selection runs on a hot path (every model call plus auxiliary tasks), so an empty/exhausted pool logged "no available entries" at INFO on *every* selection. On Windows, where multiple Hermes processes share one rotating log guarded by concurrent-log-handler's cross-process lock, that per-selection volume storms the lock (RuntimeError: Cannot acquire lock after 20 attempts), pegs a core, and stalls the asyncio event loop long enough that the Desktop backend readiness probe times out ("Timed out connecting to Hermes backend after 15000ms") even though the backend already announced HERMES_BACKEND_READY. Log the condition at most once per 60s window, re-arming on a successful selection so recovery->re-exhaustion still surfaces promptly. Same fix class as the warn-once dedup in #58265. * test(credential-pool): cover no-available-entries log throttle Assert the empty-pool INFO line logs at most once per throttle window, logs again after the window elapses, and re-arms on a successful selection so a recover->re-exhaust transition surfaces promptly. Uses a deterministic fake monotonic clock (no sleeps, no network). * perf(desktop): pre-warm opens the gateway socket too, not just the spawn Answering the review question on the PR table — why a hovered-cold switch still showed ~440ms click → WS open: getConnection-only pre-warming left the WS connect chain to the click, and its microtask continuation can only run after the click's fresh-draft React flush (unmounting a large open transcript costs ~300-400ms of render work), so the socket didn't even START connecting until the flush finished. Add openGatewayForProfile: the same spawn + connect chain as a real switch, minus activation — so the hover leaves the profile's socket fully OPEN and the click's ensureGatewayForProfile just activates it (no ws:new after the click at all; measured ws open at hover+136ms on a warm backend). No scheduleReconnect on failure: a hover is speculative, so a dead backend must not start a background retry loop — the real switch owns retry and error UX. Pruning semantics are unchanged: a hover-opened socket for an idle profile is dropped by the next pruneSecondaryGateways recompute, which just returns the click to the previous behavior. Tests updated: pre-warm asserts openGatewayForProfile is called and that activation (ensureGatewayForProfile) is NOT. * feat(desktop): promote Fireworks AI to #2 in onboarding provider picker (#66432) Mirror CANONICAL_PROVIDERS so Fireworks sits directly under Nous Portal (always visible) ahead of OpenRouter across onboarding, Settings → Providers, and the API-key catalog. * fmt(js): `npm run fix` on merge (#66445) Co-authored-by: github-actions[bot] <github-actions[bot]@users.noreply.github.com> * ci: add 2 minute timeout to osv scan (#66410) this one ran for 5 hours lol https://github.com/NousResearch/hermes-agent/actions/runs/29578577080/job/87878711479 * fix(streaming): make the single-writer fence best-effort so a missing guard can't crash a turn (#66448) A cron job ("Daily Buzz Report") died with 'AIAgent' object has no attribute '_claim_stream_writer'. The #65991 single-writer fence lives on AIAgent (run_agent.py), but the streaming paths that use it live in other modules — chat_completion_helpers (chat / anthropic / bedrock) and codex_runtime (codex responses) — and called it directly as agent._claim_stream_writer() / agent._stream_writer_is_current(). That makes those modules hard-depend on the method being present on whatever object is passed as agent. The fence is an *additive* safety net that may only ever drop a provably superseded stream, never the sole legitimate writer. But the direct calls turned any agent that doesn't expose it — a version-skewed checkout (the streaming helper module newer than run_agent), a hot-reloaded gateway mid git-pull, a duck-typed agent, or a test double — into a fatal AttributeError that aborts the whole turn (and, on cron, fails the job). Route every cross-module claim/check through agent/stream_single_writer.py. claim_stream_writer(agent) returns 0 when the fence is unavailable (or raises), and stream_writer_is_current(agent, token) treats a 0 token or an absent guard as "current" — so a guard-less agent degrades to "no fence" instead of crashing, while a real AIAgent keeps the full single-writer protection. Internal self.* uses inside run_agent are unchanged (self is always a full AIAgent there). * fix(desktop): session-scope fast mode, surface profile ownership + pinned model override Model-picker audit follow-through — closes the remaining pieces of the "switch one session, switches everywhere / can't tell whose session this is" report class: - tui_gateway: `config.set key=fast` with a session no longer writes the global agent.service_tier to config.yaml (sibling of the earlier `reasoning` scoping fix). It pins create_service_tier_override ("priority" / "" for explicit normal) so lazy builds and rebuilds keep the choice; the desktop's per-model presets were rewriting the global tier on every model pick. Fast-support validation now checks a draft's picked model, and `config.get key=fast` reads the pre-build pin. - desktop: owning-profile tag (initial chip + tooltip/aria label) on pinned rows and search results in the All-profiles sidebar, and on the chat header once a second profile exists (#66003). - desktop: composer model pill shows a pin dot + tooltip when a manual sticky pick is overriding the Settings default for new chats (#62055). Closes #66003. Addresses #62055. * refactor(desktop): derive working/attention session sets from $sessionStates $workingSessionIds and $attentionSessionIds were independently maintained atoms that updateSessionState had to manually keep in sync with the session cache (paired setSessionWorking/setSessionAttention calls, plus a rotation special-case in ensureSessionState). Make them computed() projections of $sessionStates instead, so the data flow is one-directional: gateway event → cache → $sessionStates → computed views. Transition side-effects (watchdog arm/disarm, settle grace, unread marker, compression id rotation signal) move into handleTransition, fired from publishSessionState by diffing previous vs next — one choke point instead of per-callsite bookkeeping. The watchdog's force-clear reaches the cache through setWatchdogClearFn rather than a listener set. Also: - clearAllSessionStates disarms all watchdog timers and drops settle-grace entries so a gateway switch can't leak stale timers or keep-set rows - dropSessionState disarms the dropped runtime's watchdog timer - watchdog tests now exercise the real timer→callback wiring instead of manually simulating the clear * fmt(js): `npm run fix` on merge (#66457) Co-authored-by: github-actions[bot] <github-actions[bot]@users.noreply.github.com> * fmt(js): `npm run fix` on merge (#66460) Co-authored-by: github-actions[bot] <github-actions[bot]@users.noreply.github.com> * fmt(js): `npm run fix` on merge (#66465) Co-authored-by: github-actions[bot] <github-actions[bot]@users.noreply.github.com> * perf: fast model picker + dialogs — config-load hot path, model.options off the reader thread, off-screen turns skip rendering Third profiling round (after #66033 / #66347), targeting the composer model picker and dialog opens (worktree dialog etc.), measured over CDP on real 1000+-message sessions. Backend — model.options took 4.8s cold / 1.8s warm per call, and the desktop model pill/picker blocks on it every open: - agent/credential_pool: _load_config_safe uses load_config_readonly(). Every consumer only reads, and the per-call deepcopy was the dominant cost — list_authenticated_providers calls load_pool() per provider row, and each load_pool loaded (and deep-copied) the full config again via get_pool_strategy. - hermes_cli/config: memoize ensure_hermes_home() per home path. It runs inside the config lock on EVERY load_config(), paying ~14 mkdir/chmod syscalls per call. The fast path still re-checks that the home dir exists, so a deleted home is recreated as before; profile switches hit the new path and re-run. Tests cover both. - tui_gateway/server: add model.options to _LONG_HANDLERS. It measured seconds inline on the WS reader thread — while it ran, prompt.submit and session.interrupt sat unread (same class as #21123). Together: model.options RPC 4825/1842ms → 426/230ms (measured on the live desktop backend); build_models_payload in isolation 6.2s → 0.97s cold, 0.27s warm. Desktop — every Radix dialog/popover open forced a whole-document style recalc (Presence reads getComputedStyle on mount), which on a 1300-message transcript cost ~650-730ms per open (CPU profile: getAnimationName 483ms self). The worktree dialog (⌘⇧B) paid it on every single open: - thread/list: content-visibility:auto + contain-intrinsic-size on the per-turn group wrappers. Off-screen turns now skip style recalc, layout, and paint entirely; never-rendered turns hold a placeholder height (auto: remembered real size once rendered) so scrollbar and anchoring stay stable. Verified over CDP: worktree dialog open 656- 730ms → ~200ms on the same session; stick-to-bottom pin, scroll-to- top rendering, and sticky human bubbles all intact. Also: profile-session-switch harness accepts CDP_HTTP (Chrome tends to squat on 9222). Verification: - scripts/run_tests.sh: config, credential-pool, inventory, model-switch routing, tui_gateway protocol, profiles suites green (test_profiles has one pre-existing failure on main, unrelated); new tests for the ensure_hermes_home memo. - apps/desktop: tsc clean, eslint/prettier clean, thread + session suites green (326 tests). - E2E over CDP on the live app: numbers above, plus scroll/pin sanity. * fix(desktop): stop button sends interrupt to wrong session + stale events re-arm busy (#66485) Co-authored-by: Brooklyn Nicholson <brooklyn.bb.nicholson@gmail.com> * feat(dev-sandbox): add --from DIR to seed sandbox HERMES_HOME (#66486) Adds a --from DIR flag to scripts/dev-sandbox.sh that copies an existing HERMES_HOME directory into the sandbox as the starting point before the command runs. Lets you spin up a sandbox pre-populated with your real config, sessions, skills, etc. scripts/dev-sandbox.sh --from ~/.hermes hermes desktop Design: - cp -a dir/. dest/ — preserves perms, symlinks, hidden files - Clobber guard: only seeds when sandbox HERMES_HOME is empty, so re-running --persistent doesn't blow away existing sandbox state - Validates: errors on nonexistent dir, missing arg, flag-like arg, empty --from= - Supports both --from DIR and --from=DIR forms - Backwards compatible: no --from = unchanged behavior * fix(honcho): resolve the timeout staleness check from honcho.json like the build path The staleness check added in #66052 resolved the timeout from env, config.yaml, and the default only, while the build path also reads the honcho.json host block (timeout/requestTimeout). With a timeout configured in honcho.json, the two permanently disagreed: every no-config get_honcho_client() call — i.e. every HonchoSessionManager .honcho property access — interpreted the mismatch as a config change and tore down and rebuilt the client, defeating the singleton on the hot path it was meant to protect. Teach the check to read honcho.json through the same host-aware chain as from_global_config, memoized on the file's mtime_ns so the per-call cost stays one stat(). A genuine honcho.json timeout change is now also detected, extending #57437 to that config surface. * fix(honcho): delegate the config.yaml timeout read to load_config_readonly The staleness check's bespoke mtime memo keyed only on the user config.yaml, but load_config() merges the managed-scope config (HERMES_MANAGED_DIR/config.yaml, /etc/hermes) whose leaf keys win. A managed honcho.timeout with no user config.yaml made the memo cache 'no timeout' while _build resolved the managed value — the same perpetual-rebuild mismatch this PR fixes for honcho.json. A managed timeout edit was likewise invisible while the user file's mtime stayed put. load_config_readonly() is already cached on both files' signatures plus the env-ref snapshot, so use it instead of duplicating that invalidation logic; the defensive deepcopy the old memo existed to avoid is skipped by the readonly variant. Drive the rebuild test through a real config.yaml and add a HERMES_MANAGED_DIR regression test covering stable reuse and managed-timeout edits. * fmt(js): `npm run fix` on merge (#66505) Co-authored-by: github-actions[bot] <github-actions[bot]@users.noreply.github.com> * feat(codex): stream live app-server events to TUI/desktop tool cards Extends the app-server event bridge (make_codex_app_server_event_bridge) to fire the authoritative stable-ID tool_start_callback / tool_complete_callback alongside the existing tool_progress_callback, and route item/reasoning/summaryDelta through the reasoning channel. Surfaces that render structured tool cards (TUI, desktop) — not just progress bubbles — now correlate live cards with the projected history entry after a resume, because the call ids mirror CodexEventProjector's _deterministic_call_id. Guarded per-callback so a broken display consumer can't tear down the codex turn loop. Grafted from PR #65412 by @HaiderSultanArc onto the merged bridge (the PR's parallel _codex_live_event implementation was reconciled into the bridge's existing _fire_tool_started/_fire_tool_completed helpers). * docs(codex): document live app-server display; AUTHOR_MAP entries - codex-app-server-runtime.md: add a Live display section covering the stream/reasoning/tool-card bridge and show_commentary gating. - release.py: AUTHOR_MAP entries for HaiderSultanArc, jjadeo-oss, juanfradb (the latter two for forthcoming follow-up salvages of #62396 / #18050). * fix: cap cache-scope headers at 64 chars to avoid Codex 400 error (#66045) * test(codex): cover overlength cache-scope headers Exercise the real transport path for long session ids, including stable hashing and bounded body/header cache keys. * fix(codex): harden final cache-key boundaries Fold #62349's broader provider-boundary handling into the header fix: bound top-level and xAI override keys again at preflight after middleware, preserve unrelated headers, and cover boundaries and collisions. Co-authored-by: Nick Taylor <nicktaylor@TheWorldofNick-Lappy.local> * fix(moa): surface stale presets without retries Keep invalid persisted preset names fail-closed, list the valid configured choices, and classify the local lookup failure as deterministic so it reaches Desktop immediately. * fix(mem0): migrate legacy OSS base URL aliases Normalize stale api_base keys to each mem0 provider's accepted URL field before Memory.from_config, without mutating the saved config. * fix(ci): make tests, workflows, and attribution reliable under load (#66373) * feat(attribution): conflict-free contributor mappings via contributors/emails/ directory The AUTHOR_MAP dict in scripts/release.py was a merge-conflict magnet: every concurrent salvage PR appended entries to the same lines of the same file, so parallel PRs re-conflicted on every merge to main. New system: one file per email under contributors/emails/ — filename is the commit-author email, first non-comment line is the GitHub login. File additions never conflict, so any number of PRs can add mappings concurrently. - scripts/release.py: AUTHOR_MAP is now LEGACY_AUTHOR_MAP (frozen) merged with the directory at import time (directory wins). All existing consumers (resolve_author, contributor_audit.py) unchanged. - scripts/add_contributor.py: idempotent CLI to add a mapping; refuses conflicting reassignments (incl. against the legacy map), validates email/login shapes. - contributor-check.yml: attribution gate now accepts a mapping file OR a legacy entry; failure message prints the exact add_contributor command. Also auto-resolves bare <login>@users.noreply.github.com emails is intentionally NOT added (kept id+login form only, matching previous behavior). - contributor_audit.py: guidance now points at add_contributor.py. - tests/scripts/test_contributor_map.py: 12 tests covering loader, merge precedence, CLI idempotency/conflict/validation, subprocess E2E. * feat(ci): one-shot per-file flake retry in the parallel test runner A failing test FILE is re-run once in a fresh subprocess. Pass-on-retry counts as green but is loudly reported in a '⚠ FLAKY' summary section (with both attempts' output preserved) so the flake gets fixed instead of eating a full-run rerun. Deterministic failures fail both attempts — regressions cannot be laundered green. - --file-retries N / HERMES_TEST_FILE_RETRIES (default 1, 0 disables) - E2E verified: simulated first-run-fail flake goes green with banner; deterministic failure still exits 1; retries=0 restores old behavior. This converts the dominant CI failure mode (one timing-sensitive test flaking a 4600-test shard, requiring a manual 10-minute rerun and an agent triage loop) into a self-healing retry that costs one file's runtime. * test(approval): loosen wall-clock perf bounds 0.15s -> 2.0s These guard against catastrophic regex backtracking (seconds-to-minutes class), but 0.15s is within scheduler-stall noise on loaded shared CI runners — test_max_accepted_separator_free_input_is_fast failed a CI shard this week on runner load alone. 2.0s still catches the regression class with zero flake surface. * fix(ci): job timeouts everywhere + retries on all network installs Reliability pass over every workflow: - timeout-minutes on all 21 jobs that lacked one (a hung job previously burned the 6-hour default runner budget) - ./.github/actions/retry wrapped around every network-fetching install that lacked it: pip installs (deploy-site, skills-index), npm ci (deploy-site website, upload_to_pypi web + ui-tui), uv sync (docker test deps). Deterministic build steps (npm run build) deliberately NOT retried — split into separate steps so a real build failure fails fast instead of retrying 3x. * docs(agents): document the file-retry flake policy * fix(ci): curl retries on deploy hook + skills-index probe * fix(ci): kill the remaining transient-failure classes in workflows + Dockerfile From the workflow reliability audit: - tests.yml: duration-cache restore had NO restore-keys while saves use run_id-suffixed keys — the cache never matched once, so LPT slicing always ran blind and unbalanced slices pushed heavy files toward the per-file timeout. One-line restore-keys fixes slice balancing. - Label gates (lint ci-reviewed, supply-chain mcp-catalog-reviewed): 'gh pr view || true' turned an API blip into 'label absent' → false BLOCKING failure. Now 3x retry, and API failure is reported as an API failure instead of a missing label. - detect-changes action: compare API retried before failing open (was silently running all lanes on any blip). - uv-lockfile-check: 'uv lock --check' resolves against PyPI — retried so registry blips don't read as 'lockfile stale'. - docker.yml merge job: imagetools create retried (Docker Hub eventual consistency on just-pushed digests). - Dockerfile: apt-get Acquire::Retries=3; s6-overlay ADDs converted to curl --retry 3 (ADD cannot retry; checksums still enforced); npm --fetch-retries=5; playwright chromium fetch retried 3x. - Advisory artifact uploads (per-slice durations, ci-timings report) get continue-on-error so an artifact-service blip can't fail a green test slice. * fix(tests): kill the two root-cause flakes — leaking pre-warm timer + env-dependent provider list - test_tui_gateway_server.py: session.create / non-eager session.resume arm a 50ms threading.Timer (_schedule_agent_build) that outlives its test and fires into the NEXT test's _make_agent mock, racily corrupting captured state (the recurring session_resume shard failures). Replaced the per-test whack-a-mole stub with a module-wide autouse fixture; the 3 worker-lifecycle tests that genuinely need the deferred build opt back in via @pytest.mark.real_agent_prewarm (new marker in pyproject). - test_api_key_providers.py: PROVIDER_ENV_VARS is now derived from the live PROVIDER_REGISTRY instead of a hand-list that had drifted (missing HF_TOKEN / DEEPINFRA_API_KEY) — resolve_provider('auto') tests failed on any machine with HF_TOKEN exported. E2E-verified with HF_TOKEN/DEEPINFRA_API_KEY set: 42/42 pass. * test: de-flake 30 timing-sensitive test files for loaded CI runners Root-cause fixes from the flake audit (session-DB mining + repo sweep): Event-based sync instead of sleep-sync: - title_generator: mock sets threading.Event, wait(10) replaces sleep(0.3) hoping the daemon thread got scheduled - docker zombie_reaping / profile_gateway: poll-for-state helpers replace fixed 1-3s sleeps (s6 transitions + SIGCHLD reaping are async) - process_registry tree test: select()-bounded readline replaces an unbounded blocking read (parent wedge now fails THIS test with a clear message instead of an opaque rc=124 file kill); SIGTERM grace 1s->2s (the 1s partition window mid-interpreter-startup is how a child PID escaped the live-system guard in CI) Timeout raises (loaded 8-way-sliced runners see ~5s scheduling floors; all of these complete in ms-to-1s when healthy so the raises cost nothing on green runs): - subprocess/thread waits <= 2s raised to 10-15s across mcp_tool, mcp_circuit_breaker, mcp_reconnect_retry_reset, mcp_parked_self_probe, mcp_cancelled_error_propagation, registry, clarify_gateway, interrupt, voice_cli_integration, docker_environment, session_store_lock_io, planned_stop_watcher, cli_interrupt_subagent, thread_scoped_output (joins now also assert not is_alive() so stragglers fail loudly) - wall-clock discrimination ceilings loosened where the guarded hang is 10x larger: local_background_child_hang 4s->10s, interrupt_cleanup setup 5s->20s + pgid-exit 30s->60s, mcp_stability grandchild spinup 5s->15s, protocol/gil-starvation fast-handler 0.5s->2s, iso_certify_seam 1.5s->5s, wait_for_mcp_discovery 0.1s->1s - narrow assertion windows widened: honcho first-turn wait 0.4..0.65 -> 0.25..2.0 (property is bounded-not-hung, not an exact wall-clock); compression fork-lock TTL 1s->3s (12 refresh chances per lease); compression-lock expiry margins symmetric (ttl 0.05->0.5, sleep 1.0) - telegram hung-DNS bound 1.0->1.4 (fake hang is 1.5s — must stay under) * fix(tests): repair indentation from de-flake batch edit * fix(tests): harden env isolation and replace remaining sleep-sync races The full 42k-test run and complete npm check surfaced three more classes: - Environment isolation: local ~/.honcho defaultHost and SSH_* variables leaked into Python/TUI tests. Pin the default Honcho host in the hermetic fixture, isolate the one fallback test from ~/.honcho, and blank SSH_* around terminalSetup tests. This flipped 20 false failures back to deterministic behavior on developer machines. - Background-thread sleep-sync: Honcho async writer tests patched time.sleep globally, then busy-polled with that same mocked sleep. Under full-suite load the poller could starve the writer. Each test now waits on an Event emitted by the exact flush/retry transition; 30/30 passed under 15-way contention. - Desktop streaming: the test slept 80ms and assumed a 500ms timer could not fire before its assertion. A loaded runner descheduled the test for >500ms and both chunks arrived. Producer controls now gate second-chunk and completion transitions explicitly. Also make file-retry observability complete: a self-healed flaky file now prints BOTH attempts' full output in the FLAKY summary. Two behavioral runner tests prove pass-on-retry is green+loud+traceback-preserving, while a deterministic failure remains red. * refactor(ci): use gh bot pat, better retries refactor(ci): use retry action for PR label fetch the retry action now captures stdout as a step output, so it can serve double duty: retry + output capture for commands like 'gh pr view' whose result must be consumed by later steps. Retry action gains: - 'stdout' output (heredoc-delimited to preserve newlines) - tee to temp file so stdout still streams to the job log - step id 'retry' for output reference Both lint.yml and supply-chain-audit.yml now use the retry action directly with 'command: gh pr view ...' and read steps.<id>.outputs.stdout. ci: use AUTOFIX_BOT_PAT for all gh CLI / GitHub API auth Replace secrets.GITHUB_TOKEN and github.token with secrets.AUTOFIX_BOT_PAT across all workflows and composite actions that use the gh CLI or GitHub API. The PAT has consistent permissions across fork PRs (where GITHUB_TOKEN is read-only), avoids API rate limit sharing with the default token, and is already used by js-autofix.yml for the same reasons. 19 sites swapped across 9 files: - lint.yml (3): label fetch, comment post/edit, comment update - supply-chain-audit.yml (5): scan, critical comment, unbounded dep comment, label fetch, mcp-catalog comment - lockfile-diff.yml (1): PR comment post/update - skills-index-freshness.yml (1): issue creation on degraded probe - skills-index.yml (2): index build, trigger deploy workflow - upload_to_pypi.yml (2): release view poll, release upload - ci.yml (1): timings report - deploy-site.yml (2): skills index crawl - detect-changes/action.yml (1): compare API call --------- Co-authored-by: ethernet <arilotter@gmail.com> * fmt(js): `npm run fix` on merge (#66527) Co-authored-by: github-actions[bot] <github-actions[bot]@users.noreply.github.com> * fix(cron): avoid Windows Python launcher popups * fix(cron): preserve POSIX script decoding defaults * fix(model-switch): override stale api_mode with host-mandated mode on OpenAI-direct switch Switching to a GPT-5.x model on api.openai.com while the session carried a stale chat_completions api_mode (e.g. from a prior openrouter default) left the request on /v1/chat/completions, which 400s with "Function tools with reasoning_effort are not supported" once the switched model's reasoning is applied. switch_model() only re-derived api_mode inside the provider-changed branch, so a same-provider/carryover switch kept the wrong wire protocol. Add host_mandated_api_mode(base_url): the endpoints that accept exactly one protocol (api.openai.com -> codex_responses, api.anthropic.com / *…/anthropic* -> anthropic_messages, api.kimi.com /coding -> anthropic_messages, bedrock-runtime -> bedrock_converse), matched by EXACT hostname so lookalike hosts and path-segment spoofs are rejected (#32243). switch_model() now uses it to override a stale carried api_mode, not merely fill an empty one; determine_api_mode() shares the same helper. Credit sjiangtao2024 (#15880) for the recompute-before-validation approach; this strengthens it from fill-if-empty to a host-mandated override. Co-Authored-By: sjiangtao2024 <siage@139.com> * fix(cli,gateway): sync base_url/api_mode on global model switch persist Same bug family as #47828, at the config-persistence layer instead of the in-memory agent layer: - cli.py (#25106): the --global /model handlers (both the typed-name path in _handle_model_switch and the picker path in _apply_model_switch_result) wrote model.default/model.provider to config.yaml but never touched base_url/api_mode at all. A provider switch left the OLD endpoint on disk; the next launch reconnected to the previous provider's host under the new model name. - gateway/slash_commands.py (#25107): both persist-global blocks (the picker-tap callback and the typed /model --global path) guarded the write with two INDEPENDENT ifs — `if result.base_url: ...` and `if target_provider != "custom": clear_model_endpoint_credentials(...)`. For named providers the second if always cleared stale values, masking the bug. For a custom provider with an empty resolved base_url, neither branch fired, so the previous custom endpoint's base_url/api_key/ api_mode survived untouched in config.yaml. Fix: explicit set-if-truthy/clear-if-falsy for base_url and api_mode at all four call sites, matching the already-correct pattern in tui_gateway/server.py:_persist_model_switch (fixed for #48305). Co-Authored-By: Claude Sonnet 5 <noreply@anthropic.com> * test(cli): cover picker-path persist_global=True in #25106 regression tests hermes-sweeper review on #60970 flagged that _apply_model_switch_result (the interactive-picker sibling of _handle_model_switch) was only ever tested with persist_global=False elsewhere, so the picker's global-switch base_url/api_mode persistence branch had no coverage. * fix: stop infinite loop when assistant content is a block list strip_think_blocks() ran re.sub() directly on content that could be a list of blocks (Anthropic via OpenRouter returns assistant content as [{type:text,...},{type:thinking,...}]). A list reaching re.sub raised 'TypeError: expected string or bytes-like object, got list', which the outer conversation loop swallowed and retried forever — the observed infinite 'preparing terminal...' loop that re-emitted the same assistant text every iteration. The live-turn path normalized list content to a string, but _interim_assistant_visible_text reads a *stored* history message whose content was persisted as a list and passes it straight into the shared strip_think_blocks helper. Fix at the shared choke point: coerce list/dict content to visible text (dropping reasoning blocks, which is the function's job) before any regex runs, so every caller is safe. * fix(tools/kanban): sync kanban_unblock response status with DB state * docs(kanban): clarify unblock status routing * docs(kanban): explain why an unblocked task can later land in triage The reported confusion was an unblocked task 'unpredictably' ending up in triage. unblock itself only ever routes to ready/todo; a subsequent same-cause re-block hitting BLOCK_RECURRENCE_LIMIT is what escalates to triage. Document this deterministic loop-breaker at the human-facing lifecycle level so users stop reading it as an LLM decision. * fix(ci): restore fork-safe token fallback on PR gates broken by #66373 (#66577) #66373 swapped GITHUB_TOKEN -> AUTOFIX_BOT_PAT across the workflows. That PAT is empty on fork PRs (forks get no repo secrets), which broke every fork PR two ways: 1. detect-changes classified with the empty PAT -> the compare API failed all 3 retries -> the classifier failed open and force-enabled the ci_review lane on EVERY fork PR. 2. The ci-reviewed / mcp-catalog-reviewed label gates then read labels with the same empty PAT via a hard-failing retry step -> the job failed with no recovery a fork contributor could perform (they can't self-add the label; re-running can't fix it). Restores the pre-#66373 fork-safe behavior without reverting the commit's real improvements (job timeouts, per-file flake retry, network-install retries): - detect-changes + ci.yml: token falls back to the built-in read-only github.token when AUTOFIX_BOT_PAT is empty. On main it uses the PAT (authoritative); on forks it uses github.token, which can read the public compare endpoint. (An input `default:` only applies on omission, not on an empty passed value — hence the explicit `|| github.token`.) - lint ci-review + supply-chain mcp-catalog gates: restore the inline `gh pr view ... || true` label read with the github.token fallback, dropping the hard-failing retry "Fetch PR labels" step. Graceful degrade to "label absent" on an API blip, same as before #66373. Same-repo enforcement is unchanged (byte-identical logic; the PAT is still used there). Fork PRs classify correctly and the gates read labels via the read-only token exactly as they did before the regression. * fix(desktop): trust Windows system CAs for remote gateways (#66304) * fix(desktop): trust Windows system CAs for remote gateways Load Windows-trusted roots into Node's default TLS context before Desktop probes remote backends, while preserving bundled and extra CAs. * test(desktop): cover Windows system CA installation Verify existing trust roots survive the merge and that unsupported or unavailable stores fail open without changing TLS defaults. * docs(delegation): align guidance with current contract * docs(delegation): fix stale internal batch-lifecycle comments Two internal comments in delegate_tool.py still described the superseded "N independent handles, no combined wait" model, contradicting the authoritative batch contract (one async unit, one consolidated result when all children finish). Aligns the comments with the runtime path in _execute_and_aggregate / dispatch_async_delegation_batch. * fix(desktop): expose Local / custom endpoint in Providers API-keys tab (#62818) The onboarding overlay already contains a 'Local / custom endpoint' card that writes model.provider:custom + base_url + api_key, but no reachable Desktop GUI path opens it for a fresh add. The composer model pill falls back to the gateway menu panel (Edit Models…), and Settings → Providers → API keys is env-var-driven and never lists a custom endpoint — so users following their instincts cannot add an OpenAI-compatible endpoint (Zyphra, vLLM, Ollama, …) from the GUI. Add a 'Local / custom endpoint' row to the API-keys tab that calls startManualLocalEndpoint(), landing the overlay directly on the existing custom-endpoint form. Reuses the tested onboarding flow; no new UI surface. Regression test in providers-settings.test.tsx asserts the row renders and opens the custom-endpoint flow. Fixes #62817 Co-authored-by: David Metcalfe <80915+DavidMetcalfe@users.noreply.github.com> * fix(desktop): preserve numeric and display LaTeX (#66173) * fix(cli): pass TUI Python env from dashboard chat (salvage #44797) (#66581) * fix: pass TUI Python env from dashboard chat * fix: share TUI Python env setup * fix: preserve TUI Python path semantics * chore: map contributor email for releases --------- Co-authored-by: AI on behalf of Álvaro Sánchez-Mariscal <alvaro.sanchez-mariscal@oracle.com> * fix(model-picker): show exhausted-pool providers in interactive /model picker (#66584) Salvages #66257 by @oppih (CI attribution check blocked the external branch from merging). When a provider's credential pool has entries but all are temporarily rate-limited (exhausted), list_authenticated_providers() excluded the provider from the interactive /model picker. Rate limits are per-model for many providers (e.g. Google Gemini), so an exhausted key for model-A may still work for model-B — the user should still be able to select a different model under the same provider. Adds a for_picker flag to list_authenticated_providers() that relaxes the credential-pool availability check for the picker path only, falling back to pool.has_credentials() when the pool has entries but none are currently available. The runtime resolution path (get_authenticated_provider_slugs) is unchanged, preserving the #45759 invariant that exhausted pools do not count as authenticated. Co-authored-by: oppih <oppih@users.noreply.github.com> * fix(dashboard): keep custom themes visible after embedded chat starts (#60601) * fix(dashboard): resolve dashboard-owned assets from the process launch home Profile-scoped chat / ?profile= requests install a context-local HERMES_HOME override, which made custom dashboard themes AND user dashboard-plugin extensions disappear once the embedded /chat started under a different profile than the dashboard process. Add get_process_hermes_home() (sharing _hermes_home_from_env() with get_hermes_home() so the two can't drift, and splitting the profile fallback warning into _warn_profile_fallback_once()) and use it for both the theme YAML scan and the user dashboard-plugin scan — machine-level assets that belong to the server's launch home and must not follow a transient per-request override. Genuinely profile-scoped callers (memories/backups/checkpoints/provider config) and the paired _merged_plugins_hub classification are left untouched so they keep following the override. * test(dashboard): cover process-home asset discovery under profile override - get_process_hermes_home(): env set returns that path, unset falls back to the platform default, and an active context-local override is ignored. - _discover_user_themes() and _discover_dashboard_plugins() keep returning launch-home assets while a profile override scopes the request elsewhere. * fix(dashboard): only open the chat PTY once the chat tab is active (#59551) * fix(dashboard): only open the chat PTY once the chat tab is active The dashboard mounts ChatPage persistently (hidden with CSS) on every route so the embedded chat PTY survives tab switches. But the PTY-connect effect never checked whether the chat tab was active, so it opened `/api/pty` on mount for ANY dashboard page. On a source/RPi install that spawns the whole TUI + agent bootstrap (`Installing TUI dependencies…` → `npm install`) merely by loading /sessions, /system, etc. — work the user never asked for, and the trigger behind "dashboard loses custom themes on /chat load". Gate the connect effect on a sticky activation latch: the PTY is not spawned until the chat tab has been active at least once, and stays connected across later tab switches so the persistence UX is preserved. * test(dashboard): cover chat PTY activation latch Asserts the invariant behind the fix: activation is sticky. It stays false while the chat tab has never been active (so the persistently-mounted, hidden ChatPage never opens /api/pty), flips true when the tab activates, and stays true after the user navigates away (PTY persistence). * fmt(js): `npm run fix` on merge (#66731) Co-authored-by: github-actions[bot] <github-actions[bot]@users.noreply.github.com> * fix(desktop): restore exec bit on node-pty spawn-helper for dev terminals node-pty's published npm tarball ships the POSIX `spawn-helper` with mode 0644 (no exec bit). node-pty `posix_spawnp`s that helper on macOS/Linux, so a non-executable copy fails every embedded-terminal spawn with `Error: posix_spawnp failed.`. Packaged builds are unaffected because stage-native-deps.mjs chmods the staged copy, but the dev flow (`npm run dev` -> `electron .`) resolves node-pty straight from node_modules, which nothing chmods -- so the first terminal in dev always dies. Restore the exec bit once, lazily, right before the first spawn, via a small DI-testable helper. Idempotent: already-executable copies (packaged builds) are left untouched, and stat/chmod failures are collected and logged rather than thrown so terminal startup never breaks. * fix(desktop): stop tooltips re-opening when a menu/dialog restores focus to its trigger Picking a model from the composer model pill left the pill's tooltip stuck open over the fresh selection: Radix Tooltip opens on ANY trigger focus (its isPointerDownRef guard only covers a pointerdown on the trigger itself), and Radix menus/dialogs restore focus to their trigger on close — so every mouse-driven pick ended with a phantom tip. Same pattern on every Tip-wrapped trigger that opens an overlay. Gate the focus-open to KEYBOARD focus: the trigger's own onFocus runs before Radix's composed handler and calls preventDefault() unless the trigger matches :focus-visible — composeEventHandlers skips onOpen for defaultPrevented events. Chromium keeps focus-visible modality across the menu round-trip, so a mouse pick's focus restore no longer opens the tip, while Tab-focus still shows it (a11y unchanged). Fails open if :focus-visible is unsupported. Tests cover the three branches (suppress on non-keyboard focus, keep on keyboard focus, fail open on selector error); chat/shell suites green. * fix(ci): make timings report fork-safe (missed by #66577) #66373 swapped GITHUB_TOKEN -> AUTOFIX_BOT_PAT across the workflows and #66577 restored the `|| github.token` fork fallback for detect-changes and the label gates -- but it missed the ci-timings "Collect timings and generate report" step, which still passes a bare AUTOFIX_BOT_PAT. On fork PRs that PAT is empty, so timings_report.py hard-fails at expect_env("GITHUB_TOKEN") before it can reach its own "degraded run must never redden the PR" soft-fail path. Every fork PR gets a red run from this advisory job (e.g. #66573). - ci.yml: apply the same `secrets.AUTOFIX_BOT_PAT || github.token` fallback to the timings step. github.token has `actions: read`, enough to read the run's job/step durations on forks. - timings_report.py: treat a missing/empty GITHUB_TOKEN as a degraded run (TimingsUnavailable) instead of a hard ValueError, so this whole class of failure can never redden a PR again even if a future workflow drops the token. Still writes no JSON, so no empty baseline is ever cached. * fmt(js): `npm run fix` on merge (#66741) Co-authored-by: github-actions[bot] <github-actions[bot]@users.noreply.github.com> * perf(desktop): cut startup serialization and per-turn REST amplification Hot-path pass following the switch-latency work. Four independent costs, one theme — work that runs on every boot or every turn but only needed to run on actual change: - electron: start the Python backend in parallel with the renderer load instead of on did-finish-load. The backend cold boot is the dominant startup cost and was serialized behind Chromium's load; the connection promise is shared, so the renderer's getConnection() joins the in-flight boot, and its getBootProgress() pull on mount recovers any progress events emitted before the renderer was listening. - boot/soft-switch: after the socket connects, run the independent post-connect fetches (cwd seed, config, session lists) concurrently instead of serially — profile adoption still lands first because the session fetch scopes by it. - session.info: config refetch is now gated to the foreground context and coalesced (one trailing fetch per event burst) — it used to fire two REST calls per event, including background sessions' heartbeats. model-options invalidation now requires a VALUE change vs the session's cached runtime state; the backend stamps model/provider on every event, so the presence-typed flags refetched the provider catalog once or twice per turn for a model that never changed. - turn complete: sidebar refreshes (recents + cron + messaging fan-out, each scanning profile state.dbs server-side) coalesce across near-simultaneous completions; $sessions and profile totals keep their identity when a refresh returns content-identical rows (same signature gate cron/messaging already use), and the loading flag no longer flickers over a populated list. * fix(delegate): declare stateless channel in one-shot and cron so delegate_task returns results run_agent._dispatch_delegate_task forces background=True for every top-level delegation, and async_delivery_supported() returns True for any session that never binds the capability. On runners that cannot receive a completion after their turn ends, that combination silently discards every subagent result: the model gets a dispatch handle, ends its turn, and reports 'waiting for results'. Two such runners never bind the capability: * hermes -z (one-shot) prints one final response and exits. It bypasses cli.py, so nothing drains process_registry.completion_queue (only the interactive process_loop and the gateway watchers do). * cron run_job clears the HERMES_SESSION_* routing keys, so a completion event carries session_key="" — _enrich_async_delegation_routing cannot resolve it and _inject_watch_notification drops it ("no routing metadata"). By then run_job has already shipped the job's final response via _deliver_result; there is no turn left to re-enter. Worse, get_current_session_key() can fall back to the ambient os.environ HERMES_SESSION_KEY, so a cron subagent's output can be routed into an unrelated user chat rather than merely dropped. Add declare_stateless_channel() and bind it in both runners, routing delegate_task to its existing inline/synchronous path — the same fallback the stateless HTTP adapter already relies on, and the fix suggested in #63142. The helper binds only the capability: set_session_vars() would also latch _session_context_engaged, which a pure single-process one-shot must not trigger. Also correct two agent-facing strings that hardcoded 'stateless HTTP API' as the only channel without async delivery (delegate_tool, terminal_tool); they now name the actual condition. Repro (before): hermes -z 'Use delegate_task to spawn a subagent that replies BANANA. Report its reply.' -> "Waiting for the subagent's response...", exit 0, no BANANA. After: BANANA is returned in-turn. Fixes #53027 Fixes #63142 * fix(docker): strip tini -g flags in legacy entrypoint shim A plain /usr/bin/tini → /init symlink forwarded tini's -g into s6-overlay's rc.init as the container CMD, causing boot loops after image updates that preserve old entrypoints (#66679). * test(docker): cover tini -g legacy entrypoint boot path Unit-test flag stripping without Docker, and assert the image shim rejects the rc.init '-g: not found' restart loop from #66679. * fix(gateway): route inbound-image decision off the event loop `_prepare_inbound_message_text` (async) called `_decide_image_input_mode` inline for every inbound image. That decision is synchronous and does blocking network I/O on the way to a capability answer: - `agent.models_dev.fetch_models_dev` — an HTTP GET to models.dev (15s timeout) whenever the 1-hour in-memory cache is cold or models.dev is slow. - `agent.model_metadata.query_ollama_supports_vision` — HTTP probes (`d…

…sResearch#66296) Complements the NousResearch#65637 salvage (242276b + e537ea9): the gateway session store now retries transcript appends through its own queue, but cron and CLI writers call SessionDB directly — a corrupt FTS index still hard-failed their appends until the next process restart triggered the offline repair. _execute_write now detects the FTS-corruption error class (both the generic 'database disk image is malformed' and newer SQLite's 'fts5: corrupt structure record' variant), performs a one-shot in-place rebuild by delegating to the existing rebuild_fts(), and retries the failed write. One-shot per instance so an unrecoverable database cannot loop; lock/busy jitter-retry path untouched. E2E-verified: corrupted messages_fts_data rejects appends; with this fix the same append self-heals, persists, and FTS search works again.

Complements NousResearch#66296 (self-heal on the write path): search_messages()'s main FTS5 MATCH query caught only sqlite3.OperationalError (a query-syntax error → return empty). A corrupt FTS index raises the malformed / "fts5: corrupt structure record" class, which is a sqlite3.DatabaseError — the parent of OperationalError, so it was NOT caught and propagated straight out of search_messages, crashing session/history search. The write path now rebuilds and retries on that class, but a read-only session (cron/CLI history search, or a search issued before any write) never triggers a write, so its search stayed broken until the next process restart ran the offline repair. Catch the DatabaseError corruption class on the search MATCH read too and route it through the existing one-shot _try_runtime_fts_rebuild(), then retry the query. The catch is moved outside `with self._lock` so rebuild_fts() can re-acquire the lock (mirrors _execute_write). The one-shot guard is shared with the write path, so a single instance never loops on a genuinely unrecoverable index. OperationalError syntax handling is unchanged (caught first). Adds a regression test: with a corrupted messages_fts and no post-corruption write, search_messages() rebuilds in place and returns the match; without the fix it raises DatabaseError.

The trigram MATCH branch in search_messages() had the same OperationalError-only catch that NousResearch#66420 fixed on the main FTS5 branch: a corrupt messages_fts_trigram shadow table raises the malformed / 'fts5: corrupt structure record' class (sqlite3.DatabaseError, parent of OperationalError), which propagated straight out of search_messages and crashed CJK session/history search for read-only sessions. Route that class through the shared one-shot _try_runtime_fts_rebuild() and retry the trigram query (catch moved outside self._lock so rebuild_fts() can re-acquire it, mirroring the main branch). If the rebuild is refused (guard consumed / FTS disabled / different error) or the retry fails, fall through to the existing LIKE substring fallback — which reads only the canonical messages table — instead of raising, so CJK search degrades gracefully rather than crashing. Adds two regression tests: trigram search self-heals in place after shadow-table corruption (answers from the rebuilt trigram index, not the LIKE fallback), and degrades to LIKE without raising when the one-shot rebuild was already consumed. Follow-up to NousResearch#66420; refs NousResearch#66296 NousResearch#66724

…ma v23) (NousResearch#65798) * fix(desktop): refresh repo status on session switch with unchanged cwd (#68208) fix(desktop): refresh repo status on session switch with unchanged cwd * fix(checkpoints): honor gateway config and task cwd (#68195) * fix(gateway): wire checkpoint config into agents * fix(checkpoints): resolve gateway file paths by task cwd * ci: live-updating PR review comment with structured job statuses Replace the static comment-pending + comment-results two-job pattern with a live-updating comment system that polls the GitHub Actions API every 15s, re-assembles the review comment from whatever results are available, and upserts it via the  marker. The comment updates in real time as each job finishes — no waiting for the full pipeline. Every CI job that wants to appear in the review comment emits a review_status output — a JSON array of objects, each with a source and a results array: [ { "source": "review-label-gate", "results": [ {"kind": "action_required", "title": "...", "summary": "...", "how_to_fix": "..."}, {"kind": "info", "title": "...", "summary": "..."} ] }, { "source": "ci timing", "results": [ {"kind": "warning", "title": "CI timings", "summary": "...", "detail": "...", "link": "..."} ] } ] One job can emit multiple results of different kinds. The source field is used to exclude the corresponding job from the synthesized error list (case-insensitive, hyphen-normalized matching against GitHub Actions job display names). | job | source | kind (on failure) | section | |----------------------------|--------------------------|---------------------------|----------------------| | review-labels | review label gate | action_required / info | Action required | | lockfile-diff | lockfile-diff | action_required | Action required | | ci-timings | ci timing | warning / info | Warnings | | supply-chain scan | supply chain | error / (none) | Job failures | | supply-chain dep-bounds | supply chain | action_required / (none) | Action required | | osv-scanner | osv scan | warning / (none) | Warnings | | uv-lockfile-check | uv.lock check | action_required / (none) | Action required | | history-check | unrelated histories | action_required | Action required | | contributor-check | contributor attribution | action_required | Action required | Jobs that find nothing emit [] (empty array) — no noise info items. A single comment-live job polls the GitHub Actions API every 15s, classifies jobs into (completed, pending), assembles the comment, and upserts it. Merges review_status outputs from all needs jobs via toJSON(needs.*.outputs.review_status), and downloads the ci-timings artifact when it becomes available. Shows commit SHA + message below the header. The assembler has ZERO job-specific knowledge. It just: 1. collect_from_statuses() — flattens all nested status objects into ReviewItems 2. collect_failed_jobs() — synthesizes errors for failed jobs with no declared status 3. _attach_job_urls() — fills in per-job log links for ALL items 4. render_comment() — groups by severity, renders with group headers Each item shows links inline next to the title: View report (job-emitted URL) and View job (auto-attached logs link). Each info item is its own collapsible <details> block. # ૮ >ﻌ< ა ci review running on abc1234 — commit message first line ## ❌ Job failures ### {title} · [View job](url) {summary} ## ⚠️ Action required ### {title} · [View job](url) {summary} **How to fix:** {how_to_fix} ## ⚠️ Warnings ### {title} · [View report](url) · [View job](url) {summary} {detail} <details><summary>{title}</summary> {content} </details> Still running 3 jobs: ci-timings, docker - test_assemble_review_comment.py (48 tests): collect_from_statuses, collect_failed_jobs with exclude_sources, _attach_job_urls, render_comment (group headers, inline links, commit info, per-item details, pending footer), assemble integration - test_live_comment.py (16 tests): classify_jobs pure function - test_timings_report.py (10 tests): generate_review_status nested format - test_lockfile_diff.py (6 tests) - test_classify_changes.py (32 tests, pre-existing) * ci: migrate AUTOFIX_BOT_PAT to GitHub App token Replace the long-lived fine-grained PAT (AUTOFIX_BOT_PAT) with short-lived (1-hour) installation access tokens minted via a new get-app-token composite action wrapping actions/create-github-app-token@v3.2.0. The PAT was used in 13 spots across 8 workflow files for gh CLI / GitHub API calls. The per-repo GITHUB_TOKEN (1,000 req/hr) was getting rate-limited when multiple workflows fire concurrently (deploy-site, skills-index, ci-timings, supply-chain-audit, js-autofix). App installation tokens get 5,000 req/hr per installation and are scoped to the App's permissions, not a user account. New composite action: .github/actions/get-app-token/ - Wraps actions/create-github-app-token@bcd2ba49 (v3.2.0, SHA-pinned) - Reads APP_ID + APP_PRIVATE_KEY repo secrets - Outputs a 1hr installation token via steps.app-token.outputs.token Requires two new repo secrets (set after creating the GitHub App): - APP_ID: the App's numeric ID - APP_PRIVATE_KEY: the PEM private key App installation permissions needed: contents: write (js-autofix push, pypi release upload) pull-requests: write (js-autofix PR create/merge, supply-chain comment) issues: write (skills-index-freshness issue creation) actions: write (skills-index workflow trigger) workflows: write (skills-index triggers deploy-site.yml) The AUTOFIX_BOT_PAT secret can be deleted once CI passes on this PR. The comment in js-autofix.yml noting that PAT pushes trigger downstream workflows is updated — App tokens have the same property (they are not GITHUB_TOKEN), so the concurrency-cancel loop logic is unchanged. * style(desktop): satisfy merged eslint/prettier config The SSH modules predate the stricter lint config that landed on main (curly, no-empty, perfectionist sorting, prettier). Mechanical lint:fix + fmt pass, empty catch blocks filled with the codebase's void-0 convention, and inline no-control-regex disables on the three deliberate control-char patterns (same pattern as lib/ansi.ts). * fix(ci): pass App secrets as inputs to composite action Composite actions cannot access the secrets context — the runner's template engine rejects secrets.* references at load time with 'Unrecognized named-value: secrets'. Move APP_ID and APP_PRIVATE_KEY from direct secrets.* references inside the composite action to inputs passed by each calling workflow. The fallback logic (GITHUB_TOKEN when APP_ID is empty, for fork PRs) stays in the composite action's check step. * fix(ci): add detect to all-checks-pass needs so its failure blocks merge If detect fails, all downstream sub-workflows get SKIPPED (they have needs: detect). all-checks-pass used if: always() and only checked the sub-workflows — which all showed as 'skipped' (= success) — so it passed even though the root cause (detect) failed. This made the PR mergeable despite a broken CI pipeline. Add detect to all-checks-pass needs so its failure propagates to the gate job and blocks the merge. * fix(desktop): bump skills test timeout to fix cold-start flake (#68235) Test 1 in skills/index.test.tsx pays the full cold-start cost (jsdom env init + module transform + the @/hermes/@/store/profile import graph), which pushed past vitest's 5000ms default under load — caught at 8871ms on one run, 6.6s pure test time on another. Tests 2-4 are ~30-130ms each because all that setup is already cached, so only test 1 was at risk of timing out. Bump the describe-level timeout to 15s. Verified with 10 consecutive runs, 4 of which took 5.5-6.6s of test time and would have hard-failed under the old 5s default. * feat(desktop): open multiple full app windows (electron) Add createInstanceWindow() — a full-chrome peer of the primary that renders the complete app (sidebar, routing, its own draft) against the shared backend, so several GUI windows can run at once. Mirrors the primary's window options + chatWindowWebPreferences (backgroundThrottling stays off so a streamed answer never stalls when blurred) but never overwrites the mainWindow global and doesn't respawn the backend — the renderer's getConnection() joins the running one. New windows cascade off their source via the pure, tested instanceWindowBounds(). Exposed via the hermes:window:openInstance IPC and a "New Window" File menu item. Per-window fullscreen state now targets the window itself, and titlebar/native-theme repaints reach every open chat window instead of only the primary. Retires the now-orphaned compact new-session pop-out (its only caller was ⌘⇧N, repointed in the follow-up commit): drops createNewSessionWindow, the hermes:window:openNewSession handler, and the newSession/new=1 URL flag. * feat(desktop): wire New Window to ⌘⇧N + command palette Repoint session.newWindow (⌘⇧N) from the compact new-session pop-out to openNewWindow(), which opens a full peer instance via the new openWindow bridge, and add a "New Window" entry to the ⌘K palette (shown with its hotkey hint, gated on canOpenNewWindow()). Relabel the action "New window". Drops the retired openNewSessionWindow bridge and the vestigial isNewSessionWindow()/new=1 flag; renames the shared opener helper. * fix(desktop): de-dupe cross-window cues so peers don't spam With multiple full windows, each renderer independently reacts to the same backend event, so one-shot cues fired N times: OS notifications (the per-renderer throttle can't see other windows), the turn-end sound (playCompletionSound runs on every message.complete, ungated by focus), and auto-spoken replies (double voice when a chat is open in two windows). Add a single race-free owner in the main process (electron/event-dedupe.ts): main handles IPC serially, so the first window to claim a key within a short window wins and peers stay quiet. Notifications collapse at the hermes:notify choke point; the sound and spoken replies claim via a new hermes:ambient:claim IPC (keyed by session / reply id). Off Electron the claim falls back to "emit", preserving single-window behavior. The sound's mute check runs before the claim so a muted window can't win the cue and silence an audible peer. * refactor(desktop): tidy the cross-window deduper Drop the unused DEDUPE_WINDOW_MS export and rename its interval so "window" isn't overloaded against BrowserWindow in a multi-window feature (windowMs → intervalMs). DRY the completion-sound play path. No behavior change. * nix: add cage to devDeps * fix(desktop): avoid false remote gateway reauthentication (#68250) * fix(desktop): avoid false remote gateway reauthentication Co-authored-by: Rod-fernandez <rodrigo@nxtlevelsaas.com> Co-authored-by: David Andrews (LexGenius.ai) <david@lexgenius.ai> * fix(desktop): harden remote revalidation state --------- Co-authored-by: Rod-fernandez <rodrigo@nxtlevelsaas.com> Co-authored-by: David Andrews (LexGenius.ai) <david@lexgenius.ai> * fix(desktop): keep composer draft across compression tip rotation (#68079) * fix(desktop): keep composer draft across compression tip rotation Auto-compression swaps the live stored session id while the user may still be typing. Scope the composer/queue key on the lineage root and migrate any tip-keyed draft/queue entries onto that durable key when the tip rotates so the in-progress prompt does not vanish when the response lands. * test(desktop): cover draft survival across compression tip rotation Add regression coverage for migrateSessionDraft, lineage-scoped composer keys, and the rotation path that previously wiped an in-progress draft. * fmt(js): `npm run fix` on merge (#68305) Co-authored-by: github-actions[bot] <github-actions[bot]@users.noreply.github.com> * fix(desktop): Stop parks the queue instead of firing the next queued prompt Interrupting a busy turn with the Stop button (or Esc) settles the session to idle, and the edge-independent auto-drain immediately submits the head of the composer queue. The user pressed Stop to halt the agent, but it looks like Stop skipped the current turn and kept going — and the queued text is hard to find, since its only surface is the collapsed 'N queued' pill above the composer. The old userInterruptedRef latch (a23728dcc) fixed this but was removed in #40221 because it also suppressed the drain that send-now-while-busy depends on. This reintroduces the halt with source awareness instead of a blanket latch: - Explicit halts (Stop button, composer Esc, chat-focus Esc, the streaming message's hover Stop, runtime cancel) park the session's queue before interrupting. Parked queues are skipped by both auto-drain paths (mounted ChatBar + background drainer). - Interrupts that exist to advance the queue (send-now-while-busy) unpark first, so the settle drain they rely on still flows. - The park lifts on any renewed intent: resume, a manual drain (Enter on empty composer or the per-row send arrow), queueing a new prompt, or emptying the queue. It migrates with entries on a runtime re-key and is deliberately not persisted (a fresh process starts unparked). - The queue panel expands on park, switches to 'N Queued — paused' with a pause icon, and grows a Resume action, so the held prompts are visible instead of reading as vanished. Store contract, hook wiring, and background-drain coverage included; docs updated. * fix(cli,tui): recall real paste content on up-arrow Large pastes collapse to a placeholder in the composer, but input history stored the placeholder — so up-arrow recall showed a truncated reference (CLI) or lost the content entirely (TUI, where the `[[…]]` label has no backing snip after submit). Store the expanded content in history instead: - CLI: `_inline_pastes()` expands `[Pasted text #N -> file]` into the buffer before `reset(append_to_history=True)`; also reused by the external editor (dedup). History nav suppresses re-collapse of recalled content. - TUI: `dispatchSubmission` pushes `expandSnips(pasteSnips)(full)`; idempotent on label-free text so re-submitting a recalled entry stays stable. * fix(cli): suppress CPR on POSIX local TTYs under load Delayed ESC[6n replies leak as ^[[row;colR into the classic CLI on SSH/slow PTYs (#13870) and on local POSIX TTYs under heavy subagent load. Suppress CPR on non-Windows platforms (layout hint only); keep native Windows on prompt_toolkit's default pending native coverage. Wire selection through _select_classic_cli_pt_output. * test(cli): prove local CPR leak and Application CPR-disabled wiring Add a delayed-CPR PTY harness (no SSH) plus selection/Application assertions for POSIX local and Windows preserve-default. Update the gating unit test to the new contract. * refactor: drop platform kwarg, fix PTY test cleanup - Remove redundant platform= test seam from _terminal_may_leak_cpr(); use monkeypatch.setattr(sys, 'platform', ...) consistently in both test files. - Wrap PTY tests in try/finally for fd cleanup on assertion failure. - Guard select.select() in terminal thread against OSError after fd close (fixes PytestUnhandledThreadExceptionWarning). - Trim PR-number reference from test module docstring. * docs(portal): remove retired Nous Chat references * fix(web/ddgs): isolate DuckDuckGo search in a disposable process ThreadPoolExecutor timeouts cannot fire when primp holds the GIL in native code (#68096). Run each search in a child process the parent can terminate/kill, and honor tools.interrupt between polls. * test(web/ddgs): cover GIL-hold timeout, interrupt, and worker reap Regression tests for #68096: native GIL-hold and sleep hooks must time out or interrupt promptly with no orphaned search workers. * fix: sanitize subprocess env for DDGS worker os.environ.copy() passes all Hermes secrets (gateway tokens, API keys, dashboard session tokens) into the DDGS child process. Use _sanitize_subprocess_env() to strip Hermes-managed secrets before spawning the worker. * fix(agent): pass persisted-prefix boundary when rotation flushes on cold resume (#68196) The legacy rotation branch in agent/conversation_compression.py flushes the current turn to the OLD session before ending it (#47202) via _flush_messages_to_session_db(messages) with no conversation_history boundary. On the first turn after a cold Desktop resume, the restored transcript rows live in the message list as plain dicts that have not yet been stamped with _DB_PERSISTED_MARKER — the normal turn flush that stamps them runs after preflight compression. With no boundary, _flush_messages_to_session_db builds an empty history_ids set and treats every restored row as new, durably re-appending the whole transcript to the parent session. Repeated restart/resume + threshold compression keeps growing the parent transcript. Pass messages[:_persist_user_message_idx] (the already-durable prefix that turn_context anchors before preflight runs, guarded for int/bounds) as conversation_history so the flush skips the persisted rows by identity and writes only the current turn's new messages. Adds a regression test that pre-populates SQLite, cold-loads the transcript, appends one current user row, and forces rotating compression: it fails before this change (parent grows to 5 rows) and passes after (parent holds the two originals plus the single new turn). * fix(desktop): prevent contentEditable composer input from visually collapsing to near-zero height Fix #68095 The composer input box (contentEditable div) randomly shrank to a tiny/pixelated size when typing character-by-character (paste worked fine). Root cause: during per-keystroke input, the normalizeComposerEditorDom cleanup could briefly leave the contentEditable with zero child nodes, and without intrinsic content the browser collapsed it visually despite the CSS min-height. Two-pronged fix: 1. Add min-h-[1.625rem] bracket syntax alongside the CSS variable min-height to ensure the minimum height is enforced even if the CSS variable resolution is delayed or overridden by browser defaults. 2. In normalizeComposerEditorDom, ensure the contentEditable always has at least one <br> child when empty, giving it intrinsic height that the browser cannot collapse. This is a belt-and-suspenders approach with the CSS min-height. Closes #68095 * fix(agent): circuit-break AttributeError from commit-splice and detect code skew Fix #68178 The git-install auto-updater rewrites source while the desktop backend is live. Because agent/conversation_loop.py is imported lazily on the first API call, a process can end up running two different commits spliced together — one commit's AIAgent against another commit's conversation_loop. When the interface differs, every turn fails permanently with an AttributeError, and the loop retries indefinitely, burning provider API calls (576 failures, 149 wasted API calls observed). Three-prong fix: 1. Circuit-break AttributeError on agent objects: the outer-loop error classifier now detects AttributeError targeting agent/run_agent modules and breaks immediately instead of continuing the retry loop. 2. Code skew detection for desktop/serve backend: run_agent.py now snapshots the checkout revision at import time and exposes a cheap per-iteration check that the conversation loop uses to refuse new work with a clear 'restart required' message before the lazy import can crash. 3. Informative error message: when code skew is detected, the user gets a clear explanation of the mismatch (boot revision vs current revision) and actionable guidance to restart the application. * fix(telegram): preserve fatal recovery handoff Release the current polling-recovery task's ownership before invoking the fatal-error handler. The runner bounds adapter cleanup in a child task; disconnect() cancels the tracked polling-recovery task, so retaining the current notifier in _polling_error_task would cancel the fatal callback before the runner can finish its reconnect-queue or shutdown decision. The new _handoff_polling_fatal_error() helper clears _polling_error_task only when it is the current notifier. Other recovery tasks remain tracked and are still cancelled and awaited during teardown. Covers both network retry exhaustion and polling-conflict exhaustion. Replaces the misleading "Restarting gateway" message with "Escalating to gateway recovery". Fixes #68406. * fix(telegram): widen fatal handoff to heartbeat watchdog path The wedged-recovery heartbeat watchdog (line 2526) calls _notify_fatal_error() directly from the heartbeat task. disconnect() cancels _polling_heartbeat_task unconditionally (no current_task guard, unlike _polling_error_task). Same bug class as #68406: the child disconnect cancels the heartbeat parent before the runner can queue reconnect. Widen _handoff_polling_fatal_error() to also clear _polling_heartbeat_task when it is the current task, and route the heartbeat watchdog call site through the handoff helper. Co-authored-by: Imgaojp <6065749+Imgaojp@users.noreply.github.com> * fix(tests): make the live-system-guard canary fail closed tests/test_live_system_guard_self_test.py executes real kill primitives (os.kill(-1, SIGTERM), os.killpg, pkill -f python) and depends entirely on the autouse _live_system_guard fixture in tests/conftest.py to intercept them. That makes the canary fail-OPEN: in any collection context where the file is present but its home conftest is not — a published sdist that ships tests/ but not tests/conftest.py, a tree assembled by copying test*.py (that glob does not match conftest.py), pytest --noconftest, or a foreign rootdir — the primitives fire for real, and os.kill(-1, SIGTERM) SIGTERMs every process the invoking user owns (a full desktop-session kill was reported in the field). Add an autouse fixture that refuses to run any canary test unless the guard is provably active. The one thing the canary can detect about its own safety is that the guard monkeypatches os.kill with a plain Python function, whereas the unguarded primitive is a C builtin — so the probe keys off that. Tests marked @pytest.mark.live_system_guard_bypass still opt out, matching the guard's own bypass contract (e.g. test_bypass_marker_disables_guard). With the guard loaded every canary test behaves exactly as before; without it each test refuses at setup with zero side effects. Fixes #68311 * fix(billing): rename user-facing "terminal billing" copy to Remote Spending (#68355) * fix(billing): rename user-facing "terminal billing" copy to Remote Spending The capability was renamed Remote Spending on the portal (consent CTA: "Allow Remote Spending"; per-terminal states Granted/Stopped), but the terminal, desktop, and docs still said "terminal billing" everywhere. - Feature name: Remote Spending in titles/labels, lowercase mid-sentence. - Step-up action verb is now "allow", matching the portal consent CTA. - Kill-switch-off recovery copy points at the actual control ("a billing admin can turn it on from the portal's Hermes Agent page") instead of the dead-end "manage it on the portal". - Per-terminal revoke copy uses the portal vocabulary ("stopped"). - Wire identifiers (cli_billing_enabled, cli_billing_disabled, ...) are unchanged; copy, comments, docs, and test expectations only. * fix(billing): correct the post-step-up denial diagnosis + finish the desktop rename Adversarial review findings: (1) a repeated insufficient_scope after a successful step-up is a per-terminal authorization failure, but the copy blamed the org kill-switch and pointed at the wrong recovery control — now: "Remote Spending still isn't active for this terminal — the authorization didn't take. Retry, or make this change on the portal." (2) the desktop step-up flow started in Remote Spending vocabulary but finished in "billing management access" — renamed both end states. (3) prettier formatting on the touched files (matches the post-merge fmt bot). * feat(tui): show the plan catalog in /subscription on Free (#68357) * feat(tui): show the plan catalog in /subscription on Free The server returns the tier list even with no subscription, but the overlay hid the picker behind can_change_plan && !isFree, so a Free account got only "Start a subscription" with no idea what the plans cost. Now: - Overview on Free offers "Choose a plan" whenever the catalog has enabled paid tiers. - The picker on Free lists each plan as name · price · monthly credits (no upgrade/downgrade hints — there is nothing to move from), and picking one opens the portal, where starting a subscription actually happens (card capture + checkout live there; the upgrade RPC requires an existing subscription). - Paid-plan behavior (preview → confirm → apply) is unchanged. * refactor(tui): compute the picker row suffix once Review feedback: the isFree fork duplicated the label template and run handler; only the suffix differs. * fix(tui): arm the busy guard before the Free portal handoff Adversarial review: the Free branch returned before setting busyRef, so a double-Enter could open the portal twice; and the picker narrated a handoff that openManageLink already narrates (duplicate on success, contradictory on failure). Guard first, let the helper do the talking. * fix(tui): monthly credits are dollars — label them as such The Free picker showed "1000 credits/mo" for what is $1,000 of monthly credit — render "$1,000 credits/mo" (grouped, dollar-signed). * feat(tui): render the Free-plan catalog inline in the /subscription overview Sid ruling: the upsell belongs where the user already is — no intermediate "Choose a plan" hop. On Free the overview lists each paid plan (name · $/mo · $credits/mo) as a pickable row; picking opens the portal (openManageLink narrates). The generic "Start a subscription" row survives only when the catalog is empty. The picker reverts to its original change-only form (Free never reaches it). * feat(desktop): tier catalog chips on the Subscription row Desktop parity with the TUI inline catalog (Sid ruling): accounts that can act see the plans where they already are — Free gets the upsell list (every chip opens the portal), a subscriber sees all tiers with the current one marked inert. Members and team contexts see no chips. Chips learn an optional url (portal handoff) in the shared row model. * chore(tui): fixture harness mirrors the live tier catalog The dev screenshot fixtures showed invented plans ($50 Super / $99 Ultra, "1,000 credits"); align with the real catalog ($20/$100/$200 with $22/$110/$220 monthly credits) so fixture renders cannot be mistaken for product truth. The overlay itself always reads tiers from the subscription API. * chore: trim narration comments * fmt(js): `npm run fix` on merge (#68462) Co-authored-by: github-actions[bot] <github-actions[bot]@users.noreply.github.com> * fix(relay): attach metadata.user_id on guild replies for egress fallback (#68320) The relay adapter re-attaches an egress discriminator on outbound replies so the connector can resolve the owning tenant. It captured scope_id for scoped (guild) messages and user_id for DMs, but as MUTUALLY EXCLUSIVE: a scoped inbound hit an early return, so the author's user_id was never recorded, and _with_scope only attached user_id when there was no scope_id. Guild replies therefore went out with scope_id only. That's fine while the guild has a provision-time route row. But a MANAGED Discord agent joins guilds dynamically (the shared bot is added to / removed from servers at runtime), and GATEWAY_RELAY_ROUTE_KEYS — the only thing that writes guild route rows — is a self-hosted, static field never stamped for managed agents. So their guild has no route row, the connector's guild-route lookup misses, and with no user_id on the frame there's nothing to fall back to → every guild reply is declined "discord egress declined: target not routed to an onboarded tenant" even though INBOUND resolved the same guild fine (via the author-first SharedSocketRouter.targets() fallback). Fix: capture the authentic author user_id for EVERY inbound (DM and scoped alike) and re-attach it on the outbound reply alongside scope_id. The connector consults it only on a route/scope miss, so carrying both never overrides routing-table resolution. This is the gateway half of the paired gateway-gateway change (makeDiscordTenantOf guild-route-miss author-binding fallback); together they make guild replies resolve the same observed-author way inbound already does. Tests (tests/gateway/relay/test_relay_adapter.py): a guild reply now carries both scope_id AND user_id; a scoped inbound with no author still yields scope_id only (never invents one). Verified fail-without / pass-with. * build: declare pywin32 as a direct win32 dependency hermes_cli/windows_ssh_runtime.py imports win32security/win32file/etc. directly but pywin32 only arrived transitively via concurrent-log-handler -> portalocker. Declare it with a sys_platform gate so the Windows SSH runtime doesn't depend on the logging dep chain. Review follow-up on PR #68130. * fix(desktop): preserve dragging with empty titlebar slots * Revert "fix(agent): circuit-break AttributeError from commit-splice and detect code skew" This reverts commit 14e34bde5952a6d75f5d17e9622e469c93b3b577. * fix(context): revalidate Codex OAuth context windows * test(context): document Codex cache persistence coverage * fix(context): scope Codex catalogue cache by credential * test(context): cover Codex context rollback * fix(compression): report live-resolved Codex window in the autoraise notice The autoraise banner hardcoded '272K' for the gpt-5.4/5.5/5.6 family, but the Codex /models catalog is authoritative and shifts server-side (gpt-5.6 served 372K during July 9-18, 2026 before OpenAI rolled it back). Pass the compressor's live-resolved context_length through so the notice reports the window the session actually got; the static 272K/128K text remains as the fallback when no resolved value is available. * fix(codex): send ChatGPT-Account-Id on /models probes The Codex backend returns the per-account model catalog only when the ChatGPT-Account-Id header is present. Without it, GET /backend-api/codex/models responds 200 OK with {"models":[]} and the picker silently degrades to the hardcoded fallback list — which is stale or wrong for the active plan (no GPT-5.6 family, wrong context windows). This was the upstream bug behind slow first responses and HTTP 520/120s SSE hangs: Hermes was sending invalid slugs because the probe never saw them in the catalog, and Codex's request builder also depends on the same JWT claim that's now being threaded through both probe paths. Fixes the probe-side paths in hermes_cli/codex_models.py and agent/model_metadata.py by extracting chatgpt_account_id from the OAuth JWT (mirroring the request-side logic already in auxiliary_client.py) and sending it as a header. Verified live: - _fetch_models_from_api now returns the 10-model catalog (gpt-5.6-sol, gpt-5.6-terra, gpt-5.6-luna, gpt-5.5, gpt-5.4, gpt-5.4-mini, gpt-5.3-codex-spark, 3x -pro variants) instead of []. - _fetch_codex_oauth_context_lengths resolves all 8 account models to 272K context (matches direct API probes of the same account). - end-to-end: hermes chat -m gpt-5.6-sol -q 'Reply with one word: pong' returns 'pong' cleanly via the openai-codex route. Same class of bug as PR #64760. * test(codex): cover ChatGPT-Account-Id header on /models probe Add regression tests locking in the new behavior: a JWT carrying a chatgpt_account_id claim causes the probe to send ChatGPT-Account-Id, while a malformed token omits the header instead of crashing. * fix(tools): make the tool-search context gate provider-aware (#68589) _resolve_active_context_length() called get_model_context_length() with the model id alone, so provider-enforced windows (e.g. Codex OAuth's 272K for gpt-5.x vs the direct API's 1.05M) never reached the tool-search activation gate — it sized against generic metadata for the same slug. Resolve the runtime provider for the configured model and pass provider, base_url, and api_key through. If credential resolution fails (offline, no keys), degrade to a provider+base_url-only lookup so the static provider-aware fallbacks still apply; explicit model.context_length keeps short-circuiting as before (#46620). Gap flagged during review of #16735. * feat(skills): bundle docx, xlsx, and pdf office skills; refresh powerpoint (#68595) Non-technical users asking for Word docs, spreadsheets, or PDF work had no bundled skill coverage — docx/xlsx creation required discovering and installing hub skills, and PDF manipulation had no skill at all beyond OCR extraction and nano-pdf edits. - skills/productivity/docx: create (docx-js), edit (unzip -> XML -> zip), tracked changes, comments, validation. Adapted from anthropics/skills. - skills/productivity/xlsx: openpyxl creation/editing, mandatory LibreOffice recalc gate, formula-compatibility rules, financial-model conventions. Points at optional excel-author for finance-grade work. - skills/productivity/pdf: merge/split/rotate/watermark/encrypt, form filling (AcroForm + flat overlay scripts), text/table extraction, reportlab creation, forms.md + reference.md companions. - skills/productivity/powerpoint: synced to current upstream pptx skill — richer pptxgenjs corruption footguns, template workflow, validate.py + validators + thumbnail.py, font-substitution QA guidance; drops the stale pack.py/editing.md/pptxgenjs.md workflow files. - Cross-linked ocr-and-documents, nano-pdf, excel-author via related_skills so each office skill routes to its siblings. - deliverable-mode docs mention the new skills; regenerated per-skill docs pages, catalogs, and sidebar. - tests/skills/test_office_document_skills.py: frontmatter contracts, referenced-script existence, schema-map integrity, cross-link resolution, script compilation. E2E validated: docx create->render->edit->validate, xlsx recalc (SUM + _xlfn.TEXTJOIN evaluate correctly), pdf create->merge->extract, pptx generate->validate->thumbnail. * fix(approval): raise gateway approval timeout to 300s, honest stale-tap UX, offer Always on mixed prompts (#68597) Three related messaging-approval fixes: 1. approvals.timeout default 60 -> 300. PR #63501 collapsed the gateway wait onto the canonical approvals.timeout (previously gateway_timeout=300), silently shrinking messaging approval windows to 60s. Push-notification approvals routinely arrive later than a minute; taps landed after the wait had already failed closed. 2. Stale-tap honesty: adapters resolved the approval AFTER rendering '<checkmark> Approved by <user>' (Telegram/Discord/Slack), or ignored a zero resolve count (WhatsApp Cloud/Feishu). A tap on an expired prompt claimed approval while the command had already been denied. All button paths now resolve first and render 'Approval expired - command was not run' when nothing was waiting. 3. Mixed-warning prompts (dangerous pattern + tirith finding) now offer Always: the persistence layer already permanently allowlists the pattern key and downgrades the tirith key to session scope, but the UI hid Always whenever ANY tirith warning was present. Pure-tirith prompts still withhold Always (content findings are session-max by design), and Smart-DENY overrides remain once-only. * feat(secrets): one-command token rotation + actionable startup errors for all secret sources (#68605) * feat(secrets): one-command token rotation + actionable startup errors for all secret sources When a Bitwarden machine-account token expired, users saw a raw Rust error dump (invalid_client + Location: + backtrace hints) and the only fix was manually editing .env or re-running the whole setup wizard. - New `hermes secrets bitwarden token` / `hermes secrets onepassword token`: paste a new token (masked prompt or flag), the command probes the backend BEFORE persisting — a rejected token changes nothing; a good one is written to .env and the fetch caches are cleared. - New optional SecretSource.remediation(kind, cfg) hook: startup warnings now print a '→ Run `hermes secrets <name> token`…' fix-it line after any fetch error, for bundled AND plugin sources (generic per-ErrorKind defaults in the ABC). - bws stderr is summarized to its cause line (Location:/backtrace noise dropped) and invalid_client/invalid_grant/400 identity rejects are now classified AUTH_FAILED (was INTERNAL) with a plain-English explanation naming the token env var. - op whoami probe accepts a candidate token so rotation validates the NEW credential, not the ambient one. Additive hook with defaults — no SECRET_SOURCE_API_VERSION bump. * docs: fix MDX parse error in secret-source-plugin hook table Escaped backticks around a <name> placeholder made MDX parse it as an unclosed JSX tag, breaking the docs-site build. Use a plain code span instead. * feat(desktop): configure repository discovery (supersedes #67630) (#68642) * feat(desktop): configure repository discovery * fix(config): preserve additive default migration * fix(desktop): stabilize session-actions-menu gateway mock for repo-scan subscribe projects.ts now runs $gateway.subscribe(syncReposScanning) at module load, and nanostores fires the subscriber synchronously. session-actions-menu.test.ts reaches projects.ts transitively via the session store but mocked @/store/gateway without $gateway, crashing the whole desktop vitest suite ("No \ export is defined"). Simply adding $gateway: atom(null) exposed a second issue: the synchronous subscriber calls the mock's activeGateway() during the transitive import, before the module-level const initializes (TDZ). Hoist the mock fns via vi.hoisted() so activeGateway is defined before the hoisted vi.mock factory runs, and add $gateway: atom(null) to the mock. Mirrors the self-contained mock pattern already used in projects.test.ts. Also maps the PR author's commit email for attribution. Supersedes #67630; incorporates review feedback from that PR. Co-authored-by: Rudimar Ronsoni <rudimar@outlook.com> --------- Co-authored-by: Rudimar Ronsoni <rudimar@outlook.com> Co-authored-by: Austin Pickett <austinpickett@users.noreply.github.com> * fix(desktop): ⌘W closes visible file tab when preview selection is stale (#68639) * fix(desktop): make ⌘W close visible file tab on stale preview selection When the live preview target is gone but $rightRailActiveTabId still points at preview, file tabs remain on screen while ⌘W fell through to a workspace no-op. Close the visible file tab instead. * test(desktop): cover ⌘W close for file tabs and ghost preview selection Lock the happy path and the stale-preview regression so ⌘W keeps closing the file tab the rail is actually showing. * fmt(js): `npm run fix` on merge (#68681) Co-authored-by: github-actions[bot] <github-actions[bot]@users.noreply.github.com> * feat(billing): plan chips and rows deep-link their tier (#68666) * fix(desktop): drop the decorative top-up credits bar (#68649) The bar rendered full-or-empty (value 1|0) because top-ups have no denominator — the wire carries only the current balance and the pool is open-ended, so a fill fraction is fiction. Show the amount alone; subscription credits and the monthly cap keep their bars (real denominators). * fix(ci): route critical supply-chain findings through review gate (#68833) Let the scanner report critical findings without failing. The review-label gate owns the action-required status and blocking result, allowing the ci-reviewed label rerun to clear both CI and the PR comment. * fix: `tool_calls` double-encoding on import (#68856) * nix: add `cage` to devShell * test(desktop): add pre-filled sessions support Exports createSandbox, writeMockProviderConfig, writeEnvFile, buildAppEnv, findElectron, and launchDesktop from fixtures.ts so specs can compose their own seeded-backend fixtures without duplicating the sandbox/config/launch logic. * test(desktop): auto-fail e2e tests on error banner Adds a shared test fixture (e2e/test.ts) that wraps @playwright/test's page with an error-banner guard. When any [role="alert"] element (error notification toast) appears in the DOM during a test, the test fails with the error message text. The guard uses: - A MutationObserver (injected via addInitScript) that watches for [role="alert"] elements appearing at any point during the test - A final DOM scan in afterEach for alerts still visible at teardown - Deduplication so the same error text only fires once All existing e2e specs updated to import { test, expect } from './test' instead of '@playwright/test'. No per-spec setup needed — the guard is auto-installed on every page via the extended fixture. This catches issues like the "resume failed" error banner that can appear during session loading — previously the test would pass while an error toast was silently visible on screen. * fix(state): parse tool_calls JSON string before re-serializing _insert_message_rows and append_message both do json.dumps(tool_calls) to serialize the field for SQLite storage. But when tool_calls arrives as a JSON string (from import_sessions / export_session, which store it as TEXT), json.dumps double-encodes it — wrapping the already-serialized string in quotes and escaping the inner quotes. When _rows_to_conversation later does json.loads(row['tool_calls']), the double-encoded string parses back to a plain string (not a list). _history_to_messages then iterates this string character-by-character, calling tc.get('function', {}) on each char — 'str' object has no attribute 'get'. This was a pre-existing bug (on main), but only triggered by the import_sessions path (the live agent always passes tool_calls as a Python list). The e2e error-banner guard caught it via the 'Resume failed' notification toast. Fix: in both append_message and _insert_message_rows, parse tool_calls with json.loads first if it's a string, then re-serialize. * fix(desktop): exempt boot-failure from error guard - boot-failure: add allowErrorBanners() beforeEach — these tests deliberately trigger boot errors, so error toasts are expected - test.ts: export allowErrorBanners() opt-out + reset flag in afterEach * feat(status-bar): add /battery toggle for a color-coded battery read-out Add an opt-in battery indicator to the CLI and TUI status bars, shown as the first element and colour-coded by charge (green/yellow/orange/red, or green while charging). Off by default and a no-op on machines without a battery. - agent/battery.py: shared psutil-backed reader with a short TTL cache, category bucketing, and a compact 🔋/⚡ label. Fails open to "unavailable" everywhere. - CLI: /battery [on|off|status] toggle persisted to display.battery, rendered first in every status-bar width tier. - TUI: /battery slash command, config sync, a system.battery RPC polled while enabled, and a pinned first segment in StatusRule. * fix(approval): restore session approval for Tirith-flagged commands Adds an allow_session flag to the gateway approval payload so adapters can render the session tier independently of the permanent tier. Matrix gains a session reaction (🌀) and a reaction legend; pure-tirith prompts now offer once/session/deny instead of collapsing to once/deny. Salvaged from PR #67312, adapted to the allow_permanent semantics that landed in #68597 (Always offered when any dangerous-pattern warning is persistable; pure-tirith prompts stay session-max). * fix(approval): honor allow_session across all button adapters Widen the allow_session tier from Matrix to every adapter the gateway notifies: Telegram, Discord, Slack, Feishu, and Teams gate their Session button on it; WhatsApp Cloud and qqbot accept the kwarg (no session tier in their button sets). Also thread allow_session through the plugin- escalation gate, the execute_code guard payload, and the plain-text fallback so every notify path carries the same capability flags. * test(approval): cover allow_session tiers in Matrix reaction seeding and gateway payload Update the Matrix reaction-seeding contract to the four-reaction default (once/session/always/deny), add tirith-tier (session without always) and no-session-tier cases, and assert allow_session=True in the tirith gateway payload. * fix(desktop): wrap missing sidebar icon-button tooltips (#67500) * fix(desktop): wrap sidebar icon buttons in Tip tooltips Several icon-only buttons in the sidebar (header actions, workspace menu, project menu, session actions, load-more) had aria-label but no visual tooltip on hover. Wrap them in the existing <Tip> component, matching the pattern already used elsewhere (e.g. ProfilePill). No behavioral changes -- purely wraps existing buttons. Adds vitest coverage asserting the Tip wrapper (data-slot=tooltip-trigger) for 6 of 7 files; index.tsx is a 1500+ line top-level page component and was verified manually via screenshots instead. * fix(desktop): satisfy consistent-type-imports lint rule in project-dialog test * test(desktop): update session-row mocks for restored sessionColorById * fix(desktop): compose Tip around the real trigger instead of inside it Tip was being placed as SessionActionsMenu's/PlatformAvatar's DIRECT child, which asChild then cloned instead of the actual button/span. Neither Tip nor PlatformAvatar forwarded the injected onClick/ref, so both silently dropped the wiring: - session-actions-menu.tsx: Tip now wraps DropdownMenuTrigger internally (new ooltip prop) instead of the caller wrapping its children in Tip. - platform-icon.tsx: PlatformAvatar now forwards ref and spreads rest props onto its span so a wrapping Tip's trigger actually attaches. - session-row.tsx: updated call site to use the new tooltip prop. - Added session-actions-menu.test.tsx exercising the real DropdownMenu open behavior end-to-end (no Tip/Dropdown mocks). - session-row.test.tsx no longer mocks PlatformAvatar's behavior; it now exercises the real (fixed) component for the handoff-avatar tooltip. * fix(desktop): compose Tip outside PopoverAnchor in ProjectMenu (#67500) * test(desktop): update session-row test for the tooltip-prop composition (cbbbeb2fd) * fix(desktop): satisfy consistent-type-imports in session-row.test.tsx mocks * chore: retrigger CI * test(desktop): stop mocking PlatformAvatar's behavior (#67500, third pass) The mock was re-introduced by a prior edit that fixed an unrelated lint error, silently undoing the earlier fix where this test started exercising the real (forwardRef) PlatformAvatar. Removed the mock; updated the two handoff-avatar tests to query the real component's rendered span instead of text content, since it renders a brand SVG icon for known platforms rather than the platform name as text. * fmt(js): `npm run fix` on merge (#68867) Co-authored-by: github-actions[bot] <github-actions[bot]@users.noreply.github.com> * fix(gateway): detect stale lock when macOS psutil returns valid start_time for recycled PID On macOS, the lock record's start_time is None (no /proc at creation), but psutil.Process(recycled_pid).create_time() returns a valid float for the unrelated process that now owns the PID. The old condition required both sides to be None before falling back to cmdline checking, so the recycled PID was never detected as stale. Change the fallback condition from AND to OR: when either side's start_time is missing, fall back to cmdline-based gateway detection. Fixes #53763 * fix(gateway): handle PermissionError on stale root-owned lock file When the macOS launchd service runs in a Background session, the gateway process spawns as root and creates a root-owned gateway.lock. On restart as the normal user, open() on that file raises PermissionError, crashing the gateway immediately and entering a launchd crash loop. Catch PermissionError in is_gateway_runtime_lock_active(), remove the stale lock file, and return False so the new process can start cleanly. Fixes #42685 * fix(gateway): guard acquire_gateway_runtime_lock against root-owned lock PermissionError Widen the PermissionError handling from is_gateway_runtime_lock_active (#42689) to the sibling open() in acquire_gateway_runtime_lock: a stale root-owned gateway.lock left by a launchd Background session previously crashed the acquiring process. Unlink the stale file and retry once; if the unlink or retry fails, return False cleanly instead of raising. * fix(gateway): make stale scoped-lock removal atomic via tombstone rename Replace the unlink()+O_EXCL sequence in acquire_scoped_lock with an atomic os.replace() of the stale lock to a <lock>.stale tombstone followed by the existing O_EXCL create. With plain unlink(), two racing starters could both judge the lock stale and the second unlink() would silently delete the first racer's freshly-created lock — both would then 'win'. os.replace() guarantees exactly one racer claims the stale file; the loser gets FileNotFoundError and falls through to O_EXCL, which admits at most one winner. Tombstones are cleaned up immediately; behavior is otherwise identical. * fix(gateway): detect stale gateway_state.json in `gateway status` (TTL + PID liveness) Verified: applies cleanly and the patched module compiles. Tests are described in the PR body (not bundled in this commit). Co-Authored-By: Claude Opus 4.8 <noreply@anthropic.com> * test(gateway): cover stale gateway_state.json detection (TTL + PID liveness) Co-Authored-By: Claude Opus 4.8 <noreply@anthropic.com> * fix(gateway): take over live platform-lock token holders once When --replace misses a cross-HERMES_HOME Telegram token holder, platform connect used to retry forever. Terminate a verified gateway holder once (with the takeover marker) and re-acquire the scoped lock (#65176). Co-authored-by: Cursor <cursoragent@cursor.com> * chore(contributors): map jaretbottoms@gmail.com -> jbbottoms (PR #65178 salvage) * fix(gateway): reap the replaced gateway's orphaned children on POSIX Builds on jbbottoms's #65178 takeover fix (cherry-picked as the previous commit). Windows --replace already tree-kills via taskkill /T, but the POSIX paths signalled only the recorded gateway PID — adapter subprocesses that outlived their parent kept holding scoped token locks and blocked the replacement gateway. - gateway/status.py: _snapshot_gateway_children() captures the old gateway's descendants (psutil, recursive) while it is still alive; reap_gateway_children() SIGTERMs verified orphans after the main PID is confirmed dead, waits bounded, SIGKILLs survivors. Identity-aware (psutil is_running is PID+create-time), skips zombies and children whose ppid still equals the old gateway (parent actually alive), and never raises — best-effort with debug/info logging only. - take_over_scoped_lock_holder() snapshots before terminating and reaps only on a confirmed successful handoff. - gateway/run.py: start_gateway --replace snapshots before SIGTERM and reaps after the old PID is confirmed gone, mirroring taskkill /T. - tests/gateway/test_replace_child_reap.py: reap/skip/never-raise unit coverage plus end-to-end --replace ordering (snapshot → terminate → reap) and the no---replace path never touching the old process. * chore(contributors): map emails for PRs #66906, #66420, #63398 salvage * fix(state): probe FTS5 read path in _db_opens_cleanly so partial index corruption is detected (#66724) `hermes sessions repair --check-only` opens cleanly on state.db files with partial FTS5 index corruption — base tables read fine, the rolled-back write probe from #50502 succeeds, and `PRAGMA integrity_check` returns "ok". But every session_search / /resume title resolution / feature backed by MATCH / snippet / rank queries errors out with `database disk image is malformed` because internal shadow-table segments are bad. The official repair tool then gives false confidence. Add a representative FTS5 read probe against both `messages_fts` and `messages_fts_trigram` (the latter backs title resolution). Empty MATCH strings are accepted by every FTS5 index without requiring populated content, so the probe is safe on a freshly-init'd DB; missing-table / missing-column errors fall through to the existing "not yet a populated DB" branch, matching the write-probe's behaviour. Any other OperationalError is surfaced as the check reason, which sends `hermes sessions repair` to its existing FTS 'rebuild' path (repair_state_db_schema, line 616). Single-file change in hermes_state.py::_db_opens_cleanly. No public API change. No new imports. Fixes #66724. * fix(state): also catch sqlite3.DatabaseError in FTS5 read probe (#66724) The FTS5 read probe in _db_opens_cleanly() only caught sqlite3.OperationalError. But the corruption class #66724 actually wants caught — partial shadow-table damage where MATCH / snippet / rank queries raise DatabaseError("database disk image is malformed") — is a DatabaseError, not OperationalError. Without this catch the probe crashes the caller instead of returning a reason, which is exactly the silent-fail mode the issue describes. Move the try/except inside the for-loop so each FTS table is probed independently (one table corrupted should still surface as a reason), add a separate except clause for DatabaseError that surfaces the same reason format, and use continue instead of pass so the loop still walks both tables when only one is missing on a brand-new DB. Tested by hand: with a corrupted messages_fts_trigram shadow table the function now returns 'fts5 read probe failed on messages_fts_trigram: database disk image is malformed' instead of crashing out. Without this fix it would still crash. * fix(state): preserve degraded-runtime read probe + use canonical FTS5 classifier Two follow-ups on top of f842733 (the FTS5 read probe added in #66906): 1. The original probe query used MATCH '', which FTS5 rejects with 'fts5: syntax error near '. Empty MATCH syntax is not valid FTS5. Switch to MATCH '""' — a quoted empty phrase that parses, scans zero rows, and exercises the same shadow-table read path the search tools use. The probe previously never reached the shadow segments at all on a healthy DB; the read-corruption class was only being detected because the existing write probe happens to fail first on a DatabaseError. 2. The probe's degraded-runtime branch only checked the substrings 'no such table' / 'no such column'. On a SQLite build without the fts5 module, MATCH against a legacy messages_fts table raises 'no such module: fts5' (a different OperationalError class). The substring check would misclassify that as corruption and trigger repair, whose final fallback deletes the messages_fts% schema (#66906 review). Use SessionDB._is_fts5_unavailable_error() — the canonical classifier already used by the degraded-runtime init path — to recognize both 'no such module: fts5' and 'no such tokenizer: trigram' as capability errors. Add tests covering: - Partial shadow-table damage (read-corruption class) - Repair brings reads back online - Healthy degraded DB without fts5 module stays healthy (regression for the misclassification risk) - Healthy degraded DB without trigram tokenizer stays healthy Closes #66906 review feedback Refs #66724 * fix(state): self-heal FTS corruption on the SessionDB search path too Complements #66296 (self-heal on the write path): search_messages()'s main FTS5 MATCH query caught only sqlite3.OperationalError (a query-syntax error → return empty). A corrupt FTS index raises the malformed / "fts5: corrupt structure record" class, which is a sqlite3.DatabaseError — the parent of OperationalError, so it was NOT caught and propagated straight out of search_messages, crashing session/history search. The write path now rebuilds and retries on that class, but a read-only session (cron/CLI history search, or a search issued before any write) never triggers a write, so its search stayed broken until the next process restart ran the offline repair. Catch the DatabaseError corruption class on the search MATCH read too and route it through the existing one-shot _try_runtime_fts_rebuild(), then retry the query. The catch is moved outside `with self._lock` so rebuild_fts() can re-acquire the lock (mirrors _execute_write). The one-shot guard is shared with the write path, so a single instance never loops on a genuinely unrecoverable index. OperationalError syntax handling is unchanged (caught first). Adds a regression test: with a corrupted messages_fts and no post-corruption write, search_messages() rebuilds in place and returns the match; without the fix it raises DatabaseError. * fix(state): extend search-path FTS self-heal to the CJK/trigram branch The trigram MATCH branch in search_messages() had the same OperationalError-only catch that #66420 fixed on the main FTS5 branch: a corrupt messages_fts_trigram shadow table raises the malformed / 'fts5: corrupt structure record' class (sqlite3.DatabaseError, parent of OperationalError), which propagated straight out of search_messages and crashed CJK session/history search for read-only sessions. Route that class through the shared one-shot _try_runtime_fts_rebuild() and retry the trigram query (catch moved outside self._lock so rebuild_fts() can re-acquire it, mirroring the main branch). If the rebuild is refused (guard consumed / FTS disabled / different error) or the retry fails, fall through to the existing LIKE substring fallback — which reads only the canonical messages table — instead of raising, so CJK search degrades gracefully rather than crashing. Adds two regression tests: trigram search self-heals in place after shadow-table corruption (answers from the rebuilt trigram index, not the LIKE fallback), and degrades to LIKE without raising when the one-shot rebuild was already consumed. Follow-up to #66420; refs #66296 #66724 * fix(state): add REINDEX strategy to repair stale B-tree indexes (#63386) When PRAGMA integrity_check reports 'wrong # of entries in index' for B-tree indexes (e.g. idx_sessions_handoff_state), the existing repair strategies (FTS rebuild, sqlite_master dedup, drop-FTS+VACUUM) don't address the mismatch. Add Strategy 0.5: run REINDEX to rewrite the index b-tree from canonical table rows before escalating to more destructive strategies. * test(state): exercise REINDEX repair against a REAL stale B-tree index Replace the mocked test for #63398's REINDEX strategy: the original monkeypatched _db_opens_cleanly to return the corruption string, so the REINDEX pass itself was never exercised against actual index corruption — the test would pass even if REINDEX didn't fix anything. New fixture _corrupt_btree_index() builds genuine on-disk staleness with a writable_schema hack: rewrite the index definition to a partial index (WHERE 0), REINDEX so the b-tree is rebuilt empty, then restore the full definition. integrity_check then reports the real 'wrong # of entries in index idx_messages_session' / 'row N missing from index' class from #63386 — no mocks anywhere. The rewritten test asserts end-to-end with real function calls: - the real _db_opens_cleanly detects the stale index, - repair_state_db_schema repairs it with strategy 'reindex_btree', - post-repair the detector and raw PRAGMA integrity_check both report healthy, and a query forced through the rebuilt index (INDEXED BY) sees every row. Adds a second test asserting the REINDEX strategy is non-destructive (all sessions/messages survive, readable via SessionDB). Follow-up to #63398; refs #63386 * fix(kanban): auto-repair index-only kanban.db corruption via REINDEX _guard_existing_db_is_healthy previously failed closed on ANY integrity_check failure, including the index-scoped class ('wrong # of entries in index <name>' / 'row N missing from index <name>') where the table b-trees are intact and REINDEX rebuilds the damaged indexes losslessly. Boards hit by that class were bricked until manual surgery even though SQLite can fix them in-place. Now, when integrity_check output consists ONLY of index-scoped errors (index name parsed generically from the message — no hardcoded list): 1. quarantine the corrupt bytes FIRST via the existing content- addressed _backup_corrupt_db, 2. under the caller-held cross-process init flock, REINDEX each named index (falling back to bare REINDEX if a parsed name doesn't resolve), 3. re-run integrity_check and proceed only if it comes back clean. Any non-index error class (page corruption, malformed image, freelist damage) — or a REINDEX whose re-check is still dirty — fails closed exactly as before: backup + KanbanDbCorruptError, no silent recreation. Transient OperationalError (locked/busy) still propagates raw with no quarantine. Tests build a real board DB and corrupt a live index via the writable_schema/partial-index REINDEX trick to produce the genuine 'wrong # of entries in index' shape, then assert auto-repair recovers with data intact, page corruption still raises, and a dirty re-check fails closed. * fix(kanban): cap corrupt-backup retention at 10 files per board DB Content-addressed quarantine backups dedupe identical corrupt bytes, but corruption that keeps mutating between failures (partial repairs, further damage across dispatcher retries, multi-profile fleets) mints a new sha-named backup every round — a user accumulated 124 .corrupt.*.bak files with no bound. After each NEW backup is created, prune oldest-by-mtime backups beyond _CORRUPT_BACKUP_RETENTION (module constant, default 10), including the copied -wal/-shm sidecars. The just-created backup is always exempt (copy2 preserves the source mtime, which can be older than existing backups). Pruning is best-effort and never masks the corruption error about to be raised; dedupe of identical corrupt bytes is unchanged. * feat(kanban): periodic WAL checkpoint (TRUNCATE) on the dispatcher tick Kanban connections set wal_autocheckpoint=100, but SQLite's passive autocheckpoint backs off whenever any reader holds an open snapshot — on a busy multi-process board the -wal file can grow without bound between gateway restarts. After each successful dispatch tick, while still holding the board's single-writer dispatch flock, run PRAGMA wal_checkpoint(TRUNCATE) best-effort at a coarse interval (>=5 min since this process last checkpointed that board; module-level per-path monotonic timestamp, so multi-board dispatchers checkpoint each board on its own clock). Success and busy/locked skips are both logged at DEBUG; a failing checkpoint can never fail the tick. * feat(kanban): add `hermes kanban repair` CLI verb Adds kanban_db.repair_db() — a structured, non-raising wrapper around the same narrow repair policy as the connect-time guard: probe with PRAGMA integrity_check under the board's cross-process init flock; quarantine the corrupt bytes FIRST via the content-addressed backup; REINDEX only when every integrity message is index-scoped; re-check; report ok / repaired / corrupt / missing. Locked/busy OperationalError still propagates raw (a locked healthy DB is not corruption and gets no quarantine), and a repair invalidates the per-process healthy-path cache so the next connect() re-probes. The CLI verb reports status human-readably (or --json), exits 0 for ok/repaired/missing and 1 when the DB is still corrupt (non-index corruption stays fail-closed with manual-recovery guidance). It dispatches BEFORE kanban_command's auto-init: init_db() raises KanbanDbCorruptError on a corrupt board, which previously would have made a repair verb unreachable on exactly the boards that need it. CLI tests drive the real argparse surface (build_parser + kanban_command) against real corrupted SQLite fixtures. * fix(packaging): graft web_dist in MANIFEST.in and add sdist regression test Wheels ship hermes_cli/web_dist via pyproject package-data, but the sdist did not: MANIFEST.in had no graft and .gitignore excludes web_dist, so source tarballs installed a dashboard-less package. Graft the directory and add an sdist regression test that builds the tarball and asserts index.html is inside. Salvaged from #29661; the PR's [web]-extra 404-message change was dropped per maintainer review (misleading guidance for source installs). * fix(dashboard): attempt one recovery build when --skip-build finds no dist --skip-build with a missing web_dist/index.html p…

teknium1 mentioned this pull request Jul 17, 2026

fix(gateway): retry failed transcript appends and rebuild corrupted FTS #65637

Closed

teknium1 force-pushed the fix/transcript-append-failure-visibility branch from dc6f29b to 4deb286 Compare July 17, 2026 12:58

tonydwb reviewed Jul 17, 2026

View reviewed changes

teknium1 closed this Jul 17, 2026

teknium1 force-pushed the fix/transcript-append-failure-visibility branch from 4deb286 to ec3d958 Compare July 17, 2026 13:58

teknium1 changed the title ~~fix(state): self-heal FTS corruption on the write path; surface transcript append failures~~ fix(state): self-heal FTS corruption on the SessionDB write path (cron/CLI writers) Jul 17, 2026

teknium1 reopened this Jul 17, 2026

teknium1 merged commit 9e1b1d7 into main Jul 17, 2026
33 checks passed

teknium1 deleted the fix/transcript-append-failure-visibility branch July 17, 2026 15:51

Frowtek mentioned this pull request Jul 17, 2026

fix(state): self-heal FTS corruption on the SessionDB search path too #66420

Closed

12 tasks

Navigation Menu

Uh oh!

fix(state): self-heal FTS corruption on the SessionDB write path (cron/CLI writers) - #66296

fix(state): self-heal FTS corruption on the SessionDB write path (cron/CLI writers)#66296
teknium1 merged 1 commit into
mainfrom
fix/transcript-append-failure-visibility

teknium1 commented Jul 17, 2026 •

edited

Loading

Uh oh!

alt-glitch commented Jul 17, 2026

Uh oh!

tonydwb left a comment

Uh oh!

Uh oh!

Reviewers

Assignees

Labels

Projects

Milestone

Development

Uh oh!

3 participants

Uh oh!

Conversation

teknium1 commented Jul 17, 2026 • edited Loading Uh oh! There was an error while loading. Please reload this page.

Uh oh!

Summary

Changes

Validation

Infographic

Uh oh!

alt-glitch commented Jul 17, 2026

Uh oh!

tonydwb left a comment

Choose a reason for hiding this comment

Code Review Summary

Looks Good

Uh oh!

Uh oh!

Reviewers

Assignees

Labels

Projects

Milestone

Development

Uh oh!

3 participants

teknium1 commented Jul 17, 2026 •

edited

Loading