fix(matrix): reset Olm crypto store when the access token's device ID changes - #71543
Closed
ckaznocha wants to merge 3 commits into
Closed
fix(matrix): reset Olm crypto store when the access token's device ID changes#71543ckaznocha wants to merge 3 commits into
ckaznocha wants to merge 3 commits into
Conversation
… changes The crypto store is keyed by Matrix user ID, not device ID, so swapping in a new access token (which mints a new device_id) silently inherits the previous device's Olm account. That account's identity keys can never be published under the new device ID, and the pickle key embeds the old device ID anyway — the result is stale-key mismatches and cross-signing signatures the homeserver refuses to replace, degrading E2EE in ways that are hard to diagnose (peers silently withhold room keys). _reset_crypto_store_if_device_changed() compares the store's persisted device ID against the live one at connect time and wipes the store on mismatch, so a fresh Olm account is generated for the new device instead of reusing stale key material. Co-Authored-By: Claude Sonnet 5 <noreply@anthropic.com>
alt-glitch
added
type/bug
ckaznocha
marked this pull request as ready for review
July 25, 2026 20:02
GottZ
reviewed
Jul 29, 2026
GottZ
left a comment
GottZ
left a comment
There was a problem hiding this comment.
This was generated by AI during triage.
Summary
Two PRs address distinct Matrix E2EE crypto-store failure modes: #71543 resets stale Olm state when the live device ID differs from the persisted device ID, while #71547 attempts to recover accounts and sessions after the configured pickle key changes. The supplied diffs were evaluated independently; they do not establish the behavior or ordering of a combined application.
Related pull requests
- #71543
related— (+85/-0) — merge candidate: The diff compares the persisted and live device IDs before crypto initialization and deletes the store on mismatch, directly targeting reuse of old-device Olm identity material; its tests cover mismatch, match, fresh-store, and missing-live-ID branches. The evidence does not test whether the opened store remains usable after deletion or how this path interacts with #71547. - #71547
related— (+195/-0) — keep open for revision: The diff retries account loading with known legacy keys and rewrites recoverable Olm/Megolm session rows under the current key, addressing the reported BAD_ACCOUNT_KEY transition. It does not re-pickle every stored session: rows unreadable with both keys remain unchanged despite a log message saying they are being dropped, and the supplied tests do not exercise session-row updates, unreadable rows, or the combined path with #71543.
Suggested consolidation
Merge #71543 only after validating the post-delete reconnect/initialization path. Do not close #71547 as a duplicate: it addresses a different pickle-key migration cause, but it should remain separate until its unreadable-row behavior and logging are corrected, session migration is tested directly, and combined execution with #71543 is verified.
Cross-PR triage: Reviewed 2 pull requests and 0 issues in this complex. Each diff was read against this issue; Assessment working set: 14 kB of PR diffs, 9 kB of issue/PR text, 0 verify verdicts. verdicts reflect diff content, not PR titles. Part of an automated triage batch.
teknium1
reviewed
Jul 30, 2026
teknium1
left a comment
teknium1
left a comment
Contributor
There was a problem hiding this comment.
Thanks for isolating the token/device transition and using the store API rather than deleting the database file directly.
Problems
- The new comparison is fed
client.device_id, but the token path obtains the live ID fromwhoami()and then overwrites it with configuredMATRIX_DEVICE_IDwhen present (plugins/platforms/matrix/adapter.py:1403-1411). A rotated token reporting B while configuration still contains A therefore compares persisted A to A and never resets. This misses the PR's stated token-rotation guarantee.
Suggested changes
- Preserve the live
whoami()ID for mismatch handling and define the explicit behavior when it conflicts with configuredMATRIX_DEVICE_ID(use live B for reset/initialization, or reject the mismatch). - Add a
connect()regression covering persisted A +whoami()B + configured A. The current added tests only call the helper withMagicMockstores and do not exercise this resolution path.
Automated hermes-sweeper review.
| await crypto_store.open() | ||
|
|
||
| if client.device_id: | ||
| await self._reset_crypto_store_if_device_changed( |
Contributor
There was a problem hiding this comment.
client.device_id is not always the access token's live device: the token-auth path prefers configured MATRIX_DEVICE_ID over whoami().device_id (adapter.py:1403-1411). With stale configured A and a rotated token on B, this helper receives A and skips the reset. Preserve and compare the live whoami() value, or explicitly reject that configuration mismatch.
teknium1
added
sweeper:risk-session-state
GottZ
reviewed
Jul 30, 2026
GottZ
left a comment
GottZ
left a comment
There was a problem hiding this comment.
This was generated by AI during triage.
Delta since our previous triage comment
@teknium1 corrected our assessment of #71543 by identifying that the configured MATRIX_DEVICE_ID can overwrite the live whoami() device ID, causing the proposed reset to miss the token-rotation case it claims to fix. They also extended the #71547 analysis with an ordering failure: writing the account before repickling sessions can make a partial migration appear complete and prevent retries after interruption.
Changed pull requests
- #71543
related— (+85/-0) — keep open for revision: The helper correctly compares its inputs, butconnect()passesclient.device_id, which may be stale configured value A rather than livewhoami()value B; the diff therefore misses persisted A + configured A + rotated-token B and lacks aconnect()regression for that path. - #71547
related— (+195/-0) — keep open for revision: The diff writes the account under the new key before completing the session sweep, so a sweep failure can leave legacy-key sessions behind while subsequent starts take the current-account fast path and never retry; migration must be atomic or write the account last, with a failure-and-retry regression.
Suggested consolidation
The recommendation is revised: merge neither #71543 nor #71547 until their respective contributor-reviewed correctness gaps are addressed.
Complex graph unchanged since our previous triage comment.
Cross-PR triage: Reviewed 2 pull requests and 0 issues in this complex. Each diff was read against this issue; Assessment working set: 14 kB of PR diffs, 9 kB of issue/PR text, 3 kB of discussion (4 comments), 0 verify verdicts. verdicts reflect diff content, not PR titles. Part of an automated triage batch.
…E_ID connect() resolved client.device_id as `self._device_id or resolved_device_id`, so a configured MATRIX_DEVICE_ID masked the live whoami() device. With persisted A, configured A and a rotated token reporting B, the reset compared A to A and never fired — exactly the token-rotation case this PR claims to handle. An access token is bound to one device and the homeserver only accepts key uploads for that device, so a configured value naming a different one cannot work. The live whoami() device now wins on conflict and logs an error naming both. The configured value is still preferred when whoami() reports no device. Adds a connect()-level regression for persisted A + configured A + whoami B, and corrects test_connect_uses_configured_device_id_over_whoami, whose stated premise this inverts. Co-Authored-By: Claude Opus 5 <noreply@anthropic.com>
Contributor
Author
|
@teknium1's catch is correct, and it was the whole point of the PR — fixed in d6764b6..HEAD.
Resolution chosen. Of the two options offered — use live B, or reject the mismatch — I took live B, plus an error-level log naming both IDs. Reasoning: an access token is bound to exactly one device and the homeserver only accepts key uploads for that device, so a This inverts an existing tested premise — flagging explicitly. Regression test added at One interaction worth noting for #71547: 256 passed, |
13 tasks
This was referenced Aug 3, 2026
Collaborator
|
Merged via #77336 using both of your commits cherry-picked with authorship preserved (rebase-merge). The crypto store reset and pickle key migration were combined with @webtecnica's homeserver encryption-info fallback (#71073), plus follow-up fixes for the pickle key / resolved device ID gap that connected all three PRs together. Thanks for the thorough work — the session-sweep ordering fix and the commit-marker insight were excellent. |
kshitijk4poor
added a commit
that referenced
this pull request
Aug 3, 2026
… reset, cache enc info Follow-up fixes for the combined Matrix crypto salvage (#71073, #71543, #71547): 1. Construct _pickle_key from client.device_id (resolved from whoami) instead of self._device_id (the configured value). Without this, when #71543 makes the token's real device win over a stale MATRIX_DEVICE_ID, the pickle key is built from the stale value and the Olm account is stored under a key that can never be looked up again — perpetuating the same decryption failure the PRs aim to fix. 2. Skip _migrate_legacy_crypto_pickle when the store was just deleted by _reset_crypto_store_if_device_changed — there is no account to migrate. Also check the migration return value and log a warning on failure instead of proceeding to olm.load() which fails with a cryptic BAD_ACCOUNT_KEY. 3. Add a local dict cache (_enc_info_cache) to _CryptoStateStore so the homeserver fallback in get_encryption_info() does not make a network round-trip on every is_encrypted() call. MemoryStateStore does not implement set_encryption_info, so the existing cache-back is a no-op. 4. Log homeserver encryption-info query failures at DEBUG level instead of silently returning None (which would cause OlmMachine to treat an encrypted room as unencrypted). 5. Update _CryptoStateStore docstring to mention the homeserver fallback.
OmarB97
added a commit
to OmarB97/hermes-agent
that referenced
this pull request
Aug 4, 2026
* feat(desktop): reload a tab from its right-click menu
Right-click any tab and pick Reload: the pane's content remounts in
place — effects re-run, state resets, measurements are retaken — while
the tab keeps its slot and every other tab is untouched.
A per-pane epoch atom keys the contribution inside the zone body, so
reload never rewrites the layout tree. Both tab menus offer it: the
zone strip menu (tool panels, the file tree, a fresh draft's main tab)
and the session tab menu (tiles + the loaded main tab).
* fix(approval): classify CLI/TUI approval timeouts separately from explicit denials
When an approval prompt expired without a response, every CLI-side path
collapsed the timeout into the same 'deny' choice as an explicit user
refusal, so the agent was told the user denied the action when the user
simply never answered. The gateway wait already distinguished the two
('timed out without user response... Silence is not consent.'); this
brings the CLI/TUI/ACP surfaces to parity.
- prompt_dangerous_approval(): input()-path expiry now returns a distinct
'timeout' choice (still fail-closed).
- cli.py _approval_callback + hermes_cli/callbacks.py approval_callback:
deadline expiry returns 'timeout' instead of 'deny'.
- check_all_command_guards / _run_approval_gate CLI tails: 'timeout' maps
to outcome='timeout' with a 'timed out without user response... Silence
is not consent.' BLOCKED message (matching the gateway wording);
explicit deny keeps outcome='denied' and gains user_consent=False for
shape parity.
- computer_use: 'timeout' verdict threads through the CLI adapter and
yields a 'prompt timed out — the user did not respond' error instead of
'denied by user'.
- ACP permissions bridge: FutureTimeout returns 'timeout' (other failures
still 'deny'); elicitation maps 'timeout' to 'cancel' like the gateway's
unresolved outcome; codex wire mapping documents deny/timeout→decline.
- write_approval already treats unknown choices as 'stage, not drop', so
a timeout now stages the memory write instead of silently refusing it.
Every timeout path remains fail-closed — the action never runs; only the
classification reported to the agent changes.
* fix(desktop): paste rich text with images as text, not blank attachments
Copying a Discord thread (or any rich-text selection with images) attached
one or more blank thumbnails and dropped the message text entirely.
Two causes. The clipboard's `text/html` was scraped for inline
`<img src="data:…">` regardless of whether the copy carried its own text —
and what Discord ships beside each image embed is a 32x5 blurhash
placeholder, which is exactly the blank attachment. Then, because any image
blob short-circuited the paste handler, the prose that came with it never
reached the composer.
Inline HTML images now only count for an image-only copy, and are ignored
below a thumbnail-sized floor so spacers and trackers don't attach either. A
mixed paste attaches its real images and still inserts its text.
Also registers pasteAndMatchStyle in the Edit menu — Cmd+Shift+V had no menu
entry, so the chord was never translated into an editor command anywhere in
the app.
* fix(cli): persist YOLO mode across --resume
A session's YOLO bypass lived only in the in-memory
tools.approval._session_yolo set (or the process-frozen --yolo env
var), so resuming a session in a fresh process silently reverted the
user's /yolo ON — dangerous commands started prompting again.
Persist a yolo_mode flag in the session row's model_config JSON and
restore it on every CLI resume path:
- SessionDB.set_session_yolo() merges the flag into model_config
(same lineage-preserving merge as update_session_runtime_lock);
SessionDB.session_yolo_enabled() reads it back, false on any parse
failure.
- /yolo toggle persists ON and OFF through the new helper; the
compression/branch session-id rotation carries the flag onto the
continuation row.
- --yolo launches record the flag at session creation (agent_init),
and a /yolo toggled before the lazily-created row exists is carried
into the creation-time model_config (_ensure_db_session).
- HermesCLI._restore_session_yolo() re-enables the bypass on startup
--resume/-c, the deferred init path, and mid-chat /resume, with a
visible '⚡ YOLO mode restored from session' notice. No-op under a
frozen process-wide --yolo and never enables on absent/garbage flags.
* fmt(js): `npm run fix` on merge (#77300)
Co-authored-by: github-actions[bot] <github-actions[bot]@users.noreply.github.com>
* chore: contributor email mappings for egilewski and myk0la-b
* perf(tui): bound scroll rendering and preserve anchors
* chore: AUTHOR_MAP for ckaznocha (matrix crypto PRs)
* chore: add contributor email mapping for LFDMcore (core@lfdm.co) (#77331)
* fix(tui): snapshot history after pending model switch applies (#76870)
Deferred model switches append a marker and bump history_version at
turn start; the dispatcher was snapshotting history before that
mutation, so the version-mismatch guard rejected the turn's own
result as a stale/concurrent write. Move the snapshot to after
_apply_pending_model_switch/_sync_agent_model_with_config, under
history_lock, so the turn's own preparatory mutation is included in
its baseline while the anti-stale guard still catches real external
writes.
Co-Authored-By: Claude Sonnet 5 <noreply@anthropic.com>
* chore: drop stale diagnostic report from PR #77021
teknium1 flagged ISSUE_76870_RELATORIO_CAUSA_RAIZ.md as containing
stale metadata (references an unrelated local branch) and asked to
drop the standalone report, keeping only the focused server.py fix
and regression test.
* perf(agent): precompile response and skill-scan regexes (#33208)
strip_think_blocks passed the same response-scrubbing strings through
re's pattern dispatcher on every response. Skills Guard repeated the
same work for 121 patterns against every scanned line.
Compile the existing expressions once and reuse Pattern.sub/search. Keep
each generic tool-call tag in its own paired expression so mismatched
openers retain their payload while existing stray-closer cleanup remains
unchanged.
Part of #33208
Salvaged from #32713 by @ErnestHysa.
Co-authored-by: ErnestHysa <takis312@hotmail.com>
* refactor(agent): rebuild hoisted patterns from shared tag-name tuples
Simplify-pass follow-up on the #69653 salvage: the original code built
these patterns from a name loop; hand-expanding them into 10 literals
lost that single source. Tag-name tuples restore it (adding a 6th
reasoning tag is now a one-place change), and the gnarly named-function
pattern regained a pointer to its step-1c rationale. Byte-equivalence
of every rebuilt pattern verified programmatically (alternation-order
neutrality probed: the \b and > anchors make order irrelevant).
* chore: add ATran28 to AUTHOR_MAP
at828@proton.me -> ATran28 (GH id=1445620)
Needed for PR #77270 whatsapp bridge reconnect fix.
* fix(gateway): defer in-band restart until active turns finish (#77184)
request_restart was calling stop() immediately, so the requesting turn stayed
in the drain wait set and got force-killed at restart_drain_timeout. Wait for
active work to reach zero first, then stop against an idle gateway.
* test(gateway): cover restart after-turn deferral (#77184)
* fix(tui_gateway): drain in-flight turns before finalizing sessions on compute-host shutdown
ComputeHost.shutdown() called flush_all_sessions() before its own in-flight
turn drain loop. server._finalize_session latches on session["_finalized"]
and every later call returns immediately, so that one flush was spent while
turns were still producing output: the unflushed tail was never persisted,
commit_memory_session wrote long-term memory from a truncated transcript, the
session's DB row was marked ended while it was live, on_session_end fired with
completed=False/interrupted=True against a running session, and the
active-session lease was released out from under a turn. The drain loop exists
precisely so that mid-turn work survives a teardown; finalizing first defeated
it. Reachable from all three teardown paths: the parent/orphan guard (which
os._exit(0)s immediately after), the SIGTERM/SIGINT handler, and stdin close.
Drain first, then flush. A slice of the caller's budget (_FLUSH_RESERVE_SECS,
never more than half of it so a short explicit wait still gets a real drain) is
withheld from the drain so the flush still runs when turns outlast the window:
HostSupervisor SIGKILLs the host _SHUTDOWN_TIMEOUT_SECS after SIGTERM — 10.0s,
the same value as shutdown()'s default wait — so a drain allowed to consume the
whole budget would leave the durability write racing that kill. `wait` itself is
unchanged, so total shutdown latency and the SIGTERM->SIGKILL margin are
unchanged.
* fix(tui_gateway): bound the shutdown drain sleep by the time left to it
The drain loop slept a flat 0.05s per tick, so it could overshoot its deadline
by up to one tick and spend part of the reserve withheld for
flush_all_sessions(). For a small `wait` the reserve is itself half the budget,
so a single overshoot can consume all of it: at wait=0.34 the drain budget is
0.17s but the loop requested 4 x 0.05 = 0.20s of sleep.
Clamp each tick to the remaining time. The new test asserts on the summed
*requested* sleep rather than wall-clock, which is deterministic: every sleep is
bounded by the strictly-decreasing remainder, so the total can never exceed the
drain budget regardless of how the scheduler interleaves.
* fix(tui_gateway): retain live-turn sessions unfinalized when the drain deadline expires
The drain reserves a slice of the shutdown budget so flush_all_sessions
still runs when in-flight turns outlast the window. But that flush was
unconditional: a session whose turn was still running got its one-shot
_finalize_session spent mid-turn, and the executor.shutdown(wait=False,
cancel_futures=True) immediately after does not join the turn. The
session was then permanently un-finalizable and its active-session lease
had been released out from under live work — the same persistence and
lifecycle race the drain exists to close, just relocated past the
deadline instead of removed.
Give _turn_futures a session association (Future -> sid, the same key
space as server._sessions) at both submit sites, and on deadline expiry
exclude the sids whose futures are still running from the flush. Those
sessions are retained unfinalized and therefore recoverable; sessions
with no live turn finalize exactly as before. The done-callback now pops
under the lock, since a bare dict.pop is not the drop-in set.discard was.
wait semantics, the reserve math and the bounded per-tick sleep are
unchanged, so this adds no shutdown latency. All three shutdown callers
(orphan, sigterm, and the tight stdin_closed wait=2.0 path) funnel
through this one function and are covered.
* docs: note atexit re-finalization interaction in shutdown() docstring
The PR's skip-live-sessions optimization is partially defeated by
server._shutdown_sessions() registered via atexit (server.py:1172),
which runs on SystemExit after shutdown() returns for the SIGTERM and
stdin_closed paths. The orphan path (os._exit(0)) bypasses atexit.
This is a pre-existing issue — the old finalize-first order had the
same atexit interaction. The comment documents the gap and suggests a
follow-up: gate _shutdown_sessions on not session.get('running').
* fix(desktop): memoize sidebar flatRows and row renderers to prevent scroll jitter (fixes #73629)
Fix direction inspired by PR #73674 by @drbronson with added Vitest component unit tests.
* fix(test): export VirtualSessionListProps and assert sessions array identity change
* style(desktop): eslint --fix import order in sessions-section test
Review follow-up on the #75714 salvage: perfectionist/sort-imports
would fail lint CI; autofixed.
* fix(ui-tui): pause status-chrome timers while a blocking overlay is open
`StatusRulePane` renders `StatusRule` outside the `!isBlocked` guard in
appLayout.tsx, so the status rule stays mounted underneath approval,
model-picker, pager, sessions and every other blocking overlay. Its three
timer-driven components keep firing the whole time: `FaceTicker` (glyph,
1s clock, verb rotation), `SessionDuration` (1s) and `IdleSince` (1s).
Every tick re-renders a rule nobody can see, and in an Ink TUI that churn
reads to the user as the dialog flickering.
Gate all three components' interval creation on the existing `$isBlocked`
computed store, so nothing is armed while an overlay covers the rule.
The pause alone would leave the elapsed read-outs frozen at the moment the
overlay opened, so each effect re-seeds `now` from the wall clock when it
re-arms. `SessionDuration` and `IdleSince` already did this; `FaceTicker`
gains the same re-sync. Closing a five-minute overlay now resumes at the
true elapsed time instead of the pre-overlay value.
No new store is introduced — `$isBlocked` already exists and already ORs
the current OverlayState field set.
* fix(ui-tui): gate status-rule timers on real occlusion, not $isBlocked
`$isBlocked` answers "is text input suspended", not "is the status rule
covered". appLayout uses it only to hide the input rows (appLayout.tsx:384);
`StatusRulePane` renders outside that guard, at :365 for `at="top"` and :449
for `at="bottom"`. So the previous revision paused FaceTicker /
SessionDuration / IdleSince under prompts that leave the rule fully on
screen — approval, billing, subscription, confirm, clarify, sudo and secret
all render through PromptZone in NORMAL FLOW above ComposerPane
(appOverlays.tsx:58-162, appLayout.tsx:553-568). They push the rule down;
they do not cover it. Freezing a visible clock is a worse bug than the churn
being removed.
Replace it with `$isStatusRuleOccluded`, a narrow derived store over
overlay + ui state covering only what actually paints over the rule:
- `widget` — the modal widget slot renders at viewport level
(ActiveWidgetSlot, sdk/host.tsx:209) so it can anchor the full-screen
absolute `Overlay` against the whole terminal.
- the FloatingOverlays set (modelPicker, pager, petPicker, sessions,
skillsHub, pluginsHub) — but only when `ui.statusBar === 'top'`. That
panel is `position="absolute" bottom="100%"` inside ComposerPane's
relative Box (appOverlays.tsx:387), so it grows UPWARD over the top rule
and can never reach the bottom one.
Deliberately excluded: the PromptZone flow states above; `agents` and
`journey`, which unmount the entire ComposerPane subtree (appLayout.tsx:553)
so React's effect cleanup already clears the intervals; `ambient`, an
in-flow dock; and composer completions, which share the floating grid but
are a render prop that changes per keystroke — re-arming a 1s interval on
every character would restart the countdown each time and starve the tick.
`statusBar: 'off'` needs no branch: StatusRulePane returns null for both
slots, so the timers never mount.
Tests: the store-level cases are re-split into occluding and non-occluding
sets, and an AppLayout-level `describe` mounts the real layout so the rule
sits in its true position — asserting that under approval and sudo the rule
is still rendered AND its clock advances (1m 0s to 1m 30s), that a floating
model picker suppresses the clocks with the rule at the top, and that the
same picker leaves them armed with the rule at the bottom.
* refactor(ui-tui): single source for the floating-panel kind set
Review follow-up: $isStatusRuleOccluded and FloatingOverlays each
enumerated the same six overlay kinds — adding a 7th floating panel
required updating both or the timer gate silently missed it. Extracted
hasFloatingPanel as the shared predicate (completions stays local to
FloatingOverlays; it deliberately never occludes the status rule).
Full ui-tui suite green (1487 tests).
* fix(compression): overflow handlers pass overhead-aware token size to LCM recovery (issue 441)
Root fix (Option A) for design-session "Context compression exhausted" crashes. The three
compression-retry handlers after an API overflow/413/long-context error (conversation_loop.py
~4229/4488/4747) passed the tool-BLIND messages-only estimate (approx_tokens) to _compress_context,
so hermes-lcm's forced-overflow recovery armed on the message count and missed overflows driven by
tool-schema/system overhead. Now they pass estimate_request_tokens_rough(api_messages, tools=...)
— the same overhead-aware estimator already used at :4580 — so recovery arms on the TRUE request
size; LCM's _overflow_recovery_assembly_cap self-subtracts the overhead so the full request fits.
Empirically validated on real failed session 6dddf1a67b76 (LCM engine, floor=24000/cap=248000):
observed 256,359 >= 248,000 -> arms; recovery 231,313->208,559 msg-tokens -> full request
233,605 < 272,000 FITS. Prior messages-only path did NOT arm (231K < 248K) and crashed.
Durable copy: ~/.hermes/local-patches/optionA-overflow-overhead-aware.patch (survives hermes update
reset). Upstream PR pending. classify_api_error call at :3667 intentionally unchanged (not recovery).
Co-Authored-By: Claude Opus 4.8 (1M context) <noreply@anthropic.com>
* fix(compression): route overhead-aware tokens in post-tool compress + add recovery-path tests
Post-tool compression path passed context_compressor.last_prompt_tokens (0 in the no-usage
fallback) to _compress_context instead of the overhead-aware _real_tokens computed just above
— same tool-blind bug as the overflow handlers (upstream PR #77169 review, teknium1). Also adds
production-path regression tests asserting the 413, context-overflow (two wordings), and
Anthropic long-context recovery handlers pass estimate_request_tokens_rough(..., tools=...)
(sentinel-patched) to _compress_context.
Co-Authored-By: Claude Opus 4.8 (1M context) <noreply@anthropic.com>
* fix(matrix): fallback to homeserver query for room encryption detection (#71067)
_CryptoStateStore.get_encryption_info only consulted mautrix's in-memory
MemoryStateStore, which has no record of m.room.encryption for rooms the
bot joined in the past (the raw-sync path never feeds those state events
through set_encryption_info). On a fresh crypto store this returns None
for all previously-joined rooms, so OlmMachine reports them as unencrypted,
never tracks peer devices, and silently drops all inbound messages.
Fix: pass the mautrix Client into _CryptoStateStore so get_encryption_info
can fall back to a live GET /_matrix/client/v3/rooms/{room_id}/state/
m.room.encryption query when the in-memory store returns None. The result
is cached back via set_encryption_info so subsequent lookups (and
OlmMachine device tracking) hit the fast path.
* fix(matrix): reset Olm crypto store when the access token's device ID changes
The crypto store is keyed by Matrix user ID, not device ID, so swapping in
a new access token (which mints a new device_id) silently inherits the
previous device's Olm account. That account's identity keys can never be
published under the new device ID, and the pickle key embeds the old
device ID anyway — the result is stale-key mismatches and cross-signing
signatures the homeserver refuses to replace, degrading E2EE in ways that
are hard to diagnose (peers silently withhold room keys).
_reset_crypto_store_if_device_changed() compares the store's persisted
device ID against the live one at connect time and wipes the store on
mismatch, so a fresh Olm account is generated for the new device instead
of reusing stale key material.
Co-Authored-By: Claude Sonnet 5 <noreply@anthropic.com>
* fix(matrix): let the token's own device win over a stale MATRIX_DEVICE_ID
connect() resolved client.device_id as `self._device_id or resolved_device_id`,
so a configured MATRIX_DEVICE_ID masked the live whoami() device. With
persisted A, configured A and a rotated token reporting B, the reset
compared A to A and never fired — exactly the token-rotation case this PR
claims to handle.
An access token is bound to one device and the homeserver only accepts key
uploads for that device, so a configured value naming a different one
cannot work. The live whoami() device now wins on conflict and logs an
error naming both. The configured value is still preferred when whoami()
reports no device.
Adds a connect()-level regression for persisted A + configured A +
whoami B, and corrects test_connect_uses_configured_device_id_over_whoami,
whose stated premise this inverts.
Co-Authored-By: Claude Opus 5 <noreply@anthropic.com>
* fix(matrix): migrate crypto store when the Olm pickle key changes
The Olm account pickle key is derived from the account ID plus the
configured device ID (acct:device_id). If the crypto store's account was
created before MATRIX_DEVICE_ID was set — e.g. the very first password
login, where the device ID is only known after connecting — it gets
pickled under "<acct>:default". Setting MATRIX_DEVICE_ID afterwards (a
reasonable thing to do once you know the device ID you want to pin)
changes the derived pickle key, and every subsequent unpickle attempt
fails with BAD_ACCOUNT_KEY. In optional-E2EE mode that failure is
swallowed and encryption silently stays disabled instead of surfacing an
actionable error.
_migrate_legacy_crypto_pickle() detects the BAD_ACCOUNT_KEY failure,
tries the known legacy pickle keys, and re-pickles the account (plus
every stored olm/megolm session — sessions share the same pickle key, so
migrating only the account would leave them unreadable on the next
decrypt and silently break key sharing with peers) under the current
key. It only reports failure when no known key can unpickle the
account, with a log message pointing at what changed.
Co-Authored-By: Claude Sonnet 5 <noreply@anthropic.com>
* fix(matrix): commit the migrated account only after the session sweep
The account was written under the new pickle key before sessions were
re-pickled. The account is effectively the migration's commit marker —
once it reads under the current key, the fast path short-circuits every
later startup — so a sweep that errored or was interrupted left the
remaining legacy-key sessions stranded permanently with no retry.
Sweep first, commit the account last, and return False on sweep failure
so the migration is retried on the next start.
Also corrects the unreadable-row log: it claimed rows were being dropped
while no DELETE was ever issued. Such rows are left in place (already
unusable; deleting crypto material on a guess is not worth it) and the
message now says so.
Adds session-sweep coverage, which was previously absent: rows rewritten
under the current key, rows already current left alone, unreadable rows
left in place, and a failed sweep that leaves the account uncommitted.
The existing migration test now fakes the olm C-extension so the suite
no longer requires libolm.
Co-Authored-By: Claude Opus 5 <noreply@anthropic.com>
* fix(matrix): pickle key from resolved device ID, skip migration after reset, cache enc info
Follow-up fixes for the combined Matrix crypto salvage (#71073,
#71543, #71547):
1. Construct _pickle_key from client.device_id (resolved from whoami)
instead of self._device_id (the configured value). Without this, when
#71543 makes the token's real device win over a stale
MATRIX_DEVICE_ID, the pickle key is built from the stale value and
the Olm account is stored under a key that can never be looked up
again — perpetuating the same decryption failure the PRs aim to fix.
2. Skip _migrate_legacy_crypto_pickle when the store was just deleted
by _reset_crypto_store_if_device_changed — there is no account to
migrate. Also check the migration return value and log a warning on
failure instead of proceeding to olm.load() which fails with a
cryptic BAD_ACCOUNT_KEY.
3. Add a local dict cache (_enc_info_cache) to _CryptoStateStore so the
homeserver fallback in get_encryption_info() does not make a network
round-trip on every is_encrypted() call. MemoryStateStore does not
implement set_encryption_info, so the existing cache-back is a no-op.
4. Log homeserver encryption-info query failures at DEBUG level instead
of silently returning None (which would cause OlmMachine to treat an
encrypted room as unencrypted).
5. Update _CryptoStateStore docstring to mention the homeserver fallback.
* fix(whatsapp): guard bridge reconnect against hangs and unhandled rejections
startSocket() awaits useMultiFileAuthState() and fetchLatestBaileysVersion()
before it creates a socket or registers event handlers, and the close handler
re-entered it via a bare setTimeout(startSocket, ...). That leaves two
unrecoverable failure modes on a reconnect:
- a rejection is an unhandled promise rejection (fatal on modern Node)
- a hang leaves the bridge permanently disconnected with nothing left to
retry, while its HTTP server keeps answering 503 to the gateway
The second mode was observed in the field: fetchLatestBaileysVersion() is a
plain fetch to raw.githubusercontent.com with no AbortSignal, and after a
stream:error 503 disconnect the bridge logged 'Reconnecting in 3s...' once
and then sat silent and disconnected for 27+ hours until manually restarted.
Fix, as two pure helpers in bridge_helpers.js (keeping bridge.js side-effect
free to test):
- createReconnectScheduler(): every (re)connect entry point now catches a
failed startSocket() and reschedules it instead of dying or going silent
- createVersionResolver(): bounds the version fetch with a 15s timeout and
falls back to the last known-good version (or the Baileys default before
first success) instead of pending forever
* chore: add contributor email mapping for danielblankhh
* fix(cron): stop lifecycle guard false-positives and crashes on .py/binary scripts
The gateway lifecycle guard (cron/lifecycle_guard.py) applied shell-style
tokenization and script-reference resolution to non-shell content, with two
regressions:
#77131 - every .py cron script using pathlib division was hard-blocked:
Path.home() / ".hermes" / ".env" tokenizes the bare "/" operator as an
executable path, which resolves to the filesystem root; the regular-file
check then fails closed as unsafe. Since Python runs under the
interpreter, never through a POSIX shell, the shell-script reference walk
is a false-positive generator on Python sources. check_gateway_lifecycle
now skips the walk for *.py scripts (the direct command regex still scans
the full text), and _iter_referenced_shell_scripts skips pure-separator
tokens.
#76762 - terminal commands invoking a binary by absolute path (e.g.
/usr/bin/python3) crashed the guard with ValueError: embedded null byte:
the walk read the binary's bytes, decoded them as text, and re-tokenized
machine code; the recursion then hit Path.resolve() on a NUL-bearing
path while only OSError was caught. _read_referenced_script now skips
NUL-containing files (binaries are not referenced shell scripts) and
resolve() tolerates ValueError.
Shell scripts (.sh/.bash/.zsh) keep the full deep scan; literal lifecycle
commands in .py scripts are still blocked by the direct regex. New tests
cover all four behaviors.
* fix: check NUL bytes before size limit in _read_referenced_script
On Linux, /usr/bin/python3 is >1MB, so the size check fired before
the NUL check could run — the binary was returned as unsafe=True
(blocked) instead of (None, False) (skip). Reorder: read the bounded
chunk first, check for NUL bytes (binary → skip), then check size
(oversized text → fail closed).
* fix(discord): preserve links in truncated tool previews
* refactor(discord): simplify tool preview links
* refactor(gateway): share markdown link formatting
* refactor(discord): keep link formatting adapter-local
* fix(discord): avoid truncated URL link targets
* fix(discord): suppress link embeds in tool preview markdown links
Wrap the masked-link destination in angle brackets so Discord does not
unfurl an OG-preview embed under every tool progress bubble. quote()
percent-encodes any <> inside the URL itself, so the wrapper cannot be
broken out of.
* perf(tts): pipeline sync per-sentence synthesis with playback
The universal sync fallback in stream_tts_to_speaker ran strictly serially
per sentence — synthesize, play, and only then start synthesizing the next
sentence — so every sentence boundary added a full synthesis-time of dead
air. Chunked streamers (elevenlabs/openai/gemini/xai) already avoid this;
every other provider (edge, piper, plugin providers) paid it on each reply
in voice mode and the wake-word loop.
_SyncSentencePipeline overlaps the two: one single-threaded synthesis
worker (sentences stay FIFO; providers never see concurrent calls from
this loop — same effective concurrency as before) feeds one playback
worker through a small bounded queue, so sentence n+1 synthesizes while
sentence n plays. Lookahead is bounded (backpressure + at most a couple of
temp files), stop_event short-circuits both stages, synthesis failures are
isolated per sentence, temp files are always unlinked, and the finally
block flushes the pipeline BEFORE tts_done_event fires so continuous voice
mode never reopens the mic over its own voice. synthesize/play are
resolved late so existing monkeypatch-based tests work unchanged.
Measured with a real local model provider (OmniVoice plugin, Apple
Silicon), same 3-sentence reply, playback simulated at the produced clips'
true durations, best-of-2 interleaved runs under identical load:
serial pipelined
time to first word 10.8s 4.4s
mid-reply dead air 11.2s 1.8s (second gap: 0.03s)
full reply wall 33.2s 17.0s
Tests: 4 new (timestamp-proven overlap, order + per-sentence failure
isolation, stop skips queued playback, temp-file hygiene); the existing
sync-fallback and display-callback tests pass unchanged.
Co-Authored-By: Claude Fable 5 <noreply@anthropic.com>
* fix(tui-gateway): merge agent output on model-switch history_version mismatch (#76870)
When a model switch occurs mid-turn, `_append_model_switch_marker()`
appends a marker to session history and increments `history_version`.
The turn completion guard then sees `current_version != history_version`
and discards all agent output — producing empty assistant messages in
the session DB.
Detect when the only history mutation during the turn was one or more
model-switch markers. In that case, merge the agent's new messages
into the current history (which now contains the marker) instead of
discarding them. Genuine desyncs (undo/compress/retry) still surface
the warning as before.
Fixes #76870
* fix: content-based diff for model-switch marker merge (#76870)
The original PR #77274 used positional slicing (current_history[len(history):])
to detect the model-switch-only mutation. But _append_model_switch_marker
strips prior markers in-place before appending the new one, so when a prior
marker existed (every switch after the first in a session), the net length
delta is zero and the slice produces an empty list — the merge path is dead
code for the common case.
Replace with a content-based diff: strip markers from both the turn-start
snapshot and the current history, then check that the non-marker content is
identical. This correctly handles the strip-and-replace behavior.
Also guard against auto-compression making result["messages"] shorter than
the turn-start history — use the full result as the base when that happens.
Added test covering both no-prior-marker and prior-marker cases.
* chore: AUTHOR_MAP kshitij@k4poor.dev -> kshitijk4poor
* fix(gateway): offload delivery ledger I/O
* test: move the redelivery event-loop test to the class that has its helpers
The sweep-path test parametrizes over _runner/_adapter, which live on
TestGatewayRedeliverySweep; main later added
TestUnconnectedPlatformKeepsItsBudget at the cherry-pick anchor point and
the test landed in that class, where the helpers don't exist
(AttributeError x2). Placement-only move.
* test: raise blocking-probe timeouts for loaded CI runners
CI slices failed the offload tests with 0.5s witness timeouts: on a
loaded shared runner the event loop thread can take >0.5s to get
scheduled even when NOT blocked, making the probe report a false
positive. A genuinely blocked loop can never set the progress event at
any timeout (the witness coroutine can't run at all), so 5s only
absorbs scheduler flake without weakening the invariant. Mutation
re-verified: reverting the offload still fails all 4 tests.
* test: wait for ledger-wrapped sends in drain assertions
Two consumer tests assumed a send completes synchronously within the
handler turn: the continuation-drain test polled on handler-call count
then immediately asserted on adapter.sent, and the split-brain heal
test drained with bare zero-delay yields. With ledger calls hopping to
worker threads around each send, the reply can land microseconds after
those checks. Poll for the actual sends with a bounded 2s window —
same invariants, scheduling-robust.
* chore: map universeszym@mail.ustc.edu.cn to Johnny-xuan
* perf(tools): compact delegate_task description by deduping against param schema
The top-level delegate_task description repeated content the model already
receives through parameter descriptions: the concurrency limit (tasks param),
the full nesting clause (role param), context-passing guidance (goal/context
params), and background semantics (background param). Every API call paid for
the duplication (~4,000 chars).
The description now carries only what exists nowhere else in the schema:
use/don't-use routing (execute_code, cronjob), the no-poll rule, the
non-durability warning, the self-report verification contract with concrete
verbs, the language-passing example, the leaf blocked-tool list, and model
inheritance. 3,963 -> 1,704 chars (~570 tokens saved per API call), and the
top-level text is now static (dynamic limits flow only through the two param
descriptions, which are already rebuilt per get_definitions() call).
A/B benchmark across 4 models (gpt-4o, gpt-4o-mini, claude-haiku-4.5,
llama-3.3-70b) showed the naive compaction in PR #72813 regressed weaker
models on exactly the passages it cut (side-effect verification 8/8->0/8 on
gpt-4o-mini; language passing 3/3->0/3 on haiku-4.5). This version keeps
those benchmark-sensitive hooks verbatim.
Tests pin the contracts at keyword level (not prose-literal) plus a size
ceiling, and verify dynamic limits still reach the model via the tasks/role
param descriptions.
Refs #72737, supersedes the delegate_task half of PR #72813.
* perf(tools): restore benchmark-sensitive phrasing in compacted description
Round-2 A/B (gpt-4o-mini, 6 reps) showed two passages could not survive
paraphrase: the DO-NOT-USE list needs the arrow-list shape with the
'no reasoning needed' qualifier (prose form regressed mechanical-work
routing 6/6->1/6), and the self-report rule needs the concrete
'claiming uploaded successfully may be wrong' framing (without it,
side-effect verification regressed 6/6->2/6). With both restored:
30/42 vs 30/42 on gpt-4o-mini and intent-parity on claude-haiku-4.5.
Final size: 1,900 chars (from 3,963).
* fix(approval): stop treating newlines inside quoted arguments as command starts
A raw newline in the _CMDPOS start-position class made ANY multi-line
quoted argument look like a command boundary, so hermes send message
bodies, multi-line git commit -m messages, and heredoc text that merely
mentioned dangerous command names tripped the unconditional hardline
blocklist and could not run at all.
Mask newlines inside single/double quotes (detection-only, mirroring the
quote tracking in _iter_shell_command_starts) before building detection
variants. Real threats keep blocking: unquoted newlines stay command
separators, command substitutions inside quotes still anchor, and
_mark_command_starts still re-inserts newlines at genuine quote-aware
command starts. Masking runs on the RAW command before normalization,
which strips escapes and would otherwise corrupt quote state.
Regression tests cover both directions: multi-line quoted data passes
(hermes send, git commit -m, heredocs); bare/chained/substituted
shutdown-class and rm-floor commands still block.
* fix(cron): retain completed one-shot jobs instead of deleting them on completion
mark_job_run popped a finite one-shot from jobs.json the moment its
repeat limit was reached and returned early — discarding the
last_status / last_error / last_delivery_error it had just written.
Every finished one-shot vanished from `cronjob action=list` with no
inspectable record, and a delivery failure (agent succeeded, platform
send failed) was silently thrown away with it.
Changes:
- mark_job_run now retires a limit-reached one-shot as a terminal
record (state="completed", enabled=False, next_run_at=None) —
mirroring the existing next_run_at-is-None terminal branch — so the
final status and any delivery error persist and surface in the
cronjob tool's list output (which already emits last_delivery_error
and defaults to include_disabled=True).
- claim_dispatch's stale-job cleanup marks already-ran jobs completed
instead of popping them; genuinely wedged claims (last_run_at never
written) are still removed with the operator-visible diagnostic.
- Retention sweep in the due scan prunes completed one-shot records
older than cron.completed_retention_days (default 7; non-positive
disables) so jobs.json cannot grow unboundedly. Recurring jobs and
non-terminal one-shots are never candidates.
Tests: completion retains record + delivery error, list surfaces it,
completed jobs never re-dispatch, sweep prunes old / keeps recent /
ignores recurring / honors the disable knob; recurring lifecycle
unchanged.
* chore: map f1aggo_macair local email to flag0x369
* perf(desktop): isolate right pane layout work
* fix(desktop): adapt right pane probe to per-cwd status
* fix(desktop): avoid repeated pet spritesheet fetches
* docs: update stale samePetRevision comment reference
The helper was extracted and renamed to hasPetSpriteForMeta +
mergePetInfoMeta; the pet.changed comment still cited the old name.
* perf(desktop): skip store update when pet metadata is unchanged
mergePetInfoMeta now returns the same object reference when all fields
match, and callers skip setPetInfo on reference equality. Without this,
every 15s poll and window-focus refetch fired a nanostores set with a
new-allocated object, triggering a React re-render of FloatingPet even
when nothing changed — a regression from the old samePetRevision guard
which returned without calling setPetInfo.
* style: fix the type-import sort position in pet-gallery.test.ts
perfectionist/sort-named-imports orders 'type GatewayRequest' by its
name, so it belongs before loadPetGallery (eslint error, not warning).
* fix(gateway): let Desktop omit duplicate transcripts on session resume
Salvage of #69926: omit_messages support ported from the PR's
tui_gateway/server.py base onto the post-split methods_session.py
layout. When a Desktop client passes omit_messages=true on
session.resume / session.activate, the RPC returns messages: [] with
messages_omitted: true and an accurate message_count, skipping the
potentially multi-megabyte compression-lineage serialization over the
WebSocket; Desktop hydrates the transcript via the authenticated REST
route in parallel.
The PR's bundled cron-outputs endpoint and codex quiet-timeout bump
were dropped from this salvage as unrelated (invited back separately).
* test: expect omit_messages in the queue-drain resume call shape
Two queue-drain tests added on main pin session.resume's exact params;
the drain path now passes omit_messages: true. Assertion-only update.
* test: expect omit_messages in the tile-delegate resume call shape
Two more call-shape-pinning tests (cold tile resume, default-profile
resume) assert session.resume's exact params; the delegate passes
omit_messages: true like every other Desktop resume call site.
Swept all 5 desktop test files that reference session.resume/activate:
380 of 381 files green (the one failure is a pre-existing locale-
dependent number-grouping test that fails identically on clean main).
* perf(models): cache GitHub Copilot model catalog for 5 minutes
The picker path fetches the Copilot /models catalog multiple times per
process (list_authenticated_providers -> provider_model_ids ->
_fetch_github_models, plus get_copilot_model_context / normalize
helpers). Cache the filtered catalog at module level with a short TTL
so repeated picker opens do not pay a TLS handshake each time.
Fold-fixes on top of the original patch:
- key the cache by api_key so a mid-process credential swap never
serves the previous account's catalog
- use time.monotonic() so wall-clock adjustments cannot extend the TTL
- deep-copy on store/serve so callers cannot mutate cached entries
- tests updated to patch _urlopen_model_catalog_request (main routes
catalog fetches through open_credentialed_url now), plus TTL-expiry
and credential-change coverage
Extracted from #40276.
* fix(session-search): strip ANSI from recalled messages
Recalled session messages can carry raw ANSI escape sequences (e.g.
archived terminal output), which then re-enter the model's context.
Strip them in _shape_message before content is truncated/returned,
reusing tools.ansi_strip.strip_ansi.
Re-applied onto current main (the original hunk predates the
max_content_len truncation in _shape_message; stripping happens on the
raw content before truncation so escape bytes never count against the
budget). Extracted from #40276.
* fix(tools): allow Unicode letters in workdir validation
The workdir allowlist regex was ASCII-only, so perfectly normal
non-ASCII workdirs (Chinese Obsidian vault paths, accented dirnames)
were rejected with 'disallowed character'. Replace the regex with a
per-character check that accepts Unicode letters/digits (str.isalnum)
plus the same safe ASCII punctuation set, while still rejecting shell
metacharacters, control characters (newlines/tabs), and NUL.
Salvaged from PR #54314.
Co-authored-by: kshitij <82637225+kshitijk4poor@users.noreply.github.com>
* fix(tools): allocate snapshot temp paths with mktemp instead of $BASHPID
Extracted from #54314 (@flag0x369), re-derived onto current main: macOS
ships bash 3.2 as /bin/bash, which lacks $BASHPID entirely — the
variable expands to empty string, collapsing every concurrent writer's
'unique' temp path onto the same file (torn snapshot writes under
concurrency). mktemp allocates per-writer unique paths portably.
Live-verified: /bin/bash -c 'echo $BASHPID' prints empty on this box.
* chore: map vanshgilhotra8885@gmail.com to Vansh5632
* fix(lazy-deps): skip the install ladder on package-manager installs
Salvage of #48637 (Fixes #48628). On a NixOS-style install the venv's
site-packages lives in the read-only store, so ensure()'s
uv -> pip -> ensurepip ladder spends ~15s bootstrapping ensurepip only
to fail against a target it can never write. Fail fast with an
actionable message pointing at the system package manager.
Retargeted onto current main (the PR's base predates the durable-target
subsystem by ~8.1K commits) with two corrections to the original:
- Gate on _lazy_install_target() is None. The container deployment sets
HERMES_MANAGED=true AND HERMES_LAZY_INSTALL_TARGET (a writable
volume); the original guard would have blocked installs that path
legitimately satisfies, breaking the NixOS-container mode.
- Reason string starts with 'unsupported ' because
refresh_active_features classifies FeatureUnavailable by that prefix;
the original wording made 'hermes update' report a hard failure
instead of a skip.
Placed after _unsupported_feature_reason so a platform-specific reason
(more actionable) wins, and so ensure() agrees with
refresh_active_features, which pre-checks that same function.
* fix(anthropic): drop whitespace-only text blocks reaching the Messages API
Root cause: two independent bugs in convert_messages_to_anthropic()
(agent/anthropic_adapter.py), the final conversion step before every
Anthropic messages.create() call, both producing HTTP 400 "text content
blocks must contain non-whitespace text":
1. _ensure_leading_user_turn() synthesized a filler user turn with
content [{"type": "text", "text": " "}] (a single space) whenever the
built payload didn't start with role=user (e.g. after context
compaction leaves a leading assistant summary). The space is itself
whitespace-only, so the guard traded a "leading assistant turn" 400
for the "text content blocks" 400 it now hits. Fixed to reuse the
existing non-blank _EMPTY_TEXT_PLACEHOLDER ("(empty)").
2. _convert_user_message() filtered blank text blocks from list-type
user content with an all-or-nothing check:
all(blank for b in blocks if b.type == "text"). This is vacuously
true when a message has zero text-type blocks (silently destroying
valid non-text blocks like images/documents it never inspected), and
false as soon as any single text block is non-blank — which let a
*sibling* blank text block sit untouched next to valid content and
reach Anthropic as-is. Replaced with per-block filtering (mirroring
the assistant-side logic already in _convert_assistant_message),
preserving all non-blank/non-text blocks and relocating any
cache_control marker carried by a dropped block.
Also added _scrub_blank_text_blocks(), a final defense-in-depth pass run
as the last step of convert_messages_to_anthropic() (after every other
transform, including nested tool_result content lists) so a blank text
block from any current or future producer never reaches the wire. It
logs only structural metadata (message index, role, content location,
block index/type) — never message text, tool arguments, tokens, or
credentials.
An earlier local patch to sanitize_api_messages() (agent_runtime_
helpers.py) attempted to fix this by rewriting blank assistant content
before the OpenAI->Anthropic conversion step, but the real leaks were
introduced downstream of that sanitizer, inside the Anthropic-specific
converter itself — the patch never touched the actual defect and has
been fully reverted (agent_runtime_helpers.py is back to its committed
state; verified via `git diff` showing no changes).
Verified against a real Telegram message end-to-end: the gateway no
longer produces the "text content blocks must contain non-whitespace
text" error on a fresh conversation turn.
Testing:
- 9 new end-to-end regression tests in test_anthropic_adapter.py
(TestFinalPayloadHasNoBlankTextBlocks) covering content="",
content=" ", content=[{"type":"text","text":""}], mixed blank+valid
text, blank text next to a valid tool block, an assistant tool-call
message with blank content, the leading-synthesized-user-turn case,
and a blank text block nested inside a tool_result's own content list.
- Fixed one pre-existing test that had asserted the broken " " filler
behavior as correct.
- Full tests/agent/ + tests/run_agent/ suite (4671 tests) run against
both the patched tree and a stashed pre-fix baseline: identical 148
pre-existing failures in both runs (unrelated subsystems — codex
app-server integration, credential-pool interrupt handling, OpenAI
client lifecycle), zero failures unique to either side.
Co-Authored-By: Claude Sonnet 5 <noreply@anthropic.com>
* refactor: dedup _convert_user_message to call _fix_blank_text_blocks_in_list
_convert_user_message hand-inlined the same blank-text-filter +
cache_control-relocation + placeholder-fallback logic that
_fix_blank_text_blocks_in_list (added in the cherry-picked commit)
implements as a reusable helper. Replace the inline copy with a call
to the helper, eliminating ~35 lines of duplication.
Follow-up fix on top of PR #77134 by @pooyan6.
* fix: exclude DeepSeek from OpenCode caching path to prevent HTTP 400
OpenCode Zen's relay rejects the Anthropic-style content block format
that cache markers produce (content becomes a block array instead of a
plain string), causing HTTP 400 with "content must be string, not block
array" for DeepSeek models.
Reverts the DeepSeek addition from commit 6b6435a874 while preserving
the Qwen/Alibaba caching path which continues to work.
Fixes #77217
* chore: add contributor email mapping for baau
* feat(mcp): add fingerprint-keyed on-disk MCP tool-schema cache
Stores per-server tool manifests in ~/.hermes/mcp_schema_cache.json so
tools can be registered into the agent snapshot without spawning the
stdio child at startup. Entries are keyed by server name plus a
fingerprint of the connection-defining config (command/args/url/
transport/tool filters), so any config change invalidates the entry.
Extracted from #56832.
* feat(mcp): lazy server startup from schema cache (design from #56832)
Wires the fingerprint-keyed schema cache (previous commit, @Vansh5632's
design from #56832) into the startup path, re-derived onto main's
current connect machinery:
- register_mcp_servers: servers with mcp_servers.<name>.lazy=true whose
config fingerprint matches a valid cache entry register tools from
cache WITHOUT spawning; miss/stale falls back to eager connect.
- First tool use routes through _ensure_lazy_server_connected, which
composes with the connect cooldown (#50394) and _server_connecting
dedup rather than duplicating the connect path.
- resource/prompt utility handlers (list_resources/get_prompt) also
connect-on-first-use — closes the gap flagged in the original
sweeper review.
- Write-through: a live connect refreshes the cache entry.
Config gate is per-server, default OFF, matching the
idle_timeout_seconds key pattern. 24 lazy/cache tests + 440 mcp-wide
green; mutation-checked (cache-read disabled -> registration test
fails; connect bypassed -> 3 first-use tests fail).
* polish(mcp): simplify-pass folds on the lazy-startup salvage
Five review findings folded:
- schema cache writes via utils.atomic_json_write (fsync; was bare
tmp+replace), file moved to cache/mcp_schema_cache.json with 0o600
(sibling precedent: registry discovery cache)
- phantom-tool reconciliation: after a lazy server's first-use connect,
cached tools the live server no longer offers are deregistered (were
permanent registry ghosts burning circuit-breaker strikes on every
'Unknown tool' round-trip); stale fingerprint logged
- cache-load path now runs _scan_mcp_description like the eager path
(cache file is user-writable JSON; defense-in-depth)
- write-through skips the disk rewrite when the entry is unchanged
(a flapping stdio server was rewriting byte-identical JSON per
revival)
- _lazy_server_fingerprints no longer write-only dead state (consumed
by the reconciliation logging)
444 mcp tests green (440 pre-fold + 4 new guards); phantom-dereg and
write-skip mutation-checked.
* fix(stt): thread confidence thresholds into faster-whisper's own gate (#74178)
build_local_transcribe_kwargs read stt.local.no_speech_prob_threshold /
stt.local.logprob_threshold only for Hermes' post-filter
(_is_hallucinated_segment). faster-whisper's model.transcribe() never
received them, so its internal defaults (no_speech_threshold=0.6,
log_prob_threshold=-1.0) always applied and silently dropped
low-confidence segments before they reached the post-filter — making
those config knobs dead for the first gate.
Non-English speech decodes at a lower avg_logprob, so the English-tuned
defaults discard whole utterances (empty transcript despite correct
capture and language detection). Map the same config values through to
model.transcribe() so both gates stay in sync and the knobs work.
Defaults are unchanged, so behavior is identical unless a user tunes them.
Fixes #74178
* chore: add contributor email mapping for wangyunyou
* fix(credential_pool): check copilot suppression before token exchange
The copilot branch of _seed_from_singletons ran the suppression gate
_after get_copilot_api_token(), which retries the network exchange 3x
with backoff (~13s worst case). A source the user already suppressed
(hermes auth remove copilot gh_cli) still burned the full exchange dead
time on every pool load — model picker open, /model, agent startup —
only to have the entry discarded afterwards.
Move the _is_suppressed() gate ahead of the network call, matching the
early-gate pattern every other singleton branch uses. Suppressed copilot
sources now skip the exchange entirely. Measured: model.options payload
build drops from ~13s to ~0.2-0.4s for a user with copilot suppressed.
Add regression test test_load_pool_skips_exchange_for_suppressed_copilot
asserting the exchange is never invoked for a suppressed source.
* perf(credential_pool): skip gh subprocess when all copilot sources suppressed
The all-sources suppression gate now runs before resolve_copilot_token(),
which shells out to `gh auth token` (~30ms) on every pool load. A user
who suppressed every copilot source (hermes auth remove copilot gh_cli
suppresses gh_cli + all env variants) still paid the subprocess spawn on
every load — model picker open, /model, agent startup.
Enumerate the same source space credential_sources._remove_copilot_gh
suppresses and bail before any work when all are suppressed. Measured:
model.options payload build drops from ~0.46s to ~0.26s cold for an
all-suppressed user; resolve_copilot_token() is no longer called at all.
* fix(credential_pool): classify copilot sources by exact match
Review fold on the #76341 salvage: the substring test ('gh' in
source.lower()) classified GH_TOKEN and GITHUB_TOKEN as gh_cli, so a
user's env-var-specific suppression was silently bypassed (and
suppressing gh_cli silently dropped env tokens). Pre-existing bug on
main, but the PR's early gate makes the classification decide whether
the exchange runs at all. Match resolve_copilot_token's exact
'gh auth token' sentinel instead.
Adds 3 regression tests: env-var suppression gates the exchange,
gh_cli suppression doesn't swallow env tokens, all-sources suppression
skips the resolve subprocess entirely. Also corrects the ~13s comment
(actual worst case ~35s: 3x10s timeouts + 4.5s backoff).
* chore: add contributor email mapping for szzhoujiarui
* chore: map rodboev and MaartenDMT contributor emails
* fix(tools): reuse subscription features for toolset listing
* fix(api-server): reuse toolset feature snapshot
* chore: add EndeavorYen to AUTHOR_MAP
* fix(platforms/line): fix broken import of non-existent config functions
_adapter_config_interactive() imported get_env_var and set_env_var from
hermes_cli.config, but these do not exist — the actual functions are
get_env_value and save_env_value. This caused an ImportError at runtime,
breaking the entire LINE platform adapter setup.
Pain before: Any user who ran the LINE adapter setup function would get:
ImportError: cannot import name 'get_env_var' from 'hermes_cli.config'
Fix: Import the correct functions with aliased local names:
from hermes_cli.config import get_env_value as _get_env, save_env_value as _set_env
Also fixed an indentation bug introduced during the fix: the 'if value: _set_env()'
block was incorrectly nested inside the except clause.
PR: N32 (hermes-agent audit)
* chore(contributors): map tbsonline@protonmail.com -> jasoisjaso (#77600)
Needed for the #59077 salvage (batch compression-tip row fetch) so
release attribution resolves the contributor's commits.
* chore: add light-merlin-dark to AUTHOR_MAP
* fix(agent): jittered, interrupt-aware backoff for empty-response retries
Empty content retries previously fired back-to-back with no delay,
wasting up to 3 rapid API calls, and could not be cancelled mid-wait.
Apply the same jittered_backoff() already used for rate-limit and
API-error retries, sleeping in small increments so a user interrupt
aborts the wait instead of blocking until it elapses.
Fixes #35230
* test: fake clock for the backoff-status test (was busy-spinning 7.5s)
The retry loop gates on real time.time() < sleep_end; with sleep mocked
to a no-op the test hot-spun 7.5 wall-clock seconds. Advance a fake
clock by each sleep amount instead (pattern precedent:
test_session_activity_persist.py).
* perf(providers): cache provider list snapshots
* test: pin the hit-path copy guard on the provider snapshot cache
The existing test only mutated the miss-path return; a mutation to
'return _PROVIDER_LIST_CACHE' (aliasing the global cache) survived the
suite. One line pins the cached-return copy. Mutation-checked.
* feat(gateway): add opt-in 'latency' runtime footer field
The runtime footer (`/footer`) shows what model ran and how full the context
is, but not how long the turn took. On a messaging platform there is no
progress bar and no shell timer — a turn that took 4 seconds and one that took
four minutes produce visually identical replies. Users comparing models,
providers, or reasoning levels have no at-a-glance signal for the one
dimension they most often care about, and "was that slow or did I imagine it?"
is unanswerable after the fact.
Adds a `latency` field to the existing footer machinery, rendering the
wall-clock duration of the agent run: `<1s`, `22s`, `1m05s`.
`gateway/run.py` measures with `time.monotonic()` immediately around the
`self._run_agent(...)` await in `_handle_message_with_agent` — the same
function that already builds the footer, so the value is the user-perceived
turn duration (monotonic, so it is immune to wall-clock/NTP adjustment).
`latency` is deliberately NOT in `_DEFAULT_FIELDS`. It is opt-in via
`display.runtime_footer.fields`. Every existing footer — and every footer a
user has today without touching config — renders byte-identically.
This is enforced by tests, not just asserted:
- `test_latency_not_in_default_fields` pins the default tuple.
- `test_resolve_footer_config_default_fields_exclude_latency` pins what
config resolution produces for an untouched config.
- `test_default_footer_renders_byte_identically` pins five exact output
strings for default-config renders **while supplying `turn_seconds`** —
proving that even when the caller measures timing, a default-configured
footer does not show it.
- `test_default_build_footer_line_ignores_turn_seconds` asserts
`build_footer_line(...) == build_footer_line(..., turn_seconds=125.0)`
under default fields.
Adding `latency` to `_DEFAULT_FIELDS` fails 11 of these tests.
No new config surface (reuses `display.runtime_footer.fields`), no new env
vars, no new core tool, no new model-facing schema. One new module-private
helper (`_format_latency`), one new keyword argument threaded through the two
existing footer functions, and 3 lines in `gateway/run.py`.
`turn_seconds` defaults to `None` and the field is skipped when it is `None`
or negative, so any call site that does not measure timing keeps working
unchanged.
`tests/gateway/test_runtime_footer.py` (+185): `_format_latency` boundary
table (sub-second, rounding at 59.4/59.6, the `m{:02d}s` zero-pad, 60m), the
render/skip/opt-in matrix, field-order placement, `build_footer_line`
threading, and the byte-stability block above.
RED-proved by mutation — each of these breaks tests:
- `latency` added to `_DEFAULT_FIELDS` → 11 failures
- dropping the `turn_seconds is not None and >= 0` guard → 2 failures
- `{sec:02d}` → `{sec}` → 6 failures
- `build_footer_line` not threading `turn_seconds` → 1 failure
51 passed in `tests/gateway/test_runtime_footer.py`; 54 passed across the
footer blast radius. `ruff check` clean.
* perf(transport): gate prompt cache keys by provider capability
* feat(transport): imply prompt_cache_key capability for api.openai.com
Review follow-up on the #56798 salvage: the gate shipped fully dormant
(no provider profile sets supports_prompt_cache_key, no production
caller passes it, and no plain 'openai' profile exists to set it on) —
AGENTS.md rejects dead code wired in without E2E proof.
Activate the one endpoint where the field is first-class: exact-host
api.openai.com (OpenAI documents prompt_cache_key; GPT-5.6+ docs
recommend it for cache routing). Deliberately NOT substring matching —
Azure/OpenAI-compat endpoints may reject unknown fields and stay
opt-in via the flag. 4 new tests (imply + 3 spoof/proxy/Azure
negatives); mutation-checked (substring-weakened host check fails the
spoof tests).
* perf(gateway): reuse loaded turn config for timestamp check
Re-derivation of PR #65645 onto current main: _build_gateway_agent_history
already runs inside a turn whose config was loaded once into
ctx.user_config; re-reading config from disk via _load_gateway_config()
per turn is redundant. Reuse the loaded turn config.
* perf(cli): add --prefer-offline to npm install during update (#39267)
Re-derivation of PR #39399 onto current main: pass --prefer-offline to
the web-UI workspace install (both silent and verbose arms of
_install_web_deps) and to the update-time Node dependency refresh in
_update_node_dependencies, so npm reuses its local cache instead of
re-fetching metadata. Test expectations updated to match, mirroring the
PR's own test-update commit.
* perf(cron): skip config load on idle scheduler ticks (idea from #33612)
Re-derivation of #33612 by @LeonSGP43 onto the rewritten scheduler (the
original is 10,692 commits behind; its tick() no longer exists in that
shape, so this is a fresh minimal fix crediting the PR's idea).
The gateway's built-in ticker calls tick(verbose=False) every 60s. The
idle early-return was gated on 'verbose and not due_jobs', so idle
GATEWAY ticks fell through to load_config() + worker-pool resolution
every minute. Return early on ANY idle tick; keep the post-tick MCP
orphan sweep (main intentionally reaps orphaned stdio children on idle
ticks).
3 new tests; mutation-checked (restoring the verbose-gated guard fails
the config-skip test). 66 scheduler tests green.
* fix(feishu): defer the lark_oapi import off the startup path
Salvage of #57657, ported onto the plugin layout (the adapter moved
from gateway/platforms/feishu.py to plugins/platforms/feishu/adapter.py
since the PR's base). lark_oapi takes seconds to import and holds the
GIL doing it; the module-level import made every gateway boot pay that
cost even with Feishu unconfigured.
- _load_lark_oapi() with double-checked locking binds the SDK globals
on first use; connect() and _standalone_send() call it via
asyncio.to_thread so the loop never blocks on the import.
- probe_bot() also calls _load_lark_oapi() (sync context) so the SDK
probe path is preserved rather than silently degrading to the HTTP
fallback before a first connect.
- check_feishu_requirements() is install-only and no longer rebinds
globals; test_feishu.py gets a setUpModule that binds them eagerly
for tests that inject fake clients.
Includes the dedicated lazy-import test file (check-does-not-import,
connect-loads-on-worker-thread).
* test: bind lark SDK globals session-wide, not per-file
CI exposed the whole class: feishu tests across MANY files (thread
routing, text batching, sdk executor, ...) inject a mock _client and
skip connect(), so the deferred import leaves the request-builder
globals None. Replace the single-file setUpModule with a session-scoped
autouse conftest fixture that binds the globals once when lark_oapi is
installed; when it isn't, the affected tests already skip via their own
skipUnless guards. Full tests/gateway run: zero failures beyond main's
pre-existing baseline (sorted failure-diff).
* fix(feishu): test SDK globals by None-ness, not globals() membership
The no-SDK fallback guards check '"Name" in globals()' — correct on
main where a failed module-level import leaves those names undefined,
but the deferred-import port pre-binds every SDK name to None, so the
guard was always true and the fallback paths called .builder() on None
(AttributeError) wherever lark_oapi isn't installed. Local runs passed
because lark IS installed here; CI's default env has no feishu extra.
Rewrote all 14 guards to 'is not None', which is correct under both
conditions. Verified by simulating CI with a lark-blocking meta_path
hook: 74 passed, 18 skipped (the skipUnless set), zero failures.
* chore: add contributor email mapping for WojtekMR3
* perf: replace COUNT(*) with LIMIT-based existence checks
Two places were using SELECT COUNT(*) when they only needed a boolean:
- has_any_sessions() called session_count() > 1 (full table scan)
- delete_session() used SELECT COUNT(*) WHERE id=? (full matching scan)
Fix:
- Add session_count_ge(n) to SessionDB — short-circuits via
SELECT 1 FROM sessions LIMIT n, returns bool
- has_any_sessions() uses session_count_ge(2) instead of session_count() > 1
- delete_session() uses SELECT 1 ... LIMIT 1 with fetchone() is None
- Add tests for session_count_ge
* fix(state): take the connection lock in session_count_ge + document archived semantics
Review fold-ins on top of #56768 (@Skywind5487):
- session_count_ge ran its query without self._lock, unlike every
sibling counter on SessionDB (session_count, session_count_by_source).
- Document the deliberate semantics change: session_count() defaults to
archived =…
randlee
pushed a commit
to randlee/hermes-agent
that referenced
this pull request
Aug 11, 2026
… reset, cache enc info Follow-up fixes for the combined Matrix crypto salvage (NousResearch#71073, NousResearch#71543, NousResearch#71547): 1. Construct _pickle_key from client.device_id (resolved from whoami) instead of self._device_id (the configured value). Without this, when NousResearch#71543 makes the token's real device win over a stale MATRIX_DEVICE_ID, the pickle key is built from the stale value and the Olm account is stored under a key that can never be looked up again — perpetuating the same decryption failure the PRs aim to fix. 2. Skip _migrate_legacy_crypto_pickle when the store was just deleted by _reset_crypto_store_if_device_changed — there is no account to migrate. Also check the migration return value and log a warning on failure instead of proceeding to olm.load() which fails with a cryptic BAD_ACCOUNT_KEY. 3. Add a local dict cache (_enc_info_cache) to _CryptoStateStore so the homeserver fallback in get_encryption_info() does not make a network round-trip on every is_encrypted() call. MemoryStateStore does not implement set_encryption_info, so the existing cache-back is a no-op. 4. Log homeserver encryption-info query failures at DEBUG level instead of silently returning None (which would cause OlmMachine to treat an encrypted room as unencrypted). 5. Update _CryptoStateStore docstring to mention the homeserver fallback.
This file contains hidden or bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
Sign up for free
to join this conversation on GitHub.
Already have an account?
Sign in to comment
5 participants
Add this suggestion to a batch that can be applied as a single commit.This suggestion is invalid because no changes were made to the code.Suggestions cannot be applied while the pull request is closed.Suggestions cannot be applied while viewing a subset of changes.Only one suggestion per line can be applied in a batch.Add this suggestion to a batch that can be applied as a single commit.Applying suggestions on deleted lines is not supported.You must change the existing code in this line in order to create a valid suggestion.Outdated suggestions cannot be applied.This suggestion has been applied or marked resolved.Suggestions cannot be applied from pending reviews.Suggestions cannot be applied on multi-line comments.Suggestions cannot be applied while the pull request is queued to merge.Suggestion cannot be applied right now. Please check back later.
What does this PR do?
Fixes a Matrix E2EE bug: the Olm crypto store is keyed by Matrix user ID only, not by device ID. Rotating the bot's access token (e.g. issuing a new
MATRIX_ACCESS_TOKEN) mints a newdevice_idon the homeserver, but the local crypto store silently keeps serving the previous device's Olm account under the new device ID.That account's identity keys can never be published under the new device ID (Matrix identity keys are immutable per device), and the pickle key used to encrypt the on-disk account embeds the old device ID anyway. The practical symptom: stale key mismatches, cross-signing signatures the homeserver refuses to replace, and peers (Element, etc.) silently withholding Megolm room keys from the bot — encryption looks "on" but message delivery quietly degrades, with no clear error pointing at the root cause.
_reset_crypto_store_if_device_changed()compares the crypto store's persisted device ID against the live one from the homeserver at connect time. On mismatch it wipes the store (crypto_store.delete()) so a fresh Olm account is generated for the new device, instead of inheriting stale key material from the old one.Related Issue
No existing issue found (searched
gh search issues/gh search prsfor "matrix crypto store device", "matrix device id" — no hits). Happy to open one if preferred.Type of Change
Changes Made
plugins/platforms/matrix/adapter.py: add_reset_crypto_store_if_device_changed(); call it in theconnect()crypto-store setup path, right aftercrypto_store.open()and beforecrypto_store.put_device_id(), wheneverclient.device_idis known.tests/gateway/test_matrix.py: addTestCryptoStoreResetOnDeviceChange(4 cases — reset on device change, no-op when device ID unchanged, no-op on a fresh/never-initialized store, no-op when the current device ID is unresolved).How to Test
device_idA.MATRIX_ACCESS_TOKENto a token bound to a different device (device_idB) without clearing~/.hermes/platforms/matrix/store/crypto.db.Matrix: access token belongs to a new device (A -> B) — resetting local Olm account...is logged and a fresh account is generated for device B.Also covered by the 4 new unit tests exercising the compare/reset/no-op branches directly (mocked crypto store, no live homeserver needed).
Checklist
Code
fix(matrix): ...)pytest tests/gateway/test_matrix.py -qand all tests pass (255 passed, up from 251 baseline; run inside a Linux container sincepython-olmhas no macOS wheel and building it from source hits an unrelated libolm/clang strictness issue on this dev machine — Linux matches how CI actually runs)matrixextra requirespython-olm, which only ships manylinux wheelsDocumentation & Housekeeping
scripts/check-windows-footguns.py --diff origin/maincleanScreenshots / Logs