fix(matrix): E2EE and migration bugfixes

[alt-glitch]

Closes #10094, #10725, #6016, #3487, #3806

Summary

Fixes 7 E2EE and migration bugs in the Matrix adapter, adds startup detection for stale one-time key conflicts, and includes several quality-of-life improvements for Matrix streaming and mention handling.

All fixes are E2E tested against a local Synapse homeserver with encrypted rooms.

Other improvements

Buffered streaming for Matrix

Matrix clients (Element, etc.) render the streaming cursor (▉) as a visible tofu artifact. The stream consumer now uses a buffer_only mode for Matrix that suppresses intermediate edit-based updates — text is batched and sent as complete messages instead of rapid edits that hit rate limits.

Fix ~ expansion in media file paths

MEDIA tags emitted by the agent with paths like ~/media/file.png were not expanded. Now calls os.path.expanduser() before attempting to read the file.

Fix mention stripping to not mangle file paths

The old _strip_mention removed both the full MXID (@hermes:server) and the bare localpart (hermes) from message bodies. Stripping the localpart mangled file paths like /home/hermes/media/file.png → /home//media/file.png. Now only strips the full MXID.

E2EE Bugs Fixed

1. Auto-join broken — MembershipEventDispatcher never registered

Closes: #10094, #10725 | Refs: #10135, #10732

mautrix delivers room invites as raw ROOM_MEMBER state events. The MembershipEventDispatcher that converts them into InternalEventType.INVITE was never registered on the client, so the _on_invite handler was dead code. The bot silently ignored all room invites.

Fix: Register MembershipEventDispatcher via client.add_dispatcher() in connect().

2. _CryptoStateStore reference orphaned after initial sync

Refs: #8174, #8215

connect() reassigned self._joined_rooms = set(rooms_join.keys()) after initial sync, orphaning the reference captured by _CryptoStateStore at init time. find_shared_rooms() returned [] forever, breaking Megolm session rotation on membership changes.

Fix: Mutate in place with clear() + update() instead of reassignment.

3. Dual ROOM_ENCRYPTED handler causes dedup race

Refs: #8174, #8215

Both mautrix's auto-registered DecryptionDispatcher and hermes's _on_encrypted_event fired for every ROOM_ENCRYPTED event. _on_encrypted_event won the race (zero awaits), marked event IDs in the dedup set, and the successfully-decrypted event from DecryptionDispatcher got dropped by _on_room_message's dedup check. The retry loop masked this by re-decrypting every message ~4 seconds later with spurious "could not decrypt" warnings.

Fix: Remove _on_encrypted_event entirely. DecryptionDispatcher handles decryption; the retry loop and dedup-discard hack are no longer needed.

4. No re-verification after share_keys() upload

Refs: #8174, #8215

After calling share_keys(), the method returned True without verifying the server actually accepted the new keys. Matrix homeservers treat ed25519 identity keys as immutable per device — share_keys() returns 200 but silently ignores new keys if the device already exists with different identity keys.

Fix: Re-query the server after share_keys() and fail closed if keys don't match, with an actionable error message.

5. Outbound media sent unencrypted in E2EE rooms

Refs: #9822

_upload_and_send() uploaded raw bytes and used the url key for all rooms. In E2EE rooms, media must be encrypted client-side with encrypt_attachment(), the ciphertext uploaded, and the file key (with key/iv/hashes) used instead of url.

Fix: Detect encrypted rooms via state_store.is_encrypted() and branch to the encrypted upload path using mautrix.crypto.attachments.encrypt_attachment().

6. Typing indicator lingers for 30 seconds after response

Closes: #6016 | Refs: #6020

send_typing() called set_typing(timeout=30000) but the base class stop_typing() was a no-op. The typing indicator lingered for up to 30 seconds after each response.

Fix: Override stop_typing() to call set_typing(timeout=0).

7. Audio/video/document files not cached locally

Closes: #3487, #3806

should_cache_locally only covered PHOTO, VOICE, and encrypted media. Unencrypted audio/video/documents in plaintext rooms were passed as MXC URLs requiring authentication the agent doesn't have, resulting in 401 errors.

Fix: Extend the gate to include AUDIO, VIDEO, and DOCUMENT message types.

8. Stale one-time key conflict after crypto state recovery

Refs: #8174

When crypto state is wiped but the same device ID is reused, the homeserver still holds one-time keys signed with the previous identity key. Identity key re-upload succeeds but OTK uploads fail with a signature mismatch. Peers cannot establish new Olm sessions, so all new messages are silently undecryptable.

Fix: Proactively flush OTKs via share_keys() during connect() and catch the "already exists" error, refusing E2EE with an actionable log message. Also documents the crypto recovery procedure in the Matrix setup guide.

Test plan

Unit tests

• 114 tests pass (was 111 baseline), covering all new code paths
• New test classes: TestJoinedRoomsReference, TestDeviceKeyReVerification, TestMatrixUploadAndSend, TestMatrixEncryptedEventHandler::test_connect_fails_on_stale_otk_conflict

E2E testing methodology

All fixes were verified end-to-end against a live Matrix environment:

Infrastructure:

• Homeserver: Synapse (SQLite) running as a NixOS service on a local machine (zephyr), accessible via Tailscale at zephyr.giraffe-octatonic.ts.net:8008
• Bot account: @hermes:zephyr.giraffe-octatonic.ts.net — runs as a NixOS container service with E2EE enabled via mautrix OlmMachine
• Test client: matrix-commander (matrix-nio based CLI) authenticated as @sid, used to send encrypted messages, files, and verify bot responses programmatically
• Room setup: Encrypted DM rooms created via the Synapse client API with m.megolm.v1.aes-sha2 encryption preset

Test flow for each fix:

1. Write failing test → implement fix → verify tests pass → commit
2. Rebuild NixOS container (nixos-rebuild switch && systemctl restart hermes-agent)
3. Send test messages/files via matrix-commander and verify bot behavior
4. Check journalctl -u hermes-agent for errors/warnings

E2E tests performed:

• Encrypted DM — bot decrypts and responds (👀 → ✅ → text), zero "could not decrypt" errors
• Auto-join — bot joins new room on invite within one sync cycle (verified via /members API)
• Post-sync room — bot decrypts messages in rooms joined after initial sync (Bug C verification)
• Multi-media — JSON, TXT, WAV files sent to bot, processed with zero 401 errors
• Stale OTK detection — hermes refuses E2EE with actionable error when OTK signature conflicts detected
• Full crypto wipe test — deleted crypto.db, purged device from Synapse DB, re-registered with fresh credentials, verified all fixes work from a completely clean slate

[Schnurzel700]

Hi @alt-glitch,

Thanks for the massive effort on the Matrix rewrite. I've just tested the latest version on Debian.

The good news: The MembershipEventDispatcher fix is working. The bot now successfully auto-joins rooms upon invitation.

The bad news: The bot remains silent and does not respond to messages.

Even with MATRIX_ENCRYPTION=false and a completely wiped crypto.db/store, there are no "inbound message" logs in --debug mode when a message is sent in an unencrypted room.

How can I solve this problem?

[djmaze]

@Schnurzel700 For me it works now, even in encrypted rooms. So it seems something else might be broken with your bot account.

[Schnurzel700]

@alt-glitch I've ruled out all local state issues:

Setup: Completely fresh Debian VM & brand new Matrix.org account.

The Bug: Bot joins via invite, but zero console output in --debug when messaging the bot. No Received event, no inbound message.

Symptoms: Sync loop seems to hang or disconnect after ~30s of silence.

It appears the message handler isn't being bound to the sync loop in the new mautrix implementation. The bot is effectively "deaf" on clean installs.

[Schnurzel700]

I found the issue!
Since this is a breaking bug for all new Matrix users on mautrix, could one of you please commit this 1-line change to main? It would save everyone a lot of headaches! Thanks for the great work.
I explained more in #12614

[rorar]

I found the issue! Since this is a breaking bug for all new Matrix users on mautrix, could one of you please commit this 1-line change to main? It would save everyone a lot of headaches! Thanks for the great work. I explained more in #12614

Yes please... I went crazy with a lot of forth and back
(╯°□°）╯︵ ┻━┻