Matrix gateway implementation

MATRIX_GATEWAY_PR.md

@@ -0,0 +1,197 @@
+# feat(gateway): add Matrix protocol platform adapter
+
+## Summary
+
+Full Matrix platform adapter for the Hermes gateway. Agents can be deployed as always-on Matrix bots with optional E2EE, cross-signing verification, full media support (inbound and outbound), typing indicators, and a zero-friction setup wizard that handles the entire configuration automatically in one pass.
+
+---
+
+## Files changed
+
+**23 files, ~4,650 lines**
+
+| File | Change |
+|------|--------|
+| `gateway/platforms/matrix.py` | New adapter (~1,480 lines) |
+| `hermes_cli/gateway.py` | Setup wizard + `verify-matrix` command (+1,000 lines) |
+| `tests/gateway/test_matrix.py` | 93 tests, all passing |
+| `website/docs/user-guide/messaging/matrix.md` | Full setup guide (new, 270 lines) |
+| `hermes_cli/config.py` | 10 new Matrix env vars in `OPTIONAL_ENV_VARS` |
+| `gateway/config.py` | `Platform.MATRIX` enum + env var loading |
+| `gateway/run.py` | Adapter factory, auth maps, stale library name fix |
+| `hermes_cli/main.py` | `hermes gateway verify-matrix` subcommand (+109 lines) |
+| `toolsets.py` | `hermes-matrix` toolset + added to `hermes-gateway` |
+| `tools/send_message_tool.py` | `_send_matrix()` standalone send function |
+| `tools/cronjob_tools.py`, `cron/scheduler.py` | Matrix delivery option |
+| `gateway/channel_directory.py` | Session-based discovery |
+| `agent/prompt_builder.py` | `PLATFORM_HINTS["matrix"]` |
+| `agent/redact.py` | `_MATRIX_ID_RE` pattern |
+| `hermes_cli/status.py`, `hermes_cli/setup.py` | Matrix in status/main wizard |
+| `website/docs/user-guide/messaging/index.md` | Architecture diagram, toolset table |
+| `website/docs/user-guide/security.md` | `MATRIX_ALLOWED_USERS` example |
+| `tools/terminal_tool.py` | Fix stale `minisweagent` import (pre-existing bug caught during review) |
+
+---
+
+## ADDING_A_PLATFORM.md checklist — all 16 items
+
+| # | Item | Status |
+|---|------|--------|
+| 1 | Core adapter — all required + optional methods, correct base signatures, `check_matrix_requirements()` | ✅ |
+| 2 | Platform enum + env var loading (`gateway/config.py`) | ✅ |
+| 3 | Adapter factory (`gateway/run.py`) | ✅ |
+| 4 | Authorization maps — `platform_env_map` and `platform_allow_all_map` | ✅ |
+| 5 | Session source — `build_source()` used (no new fields needed) | ✅ |
+| 6 | System prompt hints (`agent/prompt_builder.py`) | ✅ |
+| 7 | Toolset — `hermes-matrix` + added to `hermes-gateway` composite | ✅ |
+| 8 | Cron delivery (`cron/scheduler.py`) | ✅ |
+| 9 | Send message tool — `_send_matrix()` + platform routing | ✅ |
+| 10 | Cronjob tool schema — `deliver` param | ✅ |
+| 11 | Channel directory — `"matrix"` in session-based discovery | ✅ |
+| 12 | Status display (`hermes_cli/status.py`) | ✅ |
+| 13 | Gateway setup wizard — full `_setup_matrix()` with auto-login, E2EE, trust verification | ✅ |
+| 14 | ID redaction — `_MATRIX_ID_RE` in `agent/redact.py` | ✅ |
+| 15 | Docs — `matrix.md` (new), `index.md`, `security.md`, `README.md` | ✅ |
+| 16 | Tests — 93 tests in `tests/gateway/test_matrix.py` | ✅ |
+
+---
+
+## Technical design
+
+### Library: mautrix-python (not matrix-nio)
+
+matrix-nio was evaluated and found unsuitable for production E2EE bots:
+- Cross-signing unimplemented since 2020 (issue #229, open, no activity)
+- No reliable session persistence — every restart loses all Olm/Megolm sessions, causing "no session found" decrypt errors on every message after a restart
+- Last maintained 2023
+
+mautrix-python is the library used by maubot and all production mautrix bridges. It has `PgCryptoStore` for proper persistence and active maintenance.
+
+### E2EE design (mirrors maubot)
+
+- **`PgCryptoStore` on SQLite** (`~/.hermes/matrix/crypto.db`) — all Olm/Megolm sessions, device keys, and cross-signing keys survive gateway restarts. This is the fundamental fix vs matrix-nio.
+- **`resolve_trust()` guard** — cross-signing bootstrap runs once on first start only. All subsequent restarts skip it instantly via the local DB.
+- **Recovery key auto-saved** to `~/.hermes/.env` on first bootstrap so device self-signing works on every restart.
+- **Ghost device purge** — Synapse stores cross-signing public keys as device table rows. Without removal, Element tries to encrypt Olm messages to these phantom accounts and decryption fails. The adapter removes them on startup with UIA password fallback.
+- **Encrypted media** — inbound files in E2EE rooms use `content.file.url` (not `content.url`) and require `decrypt_attachment` before caching. Both handled.
+
+### Python 3.14 compatibility
+
+mautrix 0.21.0 has five incompatibilities with Python 3.14. `_patch_mautrix_py314()` applies targeted monkey-patches at import time, no-op on Python < 3.14. Two patches fix real mautrix bugs on all Python versions and run unconditionally.
+
+### Media support
+
+**Outbound** — all methods use correct base-class parameter names and upload via `/_matrix/media/v3/upload`. Supports `send_image`, `send_image_file`, `send_animation`, `send_voice`, `send_video`, `send_document`. Accepts local paths or existing `mxc://` URIs.
+
+**Inbound** — mautrix msgtype strings (`"m.image"`, `"m.audio"`, `"m.video"`, `"m.file"`) compared as strings. Downloads, decrypts if E2EE, caches, and populates `event.media_urls`/`event.media_types` following the Telegram adapter pattern exactly. Agent sees files via the standard document context note.
+
+### Typing indicators
+
+`send_typing()` → `PUT /rooms/{id}/typing` with 20s keepalive (Matrix typing expires at 30s). Clears in `send_message()`. Same pattern as nanobot's Matrix implementation.
+
+### Setup wizard (zero manual steps)
+
+`hermes gateway setup matrix`:
+1. Homeserver URL + SSL + live connectivity test
+2. Bot user ID
+3. Bot password → logs in automatically, gets token + device ID (no curl commands)
+4. E2EE deps checked/installed
+5. Cross-signing bootstrap inline, recovery key auto-saved
+6. Allowed users
+7. Trust verification inline — signs bot's master key from each allowed user's account
+
+Re-configuration: wizard offers to wipe all E2EE state for a clean start.
+
+---
+
+## Known limitations
+
+1. **`_send_matrix()` tool does not support E2EE** — one-shot HTTP client, no sync loop. Messages to encrypted rooms will fail. Documented with warning log. The running gateway handles E2EE correctly; this only affects the standalone tool.
+
+2. **SAS verification** — implemented via `olm.Sas` directly (mautrix 0.21.0 lacks built-in SAS). Tested with Element Desktop only.
+
+3. **No explicit sync reconnection backoff** — mautrix's `client.start()` has internal retry but no configurable backoff wrapper.
+
+---
+
+## Testing
+
+### Tested against
+- **Synapse** self-hosted on k3s (v1.99+), self-signed TLS via Tailscale
+- **Element Desktop** on Arch Linux
+- **Python 3.14**
+- **Media** — image, audio, document upload/download verified live against Synapse
+
+### k8s / self-hosted note
+If media upload returns `500 Internal server error`, the `media_store` volume may be owned by root. Fix:
+```bash
+kubectl exec -n matrix <pod> -- chown -R 991:991 /data/media_store
+```
+The standard Synapse k8s init container handles this on every pod start, but pre-existing volumes may need a one-time manual fix.
+
+### Needs community validation
+- **matrix.org** (public homeserver) — rate limits and UIA behavior may differ
+- **Dendrite** — different cross-signing implementation
+- **Element X** (Rust SDK) — different verification flow
+- **Python 3.11 / 3.12** — patches are no-ops; should work but unconfirmed
+- **Other clients** (FluffyChat, Nheko, Cinny) — untested
+
+### How to test
+
+```bash
+# Install E2EE dependencies
+# Setup wizard does handle this and install if not available
+pip install "mautrix[e2be]" asyncpg aiosqlite base58
+sudo pacman -S libolm   # Arch; or: apt install libolm-dev
+
+# Full guided setup (wizard handles everything)
+hermes gateway setup
+# select matrix (6)
+
+# Start
+hermes gateway restart
+
+# Send a text message, image, PDF in Element — bot should respond
+# If trust verification incomplete:
+hermes gateway verify-matrix
+```
+
+### Test results
+```
+93 passed  (tests/gateway/test_matrix.py)
+3550 passed total — same 35 pre-existing failures as main, zero regressions
+```
+
+---
+
+## Environment variables
+
+| Variable | Required | Description |
+|----------|----------|-------------|
+| `MATRIX_HOMESERVER_URL` | Yes | e.g. `https://matrix.example.org` |
+| `MATRIX_ACCESS_TOKEN` | Yes | Bot account access token (`syt_...`) |
+| `MATRIX_USER_ID` | Yes | Bot Matrix ID (`@bot:example.org`) |
+| `MATRIX_DEVICE_ID` | Recommended | Pins to one device; prevents session replay on restart |
+| `MATRIX_ALLOWED_USERS` | Recommended | Comma-separated IDs allowed to message the bot |
+| `MATRIX_HOME_CHANNEL` | Optional | Room ID for cron job delivery |
+| `MATRIX_HOME_CHANNEL_NAME` | Optional | Display name for home channel (default: "Home") |
+| `MATRIX_VERIFY_SSL` | Optional | `false` for self-signed TLS (default `true`) |
+| `MATRIX_E2EE` | Optional | `true` for E2EE (requires deps above) |
+| `MATRIX_PASSWORD` | Optional | Bot password for cross-signing UIA fallback |
+| `MATRIX_RECOVERY_KEY` | Optional | Auto-saved by gateway; enables device self-signing on restart |
+
+## Dependencies
+
+All optional — only needed when `MATRIX_E2EE=true`. Installed automatically by the setup wizard.
+
+| Package | Purpose |
+|---------|---------|
+| `mautrix[e2be]` | Matrix client + E2EE crypto |
+| `asyncpg` | SQL dialect for `PgCryptoStore` |
+| `aiosqlite` | SQLite backend (no PostgreSQL needed) |
+| `base58` | Recovery key encoding |
+| `libolm` | Olm/Megolm C library (system package) |
+
+---
+
+*Branch: `feat/matrix-gateway` | Python 3.14 | Synapse self-hosted | Element Desktop*

README.md

@@ -17,7 +17,7 @@ Use any model you want — [Nous Portal](https://portal.nousresearch.com), [Open
 
 <table>
 <tr><td><b>A real terminal interface</b></td><td>Full TUI with multiline editing, slash-command autocomplete, conversation history, interrupt-and-redirect, and streaming tool output.</td></tr>
-<tr><td><b>Lives where you do</b></td><td>Telegram, Discord, Slack, WhatsApp, Signal, and CLI — all from a single gateway process. Voice memo transcription, cross-platform conversation continuity.</td></tr>
+<tr><td><b>Lives where you do</b></td><td>Telegram, Discord, Slack, WhatsApp, Signal, Matrix, and CLI — all from a single gateway process. Voice memo transcription, cross-platform conversation continuity.</td></tr>
 <tr><td><b>A closed learning loop</b></td><td>Agent-curated memory with periodic nudges. Autonomous skill creation after complex tasks. Skills self-improve during use. FTS5 session search with LLM summarization for cross-session recall. <a href="https://github.com/plastic-labs/honcho">Honcho</a> dialectic user modeling. Compatible with the <a href="https://agentskills.io">agentskills.io</a> open standard.</td></tr>
 <tr><td><b>Scheduled automations</b></td><td>Built-in cron scheduler with delivery to any platform. Daily reports, nightly backups, weekly audits — all in natural language, running unattended.</td></tr>
 <tr><td><b>Delegates and parallelizes</b></td><td>Spawn isolated subagents for parallel workstreams. Write Python scripts that call tools via RPC, collapsing multi-step pipelines into zero-context-cost turns.</td></tr>
@@ -73,7 +73,7 @@ All documentation lives at **[hermes-agent.nousresearch.com/docs](https://hermes
 | [Quickstart](https://hermes-agent.nousresearch.com/docs/getting-started/quickstart) | Install → setup → first conversation in 2 minutes |
 | [CLI Usage](https://hermes-agent.nousresearch.com/docs/user-guide/cli) | Commands, keybindings, personalities, sessions |
 | [Configuration](https://hermes-agent.nousresearch.com/docs/user-guide/configuration) | Config file, providers, models, all options |
-| [Messaging Gateway](https://hermes-agent.nousresearch.com/docs/user-guide/messaging) | Telegram, Discord, Slack, WhatsApp, Signal, Home Assistant |
+| [Messaging Gateway](https://hermes-agent.nousresearch.com/docs/user-guide/messaging) | Telegram, Discord, Slack, WhatsApp, Signal, Matrix, Home Assistant |
 | [Security](https://hermes-agent.nousresearch.com/docs/user-guide/security) | Command approval, DM pairing, container isolation |
 | [Tools & Toolsets](https://hermes-agent.nousresearch.com/docs/user-guide/features/tools) | 40+ tools, toolset system, terminal backends |
 | [Skills System](https://hermes-agent.nousresearch.com/docs/user-guide/features/skills) | Procedural memory, Skills Hub, creating skills |

agent/prompt_builder.py

@@ -131,6 +131,16 @@ def _scan_context_content(content: str, filename: str) -> str:
         "files arrive as downloadable documents. You can also include image "
         "URLs in markdown format ![alt](url) and they will be sent as photos."
     ),
+    "matrix": (
+        "You are on Matrix, a federated, open-standard messaging protocol. "
+        "Markdown is fully supported — use it for formatting (bold, italics, "
+        "code blocks, headers, lists). You can send media files natively: "
+        "to deliver a file to the user, include MEDIA:/absolute/path/to/file "
+        "in your response. Images (.png, .jpg, .webp) appear as inline previews, "
+        "audio as playable voice messages, and other files as downloadable "
+        "attachments. You can also include image URLs in markdown format "
+        "![alt](url) and they will be sent as images."
+    ),
     "email": (
         "You are communicating via email. Write clear, well-structured responses "
         "suitable for email. Use plain text formatting (no markdown). "

agent/redact.py

@@ -81,6 +81,10 @@
 # Negative lookahead prevents matching hex strings or identifiers
 _SIGNAL_PHONE_RE = re.compile(r"(\+[1-9]\d{6,14})(?![A-Za-z0-9])")
 
+# Matrix user IDs: @localpart:homeserver  (e.g. @alice:example.org, @user:matrix.org)
+# Captures only the localpart and homeserver (not the @) for partial redaction
+_MATRIX_ID_RE = re.compile(r"(@)([^:\s@]+)(:[a-zA-Z0-9.\-]+(:[0-9]+)?)")
+
 # Compile known prefix patterns into one alternation
 _PREFIX_RE = re.compile(
     r"(?<![A-Za-z0-9_-])(" + "|".join(_PREFIX_PATTERNS) + r")(?![A-Za-z0-9_-])"
@@ -147,6 +151,16 @@ def _redact_phone(m):
         return phone[:4] + "****" + phone[-4:]
     text = _SIGNAL_PHONE_RE.sub(_redact_phone, text)
 
+    # Matrix user IDs (e.g., @alice:example.org → @al**:example.org)
+    def _redact_matrix_id(m):
+        at = m.group(1)
+        local = m.group(2)
+        server = m.group(3)
+        if len(local) <= 2:
+            return f"{at}{'*' * len(local)}{server}"
+        return f"{at}{local[:2]}{'*' * (len(local) - 2)}{server}"
+    text = _MATRIX_ID_RE.sub(_redact_matrix_id, text)
+
     return text
 
 

cron/scheduler.py

@@ -104,6 +104,7 @@ def _deliver_result(job: dict, content: str) -> None:
         "whatsapp": Platform.WHATSAPP,
         "signal": Platform.SIGNAL,
         "email": Platform.EMAIL,
+        "matrix": Platform.MATRIX,
     }
     platform = platform_map.get(platform_name.lower())
     if not platform:

gateway/channel_directory.py

@@ -60,8 +60,8 @@ def build_channel_directory(adapters: Dict[Any, Any]) -> Dict[str, Any]:
         except Exception as e:
             logger.warning("Channel directory: failed to build %s: %s", platform.value, e)
 
-    # Telegram, WhatsApp & Signal can't enumerate chats -- pull from session history
-    for plat_name in ("telegram", "whatsapp", "signal", "email"):
+    # Telegram, WhatsApp, Signal & Matrix can't enumerate chats -- pull from session history
+    for plat_name in ("telegram", "whatsapp", "signal", "email", "matrix"):
         if plat_name not in platforms:
             platforms[plat_name] = _build_from_sessions(plat_name)
 

gateway/config.py

@@ -29,6 +29,7 @@ class Platform(Enum):
     SIGNAL = "signal"
     HOMEASSISTANT = "homeassistant"
     EMAIL = "email"
+    MATRIX = "matrix"
 
 
 @dataclass
@@ -168,6 +169,9 @@ def get_connected_platforms(self) -> List[Platform]:
             # Signal uses extra dict for config (http_url + account)
             elif platform == Platform.SIGNAL and config.extra.get("http_url"):
                 connected.append(platform)
+            # Matrix uses extra dict for config (homeserver_url + user_id)
+            elif platform == Platform.MATRIX and config.extra.get("homeserver_url"):
+                connected.append(platform)
             # Email uses extra dict for config (address + imap_host + smtp_host)
             elif platform == Platform.EMAIL and config.extra.get("address"):
                 connected.append(platform)
@@ -427,6 +431,35 @@ def _apply_env_overrides(config: GatewayConfig) -> None:
                 name=os.getenv("SIGNAL_HOME_CHANNEL_NAME", "Home"),
             )
 
+    # Matrix
+    matrix_token = os.getenv("MATRIX_ACCESS_TOKEN")
+    matrix_homeserver = os.getenv("MATRIX_HOMESERVER_URL")
+    matrix_user_id = os.getenv("MATRIX_USER_ID")
+    if matrix_token and matrix_homeserver and matrix_user_id:
+        if Platform.MATRIX not in config.platforms:
+            config.platforms[Platform.MATRIX] = PlatformConfig()
+        config.platforms[Platform.MATRIX].enabled = True
+        config.platforms[Platform.MATRIX].token = matrix_token
+        matrix_device_id = os.getenv("MATRIX_DEVICE_ID", "")
+        # Store as lowercase strings so the adapter can read them consistently
+        # with a simple `.lower() not in ("false", "0", "no")` check.
+        matrix_e2ee = os.getenv("MATRIX_E2EE", "false")
+        matrix_verify_ssl = os.getenv("MATRIX_VERIFY_SSL", "true")
+        config.platforms[Platform.MATRIX].extra.update({
+            "homeserver_url": matrix_homeserver.rstrip("/"),
+            "user_id": matrix_user_id,
+            "device_id": matrix_device_id,
+            "verify_ssl": matrix_verify_ssl,
+            "e2ee": matrix_e2ee,
+        })
+        matrix_home = os.getenv("MATRIX_HOME_CHANNEL")
+        if matrix_home:
+            config.platforms[Platform.MATRIX].home_channel = HomeChannel(
+                platform=Platform.MATRIX,
+                chat_id=matrix_home,
+                name=os.getenv("MATRIX_HOME_CHANNEL_NAME", "Home"),
+            )
+
     # Home Assistant
     hass_token = os.getenv("HASS_TOKEN")
     if hass_token: