fix(gateway+setup): repair Matrix E2EE deps and stop Discord auto-enable on fresh installs (#31116) - #31236

Closed

xxxigm wants to merge 4 commits into

NousResearch:mainfrom

xxxigm:fix/31116-matrix-e2ee-asyncpg-and-platform-leak

xxxigm commented May 24, 2026 •

edited

Loading

Contributor

What does this PR do?

Fixes two independent bugs that bite a fresh Matrix-only install on Debian 13 / Proxmox LXC reported in #31116:

E2EE crashes at first connect. The wizard ran pip install 'mautrix[encryption]' and reported success, but that extra only pulls python-olm / pycryptodome / unpaddedbase64. The gateway's E2EE path imports from mautrix.crypto.store.asyncpg import PgCryptoStore (which import asyncpgs at module load) and uses Database.create("sqlite:///…") (which needs aiosqlite to register the sqlite scheme). Neither package is in the upstream extras. Result: ERROR gateway.platforms.matrix: Matrix: failed to create E2EE client: No module named 'asyncpg' on every reconnect, indefinitely.
Discord auto-enables itself on every gateway boot. The registry-driven plugin enable loop in gateway/config.py gated on entry.check_fn(). For Discord that hook lazy-installs discord.py and then unconditionally returns True — so the gateway enabled the Discord adapter with no DISCORD_BOT_TOKEN, producing [Discord] No bot token configured and a 60s reconnect spam loop. Users who only picked Matrix in the setup wizard saw their journal flooded.

Two surgical fixes wired together:

Gate plugin enablement on is_connected, not check_fn (gateway/config.py). is_connected already encodes user-intent (DISCORD_BOT_TOKEN set, IRC_SERVER set, etc.) — the same signal hermes_cli/gateway.py::_platform_status uses for the setup picker. Fall back to check_fn for plugins predating the hook so back-compat is preserved. A raising hook is treated as "not connected" instead of crashing gateway boot.
Route Matrix E2EE installs through lazy_deps.ensure(\"platform.matrix\") (hermes_cli/setup.py). The platform.matrix group in tools/lazy_deps.py already mirrors the [matrix] extra in pyproject.toml and pulls the complete dep set (mautrix[encryption], Markdown, aiosqlite, asyncpg, aiohttp-socks). Surface install failures with a pip install 'hermes-agent[matrix]' hint instead of swallowing them — so users don't see "success" followed by a broken gateway.

Related Issue

Fixes #31116

Type of Change

🐛 Bug fix (non-breaking change that fixes an issue)

Changes Made

gateway/config.py — registry-driven enable loop now prefers entry.is_connected(synthetic_cfg) over entry.check_fn(); raising hooks degrade to "not connected" instead of crashing boot.
hermes_cli/setup.py — Matrix E2EE branch installs via tools.lazy_deps.ensure(\"platform.matrix\", prompt=False); FeatureUnavailable is surfaced with a manual-install hint. The non-E2EE branch keeps its lightweight pip install mautrix fallback for plaintext-only setups.
tests/gateway/test_plugin_platform_enable_31116.py — 6 new tests covering the exact Discord-no-token reproduction, the is_connected=True happy path, the is_connected vs check_fn precedence matrix (incl. is_connected=None back-compat), and the defensive raising-hook path.
tests/hermes_cli/test_setup_matrix_e2ee_install_31116.py — 5 new tests pinning the lazy-deps allowlist (asyncpg + aiosqlite + mautrix[encryption] all present) and the wizard wiring (E2EE=yes triggers lazy_deps.ensure(\"platform.matrix\", prompt=False); install failures show a manual-install hint).

Backwards compatible: every new path defaults to historical behavior (plugins without is_connected still gate on check_fn; non-E2EE Matrix installs still use the simple subprocess), no config schema changes.

How to Test

```bash

New regression suites for the two bugs (11 tests, hermetic, ~0.8s)

./scripts/run_tests.sh tests/gateway/test_plugin_platform_enable_31116.py \
tests/hermes_cli/test_setup_matrix_e2ee_install_31116.py

Full adjacent regression (Matrix, Google Chat, ntfy, Simplex, Discord

connect, gateway-platform gating, Telegram group, Signal, Mattermost,

BlueBubbles, lazy_deps, MS Graph webhook, SMS, all setup suites):

expected: 861 passed across 19 files

./scripts/run_tests.sh tests/gateway/test_matrix.py \
tests/gateway/test_google_chat.py tests/gateway/test_ntfy_plugin.py \
tests/gateway/test_simplex_plugin.py tests/gateway/test_discord_connect.py \
tests/hermes_cli/test_gateway_platform_gating.py \
tests/gateway/test_telegram_group_gating.py tests/gateway/test_signal.py \
tests/gateway/test_mattermost.py tests/gateway/test_bluebubbles.py \
tests/tools/test_lazy_deps.py tests/gateway/test_msgraph_webhook.py \
tests/gateway/test_sms.py tests/hermes_cli/test_setup_matrix_e2ee.py \
tests/hermes_cli/test_setup.py tests/hermes_cli/test_setup_openclaw_migration.py \
tests/hermes_cli/test_setup_irc.py tests/hermes_cli/test_setup_reconfigure.py
```

End-to-end behaviour after the fix, on the issue's exact repro (Debian 13, Proxmox LXC, Matrix-only):

```
$ hermes setup
…
Enable end-to-end encryption (E2EE)? [y/N]: y
✓ E2EE enabled
Installing Matrix E2EE stack (mautrix[encryption], asyncpg, aiosqlite)...
✓ Matrix E2EE stack installed
…

$ sudo systemctl start hermes-gateway
$ journalctl -u hermes-gateway --no-pager | tail
… INFO gateway.run: Connecting to matrix...
… INFO gateway.platforms.matrix: Matrix: using access token for @bot:matrix.org (device …)
… INFO gateway.platforms.matrix: Matrix: E2EE enabled (store: ~/.hermes/matrix/crypto.db, device_id=…)
… INFO gateway.run: ✓ matrix connected
```

No more Connecting to discord... / [Discord] No bot token configured in the journal — the Discord plugin is no longer auto-enabled when the user did not configure it.

Checklist

Conventional Commits (fix(gateway):, fix(setup):, test(gateway):, test(setup):)
4 focused commits, single author (xxxigm)
11 new tests pass; 861 existing tests across 19 adjacent suites pass; no regressions introduced
Tested on macOS 15.6 (darwin 24.6.0)
No new config keys, no architecture change, no schema migration

xxxigm added 4 commits

May 24, 2026 08:32


          fix(gateway): gate plugin platforms on is_connected, not check_fn (No…

afa46fb

…usResearch#31116)

The registry-driven plugin enablement loop in `_apply_env_overrides` was
calling `entry.check_fn()` to decide whether to add a plugin platform to
`config.platforms`. For Discord, `check_fn` lazy-installs the
`discord.py` SDK and then unconditionally returns True — so the gateway
auto-enabled the Discord adapter on every fresh start, even when the
user never set `DISCORD_BOT_TOKEN` and only picked Matrix in the setup
wizard. The result was repeated `[Discord] No bot token configured`
errors and 60s reconnect spam in the journal.

Prefer `entry.is_connected(synthetic_cfg)` when the plugin defines it.
That hook checks the same env vars / config.yaml signals
`hermes_cli/gateway.py::_platform_status` already uses for the setup
picker, so what shows up in the menu and what the gateway tries to
connect to stay in sync. Fall back to `check_fn` for plugins that
predate `is_connected`. Defensively treat raising hooks as
"not connected" so a buggy third-party plugin can't crash gateway boot.


          fix(setup): install full Matrix stack via lazy_deps when E2EE is enab…

e77c46b

…led (NousResearch#31116)

The wizard's Matrix branch ran `pip install 'mautrix[encryption]'` and
declared success, but that extra only pulls `python-olm`,
`pycryptodome`, and `unpaddedbase64`. The gateway's E2EE path imports
`from mautrix.crypto.store.asyncpg import PgCryptoStore` (which
unconditionally `import asyncpg`s at module load) and uses
`Database.create("sqlite:///...")` (which requires `aiosqlite` to
register the sqlite scheme). Neither package is a transitive dep of
`mautrix[encryption]`, so the gateway crashed on every connect with
`No module named 'asyncpg'` even though the wizard said the install
succeeded.

Route E2EE installs through `tools.lazy_deps.ensure("platform.matrix")`,
which mirrors the `[matrix]` extra in `pyproject.toml` and pulls the
complete dep set (`mautrix[encryption]`, `Markdown`, `aiosqlite`,
`asyncpg`, `aiohttp-socks`). Surface install failures with a manual
`pip install 'hermes-agent[matrix]'` hint instead of swallowing them.
The non-E2EE install path keeps its lightweight `pip install mautrix`
fallback for users who only want plaintext rooms.


          test(gateway): pin registry-driven plugin enablement gate (NousResear…

c0fcc0a

…ch#31116)

Black-box regression for the plugin auto-enable change in
`gateway/config.py`. Covers:

* Discord scenario from the original ticket — no `DISCORD_BOT_TOKEN`
  must mean no Discord platform in the loaded config (pre-fix this
  silently auto-enabled, producing log-spam loops).
* Setting `DISCORD_BOT_TOKEN` re-enables Discord via the existing
  `_apply_env_overrides` path, so the new gate doesn't regress the
  happy path.
* `is_connected=True` enables, `is_connected=False` skips even if
  `check_fn` returns True (the bug shape).
* Plugins without `is_connected` keep falling back to `check_fn` for
  back-compat.
* A raising `is_connected` is treated as "not connected" instead of
  crashing gateway boot.


          test(setup): pin Matrix E2EE install path through lazy_deps (NousRese…

9a008e4

…arch#31116)

Two layers of regression coverage for the E2EE install fix:

* Allowlist sanity — `LAZY_DEPS["platform.matrix"]` must contain both
  `asyncpg` (the asyncpg crypto-store module imports it at top-level)
  and `aiosqlite` (mautrix's `Database.create("sqlite:///...")` refuses
  to start without it). Without these, routing the wizard install
  through `lazy_deps.ensure` would still leave the runtime broken.
* Wizard wiring — running `_setup_matrix` with E2EE=yes calls
  `tools.lazy_deps.ensure("platform.matrix", prompt=False)` rather
  than the old custom subprocess that only pulled
  `mautrix[encryption]`. A failing `ensure` (FeatureUnavailable) is
  surfaced as a `pip install 'hermes-agent[matrix]'` hint so users
  don't see a "success" message followed by a broken gateway.

Stubs the prompts, env, lazy_deps.ensure, and CLI helpers so the test
runs hermetically.

xxxigm force-pushed the fix/31116-matrix-e2ee-asyncpg-and-platform-leak branch from 6ae4a7b to 9a008e4 Compare

May 24, 2026 01:34

alt-glitch added type/bug comp/gateway comp/cli platform/matrix platform/discord P1 labels

alt-glitch mentioned this pull request

fix(setup): install missing Matrix E2EE companion deps #31232

Closed

ayushere mentioned this pull request

fix(setup): patch python-olm bundled libolm for macOS Apple clang #31354

Open

1 task

teknium1 mentioned this pull request

fix(gateway+setup): repair Matrix E2EE deps and stop Discord auto-enable (#31116, salvages #31236) #31699

Closed

alt-glitch mentioned this pull request

fix(matrix,gateway): Matrix E2EE installs full dep set; plugins respect is_connected #31688

Merged

teknium1 commented May 24, 2026

Contributor

Thanks for the careful diagnosis here, @xxxigm — both bugs (Matrix E2EE deps + Discord auto-enable on check_fn alone) and the test coverage you added were exactly right.

The same two fixes landed via 54e61f9 (commit 54e61f9) before this PR could be salvaged onto the latest main:

tools.lazy_deps.ensure("platform.matrix") for the E2EE wizard path — same approach as yours
is_connected gate in gateway/config.py's registry enablement loop — same approach as yours

The follow-up Google Chat ordering issue is tracked in #31703.

Closing as superseded — the analysis and behavior you pinned in your test suite are what we shipped.

teknium1 closed this

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

comp/cli comp/gateway P1 platform/discord platform/matrix type/bug