Skip to content

fix(matrix): resolve crypto store per-instance so multiplexed profiles don't share crypto.db - #89169

Closed
mjshorty wants to merge 1 commit into
NousResearch:mainfrom
mjshorty:fix/matrix-crypto-store-per-profile
Closed

mjshorty wants to merge 1 commit into
NousResearch:mainfrom
mjshorty:fix/matrix-crypto-store-per-profile

Conversation

@mjshorty

Copy link
Copy Markdown
Contributor

Summary

Fixes #89168 — the Matrix crypto-store collision under the multiplex gateway.

Root cause: plugins/platforms/matrix/adapter.py resolves the crypto store at module scope:

_STORE_DIR = _get_hermes_dir("platforms/matrix/store", "matrix/store")
_CRYPTO_DB_PATH = _STORE_DIR / "crypto.db"

With gateway.multiplex_profiles=true, the multiplexer is a single process that imports this module once; get_hermes_home() at import time resolves to the root ~/.hermes. Every profile's Matrix adapter therefore opens the same crypto.db — all bots' Olm identities collide in one store and inbound E2EE fails with Error decrypting megolm event, no session found.

Fix: replace the module-level constants with per-instance _get_store_path() / _get_store_dir() that resolve through get_hermes_dir() at connect time. Under multiplex, each profile's adapter is created and connected inside _profile_runtime_scope, so the context-local HERMES_HOME override makes every profile resolve its own store dir (<root>/profiles/<name>/platforms/matrix/store/crypto.db).

This mirrors the existing pairing-store migration (a6397c379 — "fix(gateway): align multiplex pairing stores"), which fixed the identical module→per-instance bug for the pairing store.

Changes

  • plugins/platforms/matrix/adapter.py
    • Remove module-level _STORE_DIR / _CRYPTO_DB_PATH path resolution.
    • Add MatrixAdapter._get_store_path() (crypto.db) and MatrixAdapter._get_store_dir() (store dir for mkdir/legacy pickle cleanup), both resolving per-call through get_hermes_dir.
    • Update all 7 usages (key-mismatch log, connect mkdir, E2EE setup mkdir, legacy pickle path, sqlite:/// DB URL, E2EE-enabled log, crypto_store_path status) to go through the instance methods.
    • Keep _CRYPTO_DB_PATH = None as a back-compat import alias (no external callers read it, but the name was previously importable).
  • tests/gateway/test_matrix_crypto_store_per_profile.py (new)
    • Asserts two profile homes resolve distinct crypto.db paths.
    • Asserts the path is resolved per-call (not pinned at module scope) — switching the active profile changes the resolved path.

Test Plan

MATRIX_E2EE_MODE= venv/bin/python -m pytest tests/gateway/test_matrix.py tests/gateway/test_matrix_crypto_store_per_profile.py tests/gateway/test_matrix_recovery_key_scope.py tests/gateway/test_matrix_plugin_setup.py -q

Result: 121 passed. Also ran test_multiplex_adapter_registry.py + test_multiplex_phase0.py (31 passed) — no multiplex regressions.

(Note: the existing TestMatrixRequirements::test_check_requirements_encryption_false_no_e2ee_deps_ok fails locally when the host env exports MATRIX_E2EE_MODE=required because MATRIX_E2EE_MODE is not credential-shaped and survives the hermetic conftest scrub. It passes with the var unset. That is pre-existing test-env sensitivity, unrelated to this change.)

Verification

  • New test passes: two profiles resolve different store paths (verified with set_hermes_home_override, the same contextvar the multiplexer uses).
  • grep -n '_get_store_path' plugins/platforms/matrix/adapter.py shows the new per-instance resolver.

…s don't share crypto.db

Under gateway.multiplex_profiles the gateway imports
plugins/platforms/matrix/adapter.py once; the module-level
_STORE_DIR/_CRYPTO_DB_PATH constants resolved against the root HERMES_HOME
at import time, so every profile's Matrix adapter opened the SAME crypto.db.
All bots' Olm identities landed in one store and inbound E2EE failed with
'Error decrypting megolm event, no session found'.

Replace the module-level constants with per-instance _get_store_path() /
_get_store_dir() that resolve through get_hermes_dir() at connect time.
Each profile's adapter is created and connected inside _profile_runtime_scope,
so the context-local HERMES_HOME override makes every profile resolve its
own store dir. Mirrors the pairing-store migration (a6397c3).

Adds tests asserting two profile homes resolve distinct crypto.db paths.

@MrSuddenJoy MrSuddenJoy left a comment

Copy link
Copy Markdown

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

This code looks good to me. Well done 👍🏻

@alt-glitch alt-glitch added type/bug Something isn't working comp/plugins Plugin system and bundled plugins platform/matrix Matrix adapter (E2EE) P3 Low — cosmetic, nice to have sweeper:risk-session-state Sweeper risk: may lose/corrupt/mis-associate session or context state sweeper:risk-message-delivery Sweeper risk: may drop, duplicate, misroute, or suppress messages labels Aug 18, 2026
@mjshorty

Copy link
Copy Markdown
Contributor Author

@MrSuddenJoy — follow-up on this PR from Michael Short. Thanks again for the review; the code hasn't changed since your pass.

Current state as of now:

  • Your review on the current head (7f6c9) is positive but was submitted as COMMENTED, not APPROVED — so the PR has no formal approval and is sitting open.
  • The branch is 1 commit ahead of main and 1 behind (a simple merge-conflict-free catch-up; nothing blocking).
  • No CI checks are configured on this repo, and main has no branch protection, so there's no automated gate waiting on this change.

Could you formally approve and/or pass this to a maintainer for merge consideration? No code changes are pending from our side — if a rebase/merge of main first would help unblock the merge, say the word and we'll push it.

(Note: posting via automation on Michael Short's account — he will not self-merge.)

@MrSuddenJoy

MrSuddenJoy commented Aug 21, 2026

Copy link
Copy Markdown

was submitted as COMMENTED, not APPROVED

@mjshorty thats because both other options are grayed out (not accessible) for me I have to be a contributor to have these 2 enabled.

Could you formally approve and/or pass this to a maintainer for merge consideration?

@mjshorty I would love to, as I see well-crafted code on your side, but I'm not able to technically speaking.

teknium1 added a commit that referenced this pull request Sep 2, 2026
…import

The multiplex gateway imports plugins/platforms/matrix/adapter.py once, so
the module-level _STORE_DIR/_CRYPTO_DB_PATH resolved against the root
HERMES_HOME for every profile: all bots' Olm identities landed in one
crypto.db and inbound E2EE failed with "no session found" (#89168).

connect() runs inside _profile_runtime_scope, so resolve the store dir
there via get_hermes_dir (honors the context-local HERMES_HOME) and cache
it on the instance -- diagnostics and error-log paths read outside the
scope then still report the store actually in use. Mirrors the
pairing-store fix (a6397c3).

Salvage of #89169 (per-call resolvers collapsed into one cached resolve;
dead `_CRYPTO_DB_PATH = None` alias dropped -- no external importers).
Also routes the last raw MATRIX_HOMESERVER read in check_matrix_requirements
through _startup_env_secret like its token/password neighbours (#69943).

Fixes #89168

Co-authored-by: Michael Short <18595461+mjshorty@users.noreply.github.com>
teknium1 added a commit that referenced this pull request Sep 2, 2026
…import

The multiplex gateway imports plugins/platforms/matrix/adapter.py once, so
the module-level _STORE_DIR/_CRYPTO_DB_PATH resolved against the root
HERMES_HOME for every profile: all bots' Olm identities landed in one
crypto.db and inbound E2EE failed with "no session found" (#89168).

connect() runs inside _profile_runtime_scope, so resolve the store dir
there via get_hermes_dir (honors the context-local HERMES_HOME) and cache
it on the instance -- diagnostics and error-log paths read outside the
scope then still report the store actually in use. Mirrors the
pairing-store fix (a6397c3).

Salvage of #89169 (per-call resolvers collapsed into one cached resolve;
dead `_CRYPTO_DB_PATH = None` alias dropped -- no external importers).
Also routes the last raw MATRIX_HOMESERVER read in check_matrix_requirements
through _startup_env_secret like its token/password neighbours (#69943).

Fixes #89168

Co-authored-by: Michael Short <18595461+mjshorty@users.noreply.github.com>
teknium1 added a commit that referenced this pull request Sep 2, 2026
…import

The multiplex gateway imports plugins/platforms/matrix/adapter.py once, so
the module-level _STORE_DIR/_CRYPTO_DB_PATH resolved against the root
HERMES_HOME for every profile: all bots' Olm identities landed in one
crypto.db and inbound E2EE failed with "no session found" (#89168).

connect() runs inside _profile_runtime_scope, so resolve the store dir
there via get_hermes_dir (honors the context-local HERMES_HOME) and cache
it on the instance -- diagnostics and error-log paths read outside the
scope then still report the store actually in use. Mirrors the
pairing-store fix (a6397c3).

Salvage of #89169 (per-call resolvers collapsed into one cached resolve;
dead `_CRYPTO_DB_PATH = None` alias dropped -- no external importers).
Also routes the last raw MATRIX_HOMESERVER read in check_matrix_requirements
through _startup_env_secret like its token/password neighbours (#69943).

Fixes #89168

Co-authored-by: Michael Short <18595461+mjshorty@users.noreply.github.com>
teknium1 added a commit that referenced this pull request Sep 2, 2026
…import

The multiplex gateway imports plugins/platforms/matrix/adapter.py once, so
the module-level _STORE_DIR/_CRYPTO_DB_PATH resolved against the root
HERMES_HOME for every profile: all bots' Olm identities landed in one
crypto.db and inbound E2EE failed with "no session found" (#89168).

connect() runs inside _profile_runtime_scope, so resolve the store dir
there via get_hermes_dir (honors the context-local HERMES_HOME) and cache
it on the instance -- diagnostics and error-log paths read outside the
scope then still report the store actually in use. Mirrors the
pairing-store fix (a6397c3).

Salvage of #89169 (per-call resolvers collapsed into one cached resolve;
dead `_CRYPTO_DB_PATH = None` alias dropped -- no external importers).
Also routes the last raw MATRIX_HOMESERVER read in check_matrix_requirements
through _startup_env_secret like its token/password neighbours (#69943).

Fixes #89168

Co-authored-by: Michael Short <18595461+mjshorty@users.noreply.github.com>
teknium1 added a commit that referenced this pull request Sep 2, 2026
…import

The multiplex gateway imports plugins/platforms/matrix/adapter.py once, so
the module-level _STORE_DIR/_CRYPTO_DB_PATH resolved against the root
HERMES_HOME for every profile: all bots' Olm identities landed in one
crypto.db and inbound E2EE failed with "no session found" (#89168).

connect() runs inside _profile_runtime_scope, so resolve the store dir
there via get_hermes_dir (honors the context-local HERMES_HOME) and cache
it on the instance -- diagnostics and error-log paths read outside the
scope then still report the store actually in use. Mirrors the
pairing-store fix (a6397c3).

Salvage of #89169 (per-call resolvers collapsed into one cached resolve;
dead `_CRYPTO_DB_PATH = None` alias dropped -- no external importers).
Also routes the last raw MATRIX_HOMESERVER read in check_matrix_requirements
through _startup_env_secret like its token/password neighbours (#69943).

Fixes #89168

Co-authored-by: Michael Short <18595461+mjshorty@users.noreply.github.com>
teknium1 added a commit that referenced this pull request Sep 2, 2026
…import

The multiplex gateway imports plugins/platforms/matrix/adapter.py once, so
the module-level _STORE_DIR/_CRYPTO_DB_PATH resolved against the root
HERMES_HOME for every profile: all bots' Olm identities landed in one
crypto.db and inbound E2EE failed with "no session found" (#89168).

connect() runs inside _profile_runtime_scope, so resolve the store dir
there via get_hermes_dir (honors the context-local HERMES_HOME) and cache
it on the instance -- diagnostics and error-log paths read outside the
scope then still report the store actually in use. Mirrors the
pairing-store fix (a6397c3).

Salvage of #89169 (per-call resolvers collapsed into one cached resolve;
dead `_CRYPTO_DB_PATH = None` alias dropped -- no external importers).
Also routes the last raw MATRIX_HOMESERVER read in check_matrix_requirements
through _startup_env_secret like its token/password neighbours (#69943).

Fixes #89168

Co-authored-by: Michael Short <18595461+mjshorty@users.noreply.github.com>
teknium1 added a commit that referenced this pull request Sep 2, 2026
…import

The multiplex gateway imports plugins/platforms/matrix/adapter.py once, so
the module-level _STORE_DIR/_CRYPTO_DB_PATH resolved against the root
HERMES_HOME for every profile: all bots' Olm identities landed in one
crypto.db and inbound E2EE failed with "no session found" (#89168).

connect() runs inside _profile_runtime_scope, so resolve the store dir
there via get_hermes_dir (honors the context-local HERMES_HOME) and cache
it on the instance -- diagnostics and error-log paths read outside the
scope then still report the store actually in use. Mirrors the
pairing-store fix (a6397c3).

Salvage of #89169 (per-call resolvers collapsed into one cached resolve;
dead `_CRYPTO_DB_PATH = None` alias dropped -- no external importers).
Also routes the last raw MATRIX_HOMESERVER read in check_matrix_requirements
through _startup_env_secret like its token/password neighbours (#69943).

Fixes #89168

Co-authored-by: Michael Short <18595461+mjshorty@users.noreply.github.com>
@teknium1

teknium1 commented Sep 2, 2026

Copy link
Copy Markdown
Collaborator

Thanks @mjshorty — Merged via #101252 (2e25b47) on current main.

Your commits from this PR were cherry-picked onto the salvage branch with your git authorship preserved, so the credit is yours in git log. (Your change was salvaged onto the current code layout where it no longer applied cleanly, but the commit is still yours.)

Closing this PR since the work is now on main.

@teknium1 teknium1 closed this Sep 2, 2026
melon-xf added a commit to melon-xf/hermes-agent that referenced this pull request Sep 3, 2026
…import

The multiplex gateway imports plugins/platforms/matrix/adapter.py once, so
the module-level _STORE_DIR/_CRYPTO_DB_PATH resolved against the root
HERMES_HOME for every profile: all bots' Olm identities landed in one
crypto.db and inbound E2EE failed with "no session found" (NousResearch#89168).

connect() runs inside _profile_runtime_scope, so resolve the store dir
there via get_hermes_dir (honors the context-local HERMES_HOME) and cache
it on the instance -- diagnostics and error-log paths read outside the
scope then still report the store actually in use. Mirrors the
pairing-store fix (a6397c3).

Salvage of NousResearch#89169 (per-call resolvers collapsed into one cached resolve;
dead `_CRYPTO_DB_PATH = None` alias dropped -- no external importers).
Also routes the last raw MATRIX_HOMESERVER read in check_matrix_requirements
through _startup_env_secret like its token/password neighbours (NousResearch#69943).

Fixes NousResearch#89168

Co-authored-by: Michael Short <18595461+mjshorty@users.noreply.github.com>
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

comp/plugins Plugin system and bundled plugins P3 Low — cosmetic, nice to have platform/matrix Matrix adapter (E2EE) sweeper:risk-message-delivery Sweeper risk: may drop, duplicate, misroute, or suppress messages sweeper:risk-session-state Sweeper risk: may lose/corrupt/mis-associate session or context state type/bug Something isn't working

Projects

None yet

Development

Successfully merging this pull request may close these issues.

Matrix crypto-store collision under multiplex: all profiles share one crypto.db (module-level _STORE_DIR/_CRYPTO_DB_PATH)

4 participants