feat(matrix): add LLM-callable Matrix tools and configurable read receipts by nepenth · Pull Request #4441 · NousResearch/hermes-agent

nepenth · 2026-04-01T11:34:25Z

Description

Tier 1.5: Matrix LLM-Callable Tools & Enhanced Event Handling

Builds on the Tier 1 foundation from #4338 to add LLM-callable Matrix tools and improve event handling reliability.

Depends on #4338 — should be merged after that PR.

What changed and why

New: 6 LLM-callable Matrix tools (tools/matrix_tools.py, 467 lines):

matrix_send_reaction — Send emoji reactions to messages
matrix_redact_message — Delete/redact messages with optional reason
matrix_create_room — Create Matrix rooms with presets and invites
matrix_invite_user — Invite users to rooms
matrix_fetch_history — Retrieve room message history (paginated, up to 200)
matrix_set_presence — Set online/offline/unavailable status

All tools include comprehensive input validation (prefix checks, length bounds, enum validation) and security hardening (threading lock on adapter reference, coroutine leak prevention, public room creation gating).

New: Configurable read receipt timing (MATRIX_READ_RECEIPTS env var):

immediate (default) — send read receipt on message receive
after_processing — send after agent finishes processing
disabled — never send read receipts

New: stop_typing override — properly clears typing indicator via room_typing(typing_state=False)

New: Event deduplication in _on_unknown_event — prevents duplicate processing of reaction events from older nio versions

New: Matrix platform hint in prompt builder — describes available Matrix tools and HTML formatting support

Architecture

Tools follow the Home Assistant tool pattern (tools/homeassistant_tool.py)
Adapter wired via set_matrix_adapter() / _ensure_adapter() with threading lock
Async bridge (_run_async) handles sync→async tool calls with 30s timeout and coroutine cleanup
Registered in toolsets.py as matrix toolset, included in hermes-matrix composite
Loaded via model_tools.py module loader

Security

Two security audit passes performed:

All tool inputs validated (room_id ! prefix, event_id $ prefix, user_id @ prefix, emoji length cap)
Threading lock prevents TOCTOU race on global adapter reference
future.cancel() on timeout prevents coroutine leaks
Public room creation gated behind MATRIX_ALLOW_PUBLIC_ROOMS env var (default: disabled)
No XSS regressions from Tier 1 fixes

How to test

# Run tests
pytest tests/gateway/test_matrix.py -v

# Environment variables
export MATRIX_READ_RECEIPTS=immediate    # or after_processing, disabled
export MATRIX_ALLOW_PUBLIC_ROOMS=false   # default, set true to allow

Files changed (6 files, +779/-3)

File	Changes
`tools/matrix_tools.py`	NEW — 6 tool handlers with validation
`gateway/platforms/matrix.py`	stop_typing, read receipts, dedup
`toolsets.py`	Matrix toolset registration
`model_tools.py`	Matrix tools module loader
`agent/prompt_builder.py`	Matrix platform hint
`tests/gateway/test_matrix.py`	244 new test lines

Checklist

Read Contributing Guide
Commit messages follow Conventional Commits
Searched for existing PRs (no duplicates)
PR contains only related changes
pytest tests/ -q passes (118/118)
Tests added for changes
Tested on Ubuntu 24.04
Updated tool descriptions/schemas
Considered cross-platform impact — N/A (Matrix-specific)

…tting The _markdown_to_html() method attempted to import the 'markdown' library which was never listed as a dependency, causing it to always fall through to a minimal regex fallback that only handled bold, italic, and inline code. This meant code blocks, headers, links, lists, blockquotes, and horizontal rules all rendered as raw markdown text in Matrix clients. Changes: - Add Markdown>=3.6 to the [matrix] optional dependency in pyproject.toml - Add sane_lists extension for better list handling - Rewrite the regex fallback to comprehensively handle all common markdown - Protected regions pattern prevents code content from being transformed - Import html.escape for proper HTML entity handling in code blocks

…t, presence, emote/notice Tier 1 Matrix enhancement — adds major missing features: Reactions: - Send reactions via m.reaction events (m.annotation rel_type) - Processing lifecycle: 👀 on start, ✅/❌ on complete (like Discord) - Receive reaction events (ReactionEvent + UnknownEvent fallback) - Configurable via MATRIX_REACTIONS env var (default: true) Read receipts: - Auto-send m.read + m.fully_read on incoming text and media messages - send_read_receipt() method using room_read_markers() API Message redaction: - redact_message() via room_redact() — agent can delete mistakes Room history: - fetch_room_history() via room_messages() — context loading on rejoin Room management: - create_room() with name, topic, invite list, DM flag, preset - invite_user() via room_invite() Presence: - set_presence() for online/offline/unavailable status Message types: - send_emote() for /me style actions (m.emote) - send_notice() for bot-appropriate non-alerting messages (m.notice) - Both support markdown-to-HTML conversion

…erter Security hardening for _markdown_to_html_fallback(): - HTML-escape all non-code text BEFORE applying markdown transforms. Previously, raw HTML in input (e.g. <script>, <img onerror=...>) would pass through unescaped into the formatted_body. Now all text outside of protected regions (code blocks, inline code, links) is escaped via html.escape() before any transformation occurs. - Add _sanitize_link_url() to reject dangerous URI schemes (javascript:, data:, vbscript:) and escape double-quotes in URLs to prevent href attribute breakout. - Extract and protect markdown links [text](url) BEFORE the global HTML-escape step, building <a> tags with escaped text and sanitized URLs. - Blockquote detection now handles both '>' and '>' (the escaped form) since blockquote markers are HTML-escaped in Phase 1b. - Harden the Python-Markdown library path: deregister the html_block preprocessor so raw HTML in source is escaped rather than passed through. Additional fixes: - Validate set_presence() state against allowed values (online/offline/ unavailable) before sending to homeserver. - Wrap fire-and-forget read receipt calls in _background_read_receipt() with try/except to prevent silently lost exceptions. - Escape code language class attribute in fenced blocks to prevent injection via crafted language identifiers. Verified against 12 XSS/injection vectors and 9 functional regression tests — all pass.

Tests cover: - HTML injection prevention (12 XSS vectors: script tags, img onerror, javascript:/data:/vbscript: URIs, attribute breakout, link text injection) - Extended formatting (code blocks, headers, lists, blockquotes, HR, strikethrough, links, complex mixed documents) - Link URL sanitization (scheme blocking, quote escaping) - Reactions (send, processing lifecycle, disabled state) - Read receipts (send, no-client guard) - Message redaction (redact, no-client guard) - Room management (create, invite, no-client guard) - Presence (valid state, invalid state rejected, no-client guard) - Message types (emote, notice, empty text guard) All 109 tests pass (63 existing + 46 new).

The base class calls stop_typing() when processing completes but it was a no-op. Now properly sends room_typing(typing_state=False) to clear the typing indicator in Matrix rooms.

…CEIPTS Supports three modes: - 'immediate' (default): send read receipt on message arrival - 'after_processing': send after agent finishes processing - 'disabled': never send read receipts Documented in module docstring.

The reaction handler for older matrix-nio versions was missing the dedup check that _on_reaction already had, potentially processing the same reaction event multiple times.

New tools/matrix_tools.py with six tools following the homeassistant_tool.py pattern: - matrix_send_reaction: react with emoji to messages - matrix_redact_message: delete/redact messages - matrix_create_room: create new rooms with options - matrix_invite_user: invite users to rooms - matrix_fetch_history: fetch room message history - matrix_set_presence: set bot online status All tools include input validation, proper error handling, and JSON return format. Uses _run_async bridge pattern for sync→async.

Register the adapter instance on connect() and clear it on disconnect() so the LLM-callable matrix tools have access to the running adapter.

Add standalone 'matrix' toolset and include matrix tools in 'hermes-matrix' platform toolset for full Matrix bot deployments.

Ensures matrix tools are imported and registered when the tool system initializes.

Informs the LLM about available Matrix-specific tools and HTML formatting support when running on the Matrix platform.

…ssues - Replace _run_async ThreadPoolExecutor pattern with run_coroutine_threadsafe() on the adapter's own event loop, fixing potential nio client state corruption from cross-loop access - Store event loop reference as adapter._loop in connect() - Add _ensure_adapter() null-check helper called by every handler - Restrict create_room preset to private_chat/public_chat only (trusted_private_chat grants elevated power levels to all members) - Add length limits: emoji (32), reason (500), name (255), topic (1000), status_msg (255) to prevent resource abuse via prompt injection - Silently cap over-length fields rather than rejecting

teknium1 · 2026-07-12T08:02:13Z

Thanks for the Matrix feature work. The underlying Matrix operations are already present on main, but the LLM-callable layer still needs a current-architecture implementation.

Problems

Current main has no registered implementation for any of the six matrix_* tool names; an all-repository exact-name search finds only website/docs/user-guide/messaging/matrix.md:412-417. The adapter methods exist at plugins/platforms/matrix/adapter.py:3104 and 3545-3706, so the feature should be ported rather than duplicated.
This branch edits the removed matrix-nio adapter. Main migrated Matrix to the mautrix plugin adapter in 8053d48c8df8d931d6ec21bb563d7dfa6434b3c5, now at plugins/platforms/matrix/adapter.py.
The MATRIX_READ_RECEIPTS addition is a non-secret behavioral environment variable. Repository policy requires this configuration to live in config.yaml; current main schedules receipts unconditionally at plugins/platforms/matrix/adapter.py:2705.

Suggested changes

Reimplement the wrappers through the Matrix platform plugin's tool-registration surface and reuse the existing adapter methods.
Make receipt timing a matrix config.yaml setting with coverage for all modes.

Automated hermes-sweeper review.

nepenth · 2026-07-18T00:34:03Z

Closing as superseded.

This PR targets the pre-mautrix layout (gateway/platforms/matrix.py, tools/matrix_tools.py) and is many months behind current main.

Replacement in-flight: #61218 (matrix / matrix_admin tools on current plugins/platforms/matrix/adapter.py, gateway-loop handoff, profile-safe adapter resolve), rebased onto current main 2026-07-17.

Not merged upstream; no code from this branch should be landed as-is.

Forge added 16 commits March 31, 2026 11:55

docs(matrix): add MATRIX_REACTIONS env var to module docstring

f0ad093

feat(matrix): add stop_typing override to clear typing indicator

87bb8f4

The base class calls stop_typing() when processing completes but it was a no-op. Now properly sends room_typing(typing_state=False) to clear the typing indicator in Matrix rooms.

fix(matrix): add event deduplication to _on_unknown_event handler

3489733

The reaction handler for older matrix-nio versions was missing the dedup check that _on_reaction already had, potentially processing the same reaction event multiple times.

feat(matrix): wire MatrixAdapter to matrix_tools module

81dc321

Register the adapter instance on connect() and clear it on disconnect() so the LLM-callable matrix tools have access to the running adapter.

feat(matrix): register matrix tools in toolsets

7b9cdaa

Add standalone 'matrix' toolset and include matrix tools in 'hermes-matrix' platform toolset for full Matrix bot deployments.

feat(matrix): add matrix_tools to model_tools module loader

5c6c245

Ensures matrix tools are imported and registered when the tool system initializes.

feat(matrix): add Matrix platform hint for prompt builder

478e6f9

Informs the LLM about available Matrix-specific tools and HTML formatting support when running on the Matrix platform.

fix(security): address medium-severity audit findings in matrix_tools

5c38d86

fix(matrix): harden tier 1.5 defaults

1b0195a

alt-glitch added type/feature New feature or request P3 Low — cosmetic, nice to have platform/matrix Matrix adapter (E2EE) comp/tools Tool registry, model_tools, toolsets labels May 2, 2026

nepenth mentioned this pull request Jul 18, 2026

feat(matrix): add compact matrix/matrix_admin tools #61218

Closed

nepenth closed this Jul 18, 2026

ZoloZiak mentioned this pull request Aug 27, 2026

feat(matrix): make per-message read receipts and lifecycle reactions configurable #96302

Closed

Navigation Menu

Repository navigation

feat(matrix): add LLM-callable Matrix tools and configurable read receipts - #4441

feat(matrix): add LLM-callable Matrix tools and configurable read receipts#4441
nepenth wants to merge 16 commits into
NousResearch:mainfrom
nepenth:feat/matrix-tier-1.5

nepenth commented Apr 1, 2026

Uh oh!

teknium1 commented Jul 12, 2026

Uh oh!

nepenth commented Jul 18, 2026

Uh oh!

Reviewers

Assignees

Labels

Projects

Milestone

Development

Uh oh!

3 participants

Conversation

nepenth commented Apr 1, 2026

Description

What changed and why

Architecture

Security

How to test

Files changed (6 files, +779/-3)

Checklist

Uh oh!

teknium1 commented Jul 12, 2026

Problems

Suggested changes

Uh oh!

nepenth commented Jul 18, 2026

Uh oh!

Reviewers

Assignees

Labels

Projects

Milestone

Development

Uh oh!

3 participants