Skip to content

fix(matrix): bound requests when the gateway loop stops - #130706

Draft
iainlane wants to merge 144 commits into
NousResearch:mainfrom
iainlane:codex/matrix-runtime-lifecycle
Draft

iainlane wants to merge 144 commits into
NousResearch:mainfrom
iainlane:codex/matrix-runtime-lifecycle

Conversation

@iainlane

@iainlane iainlane commented Oct 1, 2026 •

Copy link
Copy Markdown

Stack

All PRs in this section target main. Nesting records logical merge order. Every prerequisite must merge before its child. The administration branch already includes the catch-up behaviour, and both public prerequisite PRs must merge first.

Matrix tool requests can outlive gateway shutdown. A stopped but open event loop accepts a scheduled coroutine but cannot run its cancellation callback. The tool then waits indefinitely for cancellation to drain, even after its request deadline.

The shared runtime now checks loop availability before admission and while waiting for the result or cancellation. It preserves a confirmed result. Loop loss before dispatch returns an unavailable-loop error; loop loss after dispatch reports an unknown write outcome with the existing reconciliation instruction. Cancellation remains queued for the owner loop, and the persistent interrupt prevents a resumed preflight from issuing a late write. A stopped loop cannot execute cleanup, so the caller returns without claiming that the owner has drained.

Dependencies and review

Merge administration, catch-up and their prerequisite stacks before this repair. Review the one own source commit with the scoped comparison. The history is linear and preserves the original author and committer identities, dates, timezones and complete message.

Validation

The complete separate-owner-loop regression reproduces four failures and one confirmed-result pass on the exact current administration supplier, cbc39df0623bb52e9e9ae2618790fbbe02c2d38a. The causal control adds only this contribution's complete test module to that supplier. Applying the original production repair makes all five lifecycle cases pass. The complete administration, pin and read tool modules also pass, for 281 cases across four complete files. The canonical runner uses one worker, zero retries and treats ResourceWarning, RuntimeWarning, unraisable-exception and unhandled-thread warnings as errors. Every complete child output has zero resource lines.

The current branch retains all 143 signed prerequisite records unchanged and adds the single original contribution. Its complete production and test modules match the original contribution byte-for-byte. The automatic whole-snapshot merge changes only those two paths. The original author, committer, dates, timezones and full commit message are unchanged. Both frozen local merge checks, against 99a45ecc17cc51c0488ec3b8ccf7eaf483f7203d and 3637c512fc3211bdccb61b7140d09ac1018508ba, are clean.

All 8,424 tracked Python/test-runner inputs are authenticated against Git before and after each complete invocation. The canonical runner is restored exactly. A filename preflight error stopped an initial candidate attempt before any test child or runner modification; that control is preserved separately and contributes no pass count.

The two own Python files pass Ruff and Ty with the installed Matrix SDK. All three original, supplier and candidate type checks report zero findings. Ruff format reports the same pre-existing runtime file before and after the repair; the introduced test module is formatted. The administration supplier includes its independently delivered worker-loop fixture corrections. No fresh Docker or hosted CI result is claimed for this revision.

Earlier history and validation

The complete earlier administration history is retained unchanged, including all 128 captured contributor records and its attribution-only child. The published catch-up prerequisite retains its own complete contributor history. The contemporary branch uses the reviewed current supplier without replaying its existing catch-up, context, reaction or redaction code.

At the earlier published head c3f53f9cec17f09ed268a528a1e35df42bac42ec, a fresh image passed six native Matrix administration and pin cases across two files. The image digest is sha256:9ac54a7958ead018ecb36af4bbcce83afe94185bbf455bd2635e7d15860e810d. These are local validation receipts; hosted CI still requires upstream approval.

Matrix permalinks change with each triggering event. Rendering one in
the pinned session prompt made the prompt stale or changed its bytes
every turn, which breaks conversation prompt caching.

Put the link in the current model-facing user message and remove it from
the durable transcript, including queued turns. Respect effective PII
redaction when adding the note. Percent-encode room IDs, event IDs and
the via server so delimiters cannot change the matrix.to URL structure.

Preserve the Matrix and Discord source-note handling when integrating
the upstream prefix removal change. Use removeprefix after selecting
the platform-specific note, so each transport removes only its own
generated prefix.
The live Matrix exchange now checks the model request for a permalink to
the event sent by the independent client. It also verifies that the link
is absent from the system prompt, which must remain stable across turns.
Record the exchange duration separately from container setup time.
Room version 12 IDs have no server component. A permalink built only
from the room ID therefore omitted the server hint, which can prevent a
client outside the room from locating it.

Use the joined bot account's server when the room ID has no domain. The
builder still leaves via absent when neither server is known.
A room ID can identify a server that no longer participates in the
room. Its server may also include a port or IPv6 address, which the
last-colon parser truncated to an invalid via hint.

Use the joined bot account's server for inbound source links. When only
a room ID is available, parse its full server after the first colon.
The same parse also corrects the room's session scope.
In a thread, every turn's source link pointed at the thread root. The
root was chosen as a stable anchor while the link lived in the pinned
session prompt, which is per thread. The link now goes on each user
turn, so the reason no longer applies, and the model never received the
ID of the message that triggered the turn. A request to cite "this
message" produced a link to the root, which may be another user's
message.

Build the per-turn link from the triggering event in all cases. A
matrix.to link to an event inside a thread opens that thread in current
clients, and the session prompt already gives the thread root.
A room-ID permalink needs via servers so that a reader outside the room
can join through one of them. The link carried only the bot's server.
When that server was unreachable from the reader's server, the link
failed.

Choose up to three servers as the Matrix specification recommends for
room-ID permalinks. The first is the server of the highest-power joined
member at power level 50 or above; room version 12 creators count as
the highest power. The rest are the servers with the most joined
members, and IP literals are never chosen. The adapter reads members,
power levels and the create event from the mautrix state store, which
sync keeps current, so building a link makes no request. When the
store has no joined members for the room, use the bot's server and then
the room ID's server.

Server ACLs are not applied, because the state store does not keep
m.room.server_acl. Move the builder and the selection into a sibling
module, and emit one via parameter per server.
The session prompt stopped rendering the Matrix source link when the
link moved to the current user turn. Two prompt tests still asserted
that the prompt omitted the link when it was unset or redacted. No code
path produces that output any more, so neither test could fail because
of this feature. The prompt-stability test covers the remaining
contract, and the per-turn tests cover the note and its PII
suppression.

Restore the comment on the per-turn notes. strip_inbound_source_note
removes a note only when it is the first thing in the message, so a
later prefix added outside the notes would leave the note in the
persisted transcript.
Choosing source-link via servers made every inbound message await
get_power_levels and get_create on the client's state store. The
project-context isolation and voice tests gave the adapter a MagicMock
store with only get_members awaitable, so all six inbound tests in
those files failed with "'MagicMock' object can't be awaited".

Production always uses mautrix's MemoryStateStore. Give both fakes a
real MemoryStateStore in which the bot and the sender have joined the
test rooms. The real store implements every state query, so these tests
keep working when the adapter starts reading other room state. The
stores are filled with MemoryStateStore.deserialize because the helpers
are synchronous and some are called inside a running event loop.
The effective-state parent added a description to the matrix_read limit
argument: "Maximum number of events to read". In this layer the same
argument also bounds the members kind, which returns members rather
than events, and the pins kind. Say that the limit applies to events or
members so the model can tell that it bounds member lists too.
The merged matrix_read description said that events are listed oldest
first and that `errors` lists events that could not be decrypted. Both
sentences came from the history reads. A pins read returns events in
the order of the room's m.room.pinned_events list, and its errors also
cover pins that could not be fetched, are not visible messages, came
back as another event, or whose edit was redacted. A model that read
the description could tell a user that a deleted pin was an encrypted
message.

Limit the ordering sentence to room and thread reads, say that pins
follow the room's pinned order, and say that `errors` lists events that
could not be read or decrypted, with the reason. History reads already
returned non-decryption errors such as "event content changed", so the
wider wording is also accurate for them.
Room classification controls the Matrix room allowlist and mention
gates.
The previous member-count shortcut classified a bot-only room as a DM,
and an m.direct entry classified larger or unreadable rooms as DMs.
Those rooms could bypass both gates.

Classify a room as a private bot chat only when the bot and exactly one
other user are joined. Request JOIN membership explicitly: mautrix's
state store includes invited users by default. The state store may also
contain only a subset of a room's members, so use its joined IDs only
when has_full_member_list confirms completeness. Otherwise query the
homeserver with get_joined_members and apply room rules if that fails.

Keep m.direct as room identity metadata, without using it for
authorisation. Document the resulting allowlist and mention policy.

Tests reproduced the bot-only, bot-absent, multi-user, invited-user,
partial-cache and unknown-membership cases before their fixes.
The DM tests marked m.direct account data but did not supply joined
members. The two-person classifier therefore treated those fixtures
as rooms, so their threading assertions tested the wrong policy.

Give the fixture a full joined member list with the bot and one other
user. Keep the three-member variant for room behaviour. The mention
and threading tests now exercise the intended DM policy.
A room is a private bot chat only when the bot's user ID is one of its
two joined members. With an access token, the adapter replaces the
configured MATRIX_USER_ID with the ID that whoami returns. After a
password login it kept the configured string. Synapse accepts a login
identifier whose localpart differs in case from the registered user, so
a configured @Hermes:example.org logs in as @Hermes:example.org. The
membership check then never matched, and every one-to-one chat followed
room rules: with MATRIX_ALLOWED_ROOMS set its messages were dropped,
and otherwise they needed a mention.

mautrix's login stores the homeserver's user ID in client.mxid. Copy it
into the adapter after a successful login, as the whoami path does, so
the membership check uses the ID that appears in the room's member
list.
The Matrix handoff session-key test marked its DM case only through the
m.direct cache. Room classification no longer reads m.direct, so the
DM case was keyed as a group and the test failed. Give the fake client
a joined member list: the bot and one other user for the DM case, and a
third user for the room case.
The environment-variable reference still said that any room with two or
fewer joined members is a DM, and told operators to add a third member
to get room behaviour. A bot-only room, a room without the bot and a
room whose membership cannot be read now follow room rules. Describe
the current rule in the MATRIX_ALLOWED_ROOMS, MATRIX_REQUIRE_MENTION,
MATRIX_FREE_RESPONSE_ROOMS and MATRIX_AUTO_THREAD rows.

The comment in _schedule_pending_invite_joins said that a direct invite
missing from m.direct is misclassified as a group. m.direct no longer
affects classification, so drop that clause.
The E2EE crypto store keys every row (the Olm account, Olm sessions
and inbound Megolm sessions) by an account ID taken from the adapter's
user ID. After a password login the adapter now replaces the configured
MATRIX_USER_ID with the homeserver's spelling of it, so the DM check
matches the room's member list. That replacement also changed the store
key. A deployment with a password login, MATRIX_USER_ID differing in
case from the registered user, and a stable MATRIX_DEVICE_ID would open
an empty store after upgrading. It would create a new Olm account for
its existing device and could no longer decrypt history. The same
happened on an in-process reconnect, because the second login read the
replaced ID.

Keep the configured ID in its own attribute and use it for the login
identifier and, after a password login, for the store key, as before.
The access-token path keys the store by the whoami answer, as it always
has. The adapter's user ID still becomes the homeserver's spelling on
both paths, and the DM, mention, invite and self-sender checks use it.

Preserve the HTTP API supplied to the password-login fixture client and
disconnect its first real connection before reconnecting. The fixture
keeps every configured-account and canonical-identity assertion. The
shared Client constructor preserves the supplied API for this fixture.
Matrix rooms often have no m.room.name state event. Identity
resolution used the opaque room ID as the name shown to the agent
even when other occupants had readable display names.

Read member profiles from the state store, with the joined members
API as a fallback when the store is empty. Use the other occupants
to title an unnamed room. Explicit-name detection and DM
classification remain unchanged.
Matrix room identity was cached without reacting to live state
changes. Topic edits, renames and membership changes could leave
stale room context. The agent also missed important privacy or
replacement changes.

Register sync-awaited state handlers that invalidate the identity
cache. Queue one note per change kind for the next ordinary message.
Keep notes through mention drops, commands and text batching. Quote
and bound room-state values before they enter channel context.
Room notes were consumed while the adapter built an event. The gateway
could reject that event for sender policy or intercept it for a paused
session, leaving no note for the next model turn. A queued follow-up
also bypasses normal admission and missed the note entirely.

Consume notes in inbound model-message preparation, which both cold
and queued turns use after their dispatch gates. Rejected and
intercepted messages therefore leave pending notes for the next turn.

Busy-session debounce and pending-message merges also discarded
channel context from later events. Append it during both merges so
accepted context reaches the model. Request joined member profiles
explicitly so invited users do not appear in unnamed room titles.

Runner admission, paused and queued turns, busy merges, room bounds,
coalescing and invited membership have focused regressions.
A Matrix state update can change a member-derived room title, an
explicit room name or the room topic between turns. Those changes
altered the session-context prompt and agent cache signature, which
rebuilt the agent and invalidated the provider's cached prefix.

Render the Matrix room name, topic and same-sender display name from
the session's persisted origin. Keep live state for routing and room
identity. Member changes join the existing per-turn room notes, which
also report explicit name and topic edits.

A regression reproduced prompt and signature drift on the previous
head. It now verifies stable bytes and signatures after a member
rename, a join, an explicit room rename and a topic edit.
Matrix conversations pin room metadata in their initial prompt. A
reset copied the previous origin, so a new conversation still used an
old room name or topic after a state change. Pass the live Matrix
source through explicit and automatic resets, validate its route, and
persist it as the fresh session origin.

Room state notes were consumed once per room. In rooms with several
threads, the first model turn received a change and the others kept
old prompt metadata without an update. Track delivery per session and
filter notes recorded before a new session began. Keep the room and
session cursor stores bounded.

Regressions cover stable old prompts, fresh reset prompts after a
restart, two existing threads, and a thread created after the update.
Matrix prompts keep their initial room metadata for the life of a
conversation. Room state notes, however, lived only in adapter memory.
A rename, topic edit or member change while the gateway was offline
left a restored conversation with old metadata and no correction.

Persist the room state last delivered to each Matrix session. On the
next prepared model turn, compare that state with the current room
identity and add bounded correction notes to the user message. Merge
live state-event notes by kind so the same change appears once. Hash
joined members and display names to detect membership changes without
storing their list in session metadata.

A real SessionStore restart regression covers two conversations in
one room, stable prompt bytes and one correction per conversation.
A second restart confirms that delivered corrections stay consumed.
A queued Matrix message can contain room metadata from arrival time even
when the room has changed again before the model turn. Resolving the
current identity from the owning adapter makes the correction and the
recorded state describe the same room snapshot. Current state replaces
stale queued notes for name, topic and membership changes.

Room-state acknowledgement was written during message preparation. If
the gateway stopped before the user transcript row was saved, a
restored session suppressed a correction that the model never saw.
Record the state on the user row instead. A failed preparation leaves
no acknowledgement; successful and failed agent turns use their
existing transcript persistence paths.

Regressions exercise a stale queued message and a crash before the
transcript write with a real SessionStore restart.
An earlier commit in this series appended every incoming event's
channel_context when the base adapter merged a text batch, a pending
message or a busy-session debounce. Matrix room notes were the reason,
but Matrix now attaches its notes when the turn is prepared, so the
change only affected Discord, Slack and relay.

Discord backfills a thread message by scanning back to the bot's last
reply. For two batched messages, the second backfill already contains
the first backfill and the first message, so the merged turn showed
the first backfill twice and the first message three times.

Restore the previous merge, which keeps the first event's context, and
add a Discord test for two batched thread messages.
Room context reached the model through Matrix-specific code in the
gateway core. run_inbound imported the Matrix plugin behind a
Platform.MATRIX check, run_turn read an ad hoc _matrix_room_state
attribute, and session.py pinned prompt metadata, validated resets and
kept a member digest on SessionSource for Matrix only. Plugins must use
a generic surface instead.

The Matrix adapter also kept a second mechanism: notes stashed from live
state events with per-session cursors. On the production path the
runner discarded those notes in favour of the snapshot comparison, so
only tests reached them.

Add BasePlatformAdapter.prepare_turn_context. The gateway calls it
while it prepares a turn, passing the session origin and the
channel_state saved with the latest user row. The adapter returns a
TurnContextUpdate with a note for the user message and a new snapshot.
The snapshot travels on MessageEvent.channel_state and is saved in the
user row's display_metadata, so a change is acknowledged only when the
turn that reported it is saved. An adapter that sets
reports_chat_changes_in_turn keeps the session origin's chat name,
topic and user name in the session-context prompt, which stays
byte-stable while the notes report later changes.

Matrix implements the hook with its existing snapshot comparison. The
live-note stash, its take methods and their tests are gone; the
remaining tests run through a real SessionStore. The state handlers
now only invalidate the cached room identity.

build_session_context no longer pins anything, so tools read the
current chat name through HERMES_SESSION_CHAT_NAME, and the prompt copy
uses session_identity.replace_source. reset_session refreshes the
origin's name, topic and user name from the /reset source for every
platform and keeps the origin's routing fields, which removes the
Matrix guard that could raise after /reset had already evicted the
agent. format_untrusted_prompt_value becomes public because the plugin
uses it.

The room-session test helpers retain every SessionStore they create or receive until pytest teardown, then release their shared database handles. This includes restarted room sessions. The finalizer applies only to these fixture helpers; every existing test body and assertion is unchanged.
The room identity read m.room.name, m.room.topic and
m.room.canonical_alias one after another and treated any exception as
"the room has no such state". A timeout, a 5xx or a rate-limit response
therefore looked like a cleared name and topic. The next turn told the
model that the room name was now its room ID and that the topic was
cleared, and saved that snapshot as the acknowledged baseline. The
following successful read then produced a second pair of notes that
restored the real values. The turn hook also forced a fresh read on
every text message, so each reply waited for three sequential,
unbounded homeserver requests.

Only M_NOT_FOUND now means absent state. The identity reads the three
state events and the member list concurrently, and each request stops
after _ROOM_STATE_READ_TIMEOUT_SECONDS. When any read fails the
identity has no room_state snapshot, and prepare_turn_context returns
None: the turn gets no note and saves no channel_state, so the last
saved snapshot stays the baseline. The hook reuses the cached identity
that the adapter resolved for the message's source moments earlier.
The state-event handlers already invalidate that cache when the room
changes, and a restart starts with an empty cache.
Tombstone, encryption, join-rule and history-visibility notes came only
from live state events held in adapter memory. They were marked
delivered when the message was prepared, so a turn that failed before
its transcript row was saved lost them, and a restart dropped any that
had not been delivered yet. Name, topic and member changes did not have
this problem because they were derived from the saved snapshot.

Add the join rule, history visibility, encryption and tombstone to the
room snapshot. The identity reads their state events in the same
concurrent, time-limited batch as the name and topic, and the turn hook
compares them with the snapshot saved on the latest user row. Every
note kind is now acknowledged only when the row that reported it is
saved, and a restarted gateway also reports changes that happened while
it was stopped.

A baseline taken from the session origin leaves these fields unknown,
so the first turn of a session records them without a note. A saved
snapshot without the newer fields still counts as a baseline for the
name and topic.
The room-context layer changed both /new call sites to call
`reset_session(session_key, source=source)`, so a reset refreshes the
origin's chat name, topic and user name on every platform. The Telegram
topic-mode test for /new in a group still expected a call with the
session key alone, and failed with "expected call not found" from
8b5e52e onwards, although it passes on main.

The test checks that /new in a group resets the group's own session key
when DM topic mode is on. Expect the group source as well, which is the
contract that the reset call now has.
…sion

The compression-exhaustion auto-reset passes the turn's source to
reset_session, which copies that source's chat name, topic and user
name into the new session's origin. Internal wakes, such as kanban and
watch notifications, build their source from routing fields only, so
all three are None. When such a wake exhausted compression, the new
origin lost its names on every platform. On Matrix the next system
prompt showed the room ID, and the first human turn reported a rename
against the empty baseline.

Pass the source only when the turn came from a person. An internal
wake now keeps the old origin, as it did before reset_session accepted
a source, and a human turn still refreshes the origin with the current
names.
prepend_turn_context_note called SessionStore.lookup_by_session_key
directly on the event loop for every text turn on an adapter that
implements prepare_turn_context. The lookup acquires SessionStore._lock,
and other threads keep that lock while they save the routing index,
for example during a reset that AsyncSessionStore runs in a worker
thread. While a save was in progress, a Matrix text turn stopped the
whole gateway loop until the save finished.

Await the lookup through runner.async_session_store, which runs it in
a worker thread, as the other inbound callers of this method already
do.
iainlane and others added 12 commits October 2, 2026 17:54
The inbound pipeline is already larger than the project module limit.
Its cache transfer logic has separate filesystem and profile concerns
from the turn orchestration around it.

Move that existing logic into run_inbound_media before changing its
ownership policy. The pipeline and Matrix snapshots import the defining
module, and tests patch the same bindings that production reads. The
six selected ownership and quote contracts retain their behaviour.
A prepared launch-profile turn keeps the file from the quoted event
cache. Routing a later reply to another profile moved that same file
out of the launch cache, so the earlier turn lost its pixels before
model use. The retained event snapshot could not prevent a filesystem
move.

Copy quoted attachments into the routed cache and retain their source.
Publish each copy with an atomic rename so another prepared turn cannot
read a partial replacement. Authored attachments keep their existing
move policy.

The native and text quote contract now checks that every previously
prepared file remains readable throughout launch, routed, routed and
launch turns. Both successful-routing cases failed before the fix; all
four quote cases and the existing authored ownership contracts pass.
When an inbound cache transfer failed or its source file was missing,
the event kept the launch-profile path. Authored attachments then
reached media consumers in the routed profile, even though that profile
had not received the file. Quoted attachments were hidden from vision
but still occupied their original positions in the event.

Remove each failed transfer from the event before media preprocessing.
Preserve the caption and replace an inline reference to the failed file
with an unavailable marker. Rebuild the parallel media lists and quoted
dependency indices so valid siblings retain their original meaning.

The whole-event contract reproduces four failures on the unchanged
parent, with two existing ownership controls passing. All 797 tests in
the combined twelve-file effective-state and media gate pass after the
repair without retries.
The agent can inspect pinned messages but cannot change the pin list.
Add pin and unpin actions through the adapter for the current Matrix
session, with the existing room and requester policy.

Put the action in a separate Matrix administration toolset that users
must enable. Preserve the order of existing pins, serialize concurrent
updates from this adapter, and return the homeserver permission
rejection when the bot cannot write room state.

A separate matrix-nio client observed a rejected update followed by
successful pin and unpin operations that preserved an existing pin.
Focused registry and state tests also cover session scope and concurrent
updates.

Read and update pins through the room inspection context supplied by
this feature's prerequisite. The context applies admission and current
permission checks, preserves existing state content and controls the
interruptible worker. Keep a write outcome unknown once cancellation
or a transport failure prevents confirmation after the request was sent.

Include the interface and cancellation migrations with the initial
feature so this commit uses the current inspection and scheduling APIs.

Upstream compatibility (2026-10-09): keep the current Spotify catalog
plugin move while adding the Matrix administration defaults and pin table.
The removed core Spotify documentation is not restored; the complete
Matrix pin permission and session contracts remain unchanged.

Apply the published NousResearch#125882 fixture lifetime corrections to this
pin contribution. Close executor worker loops after dispatch finishes
and close the main tool loop only when the test created it. Preserve
the existing profile routing, permission fixtures, cancellation
ordering and every behavioural assertion.
_change_pin_state calls before_write() and then awaits
send_state_event under its own 10 second timeout. When that timeout
expired, or the connection dropped after the request was sent, the
exception branch returned "Matrix pin update failed: TimeoutError" or
the exception's class name. The homeserver may already have applied
the change. The tool reports an unknown outcome only when its own
30 second deadline or an interrupt ends the owning task, and a slow
homeserver reaches the inner 10 second timeout first, so the model
could tell the user that the pin failed although it exists.

_change_pin_state now records that it dispatched the write. An
exception after that point without a Matrix errcode returns the same
unknown-outcome result as the tool, with the step to read the pins
before retrying. A Matrix error response is still a definite failure,
and M_FORBIDDEN is still reported as a rejection.

The new test fails a write with a timeout and with a transport error
after before_write ran. Both returned the plain failure before this
change.
A room without a power-levels event gives the creator level 100 and
everyone else 0, but from room version 11 the create content no longer
names the creator. When a server ignores format=event and returns only
that content, the requester's level is unknown. _change_pin_state
refuses the change in that case, but every fixture returned a create
event with a sender, so no test reached the branch. Without the guard
the comparison of None with the required level raises TypeError, and
the change failed with "Matrix pin update failed: TypeError".

The requester test gains an unknown-level row: no power levels and a
content-only version 11 create response. It expects the pin-power
refusal with level None and no write, and it fails when the guard is
removed.
Pin updates read the requester's power before awaiting the current pin
list. If a moderator demotes that requester during the read, the bot
still writes with the earlier permission result. The homeserver checks
the bot's credentials, so the bot's higher power permits the update.
The shared permission reader also fetched encryption and room creation
after power, which could make inspection report an earlier power policy.

Read the pin state before evaluating the write permission, and fetch
power last in the shared permission reader. Pin and unpin now refuse a
requester whose power or required pin level changed during another state
read. Permission inspection derives its message and pin requirements
from the final power response. Existing owner, room and interrupt checks
remain in place. Matrix cannot make the requester check atomic with the
bot's state write, so a change after the final response can still race.

Two parameterized contracts reproduce revocation and raised pin power
during the pin, encryption and create reads, and check current message
requirements in both encrypted and unencrypted rooms. All 16 cases fail
on the original source and pass with the reordered reads.
Matrix room administration used a separately configured client, so
its credentials and room routing could differ from the conversation.
The agent also needed explicit controls for the bot's room lifecycle
without exposing administrative tools in every platform's schema.

Use the owning gateway adapter for opted-in Matrix sessions. Resolve
the profile's tool selection and secrets outside the gateway loop,
keep routing provenance on resumed sessions, and share one owner
snapshot across the Matrix tools. Read raw room state for admission
and power checks, restrict invites to authorized users, and update
joined-room and direct-room state when an operation succeeds.

Treat an interrupted write as an unknown outcome once the request was
sent. Include the live-session and raw-state interface migrations with
the original room lifecycle feature so the initial commit uses the
current prerequisite APIs. A separate client exercises administration
against a disposable homeserver.

Co-authored-by: Iain Lane <iain@orangesquash.org.uk>

The tests dispatch asynchronous tools from executor threads. Each
thread creates a model-tools event loop, which otherwise remains open
after the executor stops. Close that loop in the dispatch helper after
the operation and its cancellation cleanup finish. All existing
ownership, permission and cancellation assertions remain unchanged.
Administration decides at the start whether the room is a direct
chat or a group room, and every later check used that first answer.
_AdminContext.access() re-read the classification but returned the
new value without comparing it, and the checks after the permission
reads used the chat type from the first access() call.

Two refusals depend on that classification. A room outside
MATRIX_ALLOWED_ROOMS is admitted only while it is a DM, and leave and
forget skip the kick level when the requester is the only other
member. If a third member joined while the power levels or the
member list were being read, an invite or redaction was written in a
room that the allowlist excludes, and a level-0 member could make the
bot leave a group room. The pin path already refused this, because
its inspection context re-runs the access check after every read.

access() now takes the expected chat type and refuses a different
current classification with "Matrix room changed between a direct
chat and a group room". administer_matrix_room passes the first chat
type to the second access() call and calls access() again after the
permission reads, before the final selection check. The pin recheck
passes it as well.

The new test flips the room from a DM to a group room during the
power-level read (invite, redact and pin, with an allowlist) and
during the member read (leave below the kick level). Before the fix,
invite, redact and leave were written; pin was refused.
The pins parent now reports an unknown outcome when the pin write
fails after the request may have reached the homeserver. Room
administration had the same gap. Each mutation calls before_write()
and then the client call, and an exception from that call without a
Matrix errcode, such as a timeout or a dropped connection, returned
"Matrix administration failed: <type>". The homeserver may already
have created the room, sent the invite, left or forgotten the room,
or redacted the event, so the model could report a failure for a
change that happened.

administer_matrix_room now records when a mutation calls
before_write(). An exception after that point without an errcode
returns the unknown-outcome result with the action's next step. A
Matrix error response is still a definite failure with its errcode
and message. The per-action next steps move from the tool into
room_admin so that both use one table; the tool imports it when it
runs, as it imports the runtime.

The pin path had a related case in this layer: the ValueError branch
for the pre-write recheck also caught a ValueError raised by the
write itself. The branch now wraps only recheck_before_write(), so
any failure of the write goes through the parent's unknown-outcome
handling.

The new test fails the client call with a timeout and with a
transport error for each of the five actions. All ten returned the
plain failure before this change.
Administration evaluated requester power before awaiting encryption,
room creation, room access and profile selection. A moderator could
revoke that power during those reads, but the bot would still send the
mutation with the earlier decision. The homeserver checks the bot's
credentials, so it cannot enforce the requester's revoked permission.

Read power after the other permission metadata. Keep the initial refusal
and room checks, then evaluate current power again after the final
selection and access checks. The final decision has no later awaited
preflight work before dispatch. This preserves the refusal when a direct
chat becomes a group room during the initial permission reads.

Pin administration also awaited its membership and selection callback
after the parent's power check. Run that callback before reading pin
state and evaluating permission. Preserve a precise pre-dispatch refusal
if the final owner/interrupt guard raises ValueError. A write that has
already started still uses the existing unknown-outcome handling.

One parameterized contract demotes the requester during metadata reads
and final selection across invite, redact, leave, forget, pin and unpin.
Fourteen cases fail on the replayed source; four controls pass because
the repaired pin parent already reads power after metadata. All eighteen
pass after this change. Matrix cannot make a requester read atomic with
the bot's write, so a change after the final response can still race.
A homeserver can complete a mutation and then return malformed JSON.
Mautrix lets the decoding error propagate from a successful HTTP
response. JSONDecodeError inherits ValueError, so administration's
validation handler returned a plain failure even after the write marker
had run. Retrying room creation could then create another room.

Check the dispatch marker before classifying a ValueError as a refusal.
An error without a Matrix error code after dispatch now uses the existing
unknown-outcome result and the action's next step. Validation and policy
refusals before dispatch keep their existing messages. Completed writes
that fail the later ownership check still return their result and warning.
No automatic retry is introduced.

Extend the existing timeout and transport-error contract with a JSON
decoding error. All five actions return a plain failure before this
change and the unknown-outcome result afterwards.

A native contract forwards the invite through the real Mautrix decoder
to disposable Synapse, then substitutes invalid JSON in the accepted
response. Before the fix, Synapse records one invite and the tool returns
a definite failure. The contract requires the unknown-outcome result and
confirms that the invite is not retried.
@iainlane
iainlane force-pushed the codex/matrix-runtime-lifecycle branch from c3f53f9 to eee764b Compare October 2, 2026 20:34
The administration fixture replaces the room classifier with a function
that accepts only a room ID. Discovery now passes the fixed client owner
and classification requirement to that function. The allowlist helper
catches the resulting TypeError and refuses the room before the test can
change the membership during its power-level read.

Accept the production keyword arguments in the fixture. Keep the exact
original refusal and empty-write assertions for invite, redact, pin and
leave. All four cases pass on both the original prerequisite and the
composed control feature when the real late-reclassification path runs.

Historical source attribution for audit record 435.

The fixture correction from original source
a8ac3c29caef8bc2fd01e05912186da3828deb0f is already folded into
public owner 44e8969 on PR NousResearch#126289.
This empty child records the original contribution's attribution.
The source tree and all existing contributions remain unchanged.
Four routing tests pass a recording transport to RelayAdapter, whose
transport parameter requires the RelayTransport protocol. The fixture
only implements its inbound setter and outbound send, so the four
constructor calls fail static type checking even though their exercised
routing paths work.

Implement the complete structural protocol in the recording fixture.
Unused operations raise an assertion error so an unexpected call fails
the test. Keep the recorded action, platform and send result, and narrow
the captured dictionaries before the existing routing assertions.

The correction changes the inherited test fixture. It preserves the
existing contributor records and all original routing assertions.
The encrypted-file fixture returned a constant serialized payload, so
assigning its uploaded URL did not appear in later serialization. The
Matrix SDK serializes the URL stored on the encrypted-file object. The
adapter currently overwrites the outgoing URL, which concealed this
fixture difference in the existing upload test.

Read the fixture URL when serialization is called. Add an invariant
that compares the complete payload before and after URL assignment,
with only the URL overwritten. Keep the existing upload and routing
assertions.
Matrix receives ordinary m.reaction events, but reads and catch-up only
showed message events. The bundled m.relations summary cannot provide
annotation reactions.

Read bounded annotation children through the live Matrix client.
Decrypt encrypted reactions to verify their type, but use the outer
cleartext m.relates_to for the target and key even if the decrypted
payload specifies another relation. Discard redacted events and
deduplicate by sender and key. Present the actor and target in read and
catch-up context without starting a turn. Mark clipped keys and report
incomplete reads when keys, API failures or bounded pages prevent a
complete snapshot.

A five-second batch deadline keeps a slow homeserver from delaying a
turn for every earlier message. Completed lookups remain visible when
others time out. A separate Matrix client confirms that a reaction
appears in the next mentioned turn. Focused tests cover duplicate and
redacted reactions, encrypted events, long keys, lookup errors, and
the deadline.

Bind the real room-policy predicate in the read fixtures. Read admission
rechecks room policy after awaiting room identity, so the fixture must
provide the same adapter protocol as the production path.
Source-permalink and reaction-context branches each test their own prompt
contract. Their combined adapter adds the trigger permalink to catch-up
messages, so the reaction fixtures need the SDK's asynchronous state-store
methods and complete expected source notes.

Merge both feature histories without changing their existing commits.
Preserve reaction author trust, room and thread context, transcript roles,
cache privacy and the source-permalink encoding and routing rules. Use the
real MemoryStateStore in the affected fixtures and retrieve SessionSource
from the final context tuple item. Check each complete catch-up prompt
against its explicit room, trigger event and via server, including the
first event of a batched turn.

The four composed gateway files cover 343 cases. The affected two files
pass all 264 cases after the fixture adaptation; the unchanged session and
project-isolation files passed their 79 cases. Scoped Ruff reports no lint
diagnostics. Existing formatter differences remain qualified separately.
Live Docker room and thread backfill validation is still outstanding.

This dependent layer preserves both parent histories. Its two live
backfill fixture adaptations do not claim complete original 858 delivery.

Historical composition qualification

The preceding message records the original two-parent composition.
This successor reapplies its fixture integration after the individually
preserved prerequisite, SDK and reaction contributions on captured main
648a309. Its history has one parent.
The original composition message remains complete for provenance; this
commit does not create a merge or establish live Matrix acceptance.
The original feature tests predate the canonical approval lifecycle,
permalink source fields, room policy owner argument and SDK send error
classification. Their doubles omitted asynchronous state-store reads,
exact approval request IDs and terminal card edits. Several complete
expected events also omitted the newly composed source and mention
metadata, so the full feature gate failed before checking its intended
behaviour.

Update those fixture bindings and complete expected values for the
current defining owners. Keep every original parameter row, delivery
attempt, payload, cancellation and authorization assertion. Thread
reconnect expectations retain confirmed delivery and cache identity.
Catch-up expectations include the permalink note before the new message.

The complete nine-file feature boundary passes all 808 cases with the
actual installed Matrix SDK, four workers and no file retries. The full
Matrix file finishes in 30.7 seconds, including the event-loop lookup
contract that timed out on the earlier incomplete source composition.

Original858 public source qualification

The preceding 808-case result records the original composition. It does
not describe a new run against this public tree. The published backfill
composition has 343 scoped cases, using 264 final affected cases and 79
unchanged earlier successes. Its inherited formatter and type findings
remain qualified, and live Docker validation remains pending.

The other feature contracts are represented by their existing public
owners. Their fixture expectations follow the actual approval, notice,
location, send-classification, read-policy and delivered-event APIs.
Expectations from the original later composition do not establish bugs
in standalone owners that use an earlier API. No combined six-module
endpoint or repeated native gate is required for this source credit.

This source-neutral contribution preserves the complete original
message and both original contributor dates. It records the distributed
source qualification without changing the published source tree.

Current linear successor qualification

The 343-case result above describes the earlier a76019d composition.
This contribution follows the verified single-parent successor a83881c
and keeps its source tree unchanged. The accepted seven-file gateway
gate for that successor passes 362 cases. Live Matrix validation,
private integration and whole-campaign completion remain separate.
The Matrix SDK retries rate-limit responses and connection timeouts
indefinitely by default. A failed request during account or room setup
can therefore stop the whole native module until the outer runner kills
it, without reporting the setup error.

Disable those retries and bound ordinary fixture requests to fifteen
seconds. Apply the same configuration to registration,
restored accounts and the encrypted observer. The fixtures now expose
the SDK error so the failing operation can be diagnosed directly.

A real loopback HTTP server reproduces the repeated rate-limit requests
through registration and restored accounts. Both paths now make one
request and return the expected error.
Matrix turns could read message history, but the agent could not
inspect the current room state, joined members, power levels or pinned
messages. Those details are available from the receiving bot and can
affect what the agent should say or attempt.

Extend the Matrix-only read tool with bounded inspection kinds. Fetch
state and members through the owning adapter, apply its existing room
and requester policy, and resolve pinned messages with the same
decryption path as event reads. Report missing pin content per event
so the other pins remain visible. Include the event type and creator
override when presenting permission thresholds.

A separate Matrix client verifies that the model receives the room
details and pinned message from a disposable Synapse instance.

Use the effective-state parent's client-events decoder and gateway
scheduling path throughout inspection and its tests. Recheck admission
after identity resolution, derive permission defaults from raw state,
and preserve cancellation while refreshing pinned content. The feature
and these interface migrations form one change so the initial commit
uses the current prerequisite APIs.


Published read fixture worker-loop lifetime correction:

The read-tool tests dispatch through executor threads. Each worker can
create a thread-local event loop, but the fixture leaves that loop open
when the worker exits. Its selector sockets then produce strict resource
failures. The complete standalone file reproduces two such failures.

Use the published read fixture dispatcher for all seven worker calls.
It closes the worker loop in the same thread after dispatch returns or
raises. Keep every behavioural assertion and the existing cancellation
sequence unchanged. Production code and the prerequisite history do not
change.

@kvnloo kvnloo left a comment

Copy link
Copy Markdown

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Scoped exact-layer review (95638fbc → 6743967b, one commit / two files): no blocking findings. The new loop-loss path distinguishes pre-dispatch failure from a dispatched write (write_started => unknown outcome), preserves a task result that finished on the owner loop before the cross-thread future could publish it, and sets the persistent interrupt before returning so a preflight that resumes later cannot issue a fresh write. The five-stage separate-loop regression covers admission, preflight, dispatched, confirmed, and recovery cases. This review is only for the PR’s own layer; the 115-file stacked diff still depends on #125697/#126289 and their review state.

@iainlane
iainlane force-pushed the codex/matrix-runtime-lifecycle branch from 6743967 to 939b2ee Compare October 8, 2026 08:20
The fake HTTP API discarded the client_session supplied by the Matrix
adapter, so disconnect closed a mock while the real aiohttp session was
left open. A reconnect test also replaced disconnect with a stub and
left its new connection open after checking the guard. Strict resource
warnings expose both leaks.

Preserve the supplied session in the fake API. Use a wrapping disconnect
spy in the reconnect test and disconnect the replacement client after
the existing call assertion. The fake API and reconnect fixture predate
this branch, so this focused fixture maintenance is separate from the
corrections folded into their campaign contributors. Production code
and all original assertions are unchanged.

On the earlier Matrix prerequisite, prebuilt Client stubs also discarded
the supplied API. Use the same scoped constructor for the inherited
fixtures so disconnect closes the adapter-created session. This is the
earlier fixture layout; all original assertions remain unchanged.
Matrix requests can outlive gateway teardown. A stopped but open owner
loop accepts a threadsafe coroutine, yet cannot run its cancellation
callback. The tool's deadline then waits indefinitely for the owner to
drain. A result may also be complete before the stopped loop copies it
back to the worker.

Reject unavailable loops before admission and monitor loop availability
while waiting for both the result and cancellation. Keep a confirmed
owner-task result. Report an unknown outcome if a write was dispatched
before loop loss, and return a normal unavailable-loop error otherwise.
Cancellation remains queued for the owner and sets the existing interrupt
flag, so an operation cannot resume into a new write after recovery.

A real separate-loop contract reproduces four failures before this change
and preserves the confirmed-result control. All five cases and the
adjacent administration and pin tests pass, 264 tests in four files.
@iainlane
iainlane force-pushed the codex/matrix-runtime-lifecycle branch from 939b2ee to 9859260 Compare October 10, 2026 02:07

This branch has not been deployed

No deployments
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

comp/gateway Gateway runner, session dispatch, delivery comp/plugins Plugin system and bundled plugins P3 Low — cosmetic, nice to have platform/matrix Matrix adapter (E2EE) sweeper:risk-automation Sweeper risk: may affect CI, automerge, label sync, or maintainer automation sweeper:risk-compatibility Sweeper risk: may break existing users, config, migrations, defaults, or upgrades sweeper:risk-message-delivery Sweeper risk: may drop, duplicate, misroute, or suppress messages sweeper:risk-session-state Sweeper risk: may lose/corrupt/mis-associate session or context state type/bug Something isn't working

Projects

None yet

Development

Successfully merging this pull request may close these issues.

8 participants