Repository navigation
Conversation
Matrix permalinks change with each triggering event. Rendering one in the pinned session prompt made the prompt stale or changed its bytes every turn, which breaks conversation prompt caching. Put the link in the current model-facing user message and remove it from the durable transcript, including queued turns. Respect effective PII redaction when adding the note. Percent-encode room IDs, event IDs and the via server so delimiters cannot change the matrix.to URL structure. Preserve the Matrix and Discord source-note handling when integrating the upstream prefix removal change. Use removeprefix after selecting the platform-specific note, so each transport removes only its own generated prefix.
The live Matrix exchange now checks the model request for a permalink to the event sent by the independent client. It also verifies that the link is absent from the system prompt, which must remain stable across turns. Record the exchange duration separately from container setup time.
Room version 12 IDs have no server component. A permalink built only from the room ID therefore omitted the server hint, which can prevent a client outside the room from locating it. Use the joined bot account's server when the room ID has no domain. The builder still leaves via absent when neither server is known.
A room ID can identify a server that no longer participates in the room. Its server may also include a port or IPv6 address, which the last-colon parser truncated to an invalid via hint. Use the joined bot account's server for inbound source links. When only a room ID is available, parse its full server after the first colon. The same parse also corrects the room's session scope.
In a thread, every turn's source link pointed at the thread root. The root was chosen as a stable anchor while the link lived in the pinned session prompt, which is per thread. The link now goes on each user turn, so the reason no longer applies, and the model never received the ID of the message that triggered the turn. A request to cite "this message" produced a link to the root, which may be another user's message. Build the per-turn link from the triggering event in all cases. A matrix.to link to an event inside a thread opens that thread in current clients, and the session prompt already gives the thread root.
A room-ID permalink needs via servers so that a reader outside the room can join through one of them. The link carried only the bot's server. When that server was unreachable from the reader's server, the link failed. Choose up to three servers as the Matrix specification recommends for room-ID permalinks. The first is the server of the highest-power joined member at power level 50 or above; room version 12 creators count as the highest power. The rest are the servers with the most joined members, and IP literals are never chosen. The adapter reads members, power levels and the create event from the mautrix state store, which sync keeps current, so building a link makes no request. When the store has no joined members for the room, use the bot's server and then the room ID's server. Server ACLs are not applied, because the state store does not keep m.room.server_acl. Move the builder and the selection into a sibling module, and emit one via parameter per server.
The session prompt stopped rendering the Matrix source link when the link moved to the current user turn. Two prompt tests still asserted that the prompt omitted the link when it was unset or redacted. No code path produces that output any more, so neither test could fail because of this feature. The prompt-stability test covers the remaining contract, and the per-turn tests cover the note and its PII suppression. Restore the comment on the per-turn notes. strip_inbound_source_note removes a note only when it is the first thing in the message, so a later prefix added outside the notes would leave the note in the persisted transcript.
Choosing source-link via servers made every inbound message await get_power_levels and get_create on the client's state store. The project-context isolation and voice tests gave the adapter a MagicMock store with only get_members awaitable, so all six inbound tests in those files failed with "'MagicMock' object can't be awaited". Production always uses mautrix's MemoryStateStore. Give both fakes a real MemoryStateStore in which the bot and the sender have joined the test rooms. The real store implements every state query, so these tests keep working when the adapter starts reading other room state. The stores are filled with MemoryStateStore.deserialize because the helpers are synchronous and some are called inside a running event loop.
The effective-state parent added a description to the matrix_read limit argument: "Maximum number of events to read". In this layer the same argument also bounds the members kind, which returns members rather than events, and the pins kind. Say that the limit applies to events or members so the model can tell that it bounds member lists too.
The merged matrix_read description said that events are listed oldest first and that `errors` lists events that could not be decrypted. Both sentences came from the history reads. A pins read returns events in the order of the room's m.room.pinned_events list, and its errors also cover pins that could not be fetched, are not visible messages, came back as another event, or whose edit was redacted. A model that read the description could tell a user that a deleted pin was an encrypted message. Limit the ordering sentence to room and thread reads, say that pins follow the room's pinned order, and say that `errors` lists events that could not be read or decrypted, with the reason. History reads already returned non-decryption errors such as "event content changed", so the wider wording is also accurate for them.
Matrix thread continuations include an m.in_reply_to pointer for clients that do not render threads. The adapter treated that fallback pointer as a user reply, so it attributed the continuation to the thread root. Parse relations into explicit reply, thread fallback and edit fields. Use the parsed reply target for inbound context and the thread root for session routing. Malformed relation content now yields an empty relation instead of failing message handling.
A Matrix reply may omit the legacy inline quote. The gateway then knew only the target event ID and could not tell the agent what message the user referenced or who wrote it. Resolve reply targets from a bounded cache or the room event API. Apply edits to cached text and keep redactions as empty cache entries so a later reply cannot revive deleted content. Decrypt fetched events when the room uses E2EE, and label media bodies so filenames are not treated as spoken text. Attach the target author's name and authorisation verdict to the inbound event. The gateway marks fetched content from an unauthorised author as unverified background and renders Matrix quotes on one line. The adapter never fetches a remote target for a sender it rejects. Co-authored-by: Wessel <2385844+weebl2000@users.noreply.github.com> Co-authored-by: uuedxz <249543386+uuedxz@users.noreply.github.com> Co-authored-by: Simon Sharp <234941529+simonamsharp@users.noreply.github.com>
An explicit Matrix reply could identify its parent but gave the agent only a filename when that parent was an image. Threaded messages in a fresh gateway session lacked the earlier messages that explain them. Resolve reply targets through a bounded room-event cache, decrypting when needed. Download quoted images through the existing Matrix media path and attach their cached files to the triggering message. Read the thread root and recent relations for a new threaded session, then attribute and bound the prior messages before adding them to the user turn. Add external thread text after local @ reference expansion so it cannot trigger a file read. Use mautrix.api.Method.GET for the relations request because mautrix 0.21.1 treats a string method as a non-GET request and adds a body. Cache successful outbound messages by event ID for immediate replies, and preserve reply context when a later text chunk joins a batch. Mark inline fallback quotes as unverified because their author pills come from the sender and can claim the bot's identity. Expose a config.yaml limit for thread backfill and preserve room state context when both appear. Co-authored-by: Wessel <2385844+weebl2000@users.noreply.github.com>
The quoted media path and new-session thread backfill had focused unit coverage, but the tests did not exercise a media event that referenced another message without an inline quote. They also did not prepare two thread sessions from the same room. Drive both media replies through the adapter, including a failed parent fetch, and prepare separate thread turns through the gateway. These cases establish that an unavailable parent does not drop the media and that each thread receives its own earlier context. Co-authored-by: Simon Sharp <234941529+simonamsharp@users.noreply.github.com>
Legacy thread continuations can include a quoted fallback without an explicit reply target. Strip that quote before admission and command normalisation so quoted text cannot enter the current turn and a following command remains actionable. Thread backfill cached images as text. A later explicit reply then used the text entry without trying to load the image, even after a failed download. Mark cached images and retry their media load when a reply needs it. A successful outbound edit left the original event cached at its first sent fragment. Update that event only after the edit succeeds. Keep a redaction tombstone authoritative when a late replacement arrives, so an edited message cannot revive redacted content. Five red invariants now pass, including root and child image cases.
Unit tests cover relation parsing and thread backfill in isolation, but they cannot prove that a second Matrix client delivers those events through the live gateway into separate model conversations. Send two thread roots and legacy fallback continuations through the disposable Synapse service. Wait for each final reply, then inspect the fake model requests. The test checks that quoted fallback text is removed and each request contains only its own thread root and turn.
A reply lookup or thread backfill can begin before a redaction and finish after it. The cache rejected the stale result, but each caller still returned or appended its local copy. The model could therefore see redacted text or media. Return the accepted visible entry from the cache and use it for reply resolution and thread backfill. Recheck the cache after a failed lookup so an earlier cached image cannot bypass a concurrent redaction.
The reply pointer identified the quoted author on every platform, falling back to reply_to_author_id when no display name was available. WhatsApp and Signal can fill that field with the quoted participant's phone number, so a group reply sent a third party's number to the model even with privacy.redact_pii enabled. Matrix reply quotes were also cut to 500 characters and collapsed onto one line, which undid 6efca62 and dropped the end of long tracebacks, lists and code. Identify the author only when the adapter set reply_to_author_authorized, which only the Matrix adapter does after checking the quoted sender. Other platforms get the unattributed pointer again. When an attributed pointer has only an ID, the ID goes through _should_redact_pii and _hash_sender_id like the session context prompt. The quoted text is kept whole on every platform, and only the author label is reduced to one line.
Element and Element X set is_falling_back on every ordinary thread message and do not add a body fallback to it. The adapter treated every such message as containing a legacy reply fallback and removed all leading "> " lines, so a user who began a thread message with a quote lost the quote. The body was stripped a second time while resolving reply context, which also removed a user's quote that followed a real fallback. A legacy fallback always starts with the quoted sender's pill, as "> <@user:server>" or "> * <@user:server>" for an emote. Strip the leading quote block only when its first line has that form, and strip it once, when the message context is resolved.
MatrixEventContextCache.apply_edit compared the editor with the original sender only when the original event was already cached. For an older, evicted or pre-restart event, it stored the edit's m.new_content under the target's event ID with the editor as sender. Reply resolution and thread backfill then used that entry and did not fetch the real event. Any room member could therefore replace the quoted text of someone else's message or of a thread root. The Matrix spec requires clients to ignore an m.replace from a different sender. Accept an edit only when the original is cached and has the same sender. Otherwise leave the cache unchanged, so the next resolve fetches the event and reads the replacement that the server bundles, which is limited to the original sender's edits.
When a message arrives while a turn is running, the base adapter merges it into the pending event with merge_pending_message_event. The text-batch merge absorbed the incoming reply context, but this sibling path did not. A text reply to an image now has the quoted image in media_urls, so it takes the media branch: the image was kept, while reply_to_message_id and reply_to_text were dropped. The model saw an unexplained image and no reply pointer. Plain text replies merged into a pending text event lost their reply context in the same way. Absorb the incoming reply context in both merge branches, as the text-batch merge already does.
Matrix batches quick text messages into one turn, and the merged event keeps the first chunk's message_id. When thread backfill read the thread's relations, it excluded only that ID. If a user sent two quick messages into a thread without a session, the second message appeared under "[Earlier messages in this thread]" and again in the current turn. Record the IDs of the events merged into a pending event, in the text batch and in both busy-session merge branches, and exclude all of them from the backfill.
Reply resolution and thread backfill decrypt m.room.encrypted events when the client has the megolm session keys, but no test reached either branch. Thread backfill deserialises the raw event with mautrix, and the earlier test invocation could not import mautrix, so a test of that path would have skipped the event without failing. Drive both paths with real mautrix events and a crypto fake that decrypts only when the keys are available. A decrypted event supplies the quoted text or the thread entry, and a missing session leaves it out. Replacing either decryption call with a no-op fails all four cases.
The gateway core fetched earlier thread messages for a new Matrix thread
session itself. `_prepare_inbound_message_text` checked
`source.platform == Platform.MATRIX` and `hasattr(adapter,
"fetch_thread_context")`, which put Matrix-specific code in the runner
beside the generic `prepare_turn_context` hook that the room-context
layer added for room-state notes.
The two paths also disagreed about the `[New message]` marker. The
thread history went in front of the message text, and the room note
with its marker was prepended afterwards. When a room note was pending
and a message started a new thread session, the model received
<room note>
[New message]
<reply pointer>
[Earlier messages in this thread]
...
<message>
so the marker labelled the thread history as part of the new message.
Move the thread fetch into `MatrixAdapter.prepare_turn_context`. The
hook gains a `first_turn` argument, which the core sets when the
session transcript is empty, so an adapter can add history that only a
new session needs. The Matrix adapter returns the room note followed by
the thread history as one note, and the core puts the single
`[New message]` marker after all of it, before the reply pointer and
the message. The marker also follows thread history when no room note
is pending, as it already follows the Discord history backfill.
`TurnContextUpdate.channel_state` becomes optional. The adapter reads
room state only for text messages, and a failed read gives no
snapshot, but thread history is still reported in both cases; a `None`
snapshot leaves the saved state unchanged.
The core no longer checks the platform here, and any adapter can
report first-turn context through the same hook.
After the merge of NousResearch#128621, the adapter had two rules for one question: whether a leading "> " block is a legacy reply fallback or the user's own quote. NousResearch#128621 added _has_reply_fallback, which requires an <mx-reply> element at the start of the HTML body, or a plain quote block that starts with the quoted sender's pill and ends with a blank line. This branch's _strip_thread_reply_fallback used its own copy of the pill regex and checked the pill alone. The explicit-reply path in _extract_reply_context still treated any leading "> " as a fallback, because this branch's signature passed only formatted_body, and _has_reply_fallback needs the whole content. Use _has_reply_fallback for thread messages and for explicit replies, and remove _strip_thread_reply_fallback and its duplicate regex. _extract_reply_context now takes source_content as a positional argument before relates_to, as in NousResearch#128621, and reads formatted_body from it. The sender and chat_type keywords remain. Two test expectations change: - In NousResearch#128621's test_reply_strips_only_a_reply_fallback, the thread-message-own-quote case now expects no reply_to_message_id. This branch treats m.in_reply_to under is_falling_back as the thread's fallback target, not an explicit reply, so the event is not a reply. The message text, which is what that case checks, is unchanged. - test_thread_message_strips_only_the_reply_fallback gains a case for a thread message whose quote starts with a pill but has no blank line after it. The pill-only rule stripped that quote; the shared rule keeps it as the user's text. The shared rule also strips a thread message's plain quote when the HTML body starts with <mx-reply>, even without a pill, as it already does for explicit replies.
Two helpers check whether a reply's HTML body starts with an <mx-reply> fallback. _has_reply_fallback decides whether the plain-text quote is stripped from the message, and extract_mx_reply_quote returns the quote as reply_to_text. Their checks disagreed. _has_reply_fallback required format "org.matrix.custom.html" and a bare "<mx-reply>" tag. extract_mx_reply_quote ignored format and accepted any body that started with "<mx-reply". When the tag had an attribute, as in <mx-reply data-x="1">, or the event had no format field, and the plain-text quote had no pill, the message kept the quote and reply_to_text repeated it. The model then saw the quote twice. Add starts_with_mx_reply(content) to reply_context.py and call it from both helpers. It requires the HTML format and matches an <mx-reply> start tag with or without attributes, because the spec defines the fallback as a formatted_body that "begins with an <mx-reply> start tag". The match ignores case because the quote extractor's HTML parser lower-cases tag names, so the check accepts every tag that the extractor reads as <mx-reply>. extract_mx_reply_quote now takes the event content instead of the formatted body because it needs the format field.
Busy-text debounce updates the reply anchor to the latest inbound message but did not record the earlier message IDs. A new Matrix thread session therefore fetched an earlier trigger as thread history and included the same text again in the current user message. Record each previous anchor and incoming batch's merged IDs before the anchor changes. Thread-history preparation can then exclude every event already represented in the current turn. Input without a new anchor keeps its existing anchor and still contributes its merged IDs. Extend the real model-preparation invariant to both ingress and busy batching. The busy case repeated the first trigger before this change and now excludes it. Independent full-event expectations also reproduce six missing-ID cases and pass after the fix. The pending-merge contract checks unknown, unauthorized and authorized quoted authors together with media and merged IDs.
The room-context prerequisite adds channel_state as the final positional MessageEvent constructor argument. Threading inserted two fields before that argument, so an existing positional room-state dictionary became the reply-author authorization value and the event lost its room state. Append the new threading fields after channel_state. Existing positional construction keeps its original meaning, and the new authorization field defaults to None. A direct constructor probe reproduces the old misassignment and passes after reordering; all 330 tests in the seven adjacent event, thread and gateway files pass.
Resolved reply parents, cached edits and thread history removed every leading Markdown quote block. Modern Matrix clients send replies without a fallback, so an ordinary user-authored quotation disappeared from the context before the model received it. Move the adapter's existing fallback detection and splitting helpers into the shared reply-context module. All context paths now strip only a recognized legacy sender pill or a formatted mx-reply fallback. Edits use the replacement content to identify its formatting correctly. One parameterized behavior contract checks all three context paths against user quotations and both legacy fallback formats. The three user-quotation cases fail on the original head and pass after the fix.
A busy-debounced turn can begin with the thread root and finish with a later reply. The event includes both IDs, but thread backfill excluded only relation chunks and always appended the separately fetched root. The model received the root text twice: once in history and once in the current message. Check the excluded event IDs before resolving the root, just as for thread replies. Extend the real intake, debounce and model-preparation contract to batches that begin with their thread root. The busy case reproduces the duplicate on the original head and passes after the fix; ordinary thread roots still appear once in earlier history.
The Telegram adapter contains the teardown of text, photo and album buffers in a large transport facade. Reply-boundary deliveries need the same teardown policy. Move the existing cancellation method into TelegramDelayedDeliveryMixin before extending it. Keep the method body and adapter method resolution unchanged, and type the mixin receiver.
A reply boundary starts a flush outside the ordinary delivery maps. Delivery teardown therefore misses a boundary that has already popped its event and is awaiting dispatch. Cancelling a boundary before its coroutine starts can also lose the event in its private temporary map. Keep boundary events and tasks in a dedicated buffer. Include that buffer in delivery cancellation, salvage and clearing. Cancellation requeues an in-flight event through the existing flush body; buffered boundaries are salvaged before the later ordinary text batch. The event-synchronised regression reproduces the missing earlier batch and unfinished task before the fix. It checks both complete events and that the boundary task has exited when teardown returns.
When busy_text_mode=queue and a debounce state for sender B is stuck in the store (timer fired but the pending slot belongs to sender A, a different sender), any message from a third sender C hit an early `return` without being saved to the debounce store, pending_messages, or anywhere else — permanently and silently dropped. Remove the `_can_merge(existing_pending, event)` guard from the inner conflict branch in `_queue_text_debounce` and unconditionally call `merge_pending_message_event` instead. C's text is appended to the pending slot (or placed there directly if empty), matching the pre-debounce behaviour that always queued follow-ups regardless of sender. Sender attribution is best-effort when 3+ senders collide simultaneously; losing the message entirely is worse. Adds a regression test that pins the 3-sender scenario: A in pending, B stuck in the debounce store with task=None, C from a third sender — C must survive in _pending_messages and B must remain in the store for _drain_pending_after_session_command to recover. (cherry picked from commit 2a7e3c5) Relocation onto pinned upstream 343500: The current copy inserts the pending-event merge call before the third-sender branch returns. The preceding extraction moved _queue_text_debounce to BaseTextDebounceMixin. Apply the same insertion in gateway/platforms/base_text_debounce.py and retain its typed receiver. BasePlatformAdapter inherits this method. Keep the existing Base adapter and the rest of the extracted module unchanged.
Four routing tests pass a recording transport to RelayAdapter, whose transport parameter requires the RelayTransport protocol. The fixture only implements its inbound setter and outbound send, so the four constructor calls fail static type checking even though their exercised routing paths work. Implement the complete structural protocol in the recording fixture. Unused operations raise an assertion error so an unexpected call fails the test. Keep the recorded action, platform and send result, and narrow the captured dictionaries before the existing routing assertions. The correction changes the inherited test fixture. It preserves the existing contributor records and all original routing assertions.
The encrypted-file fixture returned a constant serialized payload, so assigning its uploaded URL did not appear in later serialization. The Matrix SDK serializes the URL stored on the encrypted-file object. The adapter currently overwrites the outgoing URL, which concealed this fixture difference in the existing upload test. Read the fixture URL when serialization is called. Add an invariant that compares the complete payload before and after URL assignment, with only the URL overwritten. Keep the existing upload and routing assertions.
Matrix receives ordinary m.reaction events, but reads and catch-up only showed message events. The bundled m.relations summary cannot provide annotation reactions. Read bounded annotation children through the live Matrix client. Decrypt encrypted reactions to verify their type, but use the outer cleartext m.relates_to for the target and key even if the decrypted payload specifies another relation. Discard redacted events and deduplicate by sender and key. Present the actor and target in read and catch-up context without starting a turn. Mark clipped keys and report incomplete reads when keys, API failures or bounded pages prevent a complete snapshot. A five-second batch deadline keeps a slow homeserver from delaying a turn for every earlier message. Completed lookups remain visible when others time out. A separate Matrix client confirms that a reaction appears in the next mentioned turn. Focused tests cover duplicate and redacted reactions, encrypted events, long keys, lookup errors, and the deadline. Bind the real room-policy predicate in the read fixtures. Read admission rechecks room policy after awaiting room identity, so the fixture must provide the same adapter protocol as the production path.
The concurrent-update fixture creates three handler coroutines before passing each to the processor. Cancelling the middle update while it waits for its predecessor leaves that handler coroutine unawaited, so the fixture fails under strict RuntimeWarning checks. Keep references to the fixture-created coroutines and close the cancelled handler after every processor task has finished. The existing assertions still require the first and third handlers to complete in order without executing the cancelled handler.
Direct FIFO callers such as /queue, /steer fallback and Telegram grace could bypass the 32-entry bound. Normal busy admission only logged a refusal, while /queue acknowledged input even when no event was queued. Debounced text was also outside the pending count, so a later photo or command could overtake an older accepted burst. Check admission at the shared FIFO boundary and return its result to command and busy handlers. Count buffered bursts as pending input and submit older bursts before accepting a later FIFO event. Refused external input receives a thread-aware queue-full reply; internal wakes remain unaccepted so their existing producers can retry. Earlier accepted events are never replaced to admit the new input. Copy command events without reconstructing their fields. This preserves reply, media and control context together with the existing bot-loop admission flag, so replay cannot acquire permissions or spend admission twice after stripping the command prefix. Parameterised contracts reproduce bypassed capacity, false acknowledgements, missing refusal notices, context loss and overtaking of buffered input. fix(i18n): translate the pending queue refusal The bounded pending queue reports an English-only warning when admission is refused. The other sixteen bundled language catalogues omit that key, so users receive the English fallback even after selecting their language. Add the refusal message to every bundled catalogue. Verify that runtime lookup selects the requested catalogue, that non-English results differ from the English fallback, and that all catalogue keys and placeholders remain in parity. The runtime contract failed for all sixteen translated languages before the entries were added. The complete four-file locale gate now passes all 92 cases. Source-commit: cd37cd3a25ab9c157495df61ed3a8049765e90ab Protected-source-alias: b01a5942e1b077ee7dcd7ad73f3cc0002f926ae1 Maintenance-adaptation: Apply the debounce admission method change in base_text_debounce.py, preserving its typed receiver. Add the original refusal-control branch to the current platform handler without changing the upstream selected-owner loop. Keep the predicate name and defining merge import from the earlier sources in the busy-session module; its other incoming method and declaration bytes remain unchanged. Maintenance-adaptation: Declare the existing pending-refusal collaborator interface and use pytest monkeypatch for the admission fixture overrides and dynamic marker; keep the original scenario values and assertions. Captured upstream compatibility for bounded pending admission Keep the current unquoted MessageEvent annotations in the FIFO and busy steering methods. Preserve the original boolean return values and admission behaviour, including refusal reporting and queued input.
Source-permalink and reaction-context branches each test their own prompt contract. Their combined adapter adds the trigger permalink to catch-up messages, so the reaction fixtures need the SDK's asynchronous state-store methods and complete expected source notes. Merge both feature histories without changing their existing commits. Preserve reaction author trust, room and thread context, transcript roles, cache privacy and the source-permalink encoding and routing rules. Use the real MemoryStateStore in the affected fixtures and retrieve SessionSource from the final context tuple item. Check each complete catch-up prompt against its explicit room, trigger event and via server, including the first event of a batched turn. The four composed gateway files cover 343 cases. The affected two files pass all 264 cases after the fixture adaptation; the unchanged session and project-isolation files passed their 79 cases. Scoped Ruff reports no lint diagnostics. Existing formatter differences remain qualified separately. Live Docker room and thread backfill validation is still outstanding. This dependent layer preserves both parent histories. Its two live backfill fixture adaptations do not claim complete original 858 delivery. Historical composition qualification The preceding message records the original two-parent composition. This successor reapplies its fixture integration after the individually preserved prerequisite, SDK and reaction contributions on captured main 648a309. Its history has one parent. The original composition message remains complete for provenance; this commit does not create a merge or establish live Matrix acceptance.
The original feature tests predate the canonical approval lifecycle, permalink source fields, room policy owner argument and SDK send error classification. Their doubles omitted asynchronous state-store reads, exact approval request IDs and terminal card edits. Several complete expected events also omitted the newly composed source and mention metadata, so the full feature gate failed before checking its intended behaviour. Update those fixture bindings and complete expected values for the current defining owners. Keep every original parameter row, delivery attempt, payload, cancellation and authorization assertion. Thread reconnect expectations retain confirmed delivery and cache identity. Catch-up expectations include the permalink note before the new message. The complete nine-file feature boundary passes all 808 cases with the actual installed Matrix SDK, four workers and no file retries. The full Matrix file finishes in 30.7 seconds, including the event-loop lookup contract that timed out on the earlier incomplete source composition. Original858 public source qualification The preceding 808-case result records the original composition. It does not describe a new run against this public tree. The published backfill composition has 343 scoped cases, using 264 final affected cases and 79 unchanged earlier successes. Its inherited formatter and type findings remain qualified, and live Docker validation remains pending. The other feature contracts are represented by their existing public owners. Their fixture expectations follow the actual approval, notice, location, send-classification, read-policy and delivered-event APIs. Expectations from the original later composition do not establish bugs in standalone owners that use an earlier API. No combined six-module endpoint or repeated native gate is required for this source credit. This source-neutral contribution preserves the complete original message and both original contributor dates. It records the distributed source qualification without changing the published source tree. Current linear successor qualification The 343-case result above describes the earlier a76019d composition. This contribution follows the verified single-parent successor a83881c and keeps its source tree unchanged. The accepted seven-file gateway gate for that successor passes 362 cases. Live Matrix validation, private integration and whole-campaign completion remain separate.
The Matrix SDK retries rate-limit responses and connection timeouts indefinitely by default. A failed request during account or room setup can therefore stop the whole native module until the outer runner kills it, without reporting the setup error. Disable those retries and bound ordinary fixture requests to fifteen seconds. Apply the same configuration to registration, restored accounts and the encrypted observer. The fixtures now expose the SDK error so the failing operation can be diagnosed directly. A real loopback HTTP server reproduces the repeated rate-limit requests through registration and restored accounts. Both paths now make one request and return the expected error.
Matrix turns could read message history, but the agent could not inspect the current room state, joined members, power levels or pinned messages. Those details are available from the receiving bot and can affect what the agent should say or attempt. Extend the Matrix-only read tool with bounded inspection kinds. Fetch state and members through the owning adapter, apply its existing room and requester policy, and resolve pinned messages with the same decryption path as event reads. Report missing pin content per event so the other pins remain visible. Include the event type and creator override when presenting permission thresholds. A separate Matrix client verifies that the model receives the room details and pinned message from a disposable Synapse instance. Use the effective-state parent's client-events decoder and gateway scheduling path throughout inspection and its tests. Recheck admission after identity resolution, derive permission defaults from raw state, and preserve cancellation while refreshing pinned content. The feature and these interface migrations form one change so the initial commit uses the current prerequisite APIs.
The dependency-rebinding fixtures installed a fake importer but left the package-availability check unchanged. With a complete SDK installed, ensure_matrix_deps skips the importer, so these fixtures failed without exercising their intended success and refusal paths. Report a missing Matrix package in the fixture alongside the existing installation seam. The real rebind and failed-import assertions then exercise the intended fresh-install boundary.
The Matrix adapter ignores emotes and does not register native sticker events. Emotes describe the sender's action, while stickers use a separate event type with a description and an image. The existing text handler would interpret a slash-like action body as a gateway command, so native actions need attributed text before command classification. Accept both through the existing room, sender and mention gates. Prefix emotes with their actor, preserve sticker descriptions, and download sticker pixels through the bounded media path. Use the same labels in effective reads, replies and thread history. Queued stickers can be edited or redacted while media preparation is pending. Add an optional media-event snapshot capability and retain the source paths of image descriptions, so final model preparation removes both withdrawn pixels and their derived text. Existing platform snapshots remain compatible and previous conversation context remains unchanged. The two parameterised invariants reproduce six failures on the committed parents and now cover native SDK events, gates, unavailable media and withdrawal during asynchronous preparation. Matrix and adjacent gateway checks pass against the pinned SDK source.
iainlane
force-pushed
the
codex/shutdown-snapshot
branch
from
October 7, 2026 19:51
3153f4e to
15062cd
Compare
kvnloo
reviewed
Oct 7, 2026
kvnloo
left a comment
kvnloo
left a comment
There was a problem hiding this comment.
Blocking own-layer finding on 6ab275ec..15062cd1: a queued event can fall out of every shutdown-preservation surface after it is claimed but before it becomes durable recovery work.
Production calls release_pending_dispatch(..., claimed=True) in GatewayInboundMixin._handle_message() immediately after claiming the in-memory active-session slot. It then awaits run_post_admission_hook() before _handle_message_with_agent() acquires the turn lease and calls _mark_durable_active_turn(). run_post_admission_hook() explicitly propagates CancelledError and may await arbitrary plugin hooks. If shutdown cancels the task during that await, the reservation has already been removed/marked claimed, no active-turn marker exists yet, and no user transcript row has been written. flush_adapter_pending() excludes claimed reservations and does not capture SessionState.turn.event, so the accepted queued message is absent from the new snapshot too.
That creates exactly the loss window this PR is trying to close: queued -> promoted -> claimed -> plugin hook await -> shutdown cancellation.
Please keep the reservation snapshot-eligible until the turn crosses a durable ownership boundary (for example the successful active-turn marker / lease handoff), or explicitly include this claimed-but-not-durable turn.event state in the shutdown snapshot. A regression should use the real redispatch path, block post_gateway_admission, start shutdown, and prove the original event lands in the snapshot once and is not excluded merely because the in-memory session slot was claimed.
iainlane
force-pushed
the
codex/shutdown-snapshot
branch
2 times, most recently
from
October 8, 2026 12:43
4270f1a to
acc801b
Compare
Author
|
In response to the shutdown-preservation review. The loss window is fixed in acc801b, with the shared reservation correction in #130445 at 2159c37. The reservation now remains provisional through the post-admission hook and turn preparation. I also checked the suggested active-turn-marker boundary: that token contains no input, so the snapshot uses the existing exact transcript input-owner check instead. Plugin consumption, lease refusal and normal handler completion explicitly release the reservation. The same predicate prevents active-command cancellation from restoring an already-durable input and keeps pending capacity accurate. The regression drives real queued redispatch, blocks post_gateway_admission, cancels background work and checks the complete original event once before the later queued input. Additional A/B/A SQLite controls exercise provisional, durable, foreign-profile and unreadable ownership during shutdown, cancellation and capacity accounting. All 96 cases across eight complete affected gateway files pass using the canonical runner with zero retries and strict warnings; changed-file Ruff and reservation/snapshot type checks pass. Hosted CI still requires maintainer approval. |
iainlane
force-pushed
the
codex/shutdown-snapshot
branch
2 times, most recently
from
October 8, 2026 23:33
d45af32 to
292c649
Compare
The fake HTTP API discarded the client_session supplied by the Matrix adapter, so disconnect closed a mock while the real aiohttp session was left open. A reconnect test also replaced disconnect with a stub and left its new connection open after checking the guard. Strict resource warnings expose both leaks. Preserve the supplied session in the fake API. Use a wrapping disconnect spy in the reconnect test and disconnect the replacement client after the existing call assertion. The fake API and reconnect fixture predate this branch, so this focused fixture maintenance is separate from the corrections folded into their campaign contributors. Production code and all original assertions are unchanged.
Pending-event merges and busy-text debounce share reply and attachment semantics inside the large base adapter. Telegram's ingress overrides also modify the same batching boundary. Extract those existing topics before integrating the reviewed reply-context fixes. Repoint in-tree imports and patch targets to the defining merge module. The adapter classes inherit their existing methods, so timers, tracked redispatch and delayed-delivery ownership keep their current behaviour. The fixed upstream annotation cleanup overlaps the extracted debounce state. Keep the state in its existing topical module and apply the same unquoted task annotation there. The complete state and timer behaviour remain unchanged.
iainlane
force-pushed
the
codex/shutdown-snapshot
branch
from
October 9, 2026 12:24
292c649 to
616c942
Compare
iainlane
added a commit
to iainlane/hermes-agent
that referenced
this pull request
Oct 9, 2026
The shutdown serializer used legacy attributes that MessageEvent does not define. It saved text but discarded source identity, attachment paths, complete reply context and control permissions. Captionless photos could not recover. Adapter heads, buffered text and runner tails also became separate files, so their arrival order was not durable. Write one versioned snapshot per adapter-owned session with its complete ordered events. Capture provisional dispatch reservations before each cancellation round removes them, and exclude any event claimed while cancellation unwinds. Each source binds its owning profile scope before writing. A failed adapter snapshot retains its ordered events in the existing runner FIFO for a final retry under that same source scope. Keep process-local objects and live callbacks out of the wire payload. Preserve JSON-compatible event fields, full source and routing identity, media, reply context and internal/control metadata. Invalid payloads, unsupported versions and files copied into another profile remain available for inspection instead of being partially recovered. Startup can append an explicitly labelled inspection projection through the canonical UID batch writer. It retains the structured file after projection, and stable per-event UIDs prevent duplicate rows if the process exits after SQLite commits but before the projection marker is published. Recovery does not create live events or execute queued commands, internal wakes or plugin input. Two invariant functions exercise real adapter cancellation, captionless media, security context, FIFO order, late provisional heads and real A/B/A profile stores. They also cover invalid records, cross-profile copies, failed snapshot writes and retry after a post-commit crash. (cherry picked from commit 962858b) The profile projection fixture creates a GatewayRunner inside its try block, and the lease fixture replaces the real store that its bootstrap constructor created. Closing only the profile databases or replacement store left shared references registered. Apply the published NousResearch#130492 profile-store cleanup before resetting profile scope and the NousResearch#130938 current-home registry sweep after lease-task cancellation. All original assertions and the private defining import remain unchanged.
iainlane
added a commit
to iainlane/hermes-agent
that referenced
this pull request
Oct 9, 2026
The shutdown serializer used legacy attributes that MessageEvent does not define. It saved text but discarded source identity, attachment paths, complete reply context and control permissions. Captionless photos could not recover. Adapter heads, buffered text and runner tails also became separate files, so their arrival order was not durable. Write one versioned snapshot per adapter-owned session with its complete ordered events. Capture provisional dispatch reservations before each cancellation round removes them, and exclude any event claimed while cancellation unwinds. Each source binds its owning profile scope before writing. A failed adapter snapshot retains its ordered events in the existing runner FIFO for a final retry under that same source scope. Keep process-local objects and live callbacks out of the wire payload. Preserve JSON-compatible event fields, full source and routing identity, media, reply context and internal/control metadata. Invalid payloads, unsupported versions and files copied into another profile remain available for inspection instead of being partially recovered. Startup can append an explicitly labelled inspection projection through the canonical UID batch writer. It retains the structured file after projection, and stable per-event UIDs prevent duplicate rows if the process exits after SQLite commits but before the projection marker is published. Recovery does not create live events or execute queued commands, internal wakes or plugin input. Two invariant functions exercise real adapter cancellation, captionless media, security context, FIFO order, late provisional heads and real A/B/A profile stores. They also cover invalid records, cross-profile copies, failed snapshot writes and retry after a post-commit crash. (cherry picked from commit 962858b) The profile projection fixture creates a GatewayRunner inside its try block, and the lease fixture replaces the real store that its bootstrap constructor created. Closing only the profile databases or replacement store left shared references registered. Apply the published NousResearch#130492 profile-store cleanup before resetting profile scope and the NousResearch#130938 current-home registry sweep after lease-task cancellation. All original assertions and the private defining import remain unchanged.
iainlane
added a commit
to iainlane/hermes-agent
that referenced
this pull request
Oct 9, 2026
The shutdown serializer used legacy attributes that MessageEvent does not define. It saved text but discarded source identity, attachment paths, complete reply context and control permissions. Captionless photos could not recover. Adapter heads, buffered text and runner tails also became separate files, so their arrival order was not durable. Write one versioned snapshot per adapter-owned session with its complete ordered events. Capture provisional dispatch reservations before each cancellation round removes them, and exclude any event claimed while cancellation unwinds. Each source binds its owning profile scope before writing. A failed adapter snapshot retains its ordered events in the existing runner FIFO for a final retry under that same source scope. Keep process-local objects and live callbacks out of the wire payload. Preserve JSON-compatible event fields, full source and routing identity, media, reply context and internal/control metadata. Invalid payloads, unsupported versions and files copied into another profile remain available for inspection instead of being partially recovered. Startup can append an explicitly labelled inspection projection through the canonical UID batch writer. It retains the structured file after projection, and stable per-event UIDs prevent duplicate rows if the process exits after SQLite commits but before the projection marker is published. Recovery does not create live events or execute queued commands, internal wakes or plugin input. Two invariant functions exercise real adapter cancellation, captionless media, security context, FIFO order, late provisional heads and real A/B/A profile stores. They also cover invalid records, cross-profile copies, failed snapshot writes and retry after a post-commit crash. (cherry picked from commit 962858b) The profile projection fixture creates a GatewayRunner inside its try block, and the lease fixture replaces the real store that its bootstrap constructor created. Closing only the profile databases or replacement store left shared references registered. Apply the published NousResearch#130492 profile-store cleanup before resetting profile scope and the NousResearch#130938 current-home registry sweep after lease-task cancellation. All original assertions and the private defining import remain unchanged.
iainlane
force-pushed
the
codex/shutdown-snapshot
branch
from
October 10, 2026 04:46
616c942 to
986096f
Compare
iainlane
added a commit
to iainlane/hermes-agent
that referenced
this pull request
Oct 11, 2026
The shutdown serializer used legacy attributes that MessageEvent does not define. It saved text but discarded source identity, attachment paths, complete reply context and control permissions. Captionless photos could not recover. Adapter heads, buffered text and runner tails also became separate files, so their arrival order was not durable. Write one versioned snapshot per adapter-owned session with its complete ordered events. Capture provisional dispatch reservations before each cancellation round removes them, and exclude any event claimed while cancellation unwinds. Each source binds its owning profile scope before writing. A failed adapter snapshot retains its ordered events in the existing runner FIFO for a final retry under that same source scope. Keep process-local objects and live callbacks out of the wire payload. Preserve JSON-compatible event fields, full source and routing identity, media, reply context and internal/control metadata. Invalid payloads, unsupported versions and files copied into another profile remain available for inspection instead of being partially recovered. Startup can append an explicitly labelled inspection projection through the canonical UID batch writer. It retains the structured file after projection, and stable per-event UIDs prevent duplicate rows if the process exits after SQLite commits but before the projection marker is published. Recovery does not create live events or execute queued commands, internal wakes or plugin input. Two invariant functions exercise real adapter cancellation, captionless media, security context, FIFO order, late provisional heads and real A/B/A profile stores. They also cover invalid records, cross-profile copies, failed snapshot writes and retry after a post-commit crash. (cherry picked from commit 962858b) The profile projection fixture creates a GatewayRunner inside its try block, and the lease fixture replaces the real store that its bootstrap constructor created. Closing only the profile databases or replacement store left shared references registered. Apply the published NousResearch#130492 profile-store cleanup before resetting profile scope and the NousResearch#130938 current-home registry sweep after lease-task cancellation. All original assertions and the private defining import remain unchanged.
This branch has not been deployed
This file contains hidden or bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
Sign up for free
to join this conversation on GitHub.
Already have an account?
Sign in to comment
13 participants
Add this suggestion to a batch that can be applied as a single commit.This suggestion is invalid because no changes were made to the code.Suggestions cannot be applied while the pull request is closed.Suggestions cannot be applied while viewing a subset of changes.Only one suggestion per line can be applied in a batch.Add this suggestion to a batch that can be applied as a single commit.Applying suggestions on deleted lines is not supported.You must change the existing code in this line in order to create a valid suggestion.Outdated suggestions cannot be applied.This suggestion has been applied or marked resolved.Suggestions cannot be applied from pending reviews.Suggestions cannot be applied on multi-line comments.Suggestions cannot be applied while the pull request is queued to merge.Suggestion cannot be applied right now. Please check back later.
Stack
All PRs below target
main. Nesting records merge order for composed or logical prerequisites. Every prerequisite shown must merge before its dependent. Each linked description lists earlier dependencies.Change
Shutdown records one ordered, versioned snapshot per owning session and profile. The record preserves public event fields, source provenance, attachments, reply contexts and security metadata. Canonical UID writes deduplicate overlapping transcript projections while conflicting payloads remain available for inspection. Recovery produces inspection text and retains the structured file. A promoted event remains eligible for the snapshot during post-admission hooks and turn preparation. Only its exact durable transcript input marker or a terminal handler decision removes that eligibility. Recovery does not execute restored input.
Ordinary recovery files now keep only their paths during classification and decode each payload again for replay. Canonical transcript envelopes retain their atomic per-session batches, durable message identities and identical-overlap checks. The original memory regression reproduced a 17.5 MB peak for a six-megabyte spool. Initializing the real shared recovery dependencies before the measurement still reproduces a 13.3 MB peak on the original source. The correction passes the same four-megabyte bound, recovery count, payload-value, ordering and deletion assertions.
History and review
The branch uses captured upstream
3637c512fc3211bdccb61b7140d09ac1018508bathrough the actual independently published admission prerequisite. Its 290 signed records remain exact. All three original shutdown records preserve their literal author and committer headers, dates, timezones, complete message prefixes and order. The second remains a meaningful unchanged-tree credit. The composition has 293 records in total.The only initial merge resolution uses the current builtin dictionary annotation while preserving the original complete recovery block and its comment deletion. The memory correction and regression dependency initialization are folded into the first original recovery record, with factual message appendices only in that record. All existing assertions remain exact. The earlier annotation-only source foundation and every failed control are retained.
Review the own contribution. The comparison against
mainalso includes the composed prerequisite. Merge #130445 and #129835 before this PR, then #128669.Current validation
Six complete recovery, active-text, snapshot and transcript files pass 77 cases on the corrected source. The execution uses the exact final file bytes, including explicit initialization of the two real recovery dependencies before tracing payload memory. A complete source bridge authenticates those bytes to the final Git tree. Three unchanged admission and busy-followup files retain their 36-case complete receipts. The combined qualified scope is 113 cases in nine complete files: 98 cases in eight files finish without resource emissions, and the unchanged 15-case recovery file emits the same three database shutdown warnings as the complete original control.
All failed memory control files are excluded from the passing count, including the original, annotation-only candidate, memory-only intermediate and dependency-initialized original. Their full outputs and actual inputs remain available. The canonical runner uses two workers, a 600-second file limit, zero retries, and ResourceWarning, RuntimeWarning, unraisable and thread warnings as errors. All 9,286 tracked candidate Python, configuration, lock and runner inputs are authenticated before and after; the runner is restored exactly. The installed interpreter and SDK source are bound separately.
The full 15-path type comparison has 246 original findings, 235 actual prerequisite findings and 238 final findings. Of the final findings, 237 match complete lexical, statement and relative-location occurrence counters. One unchanged complete attachment-list caller has the same finding with the inherited expected content type widened from
strtostr | None; the entire method body and caller are unchanged. That complete provider-contract disposition is recorded separately. No finding is introduced. Ruff and whitespace checks pass, and the same 14 inherited formatting paths remain. Both frozen local joins are clean. No new native platform or hosted CI result is claimed.Earlier validation
The following results concern preceding published sources and retain their original source and execution qualifications.
This composition includes the corrected #130445 cancellation-completion reconciliation. A provisional input remains reserved after a non-discarding cancellation timeout; eventual completion restores it once ahead of its successors only if its exact transcript input marker is absent. Durable input remains excluded. The three own shutdown records preserve their complete original author and committer headers and messages, including the meaningful unchanged-tree credit.
Six complete current cancellation, queue, heartbeat, active-text, snapshot and shutdown-message files pass all 84 cases with zero retries and strict runtime, unraisable-exception and thread-exception warnings. This is a fresh composition gate, source-qualified to the final tree through removal of one duplicate type import. Every complete function and test AST and every other tree path remains exact to the tested composition. The earlier 96-case, eight-file result remains revision-qualified and is not added to this count. Scoped Ruff and owned whitespace checks pass. The three affected files retain exactly 55 inherited type findings, with complete lexical spans, columns, multiplicities and statement ASTs unchanged. The helper and regression pass the formatter; the adapter module retains existing formatting debt. No new native Matrix or hosted CI pass is claimed.
The earlier formatter and fixture correction below applies to the preceding published revision.
The durable-owner helper in
gateway/platforms/base_pending.pynow passes the formatter check. Only its definition, transcript lookup and warning call changed layout. The complete module AST is unchanged; every other production source blob is exact to the preceding published revision. The correction is folded into the actual helper-introducing contribution, with all preceding signed commits unchanged. Original author and committer headers and complete message prefixes remain exact; that introducing message adds the formatter rationale, and the final snapshot message explains its fixture type correction. The meaningful unchanged-tree translation credit remains separate.The ownership fixture now declares its session store and profile-scope method on a local queue-runner subclass. All original tests, parameter rows and whole-event assertions remain exact. The complete 22-case snapshot file has a separate fresh run; the other seven complete files retain their source-qualified results. The prior 96-case, eight-file runtime scope remains qualified through the production AST bridge and unchanged surrounding source. This is not a fresh combined run. Earlier type, native and hosted results retain their recorded scope; the scoped comparison removes exactly four fixture findings, with418 before and414 after and no introduced findings. The overall type check remains non-green; no hosted execution is claimed.
The queued redispatch regression reproduced loss of the original event when shutdown cancelled a blocked post-admission hook. At the preceding published revision, eight complete affected gateway files passed all 96 cases with zero retries and strict runtime, unraisable-exception and thread-exception warnings. The tests preserve the complete original event exactly once, retain FIFO order, and exercise source-scoped A/B/A SQLite ownership during shutdown, active-command cancellation and capacity accounting. Recorded Ruff and reservation/snapshot type checks apply to that revision; this formatter correction has separate current static evidence.
The earlier validation below retains its original source and execution qualifications.
The four recovery files passed all 33 cases. The corrected complete snapshot file passed all 13 cases, for 46 distinct source-aware outcomes in five files. The earlier failed wrapper remains recorded. The canonical runner used one worker, zero retries and strict runtime, unraisable-exception and thread-exception warnings. The larger composed gate is a separate execution from the own-layer gates.
Scoped Ruff checking passes on all 46 changed Python paths. The scoped type and formatter checks remain non-green because of inherited findings; 975 findings match complete statements at the same paths and six match the original defining paths after extraction. It reports zero unqualified new findings. Existing native receipts retain their source qualifications. No fresh native Matrix execution or hosted CI pass is claimed.
Shutdown snapshots remain inspection evidence. Automatic live replay requires routing, authorisation, deletion, attachment and durable admission checks and is not enabled by this change.
The current snapshot test file also exercises fixture teardown: 22 cases pass with strict resource warnings and full subprocess output, without unclosed database, event-loop, socket or transport warnings at shutdown. The admission and profile fixtures close the stores that they create. Production code and every existing assertion are unchanged. The fixture file retains its four existing type findings and inherited formatting debt; scoped Ruff checking passes.