Repository navigation
Conversation
Signal and IRC await handle_message before they read their next input. A busy photo or queue-mode text was held for the burst window and the call returned only after the durable commit, so the burst's next input could arrive only once the burst had committed alone: two turns where main merged one, and the reader stalled about 0.38 s per busy input. An adapter now declares serial_receive_loop when it delivers a chat's next input only after handle_message returns and returning triggers no acknowledgement. For it, dispatch_busy_burst returns as soon as the input is held, as main returned once the input was merged into the pending slot. The burst still commits once and in order, and a failed commit is logged. Every other adapter keeps commit-before-return. The reader still never runs ahead of what it returned on: - a command first waits until every earlier input of that reader has committed (/stop, /new and /reset drop their own chat's held burst), so /restart cannot close admission under a held input and /new cannot miss one still committing; - a chat holds at most one uncommitted burst: the next one waits for that commit, as main's reader waited at its first commit. Flagged: Signal, IRC, email, Matrix, Mattermost, ntfy, Photon, SimpleX, WeCom, WhatsApp and Feishu. Buzz is left out: it saves its cursor only after the call returns. Refs NousResearch#134889 Fixes NousResearch#140538
This was referenced Oct 17, 2026
Author
|
Landed via #140973 (merged as |
This file contains hidden or bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
Sign up for free
to join this conversation on GitHub.
Already have an account?
Sign in to comment
Add this suggestion to a batch that can be applied as a single commit.This suggestion is invalid because no changes were made to the code.Suggestions cannot be applied while the pull request is closed.Suggestions cannot be applied while viewing a subset of changes.Only one suggestion per line can be applied in a batch.Add this suggestion to a batch that can be applied as a single commit.Applying suggestions on deleted lines is not supported.You must change the existing code in this line in order to create a valid suggestion.Outdated suggestions cannot be applied.This suggestion has been applied or marked resolved.Suggestions cannot be applied from pending reviews.Suggestions cannot be applied on multi-line comments.Suggestions cannot be applied while the pull request is queued to merge.Suggestion cannot be applied right now. Please check back later.
What does this PR do?
The problem. On Signal, IRC and the other platforms whose receive loop handles one message at a time, photos sent one by one, or a text split over two messages, while a turn is running are no longer merged into one follow-up turn. Each runs as its own turn, where Discord or Slack (and every platform before the unified runtime) gets one. The receive loop also stops reading for about 0.38 s on each such message, for every chat on that account.
Why.
dispatch_busy_burst(gateway/session_ingress_burst.py) holds a busy photo or queue-mode text for main's burst window (0.35 s debounce, 1 s cap), then commits the merged input once. Every caller returns only after that commit._sse_listenerand IRC's_receive_loopawaithandle_messagebefore they read the next message. So the burst's next input can only arrive after the burst was flushed and committed alone, and it starts a new burst. The wait is also the stall.asyncio.create_task), which a serial loop never is.The fix.
Invariant: a serial reader returns as soon as a busy input is held, so the next input of the same burst can join it, and the reader never waits out the hold window. It still never runs ahead of what it returned on: a command, and the chat's next burst, wait for those inputs to commit, as
main's reader did.serial_receive_loop = Truewhen it delivers a chat's next input only afterhandle_messagereturns, and returning triggers no acknowledgement.dispatch_busy_burstreturns once the input is held, as main returned once the input was merged into the pending slot. The burst still commits once and in order, and later input still queues behind it (holds_busy_burst). With no caller waiting, a failed burst commit is logged instead of raised.handle_messagenow runsburst_gatefirst. On a serial reader, a command flushes every held burst and waits until every pending burst commit has landed, then runs./stop,/newand/resetstill drop their own chat's held burst, as main dropped its debounce buffer. So/restartcan't close admission under a held text, and/newcan't miss a photo that is still capturing.The flag is set on every receive path of that shape I found, each with a one-line comment naming its loop: Signal, IRC, email (the IMAP poll loop), Matrix (one sync at a time), Mattermost, ntfy, Photon, SimpleX, WeCom (WebSocket), WhatsApp (the bridge poll loop, for media) and Feishu (its per-chat lock).
Design choice.
/stopand/newcancellation. A fence carried through admission would duplicate fix(gateway): keep a chat's media and quick text in arrival order #140454's route fence./approveand/deny,/queue, skill, plugin, quick and unknown commands alike. So a turn a command queues still lands after the held input. A commit never waits on the running turn, so a command sent to unblock that turn can't deadlock behind it.handle_messagecalls return, so that return is an ACK (see Known limits). LINE and WhatsApp Cloud await a POST's messages in order, but answer HTTP 200 only after them.BasePlatformAdapter:gateway/platforms/base.pyis over its line cap, so the default is read withgetattr(adapter, 'serial_receive_loop', False), and the rule is in the module docstring.handle_message's old_shared_burst_pendingcheck moved intoburst_gate. Two adapter files over the cap stay line-neutral (the flag replaces a blank line).Before / after (session authority on, a turn running):
busy_text_mode: queue: two linesfirst half\nsecond half/restartinside the window/restartwaits for that commit/newwhile a photo is still capturing/newcancels it/stopor/newwhile the chat's burst is still heldRelated Issue
Fixes #140538
Refs #134889
Overlap:
gateway/session_ingress.py(admit_message, thedispatch_shared_busydocstring),gateway/session_authority.py::admit_nativeandgateway/run_inbound.py. No file overlap. Both release a serial reader early, at different stages: fix(gateway): keep a chat's media and quick text in arrival order #140454 once a non-held input's FIFO position is fixed behind an earlier input of its chat, this PR while the input is still held for merging, before any commit. They compose: with fix(gateway): keep a chat's media and quick text in arrival order #140454 applied on top of this branch, its arrival-order tests, this PR's tests and the burst, busy and stop suites pass (9 files, 54 tests). Here a chat never has a second burst waiting behind a committing one, and/stopwaits for pending commits, so no input sits outside fix(gateway): keep a chat's media and quick text in arrival order #140454's fence when/stoparrives.Type of Change
Changes Made
gateway/session_ingress_burst.py:_acceptedreturns at once for an adapter withserial_receive_loop;_commitlogs a failed commit for such an adapter instead of raising it;burst_gate: the entry checkhandle_messageruns first. It says whether a burst owns the route, and on a serial reader it makes a command wait for every earlier input;dispatch_busy_burstwaits for the chat's committing burst before it holds the next one, on a serial reader;gateway/platforms/base.py:handle_messageawaitsburst_gatebefore routing, on the active and idle paths (net −2 lines;_shared_burst_pendingis gone).plugins/platforms/{signal,irc,email,matrix,mattermost,ntfy,photon,simplex,wecom,whatsapp,feishu}/adapter.py:serial_receive_loop = True, with the loop named.tests/gateway/test_busy_burst_serial_reader.py(new), with a realGatewayRunner,SessionAuthorityand drain over SQLite:test_signal_busy_photos_merge_without_holding_the_reader: two busy photos throughSignalAdapter.handle_message, each awaited before the next, as_sse_listenerdoes. Each call must return while its input is still held, and the follow-up must be one turn with both photos;test_irc_split_text_merges_through_the_receive_loop: two lines fed to IRC's real_receive_loopin queue mode must run as one turn;test_restart_after_a_held_text_waits_for_its_commit: a held text, then an authorized/restart(its restart stubbed to close admission). The text must be queued before admission closes;test_new_waits_for_a_burst_still_committing:/newarrives while a photo's media capture is held. The photo must commit first and be cancelled with the old conversation, not run in the new one;test_next_burst_waits_while_the_chats_last_one_commits: with the chat's first burst held in capture, the next photo'shandle_messagemust not return until that commit lands.How to Test
mainat868b9f244f, the two merge tests fail: Signal's first call returns only after its burst committed ('the reader waited out the hold window'), and IRC runsfirst halfandsecond halfas two turns. The other three pass onmainand pin its order: a reader that returns early without the gate and the per-chat wait fails them (the text is lost under/restart; the photo runs in the new conversation; the reader goes on with a burst still committing).handle_message's callers: the platform base, command bypass, busy controls,/busy,/queue,/steer, the reset deadlock, internal events, plaintext approval, guard release, restart drain, command dispatch, unknown commands, Signal formatting;scripts/check --base 868b9f244f: 12 checks ok; code health 0 blocking, 0 advisory.Independent review: one review round, then a final review-and-fix pass.
/restartclosing admission under a held text,/stopand/newmissing a burst already flushed or reached on the idle path, and an unbounded backlog behind a blocked commit. Each is fixed above and has a regression test here; the idle-path/newvariant was checked with a separate probe (it drops the held text). Yuanbao was dropped from the flag list./stop, the one-burst limit across senders and chats, the teardown limit againstmain, and composition with fix(gateway): keep a chat's media and quick text in arrival order #140454. It found no P1/P2, so the code is unchanged. Probes run on bothmainand this branch confirmed that a held text never runs after/stop, that a failing burst commit doesn't hold the next command, and thatmainloses a held input to a teardown or a drain just as this branch does. Its remaining findings are listed under Known limits.Known limits:
main. Merging there needs its cursor saved behind the commit, a separate change.hermes gateway stop, idle exit), a profile retirement or an adapter teardown in that window loses it, asmainloses an input its reader is still holding in the same debounce wait. Returning doesn't trigger an additional acknowledgement on the flagged platforms, but email, Matrix, WhatsApp and Feishu already consume or acknowledge beforehandle_message, as onmain. An in-band command can no longer overtake it.yesanswering an approval is held like any queue-mode text. So a second line from the same sender within 0.35 s merges with it and is no longer read as the answer, as on Discord and Slack today.handle_messagefrom a second task (Matrix within one sync, the text batchers), that task can run while a command or the next burst waits on a commit. Two inputs of one chat in the same sync or batch flush can then land out of order (a non-mergeable input ahead of a burst held just before it), or a photo right after a/stopcan be dropped with the held burst.Still needs a live retest:
/restart: the text runs after the restart;busy_text_mode: queue: two lines while a turn runs;Checklist
Code
pytest tests/ -qand all tests pass. Not ticked: I ran the related suites above, not the whole tree; CI is the full runpython scripts/checkpassesDocumentation & Housekeeping
session_ingress_burstdocstring and each adapter's flag commentcli-config.yaml.example: N/ACONTRIBUTING.md/AGENTS.md: N/A