Repository navigation
acpmux: carried turns arm their own cap, hold their own output and count lane reports - #27868
Conversation
… (red) Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com> Agent-Session: https://cmux.com/deeplink/agent?id=aid-c7e4fd33b45c31bb0c5f
Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com> Agent-Session: https://cmux.com/deeplink/agent?id=aid-c7e4fd33b45c31bb0c5f
… upward (red) Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com> Agent-Session: https://cmux.com/deeplink/agent?id=aid-c7e4fd33b45c31bb0c5f
…unt lane reports Follow-ups from the review of #27536: - Each carried turn arms its own turn cap timer (begin_carried), the successors run_carried starts and the turns a restart adopts included; the timer counts from the turn's start, so an adopted turn's cap is not pushed back a whole cap. - Each carry holds Claude's output once, with its own bound: Translator::holding names the newest carry, so a second carry made while the first one's prompt is still to come (its bound passed) holds again, in the in-process reader and in the agent host. - A carried turn whose steers were all lane reports is counted as one on reports only (lane_turn_on), so it reports nothing upward; a restart keeps it (carriedTo reportsOnly). Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com> Agent-Session: https://cmux.com/deeplink/agent?id=aid-c7e4fd33b45c31bb0c5f
|
Found 2 test failures on Blacksmith runners: Failures
|
|
cmux status (one comment, edited in place; each line is updated by the job that owns it) Review App of this head (app.zip, kept 1 day)
red: cmux-next checks (god files, concurrency, crash safety, l10n); NEW: |
|
Review (codex gpt-6-astra high fast, independent review): SHIP at e222db5 Reviewed exactly
Exact-head tests: run 40994153187 (clippy |
…-carry-followups # Conflicts: # cmux-tui/crates/acpmux/src/hub/turn_cap_tests.rs
|
Review (codex gpt-6.1-sol high fast, independent review): SHIP at 5568330 This head merges Reviewed
Exact-head tests: run 41013004844.
|
|
Red comparison for head 5568330 (addendum to the receipt above). I compared each red by name against the newest base runs.
|
Bugbot needs on-demand usage enabledBugbot uses usage-based billing for this team and requires on-demand usage to be enabled. A team admin can enable on-demand usage in the Cursor dashboard. |
|
Base merge only: b0cad4f merges
|
|
Review (the owner lane's independent review, at its land request): SHIP at b0cad4f Recorded by lane-gh-broker from this PR's owner lane, which asked to land this exact head after its independent review. next-autoland lands it once every red is the base's; a BLOCK at this head, a HOLD line or a hold label still stops it. |
|
Review (the owner lane's independent review, at its land request): SHIP at 7a45a16 Recorded by lane-gh-broker from this PR's owner lane, which asked to land this exact head after its independent review. next-autoland lands it once every red is the base's; a BLOCK at this head, a HOLD line or a hold label still stops it. |
|
Reds at head 7a45a16, by name (81 passed, 3 failed, 0 pending). This PR changes no file outside
None of these reds come from this PR. Review: SHIP in comment 6520646600 (the diff is unchanged, see comment 6520856518). |
|
Infra red at 7a45a16 (next-autoland lists it as "its own red"): cmux-tui artifacts / publish the Windows daemon to R2 (run 41043457329, job 124029813663, a push run of
It tried four times (11:08:55, 11:09:59, 11:12:05 and 11:16:09 UTC) with backoff, and every try hit the same error. Nothing in that job failed because of the PR's code. Under the standing rule it isn't rerun, since a rerun spends the same exhausted installation quota. The other reds are base by name (comment 6521331273). Review: SHIP (comments 6520646600 and 6520856518). |
|
Daemon-tests red at 7a45a16 (next-autoland, 14:22Z: "its own red: status / cmux-next daemon tests (same-tree)"; run 41045353403, job 124031216958):
I couldn't compare by name against a base run of this job: the REST quota is spent, and the base daemon-tests dispatches I was given (41036657085, 41024114885) skipped or timed out. So this is classified on scope and on the suite's own known issue, not on a base run with the same test. |
|
Shipped in:
|
|
Base red from a merge race with #27686: |
|
Fix: #28427. |
host.rs conflicted with #27868 (carried turns hold their own output): both sides added a HostState field (orphan, read_all), and both are kept. #27868 also made Translator::holding() return the carry's number, so the orphan's pending check reads holding().is_some(), the same meaning as the bool it replaced. Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com> Agent-Session: https://cmux.com/deeplink/agent?id=aid-0208360f23a3700be142
…feat-cmux-next tip (#28261) * ci(next): the tip suite dispatcher also runs the daemon suite on the feat-cmux-next tip A feat-cmux-next push never runs daemon-test, so the base had no "cmux-next daemon tests" verdict for hours and base_red_excuse.py could not compare any pull request's daemon red with it (#27692, #27868). The tip suite dispatcher now has a daemon leg beside the Swift one (a matrix over the two suites): a tip_daemon_suite run of cmux-next.yml, routed on the router's daemon tier since the last daemon verdict, and dispatched regardless once the tip is 20 landings past it. Runs are told apart by name and tested commit, so a same-tree run that tests a recorded sha after the tip moved still counts. Before feat-cmux-next's cmux-next.yml takes the input, the daemon dispatch warns instead of failing the tick. Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com> * fix(ci): a deferred daemon suite run is pending, and only the unknown-input 422 is a warning Review of #28261: - A tip daemon suite run whose daemon-test was skipped (its tree was still publishing) was a no-verdict run, so the next tick dispatched the same sha again and two same-tree runs could follow. It is now pending for 90 minutes (DEFERRED_GRACE_MS): its same-tree run, named for the same sha, then gives the verdict. Past the grace it is a lost run and is retried. The Swift suite has no deferred state. - Only GitHub's `Unexpected inputs provided: [...tip_daemon_suite...]` 422 is a warning; any other 422, and every Swift refusal, fails the tick. - Tests: the deferred run, its same-tree run in flight and its verdict, the grace, the 422 messages, and the feat-cmux-next router printing swift= and daemon=. Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com> * ci(rescue): a failed tip daemon suite run wakes a rescue when it completes Review of #28261: the owned-pool rescue woke on a failed `cmux-next tip Swift suite` dispatch but not on the new daemon suite's, so a runner refusal of its daemon test waited for the two-hourly sweeper and the base had no daemon-test verdict meanwhile. The `completed` wake now matches `cmux-next tip daemon suite` too (completed_target already takes any cmux-next dispatch), with tests. Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com> * fix(ci): the daemon suite's 20-landing floor counts from its newest verdict Review of #28261: the floor counted from the routing base (the last green), so a red periodic run after it made every later tick dispatch again once 20 landings had passed since the green. It now counts from the newest daemon verdict of either kind (one more compare when that is not the green); routing still diffs from the last green. Tests: a red verdict 1 landing back holds the floor, 20 past it dispatches. Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com> --------- Co-authored-by: Claude Opus 5.5 <noreply@anthropic.com>
…ied-turn cap test (base red) (#28428) * acpmux: the carried-turn cap test queues a supersede-keyed agent message (base red) carried_turns::a_carried_turn_after_a_carried_turn_has_its_own_cap failed on base since #27868 and #27686 landed together. The test queued another agent's plain send behind the third turn and expected the cap timer's notice. #27686 sends a plain agent message to the chat's inbox instead: under the cap it steers in as an inbox notice, nothing waits in the queue, and the timer correctly sends no turn-cap notice. #27868's CI ran on a base without #27686, so the two were never tested together. The test was wrong, not the code: a supersede-keyed send still queues with the inbox on (as do team sends, peer forwards and lane reports), and that is the case the cap timer covers. Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com> Agent-Session: https://cmux.com/deeplink/agent?id=aid-c7e4fd33b45c31bb0c5f * acpmux: test that an inbox message held under the turn cap gets the cap's notice (red) Red on base: the cap timer looks only at the queue, so another agent's message the inbox holds (a person's prompt waits, so no inbox notice steers in) gets no turn-cap notice, and past the cap the inbox steers none in either. Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com> Agent-Session: https://cmux.com/deeplink/agent?id=aid-c7e4fd33b45c31bb0c5f * acpmux: the turn cap's timer counts an inbox message no notice carries as waiting The cap's timer sent its notice only when another agent's message waited in the queue. Since the inbox (#27686), a plain send waits in the chat's inbox instead; one held there at the cap (behind a notice in flight, a refused one's backoff, or a prompt of the person's that waits) is never steered into a turn past its cap, so the agent was never told to end it. The timer now counts an inbox item no notice carries as waiting too. Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com> Agent-Session: https://cmux.com/deeplink/agent?id=aid-c7e4fd33b45c31bb0c5f * acpmux: test that an inbox notice refused past the cap still gets the turn the cap's notice (red) Red on base and at b4e7139: a notice that carried an inbox message (so the cap's timer saw nothing waiting) and was refused as the turn crossed its cap puts the message back, and the inbox steers none into a turn past its cap; no turn-cap notice follows. Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com> Agent-Session: https://cmux.com/deeplink/agent?id=aid-c7e4fd33b45c31bb0c5f * acpmux: the refused-notice cap test builds its inbox item with a test constructor Item's waiter is private to inbox.rs; Item::carried (cfg(test)) makes one a notice carries. Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com> Agent-Session: https://cmux.com/deeplink/agent?id=aid-c7e4fd33b45c31bb0c5f * acpmux: a refused inbox notice that puts its messages back past the cap sends the cap's notice The cap's timer skips inbox messages a notice carries. A notice refused as its turn crossed the cap (written to no agent) puts them back, but the inbox steers none into a turn past its cap, so the agent was never told to end it. notice_done now rechecks the cap (turn_cap_recheck, once per turn) after it puts them back, outside the inbox lock. Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com> Agent-Session: https://cmux.com/deeplink/agent?id=aid-c7e4fd33b45c31bb0c5f --------- Co-authored-by: Claude Opus 5.5 <noreply@anthropic.com>
…8283) * ci(next): a tip daemon suite gives the base a daemon-test verdict A feat-cmux-next push never runs daemon-test: its same-tree run has origin push, and a dispatch's tree wait marker is superseded the moment the tip moves, which in a landing burst is always before the tree publishes. So the base had no "cmux-next daemon tests" verdict for hours, and base_red_excuse.py could not compare any pull request's daemon red with it (#27692, #27868). - cmux-next.yml takes tip_daemon_suite (main's tip suite dispatcher asks for it): path routing routes only the daemon tier, and the run is named `cmux-next tip daemon suite · <tested sha>` (same_tree_sha in same-tree mode). - Its tree wait marker carries tip_suite=daemon. cmux_next_tree_notify.py starts it on the recorded sha after the tip moves (like the DEV daily) and passes tip_daemon_suite on, so the same-tree run keeps the name. Only a daemon-only dispatch on feat-cmux-next heads may carry it. - base_red_excuse.py: when its window of the newest 100 base runs holds no verdict for swift test or daemon tests, it reads the base's newest tip suite runs of that job by name (at most 6 job lists). Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com> * ci(guards): run test_cmux_tui_owned_mini_caches.py, which its registry lane names #27840 registered tests/test_cmux_tui_owned_mini_caches.py as lane "linux-guard" but no ci-guards.yml step ran it, so the execution registry validator fails every pull request into feat-cmux-next ("linux-guard lane is not run by any workflow"). The release-notary group of ci-guards now runs it, beside the other cmux-tui artifact guards. Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com> (cherry picked from commit 43312b4) * fix(ci): the tree wait marker step reads TIP_DAEMON_SUITE with a default test_a_dispatch_whose_tree_was_building_gets_its_daemon_tests_later runs the marker step's script with its own env, which has no TIP_DAEMON_SUITE, and the KeyError failed "Checks job structure" on #28283. The script reads it with .get(), and the test now covers the tip daemon suite marker: tip_suite=daemon only for a tip_daemon_suite dispatch on feat-cmux-next. Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com> * ci(guards): drop the owned-mini-caches step from this PR Review: this branch's base predates #27840, so tests/test_cmux_tui_owned_mini_caches.py is not in the branch and the step only passed on the merge commit with the base. The registry error it fixed is a base red (feat-cmux-next fails the same way); its fix goes in its own PR from a branch that has the file. This reverts commit c0c10bc. Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com> * fix(ci): the base dispatch fallback takes the newest verdict, green included Review of #28283: the fallback read only tip suite runs, so a newer ordinary dispatch that ran daemon-test (or swift-test) green was skipped and an older red tip suite run excused the head. It now reads the base's completed dispatches newest first and the first with a verdict for the job decides, a green one included. DEV daily runs and the other job's tip suite never run the job and are skipped unread; at most 6 job lists. Test: a newer green ordinary dispatch over an older red tip suite refuses. Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com> --------- Co-authored-by: Claude Opus 5.5 <noreply@anthropic.com>
…the inbox off (#28427) #27868 (7a5538d) added a_carried_turn_after_a_carried_turn_has_its_own_cap, which queues another agent's plain message behind a carried turn and waits for the turn-cap notice. Its PR head predates #27686 (cbe8e60, merged first), which sends such a message to the chat's inbox instead of the queue, so on feat-cmux-next nothing was queued and no cap notice came (red on every run). The test runs the queue path it covers (agentInbox off), as #27686 pinned the other queue tests; the rest of carried_turns.rs keeps the inbox on. Co-authored-by: Claude Opus 5.5 <noreply@anthropic.com>
…that touch acpmux (#28644) * ci(cmux-tui-spec): run acpmux's lib and hub_integration tests on PRs that touch acpmux acpmux's hub_integration suite (agent_inbox, carried_turns, inbox_digest, ...) and its lib tests (inbox, turn_cap, ...) ran only in Fast checks and on the fleet, so #27921 and #27868 landed red tests and left base red for hours (2026-10-30). The new `acpmux tests (lib, hub_integration)` job runs both on Blacksmith Linux when a change touches acpmux or its build inputs. A red pull request run is judged per test: each failing test runs again alone on the head and on the merge parent, and scripts/ci/cargo_test_verdict.py (on top of test_reds.py) passes the job only when every red is a flake or fails on the merge parent too; a crash, a cut-short binary, a compile error or a test the merge parent passes fails it. feat-cmux-next pushes run the suites plainly and save the cargo cache. Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com> Agent-Session: https://cmux.com/deeplink/agent?id=aid-7e68dca28911d1051663 * ci(acpmux tests): scope by acpmux's path dependencies and included files Review (Astra): the scope looked only at crates/acpmux, so a change to a path dependency (cmux-server, cmux-json, ...) or to a file a crate include_str!s (skills/cmux-cua/SKILL.md, schemas/...) skipped the tests that it can break. `cargo_test_verdict.py touches CRATE` now follows acpmux's path dependencies (through `workspace = true` too) and their include_str!/include_bytes! files, and the workflow's paths filters name the included files outside cmux-tui; a test fails when one is missing. Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com> Agent-Session: https://cmux.com/deeplink/agent?id=aid-7e68dca28911d1051663 --------- Co-authored-by: Claude Opus 5.5 <noreply@anthropic.com>
![Fix with [code]smith](https://pr-comments-assets.blacksmith.sh/codesmith/fix-with-codesmith-light-3x.png)
Follow-ups from the independent review of #27536 ("a turn ends when Claude Code ends it"). Its SHIP receipt listed four non-blocking notes. This PR fixes three of them; the fourth turns out to need no code change (see below).
#27536 fixed Leo's asks: "what should we do?", "should i steer", and "did ANY of the reserach/think tank/pls go look into this sessions finish???".
What changes
start_carried) armed the acpmux: turn cap, so a session taking many agents' messages can end its turn #27520 timer. The turnsrun_carriedstarted after it, and the carried turns a restart adopts (recover_carried), got none. So an agent message queued behind them never got the cap notice. Nowbegin_carriedarms the timer for every carried turn. A turn a restart adopts (a carried one, or a prompt's own inhosts.rsrecover_work) arms it too. The timer now counts from the turn's ownstarted_at, so an adopted turn's cap doesn't start over.Translator::holdingnow returns the newest carry's number. The in-process reader (agent.rs) and the agent host (host.rsread_held/holds) hold once per carry.begin_carriednever calledlane_turn_started, so a coordinator's carried turn on its lanes' reports reported upward. Reports never steer on their own: they wait in the queue (lane_deliver). So this happens only when the person presses Send now on a queued report and Claude Code hasn't read it by its end_turn. Each steer now records whether it is a lane report (Continues.reports). A carried turn counts as report-only when every steer it carried is a report (lane_turn_on). A carried turn a restart cuts before itsturn_startedkeeps the mark (carriedTo.reportsOnly). One that had already started comes back without it, but nothing depends on that: lane links live only in memory and aren't restored, so after a restart no turn of that session reports upward anyway.Note 2, "an unread turn-cap notice can be carried into the next prompt", is real but needs no code change. The notice line is already in Claude Code's queue when its turn ends. Claude Code takes it as soon as it emits the
result, socancel_async_messagewould lose that race, and neither the adapter nor the hub can pull it back. All that happens is one short carried turn on the notice ("finish the current step and end your turn"), which ends as the notice asks. The held digest goes next, and the carried turn keeps a cap of its own (turn_cap_noticedstill names the turn the notice was for).Tests (red, then green)
Red commits come first:
tests/hub_integration/carried_turns.rsa_carried_turn_after_a_carried_turn_has_its_own_cap(behavioral).tests/agent_host/carried_hold.rsa_second_carry_holds_claude_output_again(behavioral, about 11 s: it waits out the first hold's bound).src/hub/carried_turn.rsa_carried_turn_on_lane_reports_only_reports_nothing_upward. Base has no way to mark a steer as a report, so this red commit doesn't compile on base.src/hub/turn_cap_tests.rsan_adopted_turns_timer_counts_from_the_turns_startwas added with the fix.Observed / Not run
Ran through
cmux-ci test(Blacksmith Linux, 32 vCPU):d07170120b8(red tests on base), run 40993825537:cargo test -p acpmux --test hub_integration a_carried_turn_after_a_carried_turn_has_its_own_capfailed:the third turn's cap sent its notice,left: [].cargo test -p acpmux --test agent_host a_second_carry_holds_claude_output_againfailed:the second carry held nothing(read: LATE-2was read before its prompt came).e222db5536b, run 40994153187:cargo clippy -p acpmux --all-targets -- -D warningsis clean.carried_turn(12),turn_cap(12),carry(18) andlane_report(27) all pass.--test hub_integration carried_turns(10),--test agent_host carried_hold(2),--test lane_reports(17) and--test durable_agents carried(1) all pass.Not run: the full acpmux suite (left to PR CI), and a live Claude Code session.
Risk
risk:normal. acpmux daemon only, Claude carried turns. The turn cap is off by default (
turnCapMs). The hold change keeps the same 10 s bound per carry.Changelog
none
Agent session: agent session
🤖 Generated with Claude Code