Skip to content

base red: acpmux: two #27921 regressions (inbox held 30 s after every turn; a put-back message's resend answered withdrawn) - #28477

Merged
teamleaderleo merged 4 commits into
feat-cmux-nextfrom
lane/inbox-restart-base-red
Oct 30, 2026
Merged

teamleaderleo merged 4 commits into
feat-cmux-nextfrom
lane/inbox-restart-base-red

Conversation

@teamleaderleo

@teamleaderleo teamleaderleo commented Oct 30, 2026 •

Copy link
Copy Markdown
Collaborator

Fixes two acpmux base reds on feat-cmux-next. The bisect over the acpmux commits since #28369 puts both at #27921 (a9e3148): both tests pass at its parent c8f95de and fail from a9e3148 on. They are two separate bugs in #27921's code; neither test was wrong.

  • agent_inbox::an_inbox_message_from_before_the_review_tag_never_steers_into_the_review panics at agent_inbox.rs:73 with "unanswered: 1 of 3".
  • inbox_digest::a_digest_refused_before_its_turn_puts_every_message_back_and_a_restart_keeps_them panics at inbox_digest.rs:666, where the resend answers {"duplicate":true,"status":"withdrawn","stopReason":"cancelled"}.

1. Every turn end put the inbox's wake off by 30 s

finish_turn → take_turn marks the ended turn Mark::Ending until schedule_auto_resume decides whether a continuation follows. In between, finish_turn's inbox_turn_ended → inbox_review reads continuation_pending() as true. #27921 added that check: it puts the wake off by REFUSED_RETRY_MS (30 s), and nothing looks at the inbox again when the mark clears. So after every turn end, an idle chat's held message waited 30 s. The review-tag test waits 20 s for the message's own turn, which is why it panics there. The other ways a pending continuation goes leave the same 30 s wait: exhausted or disabled retries, the restored retry's off switch, Stop, and a fired retry refused before its turn.

Fix (auto_resume.rs, inbox.rs): Pending::lock() now returns a guard. When the guard drops, it bumps a watch counter (Pending::released) if the mark was a continuation that may still fire (Waiting, Firing, Ending) and no longer is. The inbox's wake timer sleeps until its time or that counter changes, whichever comes first, then reviews the inbox. inbox_review subscribes before it reads continuation_pending, so a release between the two still wakes it. Every way a continuation goes, including ways added later, lets the inbox in at once. A turn that is cut and continued still holds the inbox until its continuation runs, as #27921 intended.

2. A resend of a message a refused digest put back answered "withdrawn"

logged_prompt scans a prompt's records. #27921 widened the user_message | queued arm from None to _, so it now also takes the requeued: true record and shadows the arm that resets digest. The refused digest then stays the prompt's digest, and its queue_removed, logged after the messages go back, answers the prompt as withdrawn. That holds even though it ran later on its own turn.

Fix: the requeued arm goes ahead of the widened arm (prompt_ledger.rs), as the arm order was before #27921.

Tests

The failing tests are committed before the fix (118ff3d; the inbox test's check was corrected in 4603c82):

  • hub::inbox::tests::a_turn_that_ends_with_no_continuation_lets_the_inbox_in_at_once covers both the turn's settle and a Stop.
  • hub::prompt_ledger::tests::a_message_a_refused_digest_put_back_answers_from_its_own_turn

Results from run 41113214430, at head 4603c82 and on base src 18fa44f with the head's tests:

test head base src
a_turn_that_ends_with_no_continuation_lets_the_inbox_in_at_once ok ×3 FAILED: held past the settle (stop: false)
a_message_a_refused_digest_put_back_answers_from_its_own_turn ok FAILED, prompt_ledger.rs:457 (run 41110817062)
inbox_digest::a_digest_refused_before_its_turn_puts_every_message_back_and_a_restart_keeps_them ok ×4 FAILED, inbox_digest.rs:666
agent_inbox::an_inbox_message_from_before_the_review_tag_never_steers_into_the_review ok ×4 (2.2 s) FAILED, agent_inbox.rs:73 (20 s)
held_queue::agent_messages_joined_to_a_send_whose_agent_cannot_start_are_not_held ok ×5 ok ×5
cargo test -p acpmux --lib 2347 passed ×2
cargo test -p acpmux --test hub_integration 620 passed; one other run: 619 + prompt_supersede flake (below) 619 + the review-tag red
cargo clippy -p acpmux --all-targets -D warnings, panic ratchet clean

held_queue::…cannot_start_are_not_held timed out once in a full-suite run at be41f07, then passed 5/5 at the head and 5/5 on base, so it is not this change.

Observed / Not run

  • prompt_supersede::a_prompt_queued_before_a_restart_is_still_replaced is a flake that predates this PR: it was seen on the base tip, and it failed once in two full hub_integration runs at the head. It passed 10/10 run alone (5 at the head, 5 on base). It fails at prompt_supersede.rs:257, "the cut turn did not go on", which is acpmux: preserve retries and adopted prompt outcomes across restart (takes over #26981) #27921's restart continuation; this PR doesn't touch that. I did not diagnose it further.
  • hermetic_daemon_guard is red on base and not changed here.
  • Reviewed by Sol (gpt-6-sol, high). It blocked the first version, which reviewed the inbox only at the turn's settle and missed the other ways a continuation goes. It gave SHIP at be41f07, then SHIP at 4603c82 for the test-only delta.

Changelog

none

Risk: normal. The change is two small changes to acpmux's inbox wake and its resend lookup.

Agent session: agent session

🤖 Generated with Claude Code

teamleaderleo and others added 2 commits October 30, 2026 18:46
a_turn_that_ends_with_no_continuation_lets_the_inbox_in_at_once: a turn's
end reviews the inbox while take_turn still marks it ending, so
continuation_pending defers the wake 30 s; the settle that clears the mark
never looks again (agent_inbox review-tag test, unanswered at 20 s).

a_message_a_refused_digest_put_back_answers_from_its_own_turn: logged_prompt's
widened user_message/queued arm shadows the requeued arm, so the refused
digest stays this prompt's and its later queue_removed answers the resend as
withdrawn (inbox_digest refused-digest restart test, line 666).

Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
Agent-Session: https://cmux.com/deeplink/agent?id=aid-3db671e49c5e0014c84a
…resends as withdrawn

A turn that ends with no continuation reviews the inbox again once
schedule_auto_resume clears its ending mark: the review at the turn's end
read continuation_pending as true (take_turn had just marked it ending) and
put the wake off by REFUSED_RETRY_MS, 30 s, for every turn end.

logged_prompt reads a requeued queued record ahead of the widened
user_message/queued arm, as before #27921: the record resets the prompt's
turn and lets go of the refused digest, whose later queue_removed then no
longer answers the prompt as withdrawn.

Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
Agent-Session: https://cmux.com/deeplink/agent?id=aid-3db671e49c5e0014c84a
@github-actions

github-actions Bot commented Oct 30, 2026 •

Copy link
Copy Markdown
Contributor

cmux status (one comment, edited in place; each line is updated by the job that owns it)

Review 4603c82: HQ feed card · gallery before/after runs · checks

App of this head (app.zip, kept 1 day)

gh workflow run reload-build.yml -R manaflow-ai/cmux --ref feat-cmux-next -f tag=pr28477 -f ref=lane/inbox-restart-base-red -f platform=macos -f backend_mode=local

… not only at a turn's settle

Sol review (BLOCK at a398803): the settle was one of several ways a pending
continuation goes (exhausted or disabled retries, the restored retry's
off switch, Stop, a fired retry refused before its turn), and each left a
wake put off by REFUSED_RETRY_MS.

Pending's lock now counts each time its mark stops being a continuation that
may still fire (Waiting, Firing, Ending to anything else), and the inbox's
wake timer also wakes on that count. inbox_review subscribes before it reads
continuation_pending, so a release between the two still wakes the timer.
The explicit inbox_review in schedule_auto_resume goes; the test now covers
the settle and a Stop.

Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
Agent-Session: https://cmux.com/deeplink/agent?id=aid-3db671e49c5e0014c84a
@github-actions github-actions Bot added the risk:normal Risk tier (docs/ci/risk-tiers.md) label Oct 30, 2026
@teamleaderleo

Copy link
Copy Markdown
Collaborator Author

Owned by an agent session · session

…e still in flight

In a hub with no agent, the notice the timer sends is refused at once, and
notice_done clears in_flight again before a 10 ms look sees it (red at
be41f07 for that reason alone). A notice tried leaves in_flight, refused_at,
or a noticed or answered item.

Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
Agent-Session: https://cmux.com/deeplink/agent?id=aid-3db671e49c5e0014c84a
@teamleaderleo

teamleaderleo commented Oct 30, 2026 •

Copy link
Copy Markdown
Collaborator Author

Review (the owner lane's independent review, at its land request): SHIP at 4603c82

Recorded by lane-gh-broker from this PR's owner lane, which asked to land this exact head after its independent review. next-autoland lands it once every red is the base's; a BLOCK at this head, a HOLD line or a hold label still stops it.

@teamleaderleo
teamleaderleo merged commit ae16eb8 into feat-cmux-next Oct 30, 2026
238 of 323 checks passed
@teamleaderleo
teamleaderleo deleted the lane/inbox-restart-base-red branch October 30, 2026 19:36
@github-actions

github-actions Bot commented Oct 30, 2026 •

Copy link
Copy Markdown
Contributor

This branch was previously deployed

3 inactive deployments
artifacts — 4603c82a Deployed Oct 30, 2026 by teamleaderleo via add the Windows daemon to the tree #84668
ci-routing-agent — 4603c82a Deployed Oct 30, 2026 by github-actions[bot] via cmux-next macOS placement #50836
sentry-dsyms — 4603c82a Deployed Oct 30, 2026 by teamleaderleo via upload cmux-tui dSYMs to Sentry #84668
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

risk:normal Risk tier (docs/ci/risk-tiers.md)

Projects

None yet

Development

Successfully merging this pull request may close these issues.

1 participant