Skip to content

acpmux: a SIGTERM stop names its sender - #35147

Merged
teamleaderleo merged 6 commits into
feat-cmux-nextfrom
lane/acpmux-update-sigterm
Nov 4, 2026
Merged

teamleaderleo merged 6 commits into
feat-cmux-nextfrom
lane/acpmux-update-sigterm

Conversation

@teamleaderleo

@teamleaderleo teamleaderleo commented Nov 4, 2026 •

Copy link
Copy Markdown
Collaborator

Leo, 2026-11-04: "i feel like the restarted after nightly update does too many layout shifts/flicker renderings and takes too long to reconnect acpmux or whatever". Data: manaflow-ai/cmuxterm-hq#6338 (comments 6597127823 and 6626332314), bench in #34697.

After an update relaunch on macOS the old acpmux daemon sometimes logs stopping: SIGTERM (a restart handoff) about 1.1 s after the app quit and about 5 s before the new app starts. Its agent host is lost with it, so the next daemon resumes the streaming turn on a fresh agent. Nothing in the log says who sent the signal. The app, the bench and acpmux itself have no code that sends SIGTERM to the daemon on that path. The daemon and its hosts each lead their own session, so a process-group kill can't explain it.

This PR makes the daemon say who sent it. A SA_SIGINFO action next to tokio's own records si_pid; both go through signal-hook-registry, which runs every action registered for the signal. The action does only an atomic store, so it is async-signal-safe. The stop line then reads stopping: SIGTERM from pid N (command), parent M (command) (a restart handoff). A fleet bench run at this head (relaunch-reconnect:warm+update:4) is collecting the sender for the fix that follows.

Test

  • Red/green: cargo test -p acpmux --test daemon_lifecycle a_sigterm_stop_names_its_sender fails at the red commit (no sender on the stop line) and passes at the head.
  • Unit: daemon::term_sender::tests (ps row parsing; this process is described with its command).

Observed / Not run

  • See the run results below.

Performance

metric before after delta harness
work per SIGTERM 0 1 atomic store + 2 ps reads on the stop path stop path only code read; no hot path

Risk

Low: only the daemon's SIGTERM stop path changes, plus one extra signal action. If the action can't be registered, the daemon logs a warning and stops as before.

Changelog

none

Agent session: agent session

🤖 Generated with Claude Code


Autofix with [code]smith
Open in [code]smith · Need help on this PR? Tag @codesmith-bot with what you need.


Summary by cubic

Names the SIGTERM sender on the acpmux daemon's stop line so an unexplained SIGTERM after a NIGHTLY update relaunch on macOS can be traced.

The daemon now logs stopping: SIGTERM from pid N (command), parent M (command) instead of the plain stopping: SIGTERM (a restart handoff). A SA_SIGINFO action records the sender's pid via signal-hook-registry; the action only does an atomic store, so it is async-signal-safe. It is registered only after tokio's own stream exists, so SIGTERM's default action is never replaced. The sender's command and parent are read from ps at stop time. The lookup is a reviewed process-scan exception: it reads one kernel-named pid once on the stop path and enumerates nothing. If the action can't be registered, the daemon warns and stops as before.

Low risk: only the SIGTERM stop path changes, plus one extra signal action.

Written for commit 9a2e1da. Summary will update on new commits.

View guided diff Turn on auto-fix

teamleaderleo and others added 2 commits November 4, 2026 08:42
After a NIGHTLY update relaunch on macOS the old daemon sometimes got a
SIGTERM about 1 s after the app quit, and its log said only "SIGTERM"
(manaflow-ai/cmuxterm-hq#6338), so nothing named who sent it.

Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
Agent-Session: https://cmux.com/deeplink/agent?id=aid-cb01117327fc358e9ba1
A SA_SIGINFO action beside tokio's own records si_pid; the stop line then says
"stopping: SIGTERM from pid N (command), parent M (command)". After a NIGHTLY
update relaunch on macOS the old daemon sometimes got a SIGTERM before the new
app started and its agent host was lost with it (manaflow-ai/cmuxterm-hq#6338);
nothing in its log named the sender.

Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
Agent-Session: https://cmux.com/deeplink/agent?id=aid-cb01117327fc358e9ba1
@cursor

cursor Bot commented Nov 4, 2026

Copy link
Copy Markdown

Bugbot needs on-demand usage enabled

Bugbot uses usage-based billing for this team and requires on-demand usage to be enabled.

A team admin can enable on-demand usage in the Cursor dashboard.

@github-actions

github-actions Bot commented Nov 4, 2026 •

Copy link
Copy Markdown
Contributor

cmux status (one comment, edited in place; each line is updated by the job that owns it)

Review 18057fe: HQ feed card · gallery before/after runs · checks

App of this head (app.zip, kept 1 day)

gh workflow run reload-build.yml -R manaflow-ai/cmux --ref feat-cmux-next -f tag=pr35147 -f ref=lane/acpmux-update-sigterm -f platform=macos -f backend_mode=local

web and size checks: green

Shipped in:

@github-actions github-actions Bot added the risk:normal Risk tier (docs/ci/risk-tiers.md) label Nov 4, 2026
term_sender reads one pid, the one the kernel named as the SIGTERM's sender, once on the
stop path, to name it and its parent on the stop line. It enumerates nothing. Review note:
one kernel-named pid, read-only, off every hot path.

Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
Agent-Session: https://cmux.com/deeplink/agent?id=aid-cb01117327fc358e9ba1
@blacksmith-sh

blacksmith-sh Bot commented Nov 4, 2026

Copy link
Copy Markdown

Found 2 test failures on Blacksmith runners:

Failures

Test View Logs
nightly cmux-tui resolve/nightly cmux-tui resolve View Logs
the short nightly decide gate must use the pooled Linux route with a Blacksmith fallbac
k/
the short nightly decide gate must use the pooled Linux route with a Blacksmith fallbac
k
View Logs

Fix with [code]smith
Need help on this PR? Tag @codesmith-bot with what you need.

@teamleaderleo

Copy link
Copy Markdown
Collaborator Author

Review (codex, router-review): SHIP at 18057fe

Reviewer: codex gpt-6-astra high, fast tier.

Follow-ups: 1. They land as checklist items in this PR's one follow-up tracking issue in this repo (Follow-ups: #N ..., label follow-up; add to an open one before filing, never one issue per finding), linking this receipt; the lane that opened the PR fixes them forward.

SHIP at 18057fe

Visual verdict: no images; Leo tries the build.

  • [P1] [follow-up] [category:reliability] [finding:F1] [outcome:pending] If term_sender::record() succeeds but Tokio’s subsequent SIGTERM registration fails at daemon.rs:592-596, the signal-hook handler replaces the default disposition while next_signal(None) waits forever. SIGTERM then no longer stops the daemon. Retain the registration handle and restore/unregister on Tokio setup failure, or explicitly terminate in that case.

Focused tests were not runnable because this box has no cargo.

Follow-ups:

  • F1: Restore SIGTERM’s termination behavior when Tokio signal registration fails.

Review finding F1 (Codex SHIP at 18057fe): with our action alone, SIGTERM's default
action is replaced, so a daemon whose tokio stream failed to register would never
stop on SIGTERM. The sender action is now added only after that stream exists.

Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
Agent-Session: https://cmux.com/deeplink/agent?id=aid-cb01117327fc358e9ba1
@teamleaderleo

Copy link
Copy Markdown
Collaborator Author

Review (codex, router-review): SHIP at 4985441

Reviewer: codex gpt-6-astra high, fast tier.

Follow-ups: 1. They land as checklist items in this PR's one follow-up tracking issue in this repo (Follow-ups: #N ..., label follow-up; add to an open one before filing, never one issue per finding), linking this receipt; the lane that opened the PR fixes them forward.

SHIP at 4985441
Visual verdict: no images; Leo tries the build

  1. [P2] [follow-up] [category:correctness] [finding:F1] [outcome:pending] cmux-tui/crates/acpmux/src/daemon/term_sender.rs:24: Tokio’s earlier callback can wake the shutdown task on another thread before this callback stores SENDER. The multithreaded daemon can consequently read zero and omit the sender. Running both callbacks within one signal delivery does not guarantee ordering against other threads. Ensure sender capture precedes notification, and cover cross-thread delivery in a test.

Rust tests could not run because cargo is unavailable. The process-scan guard passed.

Follow-ups:

  • F1: Synchronize sender capture with shutdown notification.

cmux-bot[bot]
cmux-bot Bot previously approved these changes Nov 4, 2026

@cmux-bot cmux-bot Bot left a comment

Copy link
Copy Markdown

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

cmux-bot approves 4985441: an independent review passed this exact head (router-review receipt by teamleaderleo). A push needs a new review; a later BLOCK at this head dismisses this approval.

@cmux-bot
cmux-bot Bot dismissed their stale review November 4, 2026 09:59

A newer verdict blocks 9a2e1da (no independent SHIP at the head).

@teamleaderleo

Copy link
Copy Markdown
Collaborator Author

Review (codex gpt-6-astra high, router-review range e830594..9a2e1da): SHIP at 9a2e1da

Visual verdict: unchanged by this diff (no UI).

No correctness, regression, security or test findings. The head differs from the SHIP at 4985441 only by the lander's base merge.

Follow-ups: none

@cmux-bot
cmux-bot Bot temporarily deployed to ci-routing-agent November 4, 2026 10:24 Inactive
@teamleaderleo
teamleaderleo merged commit 9d33307 into feat-cmux-next Nov 4, 2026
225 of 232 checks passed
@teamleaderleo
teamleaderleo deleted the lane/acpmux-update-sigterm branch November 4, 2026 10:55
teamleaderleo added a commit that referenced this pull request Nov 4, 2026
…on an update relaunch) (#35291)

* test(acpmux): a host ended by SIGTERM names its sender (red)

After an update relaunch on macOS the old daemon got a SIGTERM from loginwindow
(#35147's stop line) and its agent host was lost; hosts log nothing, so nothing
says whether the host got that signal too (manaflow-ai/cmuxterm-hq#6338).

Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
Agent-Session: https://cmux.com/deeplink/agent?id=aid-cb01117327fc358e9ba1

* acpmux: hosts note their SIGTERM sender; the daemon carries none of the app's launch identity

- An agent host that ends on SIGTERM appends who sent it (pid, command, parent) to the
  home's agent-host-signals.log; the daemon's stop line names its sender since #35147.
- The sender is read once its action has stored it (#35147 review F1): tokio's action
  can wake the reader on another thread first.
- The app spawns its daemon without its own launch identity (__CFBundleIdentifier,
  XPC_SERVICE_NAME, XPC_FLAGS, LaunchInstanceID). After an update relaunch, loginwindow
  sent the old daemon SIGTERM 1.08 s after the app quit, while the bundle was replaced
  (fleet run 85017cc90b72d8076d0e56ff), and its agent host was lost with it.

Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
Agent-Session: https://cmux.com/deeplink/agent?id=aid-cb01117327fc358e9ba1

* acpmux: drop the launch-identity strip: loginwindow still ended the daemon and its hosts without it

Fleet bench at 2faef0a (step ef77e027a15c52e8fa79ae10): with __CFBundleIdentifier, XPC_SERVICE_NAME,
XPC_FLAGS and LaunchInstanceID stripped, update rounds 2 and 3 still logged a SIGTERM from
loginwindow (pid 179) at the daemon, and agent-host-signals.log shows it reached agent hosts
directly within 14 ms. So it is not the environment; the strip is taken out.

Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
Agent-Session: https://cmux.com/deeplink/agent?id=aid-cb01117327fc358e9ba1

---------

Co-authored-by: Claude Opus 5.5 <noreply@anthropic.com>
@teamleaderleo

Copy link
Copy Markdown
Collaborator Author

Finding outcome: comment-6627533347/F1 fixed c9f0649

This branch was previously deployed

3 inactive deployments
ci-routing-agent — 9a2e1da4 Deployed Nov 4, 2026 by github-actions[bot] via cmux-next macOS placement #69630
artifacts — 9a2e1da4 Deployed Nov 4, 2026 by cmux-bot[bot] via publish tree-addressed cmux-tui #111026
sentry-dsyms — 9a2e1da4 Deployed Nov 4, 2026 by cmux-bot[bot] via upload cmux-tui dSYMs to Sentry #111026
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

risk:normal Risk tier (docs/ci/risk-tiers.md)

Projects

None yet

Development

Successfully merging this pull request may close these issues.

1 participant