Skip to content

acpmux: a command waits for acpmux.service while it stops instead of starting a daemon in its own cgroup - #37323

Merged
cmux-bot[bot] merged 1 commit into
feat-cmux-nextfrom
fm3h--acpmux-unit-stopping-no-detached
Nov 6, 2026
Merged

cmux-bot[bot] merged 1 commit into
feat-cmux-nextfrom
fm3h--acpmux-unit-stopping-no-detached

Conversation

@teamleaderleo

@teamleaderleo teamleaderleo commented Nov 6, 2026 •

Copy link
Copy Markdown
Collaborator

When acpmux.service restarts, an acpmux command that runs while the old daemon is still stopping no longer starts its own detached daemon. It waits for the unit's next daemon instead.

The managed-unit wait (from #35174) asked systemctl is-active and fell back to a detached start on any non-zero answer. A unit that is deactivating gives a non-zero answer, and a stop takes as long as the old daemon takes to end its agents (17 s in the case below). A daemon started then runs in the calling command's cgroup, takes the lock as the old one exits, and the unit's own daemon waits behind it.

On a fleet lane host on 11-06, a review box's check (a scope with TasksMax=256 and CPUQuota=400%) hit that window at 21:34:34Z. The detached daemon served the host until 22:28. Its agent hosts and background runners never moved to their own scopes, because only a daemon in a service unit moves them. They piled into the review box's 256-task scope (pids.peak 256, 198 rejected forks), so harness starts (spawn harness process /usr/bin/systemd-run: Resource temporarily unavailable (os error 11)) and bg_start runner spawns failed. Peers also read the host as 4 CPUs, because available_parallelism honors the scope's CPU quota. The host has 32 CPUs.

The wait now reads ActiveState. It keeps waiting while the unit is active, activating, deactivating, reloading or refreshing, and takes the detached path only for inactive, failed or a probe that doesn't answer. Stops and starts are already bounded by the unit's own timeouts.

Tests: managed_unit_still_stopping_keeps_the_wait (a deactivating unit keeps the wait until its daemon serves) and managed_unit_gone_takes_the_detached_path (inactive and failed still fall back), with fake systemctl scripts like the ones the managed-unit tests already use. cargo fmt --check and cargo test -p acpmux --lib managed_unit passed on the Linux pool at 0312ccd, six runs (three parallel, three with --test-threads=1), 4/4 each. One earlier parallel run on a cold worker failed the existing managed_unit_start_avoids_a_symlinked_default_home once (its systemctl start never landed, so it took the detached path). It passed in all six reruns, and that test doesn't reach the changed probe on its success path.

Changelog

none

Agent session: agent session

🤖 Generated with Claude Code

… daemon in its own cgroup

During a restart the unit is deactivating for as long as the old daemon takes
to stop. The managed-unit wait read that as gone and started a detached daemon
in the calling command's cgroup. On a fleet lane host a review box's
check (TasksMax 256, CPUQuota 400%) did so at 21:34:34Z; that daemon held
the lock for 53 minutes, its agent hosts and background runners stayed in the
review box's scope, harness and bg_start spawns failed with EAGAIN, and peers
read the host as 4 CPUs. The wait now reads ActiveState and keeps waiting while
the unit is active, activating, deactivating, reloading or refreshing.

Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
Agent-Session: https://cmux.com/deeplink/agent?id=aid-8707001042ea634e8771
@github-actions

github-actions Bot commented Nov 6, 2026

Copy link
Copy Markdown
Contributor

cmux status (one comment, edited in place; each line is updated by the job that owns it)

Review 0312ccd: HQ feed card · gallery before/after runs · checks

App of this head (app.zip, kept 1 day)

gh workflow run reload-build.yml -R manaflow-ai/cmux --ref feat-cmux-next -f tag=pr37323 -f ref=fm3h--acpmux-unit-stopping-no-detached -f platform=macos -f backend_mode=local

@teamleaderleo

Copy link
Copy Markdown
Collaborator Author

Owned by an agent session · session

@github-actions github-actions Bot added the risk:normal Risk tier (docs/ci/risk-tiers.md) label Nov 6, 2026
@teamleaderleo

Copy link
Copy Markdown
Collaborator Author

Review (codex, router-review, started by aid-8707001042ea634e8771): SHIP at 0312ccd

Reviewer: codex gpt-6-astra high, fast tier.

SHIP at 0312ccd

Visual verdict: no images; Leo tries the build

I found no correctness issues. Focused tests could not run because cargo is unavailable in the review box.

Follow-ups: none

@cmux-bot cmux-bot Bot left a comment

Copy link
Copy Markdown

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

cmux-bot approves 0312ccd: an independent review passed this exact head (router-review receipt by teamleaderleo). A push needs a new review; a later BLOCK at this head dismisses this approval.

@cmux-bot
cmux-bot Bot merged commit c767ef8 into feat-cmux-next Nov 6, 2026
215 of 225 checks passed
@cmux-bot
cmux-bot Bot deleted the fm3h--acpmux-unit-stopping-no-detached branch November 6, 2026 23:16

This branch was successfully deployed

2 active and 1 inactive deployments
ci-routing-agent — 0312ccde Deployed Nov 6, 2026 by github-actions[bot] via cmux-next macOS placement #76004
artifacts — 0312ccde Deployed Nov 6, 2026 by teamleaderleo via publish tree-addressed cmux-tui #120057
sentry-dsyms — 0312ccde Deployed Nov 6, 2026 by teamleaderleo via upload cmux-tui dSYMs to Sentry #120057
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

risk:normal Risk tier (docs/ci/risk-tiers.md)

Projects

None yet

Development

Successfully merging this pull request may close these issues.

1 participant