Repository navigation
acpmux: a command waits for acpmux.service while it stops instead of starting a daemon in its own cgroup - #37323
Merged
cmux-bot[bot] merged 1 commit intoNov 6, 2026
Conversation
… daemon in its own cgroup During a restart the unit is deactivating for as long as the old daemon takes to stop. The managed-unit wait read that as gone and started a detached daemon in the calling command's cgroup. On a fleet lane host a review box's check (TasksMax 256, CPUQuota 400%) did so at 21:34:34Z; that daemon held the lock for 53 minutes, its agent hosts and background runners stayed in the review box's scope, harness and bg_start spawns failed with EAGAIN, and peers read the host as 4 CPUs. The wait now reads ActiveState and keeps waiting while the unit is active, activating, deactivating, reloading or refreshing. Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com> Agent-Session: https://cmux.com/deeplink/agent?id=aid-8707001042ea634e8771
teamleaderleo
temporarily deployed
to
ci-routing-agent
November 6, 2026 22:58 — with
GitHub Actions
Inactive
Contributor
|
cmux status (one comment, edited in place; each line is updated by the job that owns it) Review App of this head (app.zip, kept 1 day)
|
Collaborator
Author
|
Owned by an agent session · session |
Collaborator
Author
|
Review (codex, router-review, started by aid-8707001042ea634e8771): SHIP at 0312ccd Reviewer: codex gpt-6-astra high, fast tier. SHIP at 0312ccd Visual verdict: no images; Leo tries the build I found no correctness issues. Focused tests could not run because Follow-ups: none |
This branch was successfully deployed
This file contains hidden or bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
Sign up for free
to join this conversation on GitHub.
Already have an account?
Sign in to comment
Add this suggestion to a batch that can be applied as a single commit.This suggestion is invalid because no changes were made to the code.Suggestions cannot be applied while the pull request is closed.Suggestions cannot be applied while viewing a subset of changes.Only one suggestion per line can be applied in a batch.Add this suggestion to a batch that can be applied as a single commit.Applying suggestions on deleted lines is not supported.You must change the existing code in this line in order to create a valid suggestion.Outdated suggestions cannot be applied.This suggestion has been applied or marked resolved.Suggestions cannot be applied from pending reviews.Suggestions cannot be applied on multi-line comments.Suggestions cannot be applied while the pull request is queued to merge.Suggestion cannot be applied right now. Please check back later.
When
acpmux.servicerestarts, anacpmuxcommand that runs while the old daemon is still stopping no longer starts its own detached daemon. It waits for the unit's next daemon instead.The managed-unit wait (from #35174) asked
systemctl is-activeand fell back to a detached start on any non-zero answer. A unit that isdeactivatinggives a non-zero answer, and a stop takes as long as the old daemon takes to end its agents (17 s in the case below). A daemon started then runs in the calling command's cgroup, takes the lock as the old one exits, and the unit's own daemon waits behind it.On a fleet lane host on 11-06, a review box's
check(a scope withTasksMax=256andCPUQuota=400%) hit that window at 21:34:34Z. The detached daemon served the host until 22:28. Its agent hosts and background runners never moved to their own scopes, because only a daemon in a service unit moves them. They piled into the review box's 256-task scope (pids.peak 256, 198 rejected forks), so harness starts (spawn harness process /usr/bin/systemd-run: Resource temporarily unavailable (os error 11)) andbg_startrunner spawns failed. Peers also read the host as 4 CPUs, becauseavailable_parallelismhonors the scope's CPU quota. The host has 32 CPUs.The wait now reads
ActiveState. It keeps waiting while the unit isactive,activating,deactivating,reloadingorrefreshing, and takes the detached path only forinactive,failedor a probe that doesn't answer. Stops and starts are already bounded by the unit's own timeouts.Tests:
managed_unit_still_stopping_keeps_the_wait(adeactivatingunit keeps the wait until its daemon serves) andmanaged_unit_gone_takes_the_detached_path(inactiveandfailedstill fall back), with fakesystemctlscripts like the ones the managed-unit tests already use.cargo fmt --checkandcargo test -p acpmux --lib managed_unitpassed on the Linux pool at 0312ccd, six runs (three parallel, three with--test-threads=1), 4/4 each. One earlier parallel run on a cold worker failed the existingmanaged_unit_start_avoids_a_symlinked_default_homeonce (itssystemctl startnever landed, so it took the detached path). It passed in all six reruns, and that test doesn't reach the changed probe on its success path.Changelog
none
Agent session: agent session
🤖 Generated with Claude Code