Repository navigation
acpmux: start agent harnesses at background CPU class on macOS, so cmux and its renderers go first - #27600
Conversation
… class on macOS (red) Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com> Agent-Session: https://cmux.com/deeplink/agent?id=aid-1288bbb2b5d2f5c6b66a
Agent harnesses and the tools they run competed with cmux and its Chromium renderers at the same priority; under load a dragged pane's page trailed it. Each session's harness now starts with PRIO_DARWIN_BG, set between fork and exec (direct children and agent hosts), so everything it runs inherits it. config session.cpuClass, the daemon's ACPMUX_SESSION_CPU_CLASS and a profile's env pick normal to opt out. Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com> Agent-Session: https://cmux.com/deeplink/agent?id=aid-1288bbb2b5d2f5c6b66a
Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com> Agent-Session: https://cmux.com/deeplink/agent?id=aid-1288bbb2b5d2f5c6b66a
|
Owned by an agent session · session |
|
cmux status (one comment, edited in place; each line is updated by the job that owns it) Review App of this head (app.zip, kept 1 day)
web and size checks: green |
|
Found 6 test failures on Blacksmith runners: Failures
|
…d wait the test can release
Review: a profile's ${env:...} reference that resolves to normal was read
unresolved, so the harness started at background class anyway. The class is
now read from the command's own environment, as the harness gets it.
Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
Agent-Session: https://cmux.com/deeplink/agent?id=aid-1288bbb2b5d2f5c6b66a
Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
…rocess env, not set_var Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com> Agent-Session: https://cmux.com/deeplink/agent?id=aid-1288bbb2b5d2f5c6b66a
Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com> # Conflicts: # cmux-tui/crates/acpmux/src/cli/command.rs # cmux-tui/crates/acpmux/src/config.rs # cmux-tui/crates/acpmux/src/config/tests.rs # cmux-tui/crates/acpmux/src/hub/catalog_reload.rs
|
Review (codex gpt-6-astra high, next-review): SHIP at eb6d68c No blocking findings. |
teamleaderleo
left a comment
There was a problem hiding this comment.
Review (codex gpt-6-astra high fast, independent review): SHIP at eb6d68c
Rounds:
- 8a0720b: BLOCK. The class was read from the unresolved profile env, and the FIFO wait could hang. Both fixed in 11bc618.
- b549c72: BLOCK.
set_varwas called during tests; fixed in abd6b68. - abd6b68: SHIP.
- eb6d68c: SHIP. This head only merges feat-cmux-next (#27555's task caps); the review checked the conflict resolution in config.rs, config/tests.rs, cli/command.rs and catalog_reload.rs.
Reds, compared per test against base:
-
cmux-next/compile, and the daemon tests job built on it: the one error is
JoinConsent+App.swift:222:29: 'async' call in a function that does not support concurrency. The same error appears at the merge parent 1b67652 (fleet swift-check ac092aec), and #27758 fixed it on base afterwards. This PR changes no Swift. -
release-notary
test_the_repository_highlights_validate, and Guard status: the test flags release-notes highlights (chat-find-keeps-place, inline-pr-pills, inspector-error-colours) that lacked a tryIt or docs link at the merge parent. #25839 (0693908) fixed them on base. This PR changes no release notes. -
Linux
cmux-tui-rust-check all session: the same three tests fail at base 1b67652 with the same assertions:transcript_events::orchestration_fields_reach_subagent_events_and_session_summarysend_relay_cli a_target_nobody_here_resolves_goes_to_the_relay_and_a_session_here_never_doesvm_child_sessions_journey session_children_journey_lists_follows_and_opens_a_vm_chats_child
These come from fleet steps 7e11795e (head) and ef092519 (base).
Green at this head: macOS test cpu_class, 6 of 6 (fleet step dc5de660); macOS clippy (6c4c56d0); rust workspaces.
|
The feat-cmux-next push run https://github.com/manaflow-ai/cmux/actions/runs/40993317688 at f5e762b failed: cmux-next checks (god files, concurrency, crash safety, l10n). |
|
Shipped in:
|
|
Shipped in:
|
…e daemon guard (#27983) #27600 added tests/cpu_class.rs, which starts the agent host the same way agent_host.rs and cwd_pin.rs do (its child gets daemon_env::vars as SpawnSpec.env), but did not list it, so hermetic_daemon_guard fails on base. Agent-Session: https://cmux.com/deeplink/agent?id=aid-710a4a518c009a6ad126 Co-authored-by: Claude Opus 5.5 <noreply@anthropic.com>
…names its harness and a background one runs once more (red) Behavior tests through an in-process daemon's ACP client (macOS): the harness answers session/new above background class and runs at it once started; a tool its turn starts inherits it; an agent_start deadline error names the agent process; a start held at background class by another process is retried once. Red on the base: #27600 starts the harness at PRIO_DARWIN_BG before its exec.
… its session's start is over #27600 started every harness at PRIO_DARWIN_BG. On 2026-10-30 (load 125 on 18 cores, du and git scans of a 3.3 TB disk) a claude harness at background class spent 90 s in trust-settings reads with under 1 s of CPU, every Chief turn failed ("session/new: no answer within 60 s", "no harness session within 90s"). - The harness starts at its starter's class and clears an inherited background class at its exec (start_at_normal). Once ensure_child's start is over, the daemon puts the running harness in its class with setpriority(PRIO_DARWIN_PROCESS, pid, PRIO_DARWIN_BG); later tools inherit it. The run class rides on ChildAgent; SpawnSpec.cpu_class is gone. - A cold start that fails or misses agent_start with its harness at background class anyway runs once more, its harness's external class also cleared, and logs agent_start_retried_at_normal. - agent_start and the pool's start deadline errors name the agent process: pid, class, priority, run time, CPU time, threads running (proc_pidinfo of the one pid; no process listing). data.harness carries the same fields. - The hosted behavior test: under a background agent host (ACPMUX_QOS=background) the harness starts above it and runs below once lowered.
…names its harness and a background one runs once more (red) Behavior tests through an in-process daemon's ACP client (macOS): the harness answers session/new above background class and runs at it once started; a tool its turn starts inherits it; an agent_start deadline error names the agent process; a start held at background class by another process is retried once. Red on the base: #27600 starts the harness at PRIO_DARWIN_BG before its exec.
… its session's start is over #27600 started every harness at PRIO_DARWIN_BG. On 2026-10-30 (load 125 on 18 cores, du and git scans of a 3.3 TB disk) a claude harness at background class spent 90 s in trust-settings reads with under 1 s of CPU, every Chief turn failed ("session/new: no answer within 60 s", "no harness session within 90s"). - The harness starts at its starter's class and clears an inherited background class at its exec (start_at_normal). Once ensure_child's start is over, the daemon puts the running harness in its class with setpriority(PRIO_DARWIN_PROCESS, pid, PRIO_DARWIN_BG); later tools inherit it. The run class rides on ChildAgent; SpawnSpec.cpu_class is gone. - A cold start that fails or misses agent_start with its harness at background class anyway runs once more, its harness's external class also cleared, and logs agent_start_retried_at_normal. - agent_start and the pool's start deadline errors name the agent process: pid, class, priority, run time, CPU time, threads running (proc_pidinfo of the one pid; no process listing). data.harness carries the same fields. - The hosted behavior test: under a background agent host (ACPMUX_QOS=background) the harness starts above it and runs below once lowered.
…names its harness and a background one runs once more (red) Behavior tests through an in-process daemon's ACP client (macOS): the harness answers session/new above background class and runs at it once started; a tool its turn starts inherits it; an agent_start deadline error names the agent process; a start held at background class by another process is retried once. Red on the base: #27600 starts the harness at PRIO_DARWIN_BG before its exec.
… its session's start is over #27600 started every harness at PRIO_DARWIN_BG. On 2026-10-30 (load 125 on 18 cores, du and git scans of a 3.3 TB disk) a claude harness at background class spent 90 s in trust-settings reads with under 1 s of CPU, every Chief turn failed ("session/new: no answer within 60 s", "no harness session within 90s"). - The harness starts at its starter's class and clears an inherited background class at its exec (start_at_normal). Once ensure_child's start is over, the daemon puts the running harness in its class with setpriority(PRIO_DARWIN_PROCESS, pid, PRIO_DARWIN_BG); later tools inherit it. The run class rides on ChildAgent; SpawnSpec.cpu_class is gone. - A cold start that fails or misses agent_start with its harness at background class anyway runs once more, its harness's external class also cleared, and logs agent_start_retried_at_normal. - agent_start and the pool's start deadline errors name the agent process: pid, class, priority, run time, CPU time, threads running (proc_pidinfo of the one pid; no process listing). data.harness carries the same fields. - The hosted behavior test: under a background agent host (ACPMUX_QOS=background) the harness starts above it and runs below once lowered.
…c environment Since #27600 a harness goes to PRIO_DARWIN_BG once its start is over. The stand-ins time turns with short sleeps; on a loaded fleet Mac (load1 13-21 on 10 P-cores) a background-class Codex-steer stand-in did not end its turn in 18-30 s, so stop::a_slow_stop_does_not_fold..., forum's steered rule and steer_end's race failed only on macOS (diag: 7057c820, ad77491f, 8432fc15; the same 3 pass with the class normal: a35af9a6, 0b8d6072, 1935c6fb). daemon_env::vars sets ACPMUX_SESSION_CPU_CLASS=normal; tests/cpu_class.rs asks for background in its process and its real daemon. Product default unchanged.
… reds on loaded fleet Macs) (#30057) * test(acpmux): a test's stand-in harness runs at normal CPU class (red on macOS) * acpmux tests: stand-in harnesses keep normal CPU class in the hermetic environment Since #27600 a harness goes to PRIO_DARWIN_BG once its start is over. The stand-ins time turns with short sleeps; on a loaded fleet Mac (load1 13-21 on 10 P-cores) a background-class Codex-steer stand-in did not end its turn in 18-30 s, so stop::a_slow_stop_does_not_fold..., forum's steered rule and steer_end's race failed only on macOS (diag: 7057c820, ad77491f, 8432fc15; the same 3 pass with the class normal: a35af9a6, 0b8d6072, 1935c6fb). daemon_env::vars sets ACPMUX_SESSION_CPU_CLASS=normal; tests/cpu_class.rs asks for background in its process and its real daemon. Product default unchanged. * acpmux: the steer race unit test's stand-in runs at normal CPU class
![Fix with [code]smith](https://pr-comments-assets.blacksmith.sh/codesmith/fix-with-codesmith-light-3x.png)
Leo (2026-10-28): "Likewise, holy hell, the resizing of the Chromium elements. ... It's still the exact same as it was, like, 24 hours ago, where it's just slow as hell."
Reference: cmuxterm-hq#4947 (private).
Cause, measured. The drag lag is CPU contention, not cmux code. On a fleet bench Mac with 14 busy processes standing in for agents, a Chromium page trails its dragged pane by up to 55 ms at p95. With the same load at background class (
taskpolicy -b), it trails by 6-10 ms, and cmux's main thread spends about 45 % less CPU per drag. Profiling agrees: cmux's main thread sat idle inmach_msgin 44 of 51 samples. Agent harnesses, and the builds and tests they run, compete with cmux and its renderers at the same priority. acpmux lowered them only after a session passed its CPU budget.Fix. On macOS each session's harness now starts with
PRIO_DARWIN_BG, the class cmux-git already gives its git children (cmux-git/src/run.rs). The class is set in the harness between fork and exec, both for a direct child (agent.rs) and under an agent host (SpawnSpec.cpu_class, which the daemon decides). So the harness and every tool it starts inherit it. No running process is reniced.cpu_class.rs). The first source that sets it wins:ACPMUX_SESSION_CPU_CLASS(normalorbackground), so one build-heavy profile can opt out.ACPMUX_SESSION_CPU_CLASS.session.cpuClass(acpmux config --set session.cpu_class=normal).background.nice -n 10andnice -n 20gave no steady gain over the default class. On the heavy page they dropped 23-26, 6-36 and 19-32 frames per drag; background dropped 0-7. Only background keeps the agents off the cores the GUI needs.hub/seen.rs). Raising one again would mean renicing its running process group, and children it already started would stay low. So every session starts at background.cpu_classand starts its harness as before.Performance
Harness:
scripts/cmux-next/split-resize-bench.pythroughscripts/measure/split-spring.shon a fleet bench Mac (bench/resize-harness-tip),--pages=probe,heavy --drag-hz=120 --cpu-load=14 --cpu-load-class=default|background|utility. All runs were on the 60 Hz display. The busy processes at a class stand in for agents at that class. The heavy page is shown, as the median of 4 runs per class (fleet steps listed below).The build rows come from
scripts/measure/acpmux-overhead.sh bench=build-turn(#27582): Claude Code through acpmux runs one Bash call per turn, a cold debug build of cmux-history in a fresh target dir, withbefore=at base. The build shell's base priority was 20 before and 4 after.Runs 5 and 6 of the same harness, heavy page, page trails p95 ms / dropped / main ms (default, nice 10, nice 20, background):
These are on a scratch copy of the harness that adds
--cpu-load-class=nice10|nice20.Steps (default / background), run 1: 33cca097f4eca66b198f1e00 / 6e0128696605b93a94c61c9a; run 2: 54bbedbce6966501aad74092 / 49b59f0fa56e39d5f04e59f4; run 3: a295cbda82f1726c6d8fae0a / ac79343ddd56d8e24ece5b43; run 4: 6faf28c898ae1741ad13db10 / 2d4ab680341ccb8b6e609c5d. Utility: 8da2362168f642090f9629ee, 9c51a10f9c6ffa1e99cc3bdc. Runs 5-6 (default, nice 10, nice 20, background): 55bd43c2125b46652a0d4648, b2136d2a00ac7f2d62080e85, d80f174564b92f273602f725, 2c0cf1728c3fc31bf127044c; 22430ef0f1dc008153ce0658, 80fd10b30e1725dd44c6a7df, 09e1c94720313ec88204cd55, 854796bcc2d9540f06065267.
Tests (red, then green).
tests/cpu_class.rs, which is macOS only. A direct harness and a hosted one each start a backgroundsleep, and the test reads both processes' base priority withps -o pri=(4 is darwinbg). The two background tests failed on a fleet Mac in step 449b586ce3a30d6dfe904411.normalopt-out test, a${env:...}reference that resolves tonormal, the resolve-order unit test and the config round trip (step fcf491e6fe72a06b1b5c24dd, macOS). macOS clippy is green (0f69df34c5b3bd6abe07c1cf), and so are Linux fmt, clippy and tests (b273610db33137ec07df61c1).Risk: medium.
ACPMUX_SESSION_CPU_CLASS=normal, for a build-heavy lane), or one config value for the host (acpmux config --set session.cpu_class=normal).Agent session: agent session
🤖 Generated with Claude Code