Repository navigation
bench(acpmux): bench=build-turn, an agent turn that runs a build, at each CPU class - #27582
Conversation
… the build at each CPU class Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com> Agent-Session: https://cmux.com/deeplink/agent?id=aid-1288bbb2b5d2f5c6b66a
Bugbot needs on-demand usage enabledBugbot uses usage-based billing for this team and requires on-demand usage to be enabled. A team admin can enable on-demand usage in the Cursor dashboard. |
|
cmux status (one comment, edited in place; each line is updated by the job that owns it) Review App of this head (app.zip, kept 1 day)
red: cmux-next checks (god files, concurrency, crash safety, l10n); no finished base run on b335b10 to compare; do not merge until green; see lane-rules |
|
Owned by an agent session · session |
…'s exit Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com> Agent-Session: https://cmux.com/deeplink/agent?id=aid-1288bbb2b5d2f5c6b66a
teamleaderleo
left a comment
There was a problem hiding this comment.
Review (codex gpt-6-astra high fast, independent review): SHIP at e8103bd
Round 1 at 7ebb96f found two P2s: a raw build's failure was masked, and the Python entrypoint discarded main()'s return. Both are fixed in e8103bd, and round 2 found nothing more.
The only red, cmux-next checks, fails the same way on base feat-cmux-next (run 40958608185, job 123745178101). NO-SWIFTUI flags the same two HomeChatColors files, and the sidebar sweep flags the same two commits, 4367c16 and 6c1a5ab. This PR changes neither.
|
The feat-cmux-next push run https://github.com/manaflow-ai/cmux/actions/runs/40960009727 at 5a5ea9f failed: cmux-next checks (god files, concurrency, crash safety, l10n). |
|
Shipped in:
|
Leo (2026-10-28): "Likewise, holy hell, the resizing of the Chromium elements. ... It's still the exact same as it was, like, 24 hours ago, where it's just slow as hell."
A measurement for the fix that starts agent harnesses at a lower CPU class on macOS (follow-up PR): what that class costs an agent turn that runs a build. Fleet measure scripts run only once they are on feat-cmux-next, so the bench lands first and the fix's PR cites its numbers.
What it adds
acpmux-overhead-bench.py --mode build-turn --build-cmd CMD: Claude Code through acpmux, each run in a fresh session. The stand-in model answers the prompt with one Bash call running CMD, then "ok". Per run it records the turn's time (prompt to turn end), the build's own seconds, and the base priority (ps -o pri=) of the shell that ran the build. A darwinbg shell shows 4 and a default one 31.scripts/measure/acpmux-overhead.sh bench=build-turn [crate=cmux-history] [n=3] [before=SHA]: the Bash call is a cold debug build of one cmux-tui crate in a fresh target dir, with the step's own cargo and rustup homes and--offlineafter acargo fetch. It runs through acpmux at this ref and atbefore=, then runs the same build directly at the default class,taskpolicy -c utilityandtaskpolicy -b, interleaved (ACPMUX-BUILD-RAWlines).Proof it runs: a Linux smoke on cmux-ci test (fast checks run 40953723433) drove two build turns end to end. Turn end p50 was 203 ms with a stand-in build. Claude Code ends its request with a system message, so the stand-in looks for the last user message.
Risk: low. Bench only: no product code, and the existing modes are unchanged (the new mode returns before them).
Agent session: agent session
🤖 Generated with Claude Code
Need help on this PR? Tag
@codesmith-botwith what you need.Summary by cubic
Adds
bench=build-turn, a benchmark that measures what an agent turn running a build costs at each CPU class, so the follow-up fix that starts harnesses at a lower class can cite real numbers.acpmux-overhead-bench.py --mode build-turnruns Claude Code through acpmux in a fresh session per turn; the stand-in answers the prompt with one Bash call running--build-cmd, and records the turn's time, the build's seconds, and the shell's base priority.acpmux-overhead.sh bench=build-turnwraps a cold debug build of a cmux-tui crate as the build command, then runs the same build directly at default, utility, and background classes (taskpolicy) for comparison, interleaved.Written for commit 7ebb96f. Summary will update on new commits.