Skip to content

bench(acpmux): bench=build-turn, an agent turn that runs a build, at each CPU class - #27582

Merged
teamleaderleo merged 2 commits into
feat-cmux-nextfrom
acpmux-build-turn-bench
Oct 29, 2026
Merged

teamleaderleo merged 2 commits into
feat-cmux-nextfrom
acpmux-build-turn-bench

Conversation

@teamleaderleo

@teamleaderleo teamleaderleo commented Oct 29, 2026 •

Copy link
Copy Markdown
Collaborator

Leo (2026-10-28): "Likewise, holy hell, the resizing of the Chromium elements. ... It's still the exact same as it was, like, 24 hours ago, where it's just slow as hell."

A measurement for the fix that starts agent harnesses at a lower CPU class on macOS (follow-up PR): what that class costs an agent turn that runs a build. Fleet measure scripts run only once they are on feat-cmux-next, so the bench lands first and the fix's PR cites its numbers.

What it adds

  • acpmux-overhead-bench.py --mode build-turn --build-cmd CMD: Claude Code through acpmux, each run in a fresh session. The stand-in model answers the prompt with one Bash call running CMD, then "ok". Per run it records the turn's time (prompt to turn end), the build's own seconds, and the base priority (ps -o pri=) of the shell that ran the build. A darwinbg shell shows 4 and a default one 31.
  • scripts/measure/acpmux-overhead.sh bench=build-turn [crate=cmux-history] [n=3] [before=SHA]: the Bash call is a cold debug build of one cmux-tui crate in a fresh target dir, with the step's own cargo and rustup homes and --offline after a cargo fetch. It runs through acpmux at this ref and at before=, then runs the same build directly at the default class, taskpolicy -c utility and taskpolicy -b, interleaved (ACPMUX-BUILD-RAW lines).

Proof it runs: a Linux smoke on cmux-ci test (fast checks run 40953723433) drove two build turns end to end. Turn end p50 was 203 ms with a stand-in build. Claude Code ends its request with a system message, so the stand-in looks for the last user message.

Risk: low. Bench only: no product code, and the existing modes are unchanged (the new mode returns before them).

Agent session: agent session

🤖 Generated with Claude Code


View with [code]smith Autofix with [code]smith
Need help on this PR? Tag @codesmith-bot with what you need.


Summary by cubic

Adds bench=build-turn, a benchmark that measures what an agent turn running a build costs at each CPU class, so the follow-up fix that starts harnesses at a lower class can cite real numbers.

  • acpmux-overhead-bench.py --mode build-turn runs Claude Code through acpmux in a fresh session per turn; the stand-in answers the prompt with one Bash call running --build-cmd, and records the turn's time, the build's seconds, and the shell's base priority.
  • acpmux-overhead.sh bench=build-turn wraps a cold debug build of a cmux-tui crate as the build command, then runs the same build directly at default, utility, and background classes (taskpolicy) for comparison, interleaved.
  • Bench only: no product code, and the existing modes return before the new one, so they are unchanged.

Written for commit 7ebb96f. Summary will update on new commits.

View guided diff Turn on auto-fix

… the build at each CPU class

Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
Agent-Session: https://cmux.com/deeplink/agent?id=aid-1288bbb2b5d2f5c6b66a
@cursor

cursor Bot commented Oct 29, 2026

Copy link
Copy Markdown

Bugbot needs on-demand usage enabled

Bugbot uses usage-based billing for this team and requires on-demand usage to be enabled.

A team admin can enable on-demand usage in the Cursor dashboard.

@github-actions

github-actions Bot commented Oct 29, 2026 •

Copy link
Copy Markdown
Contributor

cmux status (one comment, edited in place; each line is updated by the job that owns it)

Review e8103bd: HQ feed card · gallery before/after runs · checks

App of this head (app.zip, kept 1 day)

gh workflow run reload-build.yml -R manaflow-ai/cmux --ref feat-cmux-next -f tag=pr27582 -f ref=acpmux-build-turn-bench -f platform=macos -f backend_mode=local

red: cmux-next checks (god files, concurrency, crash safety, l10n); no finished base run on b335b10 to compare; do not merge until green; see lane-rules

@teamleaderleo

Copy link
Copy Markdown
Collaborator Author

Owned by an agent session · session

@github-actions github-actions Bot added the risk:normal Risk tier (docs/ci/risk-tiers.md) label Oct 29, 2026

@teamleaderleo teamleaderleo left a comment

Copy link
Copy Markdown
Collaborator Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Review (codex gpt-6-astra high fast, independent review): SHIP at e8103bd

Round 1 at 7ebb96f found two P2s: a raw build's failure was masked, and the Python entrypoint discarded main()'s return. Both are fixed in e8103bd, and round 2 found nothing more.

The only red, cmux-next checks, fails the same way on base feat-cmux-next (run 40958608185, job 123745178101). NO-SWIFTUI flags the same two HomeChatColors files, and the sidebar sweep flags the same two commits, 4367c16 and 6c1a5ab. This PR changes neither.

@teamleaderleo
teamleaderleo merged commit 299ebaa into feat-cmux-next Oct 29, 2026
126 of 128 checks passed
@teamleaderleo
teamleaderleo deleted the acpmux-build-turn-bench branch October 29, 2026 21:39
@github-actions

Copy link
Copy Markdown
Contributor

The feat-cmux-next push run https://github.com/manaflow-ai/cmux/actions/runs/40960009727 at 5a5ea9f failed: cmux-next checks (god files, concurrency, crash safety, l10n).
Those jobs last passed at b455e2d. This pull request is one of 2 merged since: #27582, #27608.
If the failure is in your change, fix forward on feat-cmux-next. Pull requests run only the tiers their paths reach (docs/ci/cmux-next-tiers.md); label a batch PR full-ci to run them all before merging.

@github-actions

github-actions Bot commented Oct 29, 2026 •

Copy link
Copy Markdown
Contributor

This branch was previously deployed

1 inactive deployment
ci-routing-agent — e8103bd2 Deployed Oct 29, 2026 by teamleaderleo via cmux-tui tree preflight #78279
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

risk:normal Risk tier (docs/ci/risk-tiers.md)

Projects

None yet

Development

Successfully merging this pull request may close these issues.

1 participant