Skip to content

perf cron: web UI perf harness on Linux (web-ui part, PERF_ROW lines) - #31704

Merged
teamleaderleo merged 2 commits into
feat-cmux-nextfrom
lane/webui-perf-harness
Nov 2, 2026
Merged

teamleaderleo merged 2 commits into
feat-cmux-nextfrom
lane/webui-perf-harness

Conversation

@teamleaderleo

@teamleaderleo teamleaderleo commented Nov 2, 2026 •

Copy link
Copy Markdown
Collaborator

Leo, 2026-10-28: "and we want TONS of perf stuff... when it comes to the left scroll rail thing, i already said we want it ultra fast ultra responsive, but the same goes for EVERYTHING IN THE WEB UI"

Leo-ask: ask-8bc0f906d8

What

A shared web UI perf harness that runs in the existing perf cron (cmux-next-perf-budgets.yml + scripts/cmux-next/perf-cron.py) as a second Linux part, next to diff-viewer. It is advisory and cron-only. It adds nothing to the push, pull request or nightly path (Lawrence 2026-10-23, see ci-web.yml), adds no new job and no new runs-on: it reuses the linux job and the plan's linux_runner. No check fails on a regression. A regression goes to the one "cmux-next perf regression" issue through the same compare, confirm, bisect and issue logic the diff rows already use.

The repo CLAUDE.md says not to add CI gates, perf rows or CI steps. This adds no gate. Lawrence named the perf cron as the place for measurements ("we can have one running on a cron job"), so these rows report there, and nothing waits on them. Leo owns CI and asked for this harness (rulebook: "Lanes measure interactions with a shared web UI perf harness on Linux CI and fix the slowest first").

  1. webviews/bench/perfRow.ts: a machine-readable line beside each bench's existing human output:
    PERF_ROW {"name":"chat_switch.unread.latest","p50":..,"p95":..,"unit":"ms","target":100}.
    Targets follow one rule per kind of row, never today's number: a response (click, switch, until painted) is 100 ms, a cold load until the newest message is painted is 1000 ms, and main-thread work per frame (scroll, stream, per-token reads) is 10 ms. Row names never end in .frame, .missed or .dropped, so the cron does not judge them by display refresh (a headless runner has none).
  2. webviews/bench/web-ui-suite.ts: runs the benches below with short, CI-sized args, collects their rows, prints them slowest first relative to target, and prints one JSON report as its last line. Any bench that exits non-zero, times out or prints no row is listed in failed, so nothing passes silently. --only a,b and --out FILE are there for local runs.
  3. perf-cron.py: adds the part web-ui (where: linux, rows webui.*, needs: web-ui). supported_parts_at measures only commits that have web-ui-suite.ts. rows_of_part keeps the bench's target as the row's budget, and target_of reads it (no target means none, not the 10 ms latency default). The floor is the bench's own, or 10 % of p95. Each row carries its version, so bumping it rebases the row. If one bench fails, the part is incomplete: its rows still count, but the run is not a last-good baseline. A run with no rows fails. run-linux-part --part web-ui prints a Markdown table, slowest first against target, into the step summary, so a lane can "fix the slowest first". run-linux-part defaults to --part diff-viewer. diff-viewer's part_config hash is unchanged (9c9749c7fe76, asserted in a test), so its rows are not rebased.
  4. Workflow linux job: Rust is installed only for diff-viewer. playwright install --with-deps --only-shell chromium webkit runs only for web-ui. The step passes --part "$PART".

Benches

included rows browser
chat-open (3 rounds, 300 turns) chat_open.compact.{attach,first_row,latest,settled,long_ms} Chromium
chat-switch (3 rounds, 300 turns) chat_switch.{unread,read_ahead}.{latest,settled,long_ms} Chromium
transcript-scroll-frame (3000 rows, 2 rounds) transcript_scroll.{fling,wheel}.{react,layout}_ms WebKit
thread-rail-jump (1000 turns, 5 samples) thread_rail.{tick_click,tick_reveal,scroll}.loaded_1000.base Chromium
transcript-view-rows transcript_rows.stream.set none
edit-turns-stream edit_turns.{full,tracked}.{1000,5000,20000} none
agent-activity-groups agent_activity.groups.{snapshot,local}.{100,1000,5000} none

Excluded:

  • thread-minimap-frame-cost: an A/B check of the VS Code Minimap against its own 1 ms budget. It exits non-zero by design when over budget, or when it lacks a +canvas arm, so the suite would read that as a failed run. thread-rail-jump already covers the rail's frame cost.
  • diff-stream: the diff-viewer part already runs it.
  • thread-rail-drag.adapter: an adapter, not a bench.

transcript-scroll-frame's frame intervals are not rows, because a headless runner's frame clock is not a display's. Its React and layout work per frame are rows.

Verification

  • python3 -I scripts/cmux-next/tests/test_perf_cron.py: Ran 176 tests ... OK. There is a new WebUiTests class: rows, targets, floors, versions, regression detection, incomplete on a failed bench, slowest-first summary, and run-linux-part --part web-ui with a faked suite (done plus incomplete, or failed when there are no rows). Two existing tests were updated, because the Mac-disabled path now runs both Linux parts.
  • python3 -I scripts/cmux-next/tests/test_perf_cron_io.py: Ran 27 tests ... OK.
  • tests/test_ci_linux_single_size.py, test_ci_no_github_token_polling.py, test_ci_overflow_pick_reads_pool.py, test_ci_guard_workflow_structure.py: pass.
  • The whole suite on a Linux runner (cmux-ci test --setup webviews, fast-checks run 41382740922 at 5ff9e0f, 8 vCPU, Playwright's Chromium and WebKit): suite exit 0, all 7 benches ran, 31 rows, failed [], 119 s in total (chat-open 19 s, chat-switch 28 s, transcript-scroll-frame 51 s). The top of its slowest-first table:
   660 %  transcript_scroll.fling.react_ms  p95 66 ms (target 10)
   125 %  chat_switch.unread.latest  p95 125.2 ms (target 100)
   118 %  chat_switch.read_ahead.latest  p95 118.4 ms (target 100)
    65 %  thread_rail.tick_click.loaded_1000.base  p95 65.1 ms (target 100)
    42 %  chat_open.compact.latest  p95 419.1 ms (target 1000)

An earlier run (41381894015, at 5dff1bc) gave the same order: chat_switch 116-123 ms, thread_rail tick_click 63.7 ms, chat_open latest 430.7 ms. It also showed thread_rail.scroll at exactly 16.7 ms, the headless frame interval. The second commit takes that row's frame-work target off (the row stays, so a dropped frame still shows).
So the slowest web UI paths today are React work per frame during the transcript fling (66 ms p95 on that runner) and chat switch (about 120 ms, either arm). These are the first things to fix.

Observed / Not run

  • Observed: the unit tests above, and two full suite runs on a Linux runner.
  • Not run: the cron itself (plan, then the linux job with --part web-ui, then report). The first tick on feat-cmux-next after the merge is that end-to-end run. The runner class there is the plan's linux_runner (4 or 8 vCPU Blacksmith), so its numbers will differ from the 8 vCPU fast-checks numbers above.

Changelog

none

Agent session: agent session

🤖 Generated with Claude Code

…f harness)

The web UI benches each printed only their own human text, and only diff-stream ran on CI. Now:

- webviews/bench/perfRow.ts: a machine-readable `PERF_ROW {...}` line (name, p50, p95, unit, target,
  floor, version) beside each bench's human output, with one target rule per kind of row (response
  100 ms, cold load 1000 ms, frame work 10 ms).
- webviews/bench/web-ui-suite.ts: runs chat-open, chat-switch, transcript-scroll-frame,
  thread-rail-jump, transcript-view-rows, edit-turns-stream and agent-activity-groups with short
  args, collects their rows, prints them slowest first relative to target, and lists every bench
  that exited non-zero, timed out or printed no row.
- perf-cron.py: a second Linux part "web-ui" (rows webui.*) through the same compare, confirm,
  bisect and issue machinery as diff-viewer; a failed bench makes the part incomplete; the step
  summary lists the rows slowest first. diff-viewer's config hash is unchanged.
- The cron's linux job installs Rust only for diff-viewer and Playwright's chromium and webkit only
  for web-ui; same runner (the plan's linux_runner), still advisory and cron-only.

Leo-ask: ask-8bc0f906d8

Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
Agent-Session: https://cmux.com/deeplink/agent?id=aid-0113219d38905033a039
@github-actions

github-actions Bot commented Nov 2, 2026 •

Copy link
Copy Markdown
Contributor

cmux status (one comment, edited in place; each line is updated by the job that owns it)

Review 5dff1bc: HQ feed card · gallery before/after runs · checks

App of this head (app.zip, kept 1 day)

gh workflow run reload-build.yml -R manaflow-ai/cmux --ref feat-cmux-next -f tag=pr31704 -f ref=lane/webui-perf-harness -f platform=macos -f backend_mode=local

red: cmux-next checks (god files, concurrency, crash safety, l10n); NEW: red check: Script tests (pin-cmux-tui, embed-cef, ensure-cef store) (+1 more) is not red on the base 5f8526e; do not merge until green; see lane-rules

Shipped in:

@github-actions github-actions Bot added the risk:normal Risk tier (docs/ci/risk-tiers.md) label Nov 2, 2026
@blacksmith-sh

blacksmith-sh Bot commented Nov 2, 2026 •

Copy link
Copy Markdown

Found 12 test failures on Blacksmith runners:

Failures

Test View Logs
/tmp/tmp0ao_k0xc/summary-missing.json (live):/summary-missing.json (live): View Logs
/tmp/tmpjaymx4a7/summary-fail.json (live):/summary-fail.json (live): View Logs
[new_workspaces] debug.new_workspace timed 0 of 50 new workspace rows/
[new_workspaces] debug.new_workspace timed 0 of 50 new workspace rows
View Logs
[new_workspaces] the window draws no frames/[new_workspaces] the window draws no frames View Logs
[tabs] debug.tab_switch timed 0 of 40 switches/
[tabs] debug.tab_switch timed 0 of 40 switches
View Logs
[tabs] the window draws no frames/[tabs] the window draws no frames View Logs
main.DeveloperIdAppTests.test_download_prints_the_active_profile_only/
test_download_prints_the_active_profile_only
View Logs
stubs: /System/Library/Frameworks/Security.framework/Versions/A/Security has no stub/
Security has no stub
View Logs
stubs: CoreFoundation.tbd: install-name /System/Library/Frameworks/
CoreFoundation.framework/Versions/A/CoreFoundation, missing 1: ['_CFRetain']/
CoreFoundation, missing 1: ['_CFRetain']
View Logs
stubs: CoreFoundation.tbd: install-name /System/Library/Frameworks/
CoreFoundation.framework/Versions/A/CoreFoundation, missing 1: ['_log2']/
CoreFoundation, missing 1: ['_log2']
View Logs

...and 2 more test failures. View all on Blacksmith

Fix with [code]smith
Need help on this PR? Tag @codesmith-bot with what you need.

…target

A headless runner's frame clock sets it (16.7 ms on the first Linux run), so it is not one frame's work.

Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
Agent-Session: https://cmux.com/deeplink/agent?id=aid-0113219d38905033a039
@teamleaderleo

Copy link
Copy Markdown
Collaborator Author

Review (codex, router-review): SHIP at 5ff9e0f

Reviewer: codex gpt-6-astra high, fast tier.

Follow-ups: 1. Each lands as an issue in this repo that links this receipt; the lane that opened the PR fixes it forward.

SHIP at 5ff9e0f

Visual verdict: no images; Leo tries the build

The cron and CI tests pass (176 perf-cron tests, 27 I/O tests, and the listed CI checks). One follow-up remains:

Follow-ups:

  • [P2] [follow-up] webviews/bench/web-ui-suite.ts:53 accepts any string as a row name, so a malformed PERF_ROW with a valid p95 is counted as successful and later silently discarded by rows_of_part’s name regex. Validate the row name and other required fields in parseRows, or record the bench in failed.

@cmux-bot cmux-bot Bot left a comment

Copy link
Copy Markdown

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

cmux-bot approves 5ff9e0f: an independent review passed this exact head (router-review receipt by teamleaderleo). A push needs a new review; a later BLOCK at this head dismisses this approval.

@teamleaderleo
teamleaderleo merged commit 30cba87 into feat-cmux-next Nov 2, 2026
215 of 229 checks passed
@teamleaderleo
teamleaderleo deleted the lane/webui-perf-harness branch November 2, 2026 04:14

This branch was previously deployed

1 inactive deployment
ci-routing-agent — 5ff9e0f2 Deployed Nov 2, 2026 by teamleaderleo via wait for same-key tree publisher #98866
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

risk:normal Risk tier (docs/ci/risk-tiers.md)

Projects

None yet

Development

Successfully merging this pull request may close these issues.

1 participant