Skip to content

ci(ios): listperf, the list frame-rate rig as a fleet simulator step - #29487

Merged
teamleaderleo merged 2 commits into
feat-cmux-nextfrom
lane/ios-listperf-mode
Oct 31, 2026
Merged

teamleaderleo merged 2 commits into
feat-cmux-nextfrom
lane/ios-listperf-mode

Conversation

@teamleaderleo

@teamleaderleo teamleaderleo commented Oct 31, 2026 •

Copy link
Copy Markdown
Collaborator

Adds a listperf mode to scripts/ci/cmux-next-ios-sim.sh: the iOS list frame-rate rig (ListStressRig, cmux-next-ios/verify/listperf/) as a fleet simulator step. Base and head build and run on one simulator for a same-harness before/after.

Why: #28278 (every iOS vertical list at full frame rate) needs before/after frame-rate numbers measured at its head. The rig's rig.sh drives an AWS Mac over ssh that agent hosts no longer reach. The fleet step is the reviewed route, but no existing mode can run the rig: shots stops the app after 5 s and keeps no frame log.

What the mode does:

  • It builds the Demo app as build does.
  • base=SHA (40 hex) also builds that commit in a git worktree of the step's checkout. It reuses the head's Rust FFI build when the FFI inputs match, otherwise builds it. Base and head must have the same bundle id.
  • Each case is a final.sh label (screen and settings as final.sh runs them). For each case and side, base then head, reps=N times (1–5, default 2), the step:
    • does a clean uninstall and install;
    • launches with CMUX_MOBILE_LIST_SCREEN and its settings, capped at 300 s;
    • keeps framelog.jsonl and the rig's log lines;
    • runs verify/feedperf/summarize.py, and adds one SUMMARY.md row per window (POOLED and each -open).
  • cases=A,B picks labels. The default is ws-200-live4, chat-40-stream, chat-40, diff-2400 and tabs-30-stream.
  • Every argument is validated before anything builds.
  • The rig (ListStressRig, today on carry of #27664: keep iOS vertical lists at full frame rate #28278 and not yet on feat-cmux-next) must be in the head, and in base= when one is given. Without it the step refuses at once, before any build, rather than launching the plain app and timing out on every case.
  • A launch that exits non-zero (including the 300 s bound) or a run that measures no windows fails the step. Its evidence is still kept.

Cleanup: the base worktree is removed in the step's existing cleanup trap, and the base build lives under the step's work dir. No timer or observer is added; each launch is bounded by alarm 300.

This PR changes no app code. The controller allowlist needs the matching HQ change before the step can be submitted (iosNextSimModes, with base=, reps= and cases= checks); that is manaflow-ai/cmuxterm-hq#5683.

Prior art read: #29265 (Chief layout benchmark modes, kept beside this), #27072, #27642, #27379.

Agent session: agent session

🤖 Generated with Claude Code

…ep (base and head on one simulator)

A listperf mode runs ListStressRig cases (verify/listperf/final.sh labels) on
the Demo app and keeps each framelog.jsonl with its summarize.py lines;
base=SHA also builds that commit in a worktree of the checkout and runs every
case on both apps, base then head, on the same simulator, reps=N times.

Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
@github-actions

github-actions Bot commented Oct 31, 2026 •

Copy link
Copy Markdown
Contributor

cmux status (one comment, edited in place; each line is updated by the job that owns it)

Review 6145c08: HQ feed card · gallery before/after runs · checks

App of this head (app.zip, kept 1 day)

gh workflow run reload-build.yml -R manaflow-ai/cmux --ref feat-cmux-next -f tag=pr29487 -f ref=lane/ios-listperf-mode -f platform=macos -f backend_mode=local

Shipped in:

@github-actions github-actions Bot added the risk:normal Risk tier (docs/ci/risk-tiers.md) label Oct 31, 2026
@teamleaderleo

Copy link
Copy Markdown
Collaborator Author

Review (codex gpt-6.1-sol xhigh, next-review): BLOCK at b93df69

BLOCK at b93df69

Visual verdict: no images; Leo tries the build

  1. [P1][blocking] The new mode has no list rig to drive at this head. cmux-next-ios/verify/listperf/ and all CMUX_MOBILE_LIST_* consumers are absent; the app only selects tabs through -cmux-mobile-tab (RootView.swift:47). The exports at scripts/ci/cmux-next-ios-sim.sh:704 are ignored, so every run waits for the 300-second alarm, creates an empty log, and reports no windows instead of measurements.

  2. [P1][follow-up] feed-ref is not wired to the existing feed rig. The table only supplies FEED_STRESS=320 (scripts/ci/cmux-next-ios-sim.sh:146), but FeedStress requires CMUX_MOBILE_FEED_AUTOSCROLL to create windows and CMUX_MOBILE_FEED_EXIT_WHEN_DONE to terminate. The mode sets neither and does not pass -cmux-mobile-tab feed.

  3. [P2][follow-up] Launch, timeout, and summarization failures are swallowed at scripts/ci/cmux-next-ios-sim.sh:713 and scripts/ci/cmux-next-ios-sim.sh:722. A crashed app or malformed/missing frame log can therefore produce a green step with only a “no windows” row.

Follow-ups:

@teamleaderleo teamleaderleo removed the risk:normal Risk tier (docs/ci/risk-tiers.md) label Oct 31, 2026
@github-actions github-actions Bot added the risk:normal Risk tier (docs/ci/risk-tiers.md) label Oct 31, 2026
…e step on a failed run

A head or base without ListStressRig would launch the ordinary app and
time out on every case; refuse it before any build. A launch that exits
non-zero (the 300 s bound) or a run with no measured windows now fails
the step, with its evidence still kept.

Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
@teamleaderleo

Copy link
Copy Markdown
Collaborator Author

Review (codex gpt-6.1-sol xhigh): SHIP at 6145c08

  1. [P2] [follow-up] [scripts/ci/cmux-next-ios-sim.sh:177]: The required ListStressRig.swift is absent from this head and is not listed as omitted review material. Consequently, every listperf invocation exits with status 2 before building. Land the rig and its app integration so the documented mode can run.

  2. [P2] [follow-up] [scripts/ci/cmux-next-ios-sim.sh:737]: Matching a window label does not establish that measurements exist. A JSON window labeled chat-open with empty intervalsMs produces a label-only summary that passes this check; the resulting table contains missing metrics while the run succeeds. Require positive frame counts and the expected scroll measurements.

Shell syntax validation and all eight existing entrypoint tests passed.

Follow-ups:

  • 1: Supply the missing list stress rig and app integration.
  • 2: Reject runs without usable frame measurements.

router-review: SHIP at 6145c08 (gpt-6.1-sol xhigh); 2 follow-up(s) to file as issues

Follow-up issues: 1 is #28278 (the rig lands with it); 2 is #29565.

@teamleaderleo teamleaderleo removed the risk:normal Risk tier (docs/ci/risk-tiers.md) label Oct 31, 2026
@github-actions github-actions Bot added the risk:normal Risk tier (docs/ci/risk-tiers.md) label Oct 31, 2026
@teamleaderleo
teamleaderleo merged commit 5040256 into feat-cmux-next Oct 31, 2026
164 of 166 checks passed
@teamleaderleo
teamleaderleo deleted the lane/ios-listperf-mode branch October 31, 2026 18:15

This branch was previously deployed

1 inactive deployment
ci-routing-agent — 6145c082 Deployed Oct 31, 2026 by teamleaderleo via cmux-tui tree preflight #90715
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

risk:normal Risk tier (docs/ci/risk-tiers.md)

Projects

None yet

Development

Successfully merging this pull request may close these issues.

1 participant