Skip to content

canary: measure what one app edit costs cmuxTests, cache on and off (do not merge) - #14249

Closed
teamleaderleo wants to merge 3 commits into
mainfrom
canary/cmuxtests-incremental-probe
Closed

teamleaderleo wants to merge 3 commits into
mainfrom
canary/cmuxtests-incremental-probe

Conversation

@teamleaderleo

@teamleaderleo teamleaderleo commented Sep 24, 2026 •

Copy link
Copy Markdown
Collaborator

Measurement canary. Do not merge; I'll close it once the admission job reports.

Why

A PR that changes the app module cmux (cmux_DEV) usually recompiles all 1,091 cmuxTests files: 129 s median on 12vcpu, 199 s on 6vcpu. #14235 showed that an mtime-only touch of an app file is cheap (50 s) and that the asset-timestamp case (fixed by #14241) is a real full rebuild. It did not measure real content edits.

What it does

After the normal admission compile, it rebuilds cmux-unit in three series, each a baseline, a null rebuild, then cumulative edits:

  1. app function body only
  2. new private func in that file
  3. new internal method on GPUSpinner (one test file names it)
  4. signature change of that method
  5. new internal method on Workspace (239 test files name it)
  6. body-only edit of one cmuxTests file

Series: cached (today's CI settings), uncached (COMPILATION_CACHE_ENABLE_CACHING=NO, explicit modules on, -driver-show-incremental), uncached-implicit (also SWIFT_ENABLE_EXPLICIT_MODULES=NO). The step summary reports per-target span, SwiftCompile counts, timing summary rows and incremental remarks; raw logs are uploaded.

🤖 Generated with Claude Code


View with [code]smith Autofix with [code]smith
Need help on this PR? Tag @codesmith-bot with what you need. Autofix is disabled.


Summary by cubic

Measurement canary only—do not merge. Adds a post-compile probe that rebuilds cmux-unit after cumulative app and test edits to measure cmuxTests incremental costs with its compilation cache enabled, disabled, and disabled with explicit modules off.

  • Covers body, private member, narrow and wide internal member, signature, and test-body edits, plus builds after cmux-numeric-locale.
  • Runs targeted cache-off probes for cmuxTests while leaving other targets cached, records incremental remarks for cached rebuilds, and adds a cached-short control series.
  • Pins the macOS job to 12-vcpu macOS 26 and raises the unit-batch timeout to 240 minutes.
  • The probe is non-blocking, raw logs are uploaded, and probe logs are kept on timeout.

Written for commit 30293f8. Summary will update on new commits.

Review in cubic

@coderabbitai

coderabbitai Bot commented Sep 24, 2026

Copy link
Copy Markdown

Important

Draft PR not reviewed

Draft PRs are not automatically reviewed by default.

  • Trigger a manual review

To automatically review draft PRs, update your CodeRabbit configuration:

reviews:
  auto_review:
    drafts: true

Thanks for using CodeRabbit! It's free for OSS, and your support helps us grow. If you like it, consider giving us a shout-out.

❤️ Share

Comment @coderabbitai help to get the list of available commands.

@github-actions

Copy link
Copy Markdown
Contributor

All contributors have signed the CLA ✍️ ✅
Posted by the CLA Assistant Lite bot.

teamleaderleo and others added 3 commits September 24, 2026 21:22
…do not merge)

Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
…cmuxTests rebuilds fully

Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
@teamleaderleo

Copy link
Copy Markdown
Collaborator Author

Measurements done. Results and the fix are in #14349.

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

None yet

Projects

None yet

Development

Successfully merging this pull request may close these issues.

1 participant