Skip to content

ci: bound the SwiftPM scratch holder and cache scratch sizes - #15366

Merged
teamleaderleo merged 2 commits into
mainfrom
ci/spm-scratch-holder-bound
Sep 28, 2026
Merged

teamleaderleo merged 2 commits into
mainfrom
ci/spm-scratch-holder-bound

Conversation

@teamleaderleo

@teamleaderleo teamleaderleo commented Sep 28, 2026 •

Copy link
Copy Markdown
Collaborator

A follow-up to #14804, from its review.

Bound the holder. owned_spm_scratch.py link starts a small process that holds a shared flock on the job's SwiftPM scratch directory. Until now it exited only when the runner killed a job's leftover processes at job end. If the runner process died mid-job, the holder kept the lock until the mini rebooted, and that directory could never be pruned or evicted. hold now exits after 65 minutes (signal.alarm), just past the swift-package-tests job's 60-minute timeout.

Stop walking the whole scratch on every link. tree_stats lstat'ed every file of up to 24 GiB of scratch to size and rank each directory. Now:

  • link touches the directory's lock file, and that time is its last use, which decides prune order.
  • The size is cached in <fingerprint>.size beside the directory. It is measured again only when the directory was used after the size was written.
  • A new size is recorded only when no job holds the directory, since a held one may still be growing.
  • A link therefore walks only the directories used since the last prune.

Validation. tests/test_ci_owned_spm_scratch.py (12 tests) passes. The 3 new tests cover:

  • the holder releasing its lock after its bound;
  • the size being measured once until the directory is used again;
  • a held directory not getting a recorded size.

Changelog

none

🤖 Generated with Claude Code


View with [code]smith Autofix with [code]smith
Need help on this PR? Tag @codesmith-bot with what you need. Autofix is disabled.


Summary by cubic

Follow-up to #14804. Bounds the SwiftPM scratch holder so a runner that dies mid-job can't leave a scratch directory locked until the mini reboots, and caches each directory's size so pruning no longer walks the whole scratch on every link.

  • The holder now exits after 65 minutes, just past the swift-package-tests job's 60-minute timeout.
  • link touches the lock file as the directory's last use, and sizes are cached in <fingerprint>.size, re-measured only when a later job used the directory and no job holds it; a time that ties on coarse clocks also re-measures.
  • New tests cover the holder releasing its lock, size measured once until the directory is used again, and held directories not getting a recorded size.

Written for commit 7504716. Summary will update on new commits.

Review in cubic

…ory's size

The holder that keeps a job's shared lock on its scratch directory exited
only when the runner killed it at job end. A runner that died mid-job left it
holding the lock until reboot, so that directory could never be pruned or
evicted. It now exits after 65 minutes (signal.alarm), past the
swift-package-tests job's 60-minute timeout.

tree_stats walked every file of up to 24 GiB of scratch on every link. The
lock file, which link now touches, gives a directory's last use, and its
size is cached in <fingerprint>.size, measured again only once the directory
was used after it and no job holds it.

Co-Authored-By: Claude Opus 5.5 (1M context) <noreply@anthropic.com>
@cursor

cursor Bot commented Sep 28, 2026

Copy link
Copy Markdown

Bugbot is paused — on-demand spend limit reached

Bugbot uses usage-based billing for this team and has hit its on-demand spend limit.

A team admin can raise the spend limit in the Cursor dashboard, or wait for the next billing cycle to continue.

@coderabbitai

coderabbitai Bot commented Sep 28, 2026 •

Copy link
Copy Markdown

Warning

Review limit reached

Next included review available in 58 seconds.

Check out review usage here.

View limit details

Limit details: You’ve used all 10 included reviews currently available.

You've used all free OSS reviews for now. Wait for the free limit to reset to keep reviewing this public repository.

Learn how review limits work.

Review configuration:

⚙️ Run configuration

Configuration used: Repository: manaflow-ai/cmux/.coderabbit.yaml

Review profile: ASSERTIVE

Plan: Advanced

Run ID: 62d9ebc4-8fbb-4ad0-b057-0cc729741c1a

📥 Commits

Reviewing files that changed from the base of the PR and between 31b2619 and 7504716.

📒 Files selected for processing (3)
  • docs/ci-runners.md
  • scripts/ci/owned_spm_scratch.py
  • tests/test_ci_owned_spm_scratch.py

Thanks for using CodeRabbit! It's free for OSS, and your support helps us grow. If you like it, consider giving us a shout-out.

❤️ Share

Comment @coderabbitai help to get the list of available commands.

@github-actions

Copy link
Copy Markdown
Contributor

All contributors have signed the CLA ✍️ ✅
Posted by the CLA Assistant Lite bot.

@github-actions

github-actions Bot commented Sep 28, 2026 •

Copy link
Copy Markdown
Contributor

Dogfood build of 7504716c6b45cece0f32dce4b79e12388fcb216e

cmux DEV pr-15366-7504716c.app

The link opens this exact commit in the cmux dev menu bar app. The build starts on each push and the page waits until it is ready; a newer push replaces it. It signs in against production, so Cloud or backend changes still need a tagged build with a development backend.

@blacksmith-sh

blacksmith-sh Bot commented Sep 28, 2026

Copy link
Copy Markdown

Found 1 test failure on Blacksmith runners:

Failure

Test View Logs
test_a_size_is_measured_once_until_the_directory_is_used_again (main.Scratch.test_a
_size_is_measured_once_until_the_directory_is_used_again)/
test_a_size_is_measured_once_until_the_directory_is_used_again (main.Scratch.test_a
_size_is_measured_once_until_the_directory_is_used_again)
View Logs

Fix with [code]smith
Need help on this PR? Tag @codesmith-bot with what you need.

@github-actions

github-actions Bot commented Sep 28, 2026 •

Copy link
Copy Markdown
Contributor

CI failure attribution

CI passes on 7504716c6b (run 36426075832 attempt 1).

Written by scripts/ci/classify_failures.py (ci-failure-attribution.yml); signatures are its SIGNATURES table. A machine verdict is the runner's fault, not this PR's.

…ocks)

Co-Authored-By: Claude Opus 5.5 (1M context) <noreply@anthropic.com>
@teamleaderleo
teamleaderleo merged commit 4898cde into main Sep 28, 2026
52 checks passed
@teamleaderleo
teamleaderleo deleted the ci/spm-scratch-holder-bound branch September 28, 2026 13:09
@github-actions

Copy link
Copy Markdown
Contributor

Merge receipt for 7504716c6b: every check was green at merge (14 verified; 18 skipped by policy). Full suite runs on main after merge.

austinywang added a commit that referenced this pull request Sep 28, 2026
Resolve the scratch-age overlap using main’s complete implementation and tests from #15366. The cache repair now has no remaining delta from main.
@austinywang austinywang mentioned this pull request Sep 28, 2026
3 tasks done
rustybret pushed a commit to rustybret/bmux that referenced this pull request Sep 28, 2026
0e298fb ci: wait for the product's canonical root instead of compiling beside it (manaflow-ai#15379)
3088273 ci: UI test runs adopt compile admission's product, skip the re-upload, and report progress (manaflow-ai#15331)
b681e7e Keep a pending banner quiet once its pane is focused (manaflow-ai#15357)
03a2f6e Record that cloud_vm_sessions.attachment_count is cumulative (manaflow-ai#15321)
48258b4 fix(iroh-v2): check the team socket cap before opening the session (manaflow-ai#15340)
2638d56 Agent activity reorder follow-ups: group on-top check, search, subtitle (manaflow-ai#15362)
9ed83fd Dogfood journey: record whether a paused Cloud machine is asleep (manaflow-ai#15293)
7171ea8 Add app.tabBarVisibility to hide the pane tab bar when a pane has one tab (manaflow-ai#15294)
8743ec8 test: stop Computer Use onboarding tests waiting out the helper status deadline (manaflow-ai#15329)
6e4f1da ci: drain the snapshot's owned queue by what the machines finished since (manaflow-ai#15374)
9373164 ci: queue a pull request's admission for a root runner when Blacksmith's wait is longer (manaflow-ai#15376)
634a155 test: expect injected pane attention accent (manaflow-ai#15370)
cd030e9 Keep a named Cloud machine's prompt name instead of flipping to its slug (manaflow-ai#15288)
24ee0ee Exit 1 when cmux terminal screen wait times out (manaflow-ai#15282)
1b857ac test: cover a live Codex turn owner keeping its turn on SessionStart (manaflow-ai#13588)
56ec600 PR media: prune media of long-closed pull requests (manaflow-ai#15364)
4898cde ci: bound the SwiftPM scratch holder and cache scratch sizes (manaflow-ai#15366)

# Conflicts:
#	.github/workflows/ci-guards.yml
#	.github/workflows/ci.yml
#	.github/workflows/test-e2e.yml
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

None yet

Projects

None yet

Development

Successfully merging this pull request may close these issues.

1 participant