Repository navigation
ci: restore SwiftPM's manifest cache for package resolves - #14257
Conversation
SwiftPM keys each evaluated Package.swift on the whole process environment, so on Actions every step misses. Resolve under a fixed environment, save the cache from seed-derived-data on main, and restore it read-only in compile admission and cli-pipe-regressions. Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
|
All contributors have signed the CLA ✍️ ✅ |
|
Navigate logical layers of code changes, visualize relationships, and explore their blast radius. Warning Review limit reachedNext included review available in 2 minutes. View limit detailsLimit details: You’ve used all 10 included reviews currently available. You've used all free OSS reviews for now. Wait for the free limit to reset to keep reviewing this public repository. Review configuration: ⚙️ Run configurationConfiguration used: Repository: manaflow-ai/cmux/.coderabbit.yaml Review profile: ASSERTIVE Plan: Advanced Run ID: 📒 Files selected for processing (9)
✨ Finishing Touches📝 Generate docstrings
🧪 Generate unit tests (beta)
Thanks for using CodeRabbit! It's free for OSS, and your support helps us grow. If you like it, consider giving us a shout-out. Comment |
Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
2e5cee0 ci: restore SwiftPM's manifest cache for package resolves (manaflow-ai#14257) d8b10f4 ci: retire the persistent Mac PR compile pilot (manaflow-ai#14232) # Conflicts: # .github/workflows/ci-guards.yml # .github/workflows/ci-macos.yml # .github/workflows/ci.yml # .github/workflows/cli-pipe-regressions.yml # .github/workflows/seed-derived-data.yml
* test: require BETA notification extension registration * fix: use registered BETA notification extension identity * ci: drop compile admission's reads of the retired persistent-restore step #14257 was branched before #14232 removed the step, so its three package-cache conditions still read steps.persistent-restore, and actionlint fails on main and every open PR. Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com> (cherry picked from commit 2ea08f1) * ci: prepare signed iOS candidates before TestFlight upload * test: exclude prepared candidates from upload assignment * test: reject incomplete iOS candidates and stale BETA defaults * fix: publish only complete iOS candidates with valid BETA defaults --------- Co-authored-by: Leo Li <cheerleaderleo@outlook.com> Co-authored-by: Claude Opus 5.5 <noreply@anthropic.com>
|
Measured on real PR runs. Seed run 36034205743 saved the cache at 17:33Z: 162 entries, 712 KB. Numbers below are "Resolve Swift packages" (admission) and "Resolve Swift package dependencies" (cli-pipe-regressions) on
Signed: Manifold g1 ✨ |
Every compile admission and cli-pipe-regressions job evaluates all ~90
Package.swiftfiles from scratch while resolving packages, even when thespm-package cache is an exact hit. That accounts for about 30 s of the admission "Resolve Swift packages" step (median 66 s). This PR saves SwiftPM's evaluated-manifest cache from main and restores it read-only in both lanes.Why a plain cache restore would never hit
SwiftPM (both
swiftand xcodebuild) caches each evaluated manifest in~/Library/Caches/org.swift.swiftpm/manifests/manifest.db. Each entry's key includes:I confirmed the environment part locally with xcodebuild on a scratch package: changing one unrelated variable added a new entry. On Actions every step has a different environment (
GITHUB_RUN_ID,GITHUB_ACTION,GITHUB_OUTPUTpaths), so a restored cache would always miss. It also explains why a later xcodebuild in the same step resolves in a couple of seconds.What changes
scripts/ci/swiftpm-manifest-cache.shdoes four things:runexecs the resolve under a fixed environment:HOME, a fixedPATH,LANG,DEVELOPER_DIR,USER/LOGNAME, and any proxy variables. The command is still looked up on the caller'sPATH. No cmuxPackage.swiftreads the environment, so this can't change what a manifest declares.keyproducesswiftpm-manifests-v1-<os>-<arch>-<xcode>-<hash of every Package.swift, Package.resolved and the script>. Entries are content-keyed, so readers fall back to the newest cache for their Xcode by prefix.stagewrites a single-file copy ofmanifest.dbfor saving (about 100 entries, a few MB).installputs a restored copy in place. A missing or unreadable restore changes nothing.Where it's wired in:
seed-derived-data.yml, which already runs on main pushes inci-cache-writer, on the same pool and Xcode as PR admission. It checks for the exact key only. On a miss it starts with an empty cache, resolves at both reader paths (the canonical admission root, then the runner workspace with.spm-cacheas cli-pipe-regressions uses it), and saves. The absolute path is part of each entry's key, so each path needs its own resolve. Every save step iscontinue-on-error.ci-macos.yml) andcli-pipe-regressions.yml. Both restore read-only and run their resolve throughrun.compile-app-host-test-product.sh resolvewraps both of its xcodebuild resolves, so nightly and the seeder use the same environment.Not covered:
swift-package-testsrunsswift test, whose tests need the real environment.Evidence
Canary #14251, admission job 107720596235, 6 vCPU macOS 26, Xcode 26.6. After the real resolve, it copied the pristine package directory back into place and resolved again with the same flags as admission (
-skipPackageUpdates):The job's real first resolve (no cache yet) took 48 s from "Resolve Package Graph" to "Resolved source packages". The earlier #14235 probe measured only 3 to 7 s either way because it re-resolved an already-resolved package directory, where SwiftPM skips most manifest loading.
This PR's own CI runs can't show the gain, because nothing is seeded until
seed-derived-data.ymlruns on main after merge. After that I'll post before/after "Resolve Swift packages" and "Resolve Swift package dependencies" timings from real PR runs here.Local checks:
tests/test_ci_swiftpm_manifest_cache.sh(wired into ci-guards): pins the fixed environment, the key's inputs, and the stage/install round trip.test_ci_test_compilation_cache_seed.sh,test_seed_derived_data.py,test_ci_pull_request_caches_are_read_only.py,test_ci_workflow_guards_are_wired.py,actionlint.test_ci_canonical_build_root.pypasses with a realpathTMPDIR. With the defaultTMPDIRits two cwd assertions fail on macOS because of the/varvs/private/varsymlink, including in the build-only test this PR doesn't touch.CMUX_CI_SWIFTPM_KEEP_ENV.Signed: Manifold g1 ✨ (run_swiftpm-manifest-cache-20260924)
🤖 Generated with Claude Code
Summary by cubic
Caches SwiftPM's evaluated manifests so the "Resolve Swift packages" step no longer spends ~30 s re-evaluating all 91
Package.swiftfiles even on a warm package-cache hit.SwiftPM keys each cached manifest on the whole process environment, and every Actions step has a different one, so a plain cache restore would always miss. The new
scripts/ci/swiftpm-manifest-cache.shruns the resolve under a fixed, per-run-stable environment, content-keys the cache on the manifests,Package.resolved, and submodule pointers (the vendor/bonsplit manifest lives past one), plus the Xcode toolchain, and stages/installs the single-filemanifest.db.seed-derived-data.ymlseeds the cache from main at both paths readers resolve from (the canonical admission root and the runner workspace); compile admission and cli-pipe-regressions restore it read-only.compile-app-host-test-product.shresolves under the fixed environment too, in both the exact-hit and retry paths.swift-package-tests(its tests need the real environment) or to the build steps' own manifest loads.tests/test_ci_swiftpm_manifest_cache.shpins the fixed environment, the key's inputs, and the stage/install round trip.Written for commit 1ba930b. Summary will update on new commits.
Summary by CodeRabbit