ci: measure compiled test artifact transfer cost by teamleaderleo · Pull Request #13172 · manaflow-ai/cmux

teamleaderleo · 2026-09-20T07:43:15Z

Summary

Compiled app-host test downloads had a 12-second median, but 49 downloads took over five minutes and consumed 7.86 runner-hours in the audited four-hour cohort. Existing step timings do not distinguish large artifacts from slow transfers or retries.

Wrap the existing pinned download action in a local composite action that records artifact ID, archive size, elapsed time, runner, shard, and outcome in the job log and step summary. Effective archive throughput includes lookup, retries, and outer ZIP extraction. Reporting is best-effort; download failures still fail the job, and the existing archive checksum validation remains in place.

This adds evidence for diagnosing the long tail; it does not claim to fix the stalls or reduce runtime yet.

Validation

Actionlint passed for .github/workflows/ci.yml.
CI self-hosted runner guard, reusable-workflow permission checks, and change-area tests passed.
Reporter smoke checks passed for a successful archive, a failed partial archive, and a failed transfer with no archive. Failed transfers never report a successful throughput value.
git diff --check passed.
Hosted validation is pending; no app build is needed for this workflow-only change.

Related audit: #13095 (comment)

^{Need help on this PR? Tag @codesmith-bot with what you need. Autofix is disabled.}

Summary by cubic

Existing CI step timings don't explain why some compiled app-host test downloads take over five minutes. This wraps the download step in a local composite action that records transfer details for each run.

Records artifact ID, archive size, elapsed time, runner, shard, and outcome in the job log and step summary.
Elapsed time is measured with a monotonic clock and includes artifact lookup, retries, and outer ZIP extraction, so throughput is effective archive throughput, not raw network speed.
Reporting is best-effort; download failures still fail the job and existing archive checksum validation is unchanged.
This adds evidence for the long tail; it does not fix the stalls or reduce runtime yet.

^{Written for commit fb751ad. Summary will update on new commits.}

Summary by CodeRabbit

Chores
- Improved the CI process for retrieving compiled test artifacts.
- Added transfer timing, size, and throughput reporting to support build and test pipeline monitoring.
- Standardized artifact retrieval and merging for app-host unit tests.

coderabbitai · 2026-09-20T07:43:22Z

Warning

Review limit reached

Next included review available in 3 minutes.

Check out review usage here.

View limit details

Limit details: You’ve used all 10 included reviews currently available.

You've used all free OSS reviews for now. Wait for the free limit to reset to keep reviewing this public repository.

Learn how review limits work.

Review configuration:

⚙️ Run configuration

Configuration used: Repository: manaflow-ai/cmux/.coderabbit.yaml

Review profile: ASSERTIVE

Plan: Advanced

Run ID: 355c6231-e05b-4d43-b209-6f3441b4a452

📥 Commits

Reviewing files that changed from the base of the PR and between 1751d72 and fb751ad.

📒 Files selected for processing (2)

.github/actions/download-test-product/action.yml
.github/workflows/ci.yml

No actionable comments were generated in the recent review. 🎉

ℹ️ Recent review info

⚙️ Run configuration

Configuration used: Repository: manaflow-ai/cmux/.coderabbit.yaml

Review profile: ASSERTIVE

Plan: Advanced

Run ID: f1949a4d-6a79-4acd-9624-239c76c73677

📥 Commits

Reviewing files that changed from the base of the PR and between 02cd032 and 1751d72.

📒 Files selected for processing (2)

.github/actions/download-test-product/action.yml
.github/workflows/ci.yml

Included review availability: Your plan provides up to 10 included reviews per hour; 0 remain after this review.

📝 Walkthrough

Walkthrough

Changes

Test product transfer

Layer / File(s)	Summary
Reusable download and reporting action `.github/actions/download-test-product/action.yml`	Adds required artifact and path inputs. Downloads and merges matching archives. Records elapsed time, archive size, outcome, and effective throughput in JSON and the GitHub step summary.
App-host CI integration `.github/workflows/ci.yml`	The app-host unit-test job uses the local action with a singular artifact ID and destination path.

Priority: ⬇️ Low

Estimated code review effort: 3 (Moderate) | ~20 minutes

Change: Feature

Sequence Diagram(s)

sequenceDiagram
  participant AppHostJob
  participant DownloadTestProduct
  participant DownloadArtifact
  participant GitHubSummary
  AppHostJob->>DownloadTestProduct: invoke with artifact-id and path
  DownloadTestProduct->>DownloadArtifact: download and merge matching archives
  DownloadArtifact-->>DownloadTestProduct: return download outcome
  DownloadTestProduct->>GitHubSummary: append transfer metrics record

🚥 Pre-merge checks | ✅ 25

✅ Passed checks (25 passed)

Check name	Status	Explanation
Title check	✅ Passed	The title clearly and concisely describes the main change: measuring compiled test artifact transfer cost in CI.
Description check	✅ Passed	The description provides a detailed summary of the change, its purpose, scope, and validation results. It uses a "Validation" section instead of the template's "Testing" section and omits the review t…
Docstring Coverage	✅ Passed	No functions found in the changed files to evaluate docstring coverage. Skipping docstring coverage check. Docstring coverage is scoped to functions touched by this diff. Analyzed 0 functions across 0…
Linked Issues check	✅ Passed	Check skipped because no linked issues were found for this pull request.
Out of Scope Changes check	✅ Passed	Check skipped because no linked issues were found for this pull request.
Cmux Cloud Persistent Session And Early Input	✅ Passed	PASS. The authoritative diff changes only a local CI artifact-download action and the app-host test workflow step. It does not change Cloud terminal creation, cmux-tui clients, physical transports, re…
Cmux Swift Actor Isolation	✅ Passed	The pull-request diff changes only `.github/actions/download-test-product/action.yml` and `.github/workflows/ci.yml`. The diff contains no Swift files or Swift declarations, so it cannot introduce or …
Cmux Swift Blocking Runtime	✅ Passed	PASS: The pull request changes only `.github/actions/download-test-product/action.yml` and `.github/workflows/ci.yml`. The diff contains no Swift source or production Swift runtime change. Its timing …
Cmux Browser Automation Off-Main	✅ Passed	PASS. The pull request changes only `.github/actions/download-test-product/action.yml` and `.github/workflows/ci.yml`. The authoritative diff contains no `browser.*` socket commands and does not modif…
Cmux Expensive Synchronous Load	✅ Passed	The pull request changes only `.github/actions/download-test-product/action.yml` and `.github/workflows/ci.yml`. The diff adds a composite GitHub Action and updates artifact download inputs. It adds n…
Cmux Cache Substitution Correctness	✅ Passed	PASS: The review-scoped diff changes only `.github/actions/download-test-product/action.yml` and `.github/workflows/ci.yml`. It adds transfer measurement around the same pinned `actions/download-artif…
Cmux No Hacky Sleeps	✅ Passed	PASS. The pull request changes only `.github/actions/download-test-product/action.yml` and `.github/workflows/ci.yml`. The rule explicitly excludes GitHub Actions workflow and action YAML. The new act…
Cmux Algorithmic Complexity	✅ Passed	No algorithmic-complexity failure is introduced. The new reporter performs constant-size work: one archive stat, scalar calculations, and serialization of an 11-field fixed record. AST inspection foun…
Cmux Swift Concurrency	✅ Passed	PASS: The pull request changes only `.github/actions/download-test-product/action.yml` and `.github/workflows/ci.yml`. The authoritative diff contains no Swift files or Swift concurrency constructs. T…
Cmux Swift `@Concurrent`	✅ Passed	PASS: The scoped diff changes only `.github/actions/download-test-product/action.yml` and `.github/workflows/ci.yml`. It introduces no Swift files, Swift functions, actor isolation, or `@concurrent` a…
Cmux Swift Package Boundaries	✅ Passed	The pull request changes only `.github/actions/download-test-product/action.yml` and `.github/workflows/ci.yml`. The authoritative diff contains no Swift, SwiftPM, Xcode project, or app-target source …
Cmux Swiftpm Lockfiles	✅ Passed	PASS. The PR changes only `.github/actions/download-test-product/action.yml` and the artifact-download step in `.github/workflows/ci.yml`. The authoritative diff contains no `.gitignore`, `Package.swi…
Cmux Swift Logging	✅ Passed	PASS: The authoritative PR diff changes only `.github/actions/download-test-product/action.yml` and `.github/workflows/ci.yml`; it changes no Swift files or Swift runtime logging. The added `print` is…
Cmux User-Facing Error Privacy	✅ Passed	PASS. The authoritative diff changes only `.github/actions/download-test-product/action.yml` and `.github/workflows/ci.yml`. The new text is CI-only transfer telemetry in the job log and GitHub step s…
Cmux Full Internationalization	✅ Passed	PASS: The authoritative diff changes only `.github/actions/download-test-product/action.yml` and `.github/workflows/ci.yml`. The added text is GitHub Actions metadata, step-log output, and a CI step s…
Cmux Swiftui State Layout	✅ Passed	PASS. The pull request changes only `.github/actions/download-test-product/action.yml` and `.github/workflows/ci.yml`. No Swift, SwiftUI, or UI resource files changed, and added lines contain none of …
Cmux Architecture Rethink	✅ Passed	PASS. The authoritative diff changes only `.github/actions/download-test-product/action.yml` and `.github/workflows/ci.yml`. Both are CI/action configuration files. The added code uses Bash and Python…
Cmux Swift Auxiliary Window Close Shortcuts	✅ Passed	PASS: The pull request changes only `.github/actions/download-test-product/action.yml` and `.github/workflows/ci.yml`. The diff contains no Swift changes and no user-visible `NSWindow`, `NSPanel`, `NS…
Cmux Source Artifacts	✅ Passed	PASS. The authoritative diff changes only `.github/actions/download-test-product/action.yml` and `.github/workflows/ci.yml`. The added file is a hand-written composite GitHub Action configuration, and…
Cmux No Test Or Debug Seam In Production Source	✅ Passed	The pull request changes only `.github/actions/download-test-product/action.yml` and `.github/workflows/ci.yml`. The authoritative diff contains no Swift file under a production `Sources/` path, so it…

✨ Finishing Touches 💡 1

🧪 Generate unit tests (beta)

Commit to this branch
Create a new PR

🛠️ Fix failing CI checks 💡

Commit to this branch
Create a new PR

Thanks for using CodeRabbit! It's free for OSS, and your support helps us grow. If you like it, consider giving us a shout-out.

❤️ Share

_{Comment @coderabbitai help to get the list of available commands.}

github-actions · 2026-09-20T07:43:29Z

All contributors have signed the CLA ✍️ ✅
_{Posted by the CLA Assistant Lite bot.}

greptile-apps · 2026-09-20T12:33:53Z

Confidence Score: 5/5

The PR appears safe to merge; no actionable regression remains in the changed artifact-download path.

Summary

This PR wraps the existing compiled app-host artifact download in a local composite action that preserves download failure behavior while reporting artifact identity, archive size, elapsed time, runner, shard, outcome, and effective throughput.

Uses a monotonic nanosecond clock for transfer timing.
Keeps the download action pinned and retains the existing restoration and checksum-validation flow.
Makes telemetry reporting best-effort so observability cannot mask or replace the download result.

Diagram

%%{init: {'theme': 'neutral'}}%%
flowchart LR
    A[Compile admission] -->|artifact ID| B[Download measurement starts]
    B --> C[Pinned download-artifact action]
    C --> D[Best-effort transfer report]
    C --> E[Checksum validation and product restoration]
    D --> F[Job log and step summary]

_{Reviews (3) · Last reviewed commit: "Merge current main with the locale prefe..."}

teamleaderleo · 2026-09-20T15:10:19Z

Current-head review repair is verified at 1751d72501e66f3c9ddfa3c8c4f431fd66ab1f3f: monotonic timing/reporting smoke checks pass, as recorded in the inline reply. The remaining red web-tests/web-validation checks are a separate observed web locale failure, now tracked in #13246 with exact run/head and all three failing attempts. The relevant web source/test paths are unchanged from inspected main. No blind rerun or unrelated web patch was mixed into this telemetry PR; it remains blocked until that required gate is addressed.

teamleaderleo · 2026-09-20T18:27:46Z

The independent web failure is repaired and merged in #13255 (9e7d3be6915e416ed538a04782864547f94fe8d8). Its trace proved a delayed locale prefetch overwrote the user's newer language cookie; the fix preserves the existing browser assertions and passed required hosted web CI.

Merged current main into this branch at fb751ad8e094283d3e6dc91354fd422a83b53761. The PR still changes only the transfer-measurement composite and its CI call site. Re-ran the actual current-head reporter on success and failure outcomes: monotonic elapsed time and archive bytes are recorded, and failed transfers do not report successful throughput. Fresh required CI is running; the old failed web sample remains preserved.

Integration note for the separately owned LZFSE #13201: when that format lands, its owner must update the reporter's current app-host-products.tar.gz filename to the producer's .aar name (with populated-byte coverage), preserving the same artifact validation and measurement contract. This branch currently matches main's gzip producer.

e77a6b1 feat: add current-work reads and Find Work (manaflow-ai#13269) 6386ba5 Cache Ghostty CLI helper builds across local invocations (manaflow-ai#13206) 8f6c0ea ci: measure compiled test artifact transfer cost (manaflow-ai#13172) 55092b9 docs: add a concise guide for public CMUX writing (manaflow-ai#13257) 9e7d3be fix(web): preserve locale preference during prefetch (manaflow-ai#13255) b093335 build: skip unchanged diff sidecar builds (manaflow-ai#13212) b79d77d perf: skip unchanged bundled resource builds (manaflow-ai#13209) 4c19fcb feat: expose stable surface and workspace IDs in catalog reads (manaflow-ai#13247) 95fdfd7 ci: add safe stale run janitor (manaflow-ai#13143) 0bcf003 docs: make the contributor verification ladder explicit (manaflow-ai#13242)

ci: measure compiled test artifact transfer cost

db0bf2a

teamleaderleo mentioned this pull request Sep 20, 2026

[RFC] CI structure: thin router, reusable platform workflows, merge queue, failure ratchet #13095

Open

teamleaderleo marked this pull request as ready for review September 20, 2026 12:31

teamleaderleo requested a review from lawrencecchen as a code owner September 20, 2026 12:31

greptile-apps Bot reviewed Sep 20, 2026

View reviewed changes

Comment thread .github/actions/download-test-product/action.yml Outdated

ci: measure artifact transfer with monotonic time

1751d72

teamleaderleo mentioned this pull request Sep 20, 2026

web: locale navigation retains Korean cookie after switching to English #13246

Open

teamleaderleo mentioned this pull request Sep 20, 2026

CMUX next wave: consolidation, identity, workflows, execution, and upstream promotion teamleaderleo/Tact#85

Open

Merge current main with the locale prefetch regression fix

fb751ad

teamleaderleo enabled auto-merge (squash) September 20, 2026 18:30

teamleaderleo mentioned this pull request Sep 20, 2026

ci: pack app-host artifacts with LZFSE and verified layers #13201

Closed

teamleaderleo merged commit 8f6c0ea into main Sep 20, 2026
38 of 39 checks passed

teamleaderleo mentioned this pull request Sep 20, 2026

CMUX exposure + Settings cleanup — execution coordination teamleaderleo/Tact#79

Open

Navigation Menu

Repository navigation

ci: measure compiled test artifact transfer cost - #13172

ci: measure compiled test artifact transfer cost#13172
teamleaderleo merged 3 commits into
mainfrom
ci-measure-test-artifact-transfers

teamleaderleo commented Sep 20, 2026 •

edited by cubic-dev-ai Bot

Loading

Uh oh!

coderabbitai Bot commented Sep 20, 2026 •

edited

Loading

Review limit reached

Walkthrough

Changes

Sequence Diagram(s)

Uh oh!

github-actions Bot commented Sep 20, 2026

Uh oh!

greptile-apps Bot commented Sep 20, 2026 •

edited

Loading

Uh oh!

Uh oh!

teamleaderleo commented Sep 20, 2026

Uh oh!

teamleaderleo commented Sep 20, 2026

Uh oh!

Uh oh!

Reviewers

Assignees

Labels

Projects

Milestone

Development

Uh oh!

1 participant

Conversation

teamleaderleo commented Sep 20, 2026 • edited by cubic-dev-ai Bot Loading Uh oh! There was an error while loading. Please reload this page.

Uh oh!

Summary

Validation

Summary by cubic

Summary by CodeRabbit

Uh oh!

coderabbitai Bot commented Sep 20, 2026 • edited Loading Uh oh! There was an error while loading. Please reload this page.

Uh oh!

Review limit reached

Walkthrough

Changes

Sequence Diagram(s)

Uh oh!

github-actions Bot commented Sep 20, 2026

Uh oh!

greptile-apps Bot commented Sep 20, 2026 • edited Loading Uh oh! There was an error while loading. Please reload this page.

Uh oh!

Confidence Score: 5/5

Summary

Diagram

Uh oh!

Uh oh!

teamleaderleo commented Sep 20, 2026

Uh oh!

teamleaderleo commented Sep 20, 2026

Uh oh!

Uh oh!

Reviewers

Assignees

Labels

Projects

Milestone

Development

Uh oh!

1 participant

teamleaderleo commented Sep 20, 2026 •

edited by cubic-dev-ai Bot

Loading

coderabbitai Bot commented Sep 20, 2026 •

edited

Loading

greptile-apps Bot commented Sep 20, 2026 •

edited

Loading