Skip to content

ci: leave a merge receipt on each merged pull request - #14519

Merged
teamleaderleo merged 3 commits into
mainfrom
ci-merge-receipt
Sep 25, 2026
Merged

teamleaderleo merged 3 commits into
mainfrom
ci-merge-receipt

Conversation

@teamleaderleo

@teamleaderleo teamleaderleo commented Sep 25, 2026 •

Copy link
Copy Markdown
Collaborator

Why

main behaves like nightly: pull requests merge fast and conflict fast, and fixing breakage quickly matters more than blocking merges. But nobody records what CI had actually verified when a pull request landed. On 2026-09-25, 9 of 60 merged pull requests (15%) landed before any green ci-status on their head: #14461 merged while macOS compile admission was still running, and #14382 later broke SidebarAccessibilityTreeTests (also #14455 #14409 #14408 #14403 #14395 #14384 #14379).

What

  • .github/workflows/merge-receipt.yml: pull_request_target on closed when merged, so fork pull requests get it too. It checks out only scripts/ci/merge_receipt.py at github.workflow_sha (never pull request code), reads the GitHub API, and runs after the merge, so it cannot block or slow one.
  • scripts/ci/merge_receipt.py: reads the head commit's check runs and statuses (GraphQL statusCheckRollup), takes each check's state as of the merge time (finished before it, still running, or not started), groups the noise (guards jobs fold into one, app-host shards into one, bots and CLA hidden unless they failed), and posts one comment, edited in place on a re-run through a hidden marker. A judging check (macOS compile admission, app-host unit tests, ci-status, required checks) that was not green adds the new merged-unverified label; a re-run that finds it green removes it.
  • scripts/ci/main_regression_attribution.py (ci: name the merged pull request behind each new main full-suite failure #14436): when suspects tie on score, pull requests labeled merged-unverified are listed first; the verified ones stay in the list, since a verified head can still break main through another merge. The label never creates a suspect on its own, and the existing caps are unchanged.
  • Safety: only a github-actions comment carrying the marker is edited, and check names are escaped so a pull request's own job names cannot mention people or hide the receipt. Known limits (100-comment marker search, statuses show current state, merge queue would need gating) are noted in the script docstring.
  • Tests: tests/test_ci_merge_receipt.py, fixture-driven from real snapshots of Wire AgentChatProseStreamWakeDriver.swift into the app target #14461 (unverified) and ci: weigh a package source change when an owned Mac picks its starting build #14433 (all green), no network. Registered on the linux-guard lane and in the guards ci group, with workflow_guard_groups.py owners for the script, workflow, fixtures, and the attribution script.

Sample receipt, dry run on #14461:

**Merge receipt** for `e9426f528e`, merged 2026-09-25 10:23:09 UTC
- Not verified at merge: ci-status (not reported), macOS compile admission (in progress)
- Verified: Web complexity, web-validation, Fast static checks, GhosttyKit release check, guards (17), linux-preflight, macOS admission gate, Testbox broker trust boundary
- Skipped by policy: browser, Claude wrapper regressions, remote-daemon, suite-coverage, swift-package-tests, web, web-build, web-database-tests, web-tests
- Full suite: runs on main after merge.

Labeled `merged-unverified`: if main breaks near this merge, look here first.

An all-green merge (#14433) gets one line: "every check was green at merge (14 verified; 15 skipped by policy). Full suite runs on main after merge."

How validated

🤖 Generated with Claude Code

A pull request can merge before every check on its head finishes; on
2026-09-25, 9 of 60 did. A post-merge workflow now comments once on each
merged pull request with what was verified at merge, what was still running
or missing, and what policy skipped, reading each check as of the merge time.
When a judging check (compile admission, app-host unit tests, ci-status,
required checks) was not green, the pull request gets the merged-unverified
label, and main regression attribution prefers such pull requests in a tie.

Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
@coderabbitai

coderabbitai Bot commented Sep 25, 2026 •

Copy link
Copy Markdown

Warning

Review limit reached

Next included review available in 1 minute.

Check out review usage here.

View limit details

Limit details: You’ve used all 10 included reviews currently available.

You've used all free OSS reviews for now. Wait for the free limit to reset to keep reviewing this public repository.

Learn how review limits work.

Review configuration:

⚙️ Run configuration

Configuration used: Repository: manaflow-ai/cmux/.coderabbit.yaml

Review profile: ASSERTIVE

Plan: Advanced

Run ID: bf6cc2ce-bdc9-4604-b014-13616a8a6f70

📥 Commits

Reviewing files that changed from the base of the PR and between 6f42d6c and f844576.

📒 Files selected for processing (10)
  • .github/workflows/ci-guards.yml
  • .github/workflows/merge-receipt.yml
  • scripts/ci/main_regression_attribution.py
  • scripts/ci/merge_receipt.py
  • scripts/ci/workflow_guard_groups.py
  • tests/fixtures/merge_receipt/pr14433.json
  • tests/fixtures/merge_receipt/pr14461.json
  • tests/test-execution.toml
  • tests/test_ci_main_regression_attribution.py
  • tests/test_ci_merge_receipt.py

Thanks for using CodeRabbit! It's free for OSS, and your support helps us grow. If you like it, consider giving us a shout-out.

❤️ Share

Comment @coderabbitai help to get the list of available commands.

@github-actions

Copy link
Copy Markdown
Contributor

All contributors have signed the CLA ✍️ ✅
Posted by the CLA Assistant Lite bot.

teamleaderleo and others added 2 commits September 25, 2026 06:54
Only a github-actions comment carrying the marker is edited, check names are
escaped so a pull request's own job names cannot mention people or hide the
receipt, and a failed gh call prints GitHub's error. Attribution now lists
merged-unverified pull requests first in a tie instead of dropping the rest,
since a verified head can still break main through another merge. Known
limits are noted in the script's docstring.

Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
GitHub still links @user and #123 after a backslash, so a zero-width space
follows each @ and # instead (checked with the markdown API).

Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
@cursor

cursor Bot commented Sep 25, 2026

Copy link
Copy Markdown

Bugbot is paused — on-demand spend limit reached

Bugbot uses usage-based billing for this team and has hit its on-demand spend limit.

A team admin can raise the spend limit in the Cursor dashboard, or wait for the next billing cycle to continue.

@teamleaderleo
teamleaderleo merged commit 37c8535 into main Sep 25, 2026
48 checks passed
@teamleaderleo
teamleaderleo deleted the ci-merge-receipt branch September 25, 2026 11:01
@github-actions

Copy link
Copy Markdown
Contributor

Merge receipt for f8445764e0: every check was green at merge (8 verified; 11 skipped by policy). Full suite runs on main after merge.

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

None yet

Projects

None yet

Development

Successfully merging this pull request may close these issues.

1 participant