Skip to content

ci(fork): pull requests skip Fork CI jobs their changes cannot affect - #1135

Merged
incognitojam merged 3 commits into
ci/fork-ci-coverage-gapsfrom
ci/fork-ci-change-gates
Oct 29, 2026
Merged

incognitojam merged 3 commits into
ci/fork-ci-coverage-gapsfrom
ci/fork-ci-change-gates

Conversation

@incognitojam

@incognitojam incognitojam commented Oct 29, 2026 •

Copy link
Copy Markdown
Owner

Note

Pull requests now run only the Fork CI jobs their changed files can affect. Replaying the last 40 merged PRs through the classifier gives an estimated 30% fewer runner-minutes. Main and intake pushes still run everything. Lint now runs on every PR, docs-only ones included.

Fork CI had one switch: any change outside *.md and docs/ started every lane. A typical full run uses about 45 runner-minutes, and on a narrow PR most of that is spent on suites the change can't reach. Upstream gates its CI by changed packages in pingdotgg#19587. The fork already carries that classifier, .github/scripts/ci-changes.cjs, but only the disabled inherited ci.yml used it.

What changes

  • Fork Changes runs the classifier. It maps changed files to workspace packages and their dependents, and each job is gated on the result:
    • Typecheck and build: any affected package, or a desktop, server or web change.
    • Linux browser calibration: desktop or web changes.
    • Server shards, the previous-nightly upgrade test and Windows device tests: server (@styal/cli) changes.
    • Workspace suites:
      • Web 1/2 and 2/2 for web or client-runtime changes. Web 2/2 also runs for Rust crate changes.
      • Clients for desktop or mobile changes.
      • E2E for e2e, server or web changes.
      • Other tests only the affected remaining packages.
    • Release smoke: changes to scripts/.
  • Full runs where they matter. Pushes to main and intake branches still run every job, so main stays the backstop for anything the gate misses. Package manifests, paths the classifier doesn't recognise, and an unreadable diff also run everything.
  • Fork Lint runs for every change, so docs-only and ledger-only PRs now get format and lint checks too, as upstream does. When the ledger or the provider canary workflow changes, it also runs their tests. Before, those ran in a separate path inside Fork Changes.
  • The required checks keep their names. The aggregate jobs (Fork Check, Fork Test (Server), Fork Test (Workspace) and Fork Release Smoke) now run even after a cancellation. They accept a skipped job only when the classifier said it wasn't needed. Fork Release Smoke becomes an aggregate over a new Fork Release Smoke (Run) job, so a skipped run can't pass as a success.

Validation

  • Replay of the last 40 merged PRs. The estimate uses each job's median duration. Before: 1,831 runner-minutes. After: 1,292. Saved: 540 (29.5%).
    • 15 PRs touched workflows, manifests or shared paths and would still run everything.
    • Mobile-only PRs drop to Clients, Other and typecheck, saving about 34 minutes each.
    • Web-only PRs skip the server shards and Windows device tests, saving about 20 minutes.
    • Server-only PRs skip both web shards and Clients, saving about 15 minutes.
  • Failures a skip would have hidden. The replay checked each PR's real CI failures against the jobs it would skip. One case came up: fix(mobile): composer attachment uploads retry after a connection failure #1099, a mobile-only change, hit an unrelated server storage-cleanup assertion that the gate would now skip. Main's full run still catches failures like that.
  • Classifier tests. scripts/ci-changes.test.ts checks the classifier's decisions for representative paths. It also runs each aggregate job's jq rule against simulated job results, and checks that failures, cancellations, and skips the classifier didn't allow all fail.
  • Not exercised by this PR's CI. This PR edits fork-ci.yml and the classifier, so its own Fork CI run takes the run-everything path. The gated path first runs on the next narrow PR after merge.

Written by an agent (Claude Code, claude-opus-5-5), with implementation by Codex (gpt-6.1-sol).

@github-actions

github-actions Bot commented Oct 29, 2026 •

Copy link
Copy Markdown

Upstream divergence

Both sides are compared with upstream at the in-order tip b912bd2da0. Excludes .repos/ and pnpm-lock.yaml.

Measure Base This PR Change
Upstream files the fork changes 1,742 1,742 0
Lines changed in them 97,776 97,826 +50
Upstream files the fork removes 49 49 0
Fork-only files 694 693 −1
Fork-only lines 124,878 125,058 +180
Upstream files whose difference changes (1)
File Change Now differs by
.github/scripts/ci-changes.cjs +50 68 lines

@github-actions

github-actions Bot commented Oct 29, 2026 •

Copy link
Copy Markdown

Thread transfer impact

✅ Thread transfer remains within every enforced ceiling.

ℹ️ No successful main baseline artifact is available yet. This run establishes the initial measurement.

Provider Metric Main baseline This PR Impact PR ceiling
Codex Total thread wire — 5.0 KiB — 6.8 KiB ✅
Codex Thread snapshot wire — 3.8 KiB — 4.9 KiB ✅
Codex Live turn WebSocket wire — 1.2 KiB — 2.0 KiB ✅
Codex Live turn WebSocket decoded — 20.9 KiB — 29.3 KiB ✅
Codex Live turn messages — 2 — 8 ✅
Claude Total thread wire — 5.0 KiB — 6.8 KiB ✅
Claude Thread snapshot wire — 3.8 KiB — 4.9 KiB ✅
Claude Live turn WebSocket wire — 1.2 KiB — 2.0 KiB ✅
Claude Live turn WebSocket decoded — 21.2 KiB — 29.3 KiB ✅
Claude Live turn messages — 2 — 8 ✅

Baseline: unavailable · PR result: 64af9d2 · Source CI: success

Scenario and decoded snapshot size

10 historical turns, 5 command tools per turn, 878.9 KiB retained MCP result per historical turn, and a 1.05 MiB retained result in the measured turn.

  • Codex decoded thread snapshot: 108.5 KiB
  • Claude decoded thread snapshot: 108.8 KiB

Updated in place by a trusted workflow. PR artifacts are strictly validated and never executed.

incognitojam and others added 3 commits October 29, 2026 00:43
Fork Changes now runs .github/scripts/ci-changes.cjs, which maps changed
files to workspace packages and their dependents, and gates each Fork CI
job on the result. The classifier also decides which Fork Test
(Workspace) suites run, and which packages the Other suite tests.

Pushes to main and intake branches still run every job. Unknown paths,
package manifests and an unreadable diff also run everything. Fork Lint
runs for every change, including docs-only ones, and runs the ledger and
provider canary tests when those files change.

The required aggregate checks run even after a cancellation, and accept a
skipped job only when the classifier said it was not needed.

Co-authored-by: maria <maria@kuuro.net>
@incognitojam
incognitojam force-pushed the ci/fork-ci-change-gates branch from 611b200 to 64af9d2 Compare October 29, 2026 00:43
@incognitojam
incognitojam changed the base branch from main to ci/fork-ci-coverage-gaps October 29, 2026 00:44
@incognitojam
incognitojam added this pull request to stack #1139 October 29, 2026 00:44
@incognitojam
incognitojam merged commit dab6791 into main Oct 29, 2026
48 of 49 checks passed
@incognitojam
incognitojam deleted the ci/fork-ci-change-gates branch October 29, 2026 00:54
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

Projects

None yet

Development

Successfully merging this pull request may close these issues.

1 participant