(MOT-4183) fix(ci): stabilize integration and Rust caches

[ytallo]

Summary

• run worker fmt, lint, tests, PR checks, and interface smoke only for workers directly changed by the pull request
• run the full harness integration E2E suite only when harness-stack inputs change
• move the integration job into a reusable workflow with restore-only caches for pull requests
• warm the harness Rust cache plus shared engine and integration caches from trusted main, scheduled, and manual runs
• remove caches belonging to closed pull requests and report the repository cache budget

Why

Pull-request caches were scoped to individual merge refs and could not be reused by sibling PRs or main. The repository had more than 11 GB of active caches against a 10 GiB operating budget, while a cache miss pushed the integration job beyond 20 minutes.

The default branch did not run the main CI workflow after merges, so it never produced reusable caches. Harness changes also expanded every runtime dependency from iii.worker.yaml into the worker matrices, causing unrelated jobs such as shell formatting to run even though those workers do not compile against the local harness source.

This change makes main the cache producer, keeps PR jobs restore-only, and separates integration-stack coverage from per-worker checks.

Validation

• 127 GitHub script tests passed
• 14 discovery and worker-selection tests passed
• a forced harness warm selects only rust: [harness], with empty Node and Python matrices
• actionlint v1.7.12 passed
• Python bytecode compilation passed
• git diff --check passed
• cache API filtering identified refs/pull/518/merge as a closed PR without deleting it during development

Fixes MOT-4183
Refs MOT-4107

Summary by CodeRabbit

• New Features

  • Added dedicated harness integration end-to-end testing for relevant infrastructure, workflow, and worker changes.
  • Added cache-warming automation for Rust builds, integration testing, and frontend assets.
  • Added explicit worker selection for targeted CI runs.

• Bug Fixes

  • Integration tests now run only when integration-related changes are detected.
  • Improved cache cleanup and budget monitoring for pull requests and scheduled runs.
  • Added validation for integration reports and test artifacts.

[vercel]

The latest updates on your projects. Learn more about Vercel for GitHub.

Project	Deployment	Actions	Updated (UTC)
workers	Ready	Preview, Comment	Jul 23, 2026 12:21am
workers-tech-spec	Ready	Preview, Comment	Jul 23, 2026 12:21am

[coderabbitai]

Caution

Review failed

The pull request is closed.

ℹ️ Recent review info

⚙️ Run configuration

Configuration used: Organization UI

Review profile: CHILL

Plan: Pro Plus

Run ID: e2741cbc-b47f-49c1-a07e-54e5ed79250b

📥 Commits

Reviewing files that changed from the base of the PR and between 81d14fd and 8a0bb0e.

📒 Files selected for processing (5)

• .github/scripts/discover_changed_workers.py
• .github/scripts/tests/test_discover_changed_workers.py
• .github/workflows/_harness-integration.yml
• .github/workflows/cache-warm.yml
• .github/workflows/ci.yml

---

📝 Walkthrough

Walkthrough

Worker discovery now distinguishes integration changes, supports explicit worker forcing, and drives conditional harness integration. New reusable and cache-warming workflows build, test, verify, cache, and report integration artifacts.

Changes

Integration-aware CI and cache workflows

Layer / File(s)	Summary
Integration-aware worker selection .github/scripts/discover_changed_workers.py, .github/scripts/tests/test_discover_changed_workers.py	Integration paths and forced workers affect selection and integration_changed; dependency fan-out is removed and covered by updated tests.
Reusable harness integration workflow .github/workflows/_harness-integration.yml	The reusable workflow validates the pinned engine, builds Rust and Console components, runs E2E checks, verifies report hashes, and uploads artifacts.
Cache warming and budget management .github/workflows/cache-warm.yml	Cache cleanup, worker warming, integration warming, and cache-budget enforcement are added.
CI integration routing and cache configuration .github/workflows/ci.yml	CI exposes integration_changed, conditionally invokes the reusable workflow, and updates Rust cache settings.

Estimated code review effort: 4 (Complex) | ~60 minutes

Sequence Diagram(s)

sequenceDiagram
  participant ChangedWorkerDiscovery
  participant CIWorkflow
  participant HarnessIntegrationWorkflow
  participant IntegrationEngine
  ChangedWorkerDiscovery->>CIWorkflow: emit integration_changed
  CIWorkflow->>HarnessIntegrationWorkflow: invoke when integration_changed is true
  HarnessIntegrationWorkflow->>IntegrationEngine: checkout and build pinned revision
  HarnessIntegrationWorkflow->>HarnessIntegrationWorkflow: run harness and Console E2E tests
  HarnessIntegrationWorkflow->>CIWorkflow: publish verified reports and artifacts

Loading

Poem

I’m a rabbit with a workflow key,
Sorting workers neatly as can be.
Engines build and tests take flight,
Caches warm beneath moonlight.
Reports hop home, checksums bright—
Integration runs just right!

✨ Finishing Touches

📝 Generate docstrings

• Create stacked PR
• Commit on current branch

🧪 Generate unit tests (beta)

• Create PR with unit tests
• Commit unit tests in branch fix/ci-cache-stability

Warning

There were issues while running some tools. Please review the errors and either fix the tool's configuration or disable the tool if it's a critical failure.

🔧 Checkov (3.3.8)

.github/workflows/_harness-integration.yml

Traceback (most recent call last):
File "/usr/local/bin/checkov", line 2, in
from checkov.main import Checkov
ModuleNotFoundError: No module named 'checkov'

.github/workflows/cache-warm.yml

Traceback (most recent call last):
File "/usr/local/bin/checkov", line 2, in
from checkov.main import Checkov
ModuleNotFoundError: No module named 'checkov'

.github/workflows/ci.yml

Traceback (most recent call last):
File "/usr/local/bin/checkov", line 2, in
from checkov.main import Checkov
ModuleNotFoundError: No module named 'checkov'

---

Thanks for using CodeRabbit! It's free for OSS, and your support helps us grow. If you like it, consider giving us a shout-out.

❤️ Share

• X
• Mastodon
• Reddit
• LinkedIn

_{Comment @coderabbitai help to get the list of available commands.}

[github-actions]

skill-check — worker

0 verified, 47 skipped (no docs/).

Layer	Result
structure	✓
vale	✓
ai	✓
render	✓

Four for four. Nicely done.