docs: inventory the CI lists that must track something else

[teamleaderleo]

Problem

Five of today's incidents were the same bug wearing different clothes. Something in the tree had to be updated in lockstep with something else, the pull request that forgot was green on its own, and main went red for everyone:

• the guard step ownership manifest (Devex: add edit-weighted build graph health report #13535, main red 88 minutes; ci: derive guard step ownership from ci-guards.yml #13642 derived it from ci-guards.yml's own if: conditions)
• the Python test execution registry, stale three times in one day (ci: register the nine tests main's execution registry does not know about #13710, ci: register the R2 cache census in the test execution registry #13731, ci: register the settings JSONC and test-wiring regressions #13739)
• ci: wire the test-wiring sync guard and register the last two unlisted tests #13738 and ci: register the settings JSONC and test-wiring regressions #13739 each adding the same registry entry — both green alone, the merge conflict-free and invalid
• macos / macOS status rejecting an unset compile_admitted instead of defaulting (ci: read an unset compile_admitted as compile in macOS status #13727)
• fork pull requests getting no repository variables, so an empty value picked the most expensive path (ci: default the PR suite policy and R2 cache for fork pull requests #13717)

Each was fixed where it was found. Nothing wrote down that they are one class, so the next copy of the same shape keeps landing.

Resulting behavior

docs/ci/derived-not-declared.md inventories 30 of these across .github/, scripts/ci/ and tests/ — for each: what it duplicates, what drift costs, whether it fails only the offending pull request or everyone, and whether it could be derived instead. Ordered by blast radius times likelihood, with a section on why ci.yml's trigger set makes most of these merge-queue problems rather than main problems, and which five workflows do run on push: main and so are everyone's problem immediately.

Three rules fall out: derive from the source of truth; fail only the pull request that causes the drift; treat missing configuration as the cheap default, never the expensive one. Plus the corollary the #13738/#13739 collision makes concrete — an append-only file where two pull requests can add the same line merges cleanly into an invalid file, because git cannot see a semantic conflict between two appends to different regions.

Two rows are implemented alongside this doc, in their own pull requests: #13775 derives the Linux guard route inputs from ci-guards.yml, and #13777 writes the cheap default next to 21 bare vars. reads and adds a guard. Everything else carries a one-line note on why not and what it would take.

Validation and remaining gap

Four rows were verified against the tree rather than reasoned about: ci-artifact-transport.yml really does guard nine paths on pull requests and not on push: main; .github/review-bot-rules/README.md indexes 27 of 29 rule files and is missing test-determinism.md; ROUTING_POLICY_PATHS omits tests/test_ci_source_lint_guard_structure.py though it lists the other four *_guard_structure.py files; 43 distinct repository variables are in use and docs/ci-runners.md documents 27.

This is a documentation change only — no workflow, script or test is touched, and docs/**.md routes as plain documentation.

Remaining gap: rows 1 and 2 (branch-protection required checks, and workflow display names that merge-group-fail-fast.yml matches on) are the two that can hang the merge queue for everyone with no red check anywhere, and neither is derivable from the tree — GitHub owns both. The doc says what reconciling them would need; it does not solve them.

Part of #13095.

🤖 Generated with Claude Code

---

^{Need help on this PR? Tag @codesmith-bot with what you need. Autofix is disabled.}

---

Summary by cubic

Adds docs/ci/derived-not-declared.md, documenting the CI lists that duplicate another source of truth — the same bug shape behind five incidents today where a hand-maintained list went stale, the fixing pull request was green on its own, and main went red for everyone.

• Catalogs 30 lists across .github/, scripts/ci/, and tests/: what each duplicates, what drift costs, who it fails, and whether it can be derived.
• Records the three rules the incidents imply: derive from the source of truth, fail only the pull request that causes the drift, and treat missing configuration as the cheap default.
• Three rows are implemented in their own pull requests; two rows are underivable because GitHub owns the source of truth, so the doc says what reconciling them would need.
• Documentation only — no workflow, script, or test is touched.

^{Written for commit 0cd6a66. Summary will update on new commits.}

Summary by CodeRabbit

• Documentation
  • Added a postmortem and inventory of CI incidents caused by manually maintained lists drifting from their sources of truth.
  • Documented CI workflow impact, two implemented fixes, and seven potential follow-up items.

[github-actions]

All contributors have signed the CLA ✍️ ✅
_{Posted by the CLA Assistant Lite bot.}

[coderabbitai]

Navigate logical layers of code changes, visualize relationships, and explore their blast radius.

Note

Currently processing new changes in this PR. This may take a few minutes, please wait...

⚙️ Run configuration

Configuration used: Repository: manaflow-ai/cmux/.coderabbit.yaml

Review profile: ASSERTIVE

Plan: Advanced

Run ID: ae87d869-0cfb-4d3e-95cd-196e42fff392

📥 Commits

Reviewing files that changed from the base of the PR and between ce6a31e and 0cd6a66.

📒 Files selected for processing (1)

• docs/ci/derived-not-declared.md

 __________________________________________________
< Veni, Vidi, Validavi. I came, I saw, I reviewed. >
 --------------------------------------------------
  \
   \   \
        \ /\
        ( )
      .( o ).

✨ Finishing Touches

🧪 Generate unit tests (beta)

• Commit to this branch
• Create a new PR

---

Thanks for using CodeRabbit! It's free for OSS, and your support helps us grow. If you like it, consider giving us a shout-out.

❤️ Share

• X
• Mastodon
• Reddit
• LinkedIn

_{Comment @coderabbitai help to get the list of available commands.}