ci: absent prerequisites SKIP with a named reason, and skipping is bounded

.github/workflows/ci.yml

@@ -5,10 +5,18 @@ name: CI
 # for weeks without anyone knowing: 9 pytest failures plus a module selftest broken since
 # 2026-06-18. A clean-machine CI run is the single strongest reason this code is in a repo at all.
 #
-# The suite is STATE-INDEPENDENT: verified 2026-08-21 by running it with ORCH_STATE_DIR pointed at
-# an empty directory — 330 passed. So CI needs no Brain database, no capability ledger and no
-# seeded state, which is what keeps the owner's rule (no evaluation databases stored remotely)
-# compatible with having CI at all.
+# CORRECTED 2026-08-21 after the first run came back red. The claim here was that the suite is
+# STATE-INDEPENDENT, "verified by running it with ORCH_STATE_DIR pointed at an empty directory —
+# 330 passed". That verification moved the WRONG KNOB. There are two: ORCH_STATE_DIR (the audit
+# cache, firing monitor, redirect sweep) and ORCH_LOCAL_RUNTIME (the capability LEDGER and the
+# Brain). Only the second one holds the state those checks read, so the experiment left the
+# owner's populated 40-row ledger in place and proved nothing about a fresh machine. On a real
+# runner the ledger bootstraps to the 14 rows the code declares, and 21 checks that read the
+# other 26 failed.
+#
+# Both knobs are now set, so the run really is state-free, and the checks that need this
+# instance's registration history SKIP with the missing row named instead of failing. verify.py
+# bounds that skipping with a ceiling — see .verify-floor.json.
 
 on:
   push:
@@ -42,14 +50,20 @@ jobs:
 
       - name: Verify
         env:
-          # State is machine-local by design. A fresh directory proves the suite does not depend on
-          # this developer's Brain, ledger or cadence stamps.
+          # State is machine-local by design, and it lives behind TWO variables. Setting only one
+          # is what made the "state-independent" claim above wrong for two months of one day.
           ORCH_STATE_DIR: ${{ runner.temp }}/orch-state
+          ORCH_LOCAL_RUNTIME: ${{ runner.temp }}/orch-runtime
         run: |
-          mkdir -p "$ORCH_STATE_DIR"
+          mkdir -p "$ORCH_STATE_DIR" "$ORCH_LOCAL_RUNTIME"
           python3 verify.py
 
       # verify.py already fails on: any pytest failure, ZERO tests collected, a collection count
-      # below the recorded floor, a module selftest that exits 0 without saying anything, and any
-      # of the five capability gates. Deliberately no `|| true` anywhere — an exit code that cannot
-      # fail is the defect this repo exists to stop repeating.
+      # below the recorded floor, passed+skipped dropping below the floor, MORE SKIPS THAN THE
+      # AGREED CEILING, a module selftest that exits 0 without saying anything, a selftest or gate
+      # that exits 0 having skipped without naming what is missing, and any of the five capability
+      # gates. Deliberately no `|| true` anywhere — an exit code that cannot fail is the defect
+      # this repo exists to stop repeating.
+      #
+      # It also prints every skip and its reason, so a green run here always states what it did
+      # not check. If that list grows, the ceiling turns it into a red rather than a footnote.

.verify-floor.json

@@ -1,5 +1,8 @@
 {
  "collected": 330,
  "passed": 330,
- "note": "floor recorded by verify.py --update-floor; a later run collecting fewer tests FAILS, because silently running fewer tests looks exactly like passing"
+ "skipped_max": 24,
+ "selftest_skipped_max": 7,
+ "gate_skipped_max": 2,
+ "note": "Recorded by verify.py --update-floor, except the *_max ceilings, which are edited BY HAND and never re-measured. `collected` catches tests that stopped being collected; `passed` is compared against passed+skipped, so a check may move between passing and consciously-skipped but the two together may never shrink. The *_max ceilings bound the skipped side: 24/7/2 is exactly what a machine with none of this instance's local prerequisites skips (a GitHub runner: no agent CLIs, no ~/.codex/skills, no /Applications/ChatGPT.app, no populated capability ledger), measured 2026-08-21. On the owner's machine all prerequisites exist and nothing skips at all. Raising a ceiling is a deliberate act: it means agreeing that one more thing is allowed to go unchecked, so say which and why in the commit."
 }

CLAUDE.md

@@ -75,6 +75,24 @@ Do not create a second event log, model registry, or capability inventory.
   real pytest, reads the COUNTS rather than the exit status, enforces a collection floor so tests
   silently ceasing to run cannot look like tests passing, treats a silent zero-exit selftest as a
   failure, and runs the five capability gates. CI runs the same command on a clean machine.
+- **A check whose PREREQUISITE is absent skips with the missing thing NAMED, and skipping is
+  bounded.** Some checks need what only a running instance has: the populated capability ledger,
+  an installed agent CLI, `~/.codex/skills`, the version-capable Codex binary. Those gates live in
+  `env_prereq.py` — detect the prerequisite, never `$CI`, so the same code is right on any machine.
+  Three rules, and they are enforced, not advisory: every skip carries a reason naming what is
+  missing; `verify.py` prints all of them so a green run always states what it did not check; and
+  `.verify-floor.json` caps the number of skipped tests, selftests and gates, so skipping one more
+  thing than agreed is a RED, not a footnote. Raising a ceiling means agreeing that one more thing
+  goes unchecked — do it deliberately and say why. When a check fails only because a stub leaked
+  (a monkeypatched `Popen` catching a model-catalog probe, say), the fix is isolation, not a skip:
+  that makes CI run MORE. **Never turn a real failure into a skip**, and never add a skip without
+  a reason string — a reason-less skip is indistinguishable from a pass, which is this repo's
+  founding defect wearing a different hat.
+- **State lives behind TWO variables and they are not the same.** `ORCH_STATE_DIR` holds the audit
+  cache, firing-monitor and redirect-sweep state; `ORCH_LOCAL_RUNTIME` holds the capability LEDGER
+  and the Brain. Pointing only the first at an empty directory and concluding "the suite is
+  state-independent" is exactly the mistake that made the first CI run red — the ledger never
+  moved. Set both when testing a fresh-machine claim.
 - **The split is TOOL vs EVIDENCE.** Generic capabilities, gates and tests are committed. This
   instance's evidence is not: `IMPROVEMENT_BACKLOG.md`, `CAPABILITY_USEFULNESS.md`,
   `LOCAL_POLICY.md`, `*.local.md`, `experiments/`, `ux_reviews/`, `data/`, `Audits/`. When adding a

README.md

@@ -40,6 +40,13 @@ HANDOFF:   ~/.codex/handoff/         (heartbeat orchestrator.json — legacy lan
   canonical tree — so canonical edits are yours alone, but always re-sync so the schedule sees them.
 - **Every module has a `--selftest`.** Run it after editing that module; it is the project's test
   suite (there is no separate pytest tree). `python3 <module>.py --selftest`.
+- **`python3 verify.py` is the whole verdict.** Real pytest plus every module selftest plus the five
+  capability gates, judged on the COUNTS rather than exit codes, against a recorded floor in
+  `.verify-floor.json`. It also bounds SKIPPING: a check needing something only a running instance
+  has (the populated capability ledger, an installed agent CLI, `~/.codex/skills`) skips with the
+  missing thing named — see `env_prereq.py` — and the floor file caps how many such skips are
+  allowed, so quietly checking less is a red. Every skip and its reason is printed, so a green run
+  always states what it did not check. On a machine with all prerequisites nothing skips at all.
 - **Activation is evidence-backed.** `features.py` describes reusable code maturity;
   `capabilities.py` is the activation authority. An `active` declaration must prove its matcher,
   invocation, artifact consumer, outcome sink, expiry, kill switch, and rollback. Each active tick

adapters.py

@@ -733,15 +733,17 @@ def _selftest():
     old_codex_sandbox = os.environ.pop("CODEX_SANDBOX", None)
     old_codex_bypass = os.environ.pop("ORCH_CODEX_BYPASS_INNER_SANDBOX", None)
     try:
-        _selftest_inner()
+        _selftest_inner(gaps=[])
     finally:
         if old_codex_sandbox is not None:
             os.environ["CODEX_SANDBOX"] = old_codex_sandbox
         if old_codex_bypass is not None:
             os.environ["ORCH_CODEX_BYPASS_INNER_SANDBOX"] = old_codex_bypass
 
 
-def _selftest_inner():
+def _selftest_inner(*, gaps: list[str] | None = None):
+    import env_prereq                       # imported here: env_prereq reads this module
+    gaps = gaps if gaps is not None else []
     c = build_command("cursor", "do x")
     # Composer is PINNED, not implied: omitting --model selects `auto`, which routes across every
     # frontier model cursor sells. Owner policy is Composer only (2026-08-08).
@@ -772,23 +774,30 @@ def _selftest_inner():
     # Non-tier modes still pass NO --model and keep the legacy lane tag.
     assert "--model" not in build_command("codex", "x", mode="assess"), "assess must not pin a model"
     assert model_identity("codex", None) == "codex:full:default"
-    profile_commands = {}
-    for profile in execution_profiles.profiles_for_agent("codex"):
-        cmd = build_command("codex", "x", mode="full", profile=profile, transport="local")
-        assert cmd[0] == str(CODEX_PROFILE_BIN), cmd
-        assert cmd[cmd.index("--model") + 1] == profile["requested_model"], cmd
-        assert cmd[cmd.index("--sandbox") + 1] == "workspace-write", cmd
-        assert cmd[cmd.index("-c") + 1] == f'model_reasoning_effort="{profile["reasoning_effort"]}"', cmd
-        profile_commands[profile["profile_id"]] = cmd
-        assess = build_command(
-            "codex", "x", mode="assess", profile=profile, transport="offload",
-            permission_mode="read-only",
-        )
-        assert assess[assess.index("--sandbox") + 1] == "read-only", assess
-        assert "--json" not in assess and assess[assess.index("--model") + 1] == profile["requested_model"], assess
-    assert {
-        cmd[cmd.index("--model") + 1] for cmd in profile_commands.values()
-    } == {"gpt-5.6-sol", "gpt-5.6-terra", "gpt-5.6-luna"}
+    # An EXACT profile resolves the version-capable Codex binary and `profile_codex_binary()`
+    # fails closed rather than falling back to PATH — deliberately, since a profile that cannot
+    # pin its version is not an exact profile. So this SECTION needs that binary installed; the
+    # default lives inside a macOS app bundle and cannot exist on a Linux runner. Everything else
+    # in this selftest runs anywhere.
+    if env_prereq.runnable(gaps, env_prereq.codex_profile_binary_absent()):
+        profile_commands = {}
+        for profile in execution_profiles.profiles_for_agent("codex"):
+            cmd = build_command("codex", "x", mode="full", profile=profile, transport="local")
+            assert cmd[0] == str(CODEX_PROFILE_BIN), cmd
+            assert cmd[cmd.index("--model") + 1] == profile["requested_model"], cmd
+            assert cmd[cmd.index("--sandbox") + 1] == "workspace-write", cmd
+            assert cmd[cmd.index("-c") + 1] == f'model_reasoning_effort="{profile["reasoning_effort"]}"', cmd
+            profile_commands[profile["profile_id"]] = cmd
+            assess = build_command(
+                "codex", "x", mode="assess", profile=profile, transport="offload",
+                permission_mode="read-only",
+            )
+            assert assess[assess.index("--sandbox") + 1] == "read-only", assess
+            assert "--json" not in assess and assess[assess.index("--model") + 1] == profile["requested_model"], assess
+        assert {
+            cmd[cmd.index("--model") + 1] for cmd in profile_commands.values()
+        } == {"gpt-5.6-sol", "gpt-5.6-terra", "gpt-5.6-luna"}
+    env_prereq.report_gaps("adapters.py", gaps)
     codex_cwd = HOME / ".codex" / "orchestrator" / "worktrees" / "selftest"
     ccwd = build_command("codex", "x", cwd=codex_cwd)
     assert "--cd" in ccwd and ccwd[ccwd.index("--cd") + 1] == str(codex_cwd), ccwd

capability_activation_audit.py

@@ -43,6 +43,7 @@
 import json
 import os
 import re
+import shutil
 import subprocess
 import sys
 import time
@@ -444,9 +445,19 @@ def _fleet_label_index(*, use_cache: bool = True) -> dict:
         except (OSError, ValueError):
             pass
     index = {}
+    # A FAILED gh call already means "unknown for this repo" (unauthenticated, offline, no
+    # access), and the loop skips it. An ABSENT gh binary meant an uncaught FileNotFoundError
+    # that took the whole audit down — same information, opposite outcome. Named here, and it
+    # short-circuits: with no gh at all there is nothing to ask 12 times.
+    if not shutil.which("gh"):
+        return {"generated_at": time.time(), "repos": {},
+                "unreadable": "gh CLI not installed; fleet label vocabulary unknown"}
     for full in getattr(backlog, "SUPPORTED_REPOS", []):
-        proc = subprocess.run(["gh", "label", "list", "--repo", full, "--limit", "300",
-                               "--json", "name"], capture_output=True, text=True, timeout=120)
+        try:
+            proc = subprocess.run(["gh", "label", "list", "--repo", full, "--limit", "300",
+                                   "--json", "name"], capture_output=True, text=True, timeout=120)
+        except (OSError, subprocess.SubprocessError):
+            continue                       # unknown for this repo, exactly like a nonzero exit
         if proc.returncode != 0:
             continue
         try: