review: move deep-reasoning reviewer roles to Fable 5

.changeset/review-fable-deep-reasoning.md

@@ -0,0 +1,5 @@
+---
+"review": minor
+---
+
+Move the `correctness-reviewer` from Opus 4.8 to Fable 5 (`claude-fable-5`). Bug-finding recall is the workflow's load-bearing metric, and the 2026-07-20 pooled A/B's recall gain (83% to 90%, true misses down 43%) concentrated in correctness-adjacent rows, so the 2x per-token price goes to the one role where the measurement supports it. Deliberately NOT moved, per review: `claim-validator` stays Opus at xhigh (the Fable arm measurably did not improve the precision gate: noise 43% to 49%, one wrong blocking flag on a clean case); the opt-in whole-change reviewers (`holistic`, `completeness`, `test-adequacy`) stay Opus (the A/B measured the full-Fable bundle, so their contribution is unattributed, and both known consumers enable them in ROUTING so they are not free in practice); the procedural roles (orchestrator, `thread-reconciler`, `skill-auditor`, `conventions`) stay Opus (instruction-following over staged context, and the orchestrator is the token-heaviest agent under a 20-minute timeout); and the specialist lenses stay Opus because Fable's cyber safety classifiers can refuse benign security-focused analysis, and a refused security lens would be a silent coverage hole. `first-principles` already ran Fable 5. The firewall pricing pin (awf v0.27.27) and the `models:` cost block already cover `claude-fable-5`, so no sandbox or pricing change is needed.

.github/workflows/review-eval-drift.yml

@@ -35,16 +35,23 @@ on:
         # case-arm-runs (~$0.82 each) with the tail of the corpus
         # budget-skipped, which also contaminated the noise-floor bands
         # (asymmetric case mix). The 23-case corpus (both new mutation cases
-        # are live-tagged) needs 23 x 3 repeats x 2 arms = 138 case-arm-runs,
-        # ~$113; 120 clears it while staying a hard stop against runaways,
-        # though the headroom is thin: the ~$0.82/run mean comes from a run
-        # whose tail was budget-skipped, so it may understate. Repeats stay
-        # at 3: they are
-        # the power axis, and per-case repeat tiering would make every
-        # noise-floor sample a different case mix.
+        # are live-tagged) needed 23 x 3 repeats x 2 arms = 138 case-arm-runs,
+        # ~$113; 120 cleared it while staying a hard stop against runaways.
+        # Repeats stay at 3: they are the power axis, and per-case repeat
+        # tiering would make every noise-floor sample a different case mix.
+        #
+        # Re-sized 2026-07-21 for the Fable-5 correctness-reviewer plus the
+        # discrimination cases added alongside it: 30 live cases (2 clean +
+        # 5 golden + 6 incidents + 7 smoke + 10 synthetic-mutations) x 3
+        # repeats x 2 arms = 180 case-arm-runs. The de-scoped roster measured
+        # ~$1.08 per case-arm-run in the pooled A/B (runs
+        # 29855626692-29855643020: $97.19 / 90), so a drift week with both
+        # arms on this roster is ~$194; 220 clears it with real headroom,
+        # which matters now that a budget-skipped tail is a red run (the
+        # caseAsymmetry gate). If the swap is reverted, dial back toward 160.
         description: "Total hard budget across both arms and all repeats"
         required: false
-        default: "120"
+        default: "220"
 
 permissions:
   # write: the report step commits the aggregate under
@@ -71,7 +78,7 @@ jobs:
         env:
           ANTHROPIC_API_KEY: ${{ secrets.ANTHROPIC_API_KEY }}
           REPEATS: ${{ inputs.repeats || '3' }}
-          MAX_USD: ${{ inputs.max_usd || '120' }}
+          MAX_USD: ${{ inputs.max_usd || '220' }}
         run: |
           if [ -z "$ANTHROPIC_API_KEY" ]; then
             echo "ANTHROPIC_API_KEY secret not configured; skipping the drift run." >&2

workflows/review/README.md

@@ -297,21 +297,32 @@ sub-agent models — this table is the human-facing summary:
 | orchestrator | `claude-opus-4-8` | high | Owns every GitHub/safe-output decision |
 | `pattern-triage` | `claude-sonnet-4-6` | medium | Cheap first-pass triage |
 | `thread-reconciler` | `claude-opus-4-8` | medium | Reconciliation |
-| `correctness-reviewer` | `claude-opus-4-8` | high | Whole-change reviewer |
+| `correctness-reviewer` | `claude-fable-5` | high | Whole-change reviewer; bug-finding recall is the load-bearing metric |
 | `skill-auditor` | `claude-opus-4-8` | high | Whole-change reviewer |
 | `holistic` | `claude-opus-4-8` | high | Opt-in whole-change reviewer (`enable` in `ROUTING`) |
 | `completeness` | `claude-opus-4-8` | high | Opt-in whole-change reviewer (`enable` in `ROUTING`) |
 | `test-adequacy` | `claude-opus-4-8` | high | Opt-in whole-change reviewer (`enable` in `ROUTING`) |
 | `conventions` | `claude-opus-4-8` | medium | Opt-in advisory targeted check (`enable` in `ROUTING`) |
 | `first-principles` | `claude-fable-5` | high | Opt-in advisory-only; reviews the change's justification |
-| `claim-validator` | `claude-opus-4-8` | xhigh | Adversarial claim validation |
+| `claim-validator` | `claude-opus-4-8` | xhigh | Adversarial claim validation; stays Opus (the Fable arm did not improve precision) |
 | specialist lenses | `claude-opus-4-8` | high | Opt-in via `lens=` in `ROUTING`; the security & auth lens is xhigh |
 
 Only the orchestrator and the default roster (`pattern-triage`,
 `correctness-reviewer`, `skill-auditor`, `thread-reconciler`, `claim-validator`)
 run by default; every other row is opt-in via `ROUTING` and earns its line through
-the eval suite. Per-role Fable-5 / Sonnet experiment arms are eval-suite
-measurements to run after the suite exists.
+the eval suite. Exactly two roles run Fable 5: `first-principles` (from day
+one, for perspective diversity) and `correctness-reviewer` (the 2026-07-20
+A/B's recall gain concentrated in correctness-adjacent rows, and bug-finding
+recall is the load-bearing metric). Everything else stays on Opus 4.8
+deliberately: the A/B measured the full-Fable bundle, so the other roles'
+contribution is unattributed while their cost is not (both known consumers
+enable the whole-change reviewers in `ROUTING`, so they are not free in
+practice); the Fable `claim-validator` measurably did not improve the
+precision gate (noise 43% -> 49%); and the specialist lenses stay Opus
+because Fable's cyber safety classifiers can refuse benign security-focused
+analysis, and a refused security lens would be a silent coverage hole. Any
+further per-role promotion (or Sonnet step-down) earns its line through its
+own eval-suite arm.
 
 ### Feedback signal: thumbs sweep and live counters
 

workflows/review/eval/README.md

@@ -10,7 +10,7 @@ comment is the reference for its internals.
 | --- | --- | --- | --- |
 | Deterministic suite (vitest) | The pipeline code: router, gates, verdict, rendering, matching, aggregation. Replays recorded findings; no model calls. | $0 | Every push (`pnpm test --run`) |
 | Live A/B, smoke subset | One PR's marginal `review.md` delta: real model sub-agents run from BOTH the base branch's and the PR's review.md over live-tagged smoke cases. A tripwire, not a measurement. | ~$10/PR | Every PR touching `workflows/review/**` |
-| Powered / scheduled runs | Recall effects, priced with repeats and binomial intervals; run-to-run wobble; cumulative drift vs main. | ~$29-85 | `workflow_dispatch`, weekly cron |
+| Powered / scheduled runs | Recall effects, priced with repeats and binomial intervals; run-to-run wobble; cumulative drift vs main. | ~$29-220 | `workflow_dispatch`, weekly cron |
 
 Single-run percentage deltas below the measured noise floor mean nothing
 (see "Reading a report" below). Any recall claim needs a powered run.
@@ -66,13 +66,16 @@ gh workflow run review-eval-ab.yml --ref <branch> \
   -f cases=adversarial-injection-approve,golden-request-changes-authz \
   -f repeats=10 -f max_usd=45
 
-# Cumulative measurement vs production (~$60): full corpus x3 against main
+# Cumulative measurement vs production (~$170 at post-Fable-swap measured
+# rates: ~$0.80/case-arm-run on the all-Opus baseline arm, ~$1.08 on the
+# Fable-correctness candidate arm, 30 live cases x3): full corpus x3
 gh workflow run review-eval-ab.yml --ref <branch> \
-  -f base_ref=origin/main -f full=true -f repeats=3 -f max_usd=85
+  -f base_ref=origin/main -f full=true -f repeats=3 -f max_usd=200
 
-# Noise floor / wobble control: identical arms, full corpus x3
+# Noise floor / wobble control (~$194 measured with both arms on the
+# Fable-correctness roster): identical arms, full corpus x3
 gh workflow run review-eval-ab.yml --ref <branch> \
-  -f base_ref=origin/<branch> -f force_arms=true -f full=true -f repeats=3 -f max_usd=85
+  -f base_ref=origin/<branch> -f force_arms=true -f full=true -f repeats=3 -f max_usd=220
 
 # Pool reports across dispatches (run ids or local paths)
 pnpm dlx tsx workflows/review/eval/aggregate.ts <run-id> <run-id> ... [--out <path>]

workflows/review/review.md

@@ -175,7 +175,7 @@ engine:
   model: claude-opus-4-8
 timeout-minutes: 20
 
-# claude-fable-5 (the first-principles reviewer's pinned model) is not in the
+# claude-fable-5 (pinned by first-principles and correctness-reviewer) is not in the
 # AI-credits pricing table of the firewall api-proxy that gh-aw <= v0.81.x pins
 # (gh-aw-firewall v0.27.11), and the proxy rejects any un-priced model with a 400,
 # so the first-principles dispatch fails on every run where it is enabled. Two
@@ -1717,9 +1717,11 @@ return `{"findings": [], "hunts": [...]}` with the hunt states still recorded.
 ---
 name: correctness-reviewer
 description: Classifies each changed file's risk and reviews the diff for correctness defects; returns JSON.
-model: claude-opus-4-8
+model: claude-fable-5
 # effort: high — launch default (whole-change reviewer). gh-aw has no per-agent
 # effort field yet; the per-role model/effort table lives in the README.
+# Fable 5: bug-finding recall is this workflow's load-bearing metric, and
+# stronger real-defect detection is Fable's headline gain over Opus 4.8.
 ---
 You are a correctness-focused code reviewer. You have **no GitHub access** — read the
 diff and file list from disk and return your result as JSON only.
@@ -2114,7 +2116,11 @@ Return ONLY this JSON object (no prose, no code fence):
 name: claim-validator
 description: Re-checks each candidate review comment against the actual code and the repo's best-practice skills, and drops or corrects the ones that are wrong; returns JSON.
 model: claude-opus-4-8
-# effort: xhigh — launch default (claim-validator).
+# effort: xhigh — launch default (claim-validator). Deliberately NOT moved to
+# Fable 5 with the correctness reviewer: in the 2026-07-20 pooled A/B the
+# Fable validator did not offset the higher flag rate (noise 43% -> 49%, one
+# wrong blocking flag on a clean case), so the precision gate stays on Opus
+# until an arm shows otherwise; prompt tightening is the queued follow-up.
 ---
 You are a skeptical validator. Other reviewers proposed the comments in
 `/tmp/gh-aw/review/claims.json`; your job is to catch the ones that are **wrong** —
@@ -2496,16 +2502,17 @@ If the changed behavior is adequately tested, return {"findings": []}.
 name: first-principles
 description: A diverse-perspective, advisory-only sanity check on whether the change should exist as written; returns findings as JSON.
 model: claude-fable-5
-# effort: high — launch default. Runs on Fable 5 (claude-fable-5) day one for a
-# genuinely different perspective. Advisory-only, never blocks.
+# effort: high — launch default. Ran on Fable 5 (claude-fable-5) from day one;
+# the correctness reviewer joined it after the 2026-07-20 A/B. Advisory-only,
+# never blocks.
 ---
 You are the **first-principles** reviewer. Your single mandate is to review the
 **justification for the change, not the change itself**: where `holistic` asks
 whether the diff hangs together, you step outside the change's own framing and ask
 whether it **should exist as written**. Your primary input is the stated rationale —
 the PR title/description and the problem it claims to solve — read against the diff,
-not the diff line by line. You run on a different model (Fable 5) on purpose,
-to bring a perspective the other reviewers do not. You have **no GitHub access** — read
+not the diff line by line. You are prompted for a deliberately different
+perspective than the other reviewers, so bring one. You have **no GitHub access** — read
 from disk and return JSON only.
 
 **You are advisory-only and you never block.** Every finding you return MUST carry a