fix(verify): land evidence-acquisition and the unblock() guard it depends on by stranske · Pull Request #34 · stranske/Orchestrator

stranske · 2026-08-23T04:22:19Z

python3 verify.py was RED on main with three failures, all from one ledger row:

test_capability_admission.py::test_new_capabilities_carry_all_required_parts
  -> {'evidence-acquisition': ['caller_exists', 'heartbeat', 'fixture']}
test_capability_set_coverage.py::test_every_capability_has_a_recurrence_fixture
  -> ['evidence-acquisition']
test_model_tier_resolution.py::test_every_capability_has_a_heartbeat_call_site
  -> ['evidence-acquisition']

The row was not orphaned

evidence-acquisition is registered in the live ledger, but its code was not in this tree — which
reads as "a ledger row registered without its implementation". It wasn't. The module exists on
claude/profile-per-agent (commit e73b47d, pushed, no PR). The ledger is shared machine-local
state; code is branch-isolated. So a sibling branch's registration turns main red.

That distinction mattered: retiring the row or waiving the checks would each have discarded live,
complete work — and would have hidden the bug below.

The lane has a hard dependency that main did not have

evidence_acquisition.py delegates the "is this worth feeding" decision to
capabilities.unblock() instead of reimplementing it. The _has_default_off_switch guard it relies
on is in the parent commit (69a1c38). Landing the lane alone reproduced the exact latched-gate
defect its own commit message claims to avoid:

	`feedable`	state
guard absent	2 (`range-lane-rollout`, `synthesis-promotion`)	`no_candidates`
guard present	0	`nothing_to_feed` — matches the ledger's recorded event

Both capabilities it wanted to feed are held by a documented default-off switch. Feeding a
switched-off capability manufactures work it cannot execute → no durable reuse → the debt never
falls → it is fed again next cycle, forever. The drain is blocked by the switch the feed ignored.

The lane's own selftest asserts "default-off switch must not be fed" and passed on main without
the guard — its synthetic fixture was excluded by a different unblock() branch. A test passing
for the wrong reason is why this needed the live-ledger comparison, not just a green selftest.

What landed

Cherry-picked with -x from claude/profile-per-agent, scoped to the dependency:

052ca2b — e73b47d verbatim: evidence_acquisition.py + its caller (cadence_registry.py,
orchestrate.sh), heartbeat, and recurrence fixture.
f5c7052 — 69a1c38 capabilities.py only. That commit's unrelated adapters.py auth-probe
change stays on its branch.

Pure addition: 5 files, +348, −0. All eight admission parts pass, enforced and unwaived — no
WAIVERS entry was needed. No failure was converted into a skip (skip counts are 0).

pytest:     368 passed, 0 failed, 0/24 max skipped (368 collected; floor 366)
selftests:  83 of 83 modules ran, 0/7 max skipped
activation audit   ok  CAN FIRE:   43 of 43  (100%)
recurrence replay  ok  WOULD FIRE:  43 of 48
set coverage       ok  all 6 checks over ALL 43 ledger capabilities
admission          ok  all 8 admission checks passed — 7 enforced, 3 legacy debt, commitments clean
ledger validate    ok  "valid": true

VERIFIED — 368 tests actually executed and passed, 83 selftests spoke, 5 of 5 gates green

Notes for the merger

The mirror is deliberately not synced — orch-sync-mirror.sh is the manual circuit breaker
between an agent's change and the dispatcher. Run it after merge, then re-run verify.py from the
mirror.
claude/profile-per-agent still holds 20 unmerged commits. Its own PR will see these two patches
already applied.

🤖 Generated with Claude Code

…d for want of evidence `unblock()` names the capabilities whose only blocker is missing evidence, but nothing acted on that list — the report was read by no one and the debt did not move. This adds the lane that acts on it. Bounded by construction: one feed per cycle, three items per capability, shadow unless ORCH_EVIDENCE_ACQUISITION is set. It never feeds a capability whose switch is default-off or whose blocker is a failure rather than absent evidence — a default-off switch cannot be fed, and feeding a failing one would train on noise. The empty states are distinct, because "nothing to feed" and "blocked from feeding" look identical in a count: `nothing_to_feed — feedable 0 / capped 1 / candidates 0 / fed 0` reports the blocking AND drainable quantities in one line, per the latched-gate rule. Declared as a cadence lane (`tick_phase`), not task-routed. The first registration used a task_type matcher, which made capability_activation_audit classify it task_routed and judge reachability on issue labels that can never invoke a cadence step — it reported caller_exists FAIL and was right to. A matcher describes how a capability is INVOKED, not the work it nominates. Co-Authored-By: Claude Opus 5 <noreply@anthropic.com> (cherry picked from commit e73b47d)

… lane depends on Cherry-picked from 69a1c38 on claude/profile-per-agent (capabilities.py only; that commit's unrelated adapters.py auth-probe change stays on its branch). The preceding commit landed evidence_acquisition.py, which DELEGATES the "is this worth feeding" decision to capabilities.unblock() rather than reimplementing it. That guard was not on main, so the lane arrived here with the exact latched-gate defect its own commit message claims it avoids: guard absent: feedable 2 / no_candidates <- range-lane-rollout, synthesis-promotion guard present: feedable 0 / nothing_to_feed <- matches the ledger's recorded event Both capabilities it wanted to feed are held by a documented default-off switch. Feeding a switched-off capability manufactures work it cannot execute, so no durable reuse is produced, so the evidence debt never falls, so it is fed again next cycle — forever. The drain is blocked by the very switch the feed ignored. The lane's own selftest asserted "default-off switch must not be fed" and PASSED on main without the guard, because its synthetic fixture was excluded by a different unblock() branch. A test passing for the wrong reason is why this needed the live-ledger check, not just a green selftest. Co-Authored-By: Claude Opus 5 <noreply@anthropic.com>

coderabbitai · 2026-08-23T04:22:54Z

Warning

Review limit reached

You’ve reached a temporary PR review limit under our Fair Usage Limits Policy.

Your current included review allowance is based on your included PR review attempts over the past 7 days.

Next review available in: 45 minutes

Limit details: You’ve used the included review currently available. Your 70 included PR review attempts over the past 7 days set your current allowance at 1 review per hour.

Your organization has reached its usage spending cap. Adjust your spending cap in the billing tab.

How can I continue?

Wait for the limit to reset, then comment @coderabbitai review or push new commits to the PR.

An organization admin can change what happens after included review limits in Billing.

How do review limits work?

CodeRabbit enforces per-developer PR review limits within each organization.

For paid Pro and Pro+ reviews, CodeRabbit uses a developer's included PR review attempts over the past 7 days to set the current hourly allowance. At typical activity levels, the full plan allowance applies. Higher sustained activity can lower the allowance until earlier attempts leave the 7-day window.

Please refer docs for additional details.

Review details

⚙️ Run configuration

Configuration used: Organization UI

Review profile: ASSERTIVE

Plan: Pro

Run ID: 1d936ecd-d428-4fea-8abf-1a244bde1825

📥 Commits

Reviewing files that changed from the base of the PR and between 2215f1d and f5c7052.

📒 Files selected for processing (5)

cadence_registry.py
capabilities.py
capability_recurrence_check.py
evidence_acquisition.py
orchestrate.sh

_{Comment @coderabbitai help to get the list of available commands.}

`.verify-floor.json` recorded 366 while main collects 368. The drift came from #34 (evidence-acquisition landed) and #37 (tick capability evidence), each measured against a branch cut before the other merged — the same mistake the file's own note already documents from the 345->353 episode. A floor BELOW reality is the permissive direction this file exists to close: two tests could have stopped being collected and still cleared the check, which is precisely the hole it guards. Measured on the merge result, not on a branch: 368 passed, 0 failed, 0 skipped, 83/83 selftests, 43/43 can-fire, 5/5 gates. No ceiling moved and nothing new is skipped. Co-authored-by: Tim Stranske <tim@stranskemo.com> Co-authored-by: Claude Opus 5 <noreply@anthropic.com>

… silent `.verify-floor.json` has been found BELOW reality four times -- 21 low at the worst, then 8, then 1, then 2 -- and each was hand-raised after the fact by whoever happened to look. The stale number was never the defect. `_floor_problems` fired only on `collected < floor`, so a PR could add tests and never touch the file: silently green, with the floor left permissive by exactly the number added. #34 and #37 each did precisely that, which is what PR #50 then had to clean up by hand. `collected` is now an equality. Too few tests still fails; too many fails as well, printing the two integers to write and telling you to rebase first. That also makes the concurrent case self-enforcing, which is the part discipline could not fix. Once every test-adding branch must edit these same two lines, two concurrent branches CONFLICT IN GIT -- the second cannot merge without rebasing onto the first and re-measuring on the actual merge result. This change demonstrated that on itself: #42 merged underneath it, moved the floor 368 -> 387 and the ceiling 24 -> 26, and the resulting conflict forced the rebase that produced the 387 recorded here. Git's own conflict detection is what enforces "measure on the merge result, not on the branch", the rule the note in that file had to restate three times with nothing behind it. `passed` deliberately stays a MINIMUM on passed+skipped. Only collection is machine-invariant -- a skipped test is still collected, measured the same day at 368 on both CI and this machine with pass/skip splits of 344/24 against 368/0. Making that one strict too would fail a machine for honestly naming a missing prerequisite. Two further fixes in the same change: * `--update-floor` no longer REPLACES the note. It appends. The note is the only record of which prerequisite justifies each ceiling, so overwriting it destroyed the rationale on every use -- the file had to carry a warning about its own tool. * Drift does not block `--update-floor`. The first draft made it a latched gate: a floor behind reality became a problem, and the guard was `not problems`, so the one remedy the error message named was refused for the existence of the very condition it clears. `_blocks_floor_update` lets drift through while real failures still block, sharing `DRIFT_PREFIX` so message and predicate cannot diverge. Selftests cover both directions, skip-invariance, the unset-floor case, the unblock predicate and note preservation, each with a deliberate-break demonstration. Two of those tests were themselves defective and the break demo is what caught it: the latched-gate assert exercised `_blocks_floor_update` in isolation and stayed green when the CALL SITE was reverted (built-but-not-wired), and the source-text assert written to fix that searched for a literal that appears in its own line, so it could never fail. The needle is now built from two fragments and the wiring is asserted at the guard. Verified: 387 passed, 0 failed, 0/26 skipped, 387 collected (floor 387), 83/83 selftests, 43/43 can-fire, 5/5 gates. Ceilings untouched at 26/7/2. black clean at line-length 100; ruff at main's baseline of 6 pre-existing findings, none added. Co-Authored-By: Claude Opus 5 <noreply@anthropic.com>

… silent `.verify-floor.json` has been found BELOW reality four times -- 21 low at the worst, then 8, then 1, then 2 -- and each was hand-raised after the fact by whoever happened to look. The stale number was never the defect. `_floor_problems` fired only on `collected < floor`, so a PR could add tests and never touch the file: silently green, with the floor left permissive by exactly the number added. #34 and #37 each did precisely that, which is what PR #50 then had to clean up by hand. `collected` is now an equality. Too few tests still fails; too many fails as well, printing the two integers to write and telling you to rebase first. That also makes the concurrent case self-enforcing, which is the part discipline could not fix. Once every test-adding branch must edit these same two lines, two concurrent branches CONFLICT IN GIT -- the second cannot merge without rebasing onto the first and re-measuring on the actual merge result. This change demonstrated that on itself TWICE inside an hour: #42 landed underneath it (floor 368 -> 387, ceiling 24 -> 26) and then #59 did (387 -> 391), and each conflict forced a rebase and a fresh measurement. Under the old one-directional rule both would have merged green with a floor below reality. Git's own conflict detection is what enforces "measure on the merge result, not on the branch", the rule the note in that file had to restate three times with nothing behind it. `passed` deliberately stays a MINIMUM on passed+skipped. Only collection is machine-invariant -- a skipped test is still collected, measured the same day at 368 on both CI and this machine with pass/skip splits of 344/24 against 368/0. Making that one strict too would fail a machine for honestly naming a missing prerequisite. Two further fixes in the same change: * `--update-floor` no longer REPLACES the note. It appends. The note is the only record of which prerequisite justifies each ceiling, so overwriting it destroyed the rationale on every use -- the file had to carry a warning about its own tool. * Drift does not block `--update-floor`. The first draft made it a latched gate: a floor behind reality became a problem, and the guard was `not problems`, so the one remedy the error message named was refused for the existence of the very condition it clears. `_blocks_floor_update` lets drift through while real failures still block, sharing `DRIFT_PREFIX` so message and predicate cannot diverge. Selftests cover both directions, skip-invariance, the unset-floor case, the unblock predicate and note preservation, each with a deliberate-break demonstration. Two of those tests were themselves defective and the break demo is what caught it: the latched-gate assert exercised `_blocks_floor_update` in isolation and stayed green when the CALL SITE was reverted (built-but-not-wired), and the source-text assert written to fix that searched for a literal that appears in its own line, so it could never fail. The needle is now built from two fragments and the wiring is asserted at the guard. Verified on the merge result: 391 passed, 0 failed, 0/26 skipped, 391 collected (floor 391), 84/84 selftests, 43/43 can-fire, 5/5 gates. Ceilings untouched at 26/7/2; floor stays at main's 391, since this adds selftest assertions rather than pytest tests. black clean at line-length 100; ruff at main's baseline of 6 pre-existing findings, none added. Co-Authored-By: Claude Opus 5 <noreply@anthropic.com>

… silent `.verify-floor.json` has been found BELOW reality four times -- 21 low at the worst, then 8, then 1, then 2 -- and each was hand-raised after the fact by whoever happened to look. The stale number was never the defect. `_floor_problems` fired only on `collected < floor`, so a PR could add tests and never touch the file: silently green, with the floor left permissive by exactly the number added. #34 and #37 each did precisely that, which is what PR #50 then had to clean up by hand. `collected` is now an equality. Too few tests still fails; too many fails as well, printing the two integers to write and telling you to rebase first. That also makes the concurrent case self-enforcing, which is the part discipline could not fix. Once every test-adding branch must edit these same two lines, two concurrent branches CONFLICT IN GIT -- the second cannot merge without rebasing onto the first and re-measuring on the actual merge result. This change demonstrated that on itself repeatedly: #42 landed underneath it (floor 368 -> 387, ceiling 24 -> 26, verify.py reformatted), then #59 (387 -> 391), then #61, all in one afternoon. Each conflict forced a rebase and a fresh measurement; under the old one-directional rule each would have merged green with a floor below reality. Git's own conflict detection is what enforces "measure on the merge result, not on the branch", the rule the note in that file had to restate three times with nothing behind it. `passed` deliberately stays a MINIMUM on passed+skipped. Only collection is machine-invariant, measured across machines at 391 collected on both, with pass/skip splits of 365/26 on CI against 391/0 locally. Making that one strict too would fail a machine for honestly naming a missing prerequisite. Two further fixes in the same change: * `--update-floor` no longer REPLACES the note. It appends. The note is the only record of which prerequisite justifies each ceiling, so overwriting it destroyed the rationale on every use -- the file had to carry a warning about its own tool. * Drift does not block `--update-floor`. The first draft made it a latched gate: a floor behind reality became a problem, and the guard was `not problems`, so the one remedy the error message named was refused for the existence of the very condition it clears. `_blocks_floor_update` lets drift through while real failures still block, sharing `DRIFT_PREFIX` so message and predicate cannot diverge. Selftests cover both directions, skip-invariance, the unset-floor case, the unblock predicate and note preservation, each with a deliberate-break demonstration. Two of those tests were themselves defective and only the break demo caught it: the latched-gate assert exercised `_blocks_floor_update` in isolation and stayed green when the CALL SITE was reverted (built-but-not-wired), and the source-text assert written to fix that searched for a literal that appears in its own line, so it could never fail. The needle is now built from two fragments and the wiring is asserted at the guard. Verified on the merge result: 391 passed, 0 failed, 0/26 skipped, 391 collected (floor 391), 84/84 selftests, 43/43 can-fire, 5/5 gates. Ceilings untouched at 26/7/2; the floor stays at main's 391, since this adds selftest assertions rather than pytest tests. black clean at line-length 100; ruff at main's baseline of 6 pre-existing findings, none added. Co-Authored-By: Claude Opus 5 <noreply@anthropic.com>

… silent `.verify-floor.json` has been found BELOW reality four times -- 21 low at the worst, then 8, then 1, then 2 -- and each was hand-raised after the fact by whoever happened to look. The stale number was never the defect. `_floor_problems` fired only on `collected < floor`, so a PR could add tests and never touch the file: silently green, with the floor left permissive by exactly the number added. #34 and #37 each did precisely that, which is what PR #50 then had to clean up by hand. `collected` is now an equality. Too few tests still fails; too many fails as well, printing the two integers to write and telling you to rebase first. That also makes the concurrent case self-enforcing, which is the part discipline could not fix. Once every test-adding branch must edit these same two lines, two concurrent branches CONFLICT IN GIT -- the second cannot merge without rebasing onto the first and re-measuring on the actual merge result. This change demonstrated that on itself, at length: six merges landed on main during the two hours it took to write, moving the floor 368 -> 387 -> 391 -> 402 and the ceiling 24 -> 26. Under the old one-directional rule every one of them would have left the floor below reality and merged green. Git's own conflict detection is what enforces "measure on the merge result, not on the branch", the rule the note in that file had to restate three times with nothing behind it. `passed` deliberately stays a MINIMUM on passed+skipped. Only collection is machine-invariant, measured across machines at 391 collected on both, with pass/skip splits of 365/26 on CI against 391/0 locally. Making that one strict too would fail a machine for honestly naming a missing prerequisite. Two further fixes in the same change: * `--update-floor` no longer REPLACES the note. It appends. The note is the only record of which prerequisite justifies each ceiling, so overwriting it destroyed the rationale on every use -- the file had to carry a warning about its own tool. * Drift does not block `--update-floor`. The first draft made it a latched gate: a floor behind reality became a problem, and the guard was `not problems`, so the one remedy the error message named was refused for the existence of the very condition it clears. `_blocks_floor_update` lets drift through while real failures still block, sharing `DRIFT_PREFIX` so message and predicate cannot diverge. Selftests cover both directions, skip-invariance, the unset-floor case, the unblock predicate and note preservation, each with a deliberate-break demonstration. Two of those tests were themselves defective and only the break demo caught it: the latched-gate assert exercised `_blocks_floor_update` in isolation and stayed green when the CALL SITE was reverted (built-but-not-wired), and the source-text assert written to fix that searched for a literal that appears in its own line, so it could never fail. The needle is now built from two fragments and the wiring is asserted at the guard. Verified on the merge result: 402 passed, 0 failed, 0/26 skipped, 402 collected (floor 402), 84/84 selftests, 43/43 can-fire, 5/5 gates. Ceilings untouched at 26/7/2; the floor stays at main's 402, since this adds selftest assertions rather than pytest tests. black clean at line-length 100. Co-Authored-By: Claude Opus 5 <noreply@anthropic.com>

…both stranded post-merge (#84) Both branches were held back from the branch cleanup because their tips carried commits pushed AFTER their PR merged, so "PR merged" did not mean "work landed". Verified per-symbol rather than by diff size — both branches are thousands of lines behind main, so a raw diff conflates stale with unlanded. #42 / commit 4e0d6ae — `adapters.py` catalog resolution. Main has `advertised_models` and NONE of the generalisation around it: `advertised_catalog`, `_advertised_catalog`, `_cached_catalog`, `agy_log_for`, `AGY_LOG_SUFFIX`, `CATALOG_ROUTING_TAGS`, `_catalog_model_id` were all absent. This is learning-loop provenance code (CLAUDE.md 2: "never treat a generic trace model as provider resolution"), and its whole point is that THE CATALOG IS THE AUTHORITY — a label resolves against the ids the CLI actually advertises, with routing TAGS (`auto`, `default`, `cli-default`) refused as non-identities. The commit's own note records that `VENDOR_MODEL_RE` rejects 42 of 204 real cursor ids, so shape-matching an id the CLI itself advertised is both redundant and wrong. Cherry-picked; `adapters.py`, `dispatcher.py` and `ledger_reconcile.py` applied clean. Two conflicts: * `.verify-floor.json` — took main's. A floor is a property of the MERGE RESULT, never carried in from a branch, so it is re-measured below. * `test_feedback_model_provenance.py` — TWO DIFFERENT tests in one region: main's `test_late_sweep_completes_terminal_attempts_never_one_in_flight` (from #63) and the branch's `test_gemini_provenance_reads_the_per_run_log_before_the_conversation_store`. Kept BOTH; they are independent. #34 / commit c1dc9a7 — README item 11 for `evidence_acquisition.py`, which main documented nowhere (zero occurrences). Every factual claim was re-verified against main's code rather than trusted: `capabilities.unblock()` exists; `ORCH_EVIDENCE_ACQUISITION_MAX_FEEDS`/`_MAX_ITEMS` default to 1 and 3; `LIVE_FLAG = "ORCH_EVIDENCE_ACQUISITION"` with SHADOW as the documented default; and the quoted summary line matches the format string verbatim (`feedable {n} / capped {n} / candidates {n} / fed {n}`). It is the drainable-vs-blocking line the latched-gate rule asks for, and it was the only place that reported it. DELIBERATE-BREAK -> REVERT: emptying `CATALOG_ROUTING_TAGS` fires `assert model_id_for_label("cursor", "Auto (default)") is None` in adapters' OWN selftest; reverted clean. Worth recording that `pytest test_feedback_model_provenance.py` did NOT catch that break — the guard is covered by a `--selftest`, not by a test_*.py, which is precisely why `verify.py` is the gate and a pytest subset is not. A redundant pytest test written before checking was dropped. FLOOR 427 -> 428, one new test, note appended not replaced. Verified FRESH-STATE (both ORCH_STATE_DIR and ORCH_LOCAL_RUNTIME at empty dirs, reproducing CI): VERIFIED — 420 passed, 0 failed, 79 selftests, 3/5 gates green, 8 tests + 5 selftests + 2 gates skipped for named prerequisites; 420 + 8 = 428 = floor. ruff + black -l 100 clean. NOT FIXED HERE, and not caused here: on this machine `test_capabilities.py`'s `test_gate_blocks_execution_is_opt_in_and_narrow` and `test_evidence_gate_kind_is_not_blanket_observer` fail on PRISTINE main too — the hourly fleet tick mutated the machine-local ledger and range-lane-rollout now classifies `matched_not_invoked` instead of `deliberately_gated`. Ledger STATE, not code; they skip with a named reason under a fresh ledger, which is what CI uses. Co-authored-by: Tim Stranske <tim@stranskemo.com> Co-authored-by: Claude Opus 5 <noreply@anthropic.com>

Tim Stranske and others added 2 commits August 22, 2026 23:14

stranske merged commit cf901dd into main Aug 23, 2026
2 checks passed

This was referenced Aug 23, 2026

feat(profiles+provenance): a profile per agent, and the provenance chain that makes per-agent identity trustworthy #42

Merged

fix(verify): raise the floor to the merge reality (366 -> 368) #50

Merged

stranske mentioned this pull request Aug 23, 2026

fix(verify): make the collection floor an EQUALITY so drift cannot be silent #56

Merged

This was referenced Aug 23, 2026

feat(ci): give the collection floor an automatic drain on main #83

Merged

fix(salvage): #42's catalog-resolution commit and #34's README item, both stranded post-merge #84

Merged

Navigation Menu

fix(verify): land evidence-acquisition and the unblock() guard it depends on - #34

fix(verify): land evidence-acquisition and the unblock() guard it depends on#34
stranske merged 2 commits into
mainfrom
claude/sad-grothendieck-20324b

stranske commented Aug 23, 2026

Uh oh!

coderabbitai Bot commented Aug 23, 2026

Review limit reached

Uh oh!

Uh oh!

Reviewers

Assignees

Labels

Projects

Milestone

Development

Uh oh!

1 participant

Conversation

stranske commented Aug 23, 2026

The row was not orphaned

The lane has a hard dependency that main did not have

What landed

Notes for the merger

Uh oh!

coderabbitai Bot commented Aug 23, 2026

Review limit reached

Uh oh!

Uh oh!

Reviewers

Assignees

Labels

Projects

Milestone

Development

Uh oh!

1 participant