feat(propensity): add the repair channel — the loop's third action - #65
Conversation
|
Important Review skippedAuto reviews are disabled on base/target branches other than the default branch. Please check the settings in the CodeRabbit UI or the ⚙️ Run configurationConfiguration used: Path: .coderabbit.yaml Review profile: ASSERTIVE Plan: Pro Run ID: You can disable this status message by setting the Use the checkbox below for a quick retry:
Comment |
Workflow source neededPR #65 needs either a linked GitHub issue or one valid non-issue Workflow Source before PR metadata automation can manage it safely. Please do one of:
Once a valid source is present, this warning will not be reposted. |
209e1d5 to
0b9f5c3
Compare
Automated Status SummaryHead SHA: 266f59d
Coverage Overview
Updated automatically; will refresh on subsequent CI/Docker completions. Keepalive checklistScopeNo scope information available Tasks
Acceptance criteria
|
|
Runner dispatch state for autofix on PR #65. Do not edit. |
|
Runner dispatch state for codex on PR #65. Do not edit. |
|
Workflow state fingerprint for Agents Gate Followups. Do not edit. |
|
Workflow state fingerprint for Keepalive Loop Reporter. Do not edit. |
f1cf531 to
1e3e868
Compare
a226820 to
cac4c66
Compare
The loop had exactly two actions, PROMOTE (widen a binding) and DEMOTE (narrow
one), and therefore COULD NOT REPRESENT "this capability is worth having and is
broken." The only available response to a broken capability was to stop offering
it -- which silences the thing that should be fixed and loses a capability worth
keeping.
THE LIVE CASE. `repo-playbook` sits at one useful and one not-useful verdict, and
the Fine-Art-Archive audit documented exactly WHY: its useful content is gated
behind `task_type: implement/testgen/mechanical`, so a `review` consult receives
308 characters, one clause of which is factually wrong -- it tells auditors a
repository's default branch is something it is not. Demotion silences that. A
repair proposal names it, with the words attached.
TWO INPUTS:
1. `not_useful` verdicts WITH THEIR EVIDENCE CARRIED FORWARD. That is the whole
difference between a flag and a repair: "0.5, one bad verdict" is a number,
"308 characters, one clause factually wrong about the default branch" is an
action.
2. Declines whose KIND indicates a DEFECT -- `decline_kind_repairable`:
`wrong_match` (the matcher may be wrong) and `precondition_unmet`. Explicitly
NOT `no_landing_zone`: nobody's fault, the match was correct, the capability
is working; a repair proposal there asserts a defect that does not exist.
`scope_too_small` is also excluded, for an arithmetic reason rather than a
judgement -- its fix is narrowing the DECLARATION, which IS the demotion
path, and one decline must not argue for unbinding and rebuilding at once.
`repairable` IS A SECOND PROPERTY OF THE KIND, declared once beside `demotable`
in `DECLINE_KINDS` and read by one lookup. They are independent questions, and the
pair that proves it is `precondition_unmet`: NOT demotable, IS repairable. Before
this channel existed that kind had no action at all -- 11 on the live ledger,
recorded and inert forever.
REPORT-ONLY, NEVER APPLIED, AND IT QUEUES NOTHING FOR ANYONE (CLAUDE.md §3).
Proposals are a field in a report the cadence step already writes: nothing waits
on a human, nothing expires against a human, no human action can fall behind.
ATTENTION COST: 13 rows in an existing hourly/6-daily report, zero actions
required, expiring on their own with WINDOW_DAYS. 0 minutes/week.
LATCHED-GATE ANSWERS (a proposal set is a gate, so it owes all three in writing;
they are in the module and in ARCHITECTURE.md):
1. WHAT DECREMENTS IT? `record_repair` -- a named mechanism writing a durable
marker with the fix and its artifact, after which a proposal counts only
defect evidence NEWER than that marker. Not "time passes", not "someone
notices". Window expiry is a SECOND drain on the same WINDOW_DAYS constant.
THE FIRST DRAFT HAD NO MARKER: defect evidence stayed in the 90-day window,
so fixing the capability did not clear its proposal for three months. That
is the latch, and asking question 1 -- not testing -- is what caught it.
2. CAN THE DRAIN RUN WHILE THE GATE IS NON-EMPTY? Yes, unconditionally.
`record_repair` requires nothing a standing proposal forbids, and a proposal
is report-only on both sides: it never withholds the capability from
`rank()`, never lowers its propensity, never blocks a consult.
3. SAME WINDOW BOTH WAYS? Yes -- WINDOW_DAYS, the one constant `usefulness()`,
`propensity()` and `surface_decline_counts()` already share, bounds the
defect evidence counted AND the repair markers that clear it.
Runtime rule: every proposal carries `defect_evidence_total` (measuring),
`defect_evidence_since_repair` (blocking) and `repairs_recorded` (drainable), and
`report()` carries `repairs_recorded` EVEN WHEN THE PROPOSAL LIST IS EMPTY --
an empty list cannot say whether anything is accumulating.
And the tie-break FAILS TOWARD MOTION: ledger timestamps are second-granular, so
freshness compares `>=`, not `>`. A defect recorded in the same second as a repair
is unorderable and must RE-OPEN the proposal (one report line) rather than vanish
(the finding). Asserted directly, and the strict-`>` break fails that assertion by
name.
WHAT THIS DELIBERATELY DOES NOT DO. It never applies a repair, never opens an
issue, never queues anything for the owner. It does not touch any posterior --
asserted. `worth_having` is REPORTED, never a filter: filtering on it would hide
the broken-and-unwanted population, which is the demotion path's business, and
both readings must stay visible.
Seven break->revert demonstrations, each confirmed to discriminate on a named
assertion: making `no_landing_zone` repairable (a correct match gets proposed),
making `precondition_unmet` non-repairable (the kind with no other channel goes
silent again), dropping the marker from the freshness test (the latch), flipping
the tie-break to a strict `>` (a same-second defect vanishes), dropping the
artifact guard on `record_repair` (a claimed repair clears a proposal), dropping
the carried-forward evidence (the proposal becomes a flag), and using
`worth_having` as a filter.
THE TEST SEAM IS ON `record_usefulness`, NOT `record_decline`, and CI is why. The
selftest needs to place events in time, because ledger timestamps are
second-granular and a chronology compressed into one second cannot distinguish
"the action drained it" from "the tie-break went this way". The seam was added to
`record_decline` first, and `mcp_server`'s contract selftest failed on CI: it
asserts every keyword-only parameter of `record_decline` is advertised by the
`capability_decline` MCP tool, and an MCP caller must never be able to backdate a
decline. The seam moved to `record_usefulness` (not MCP-exposed) and the
chronology now uses timestamped `not_useful` verdicts, which are the same class of
defect evidence.
Live output: 13 proposals, 5 worth having and broken, `repo-playbook` among them
carrying its own 308-character evidence.
CLI: `repair` (report) and `record-repair --capability X --fix ... --artifact ...`
(the drain). `orchestrate.sh` prints the counts beside the provenance mix.
No new pytest tests, so `.verify-floor.json` is unchanged.
Co-Authored-By: Claude Opus 5 <noreply@anthropic.com>
cac4c66 to
8292d4c
Compare
…e-measured #60 landed the `.github/workflows/autofix-versions.env` pin file that every Python check on this PR was dying on, at the shared `Install dependencies` step before any tool ran. It also brings this repo's first `ruff.toml` and `mypy.ini`, so this change is linted for real for the first time rather than under Ruff's 88-column defaults. Checked against the config that actually landed, not the branch preview: ruff check ledger_reconcile.py test_feedback_model_provenance.py -> passed ruff check . (the Gate's own command) -> passed black --line-length 100 --check on both files -> unchanged `.verify-floor.json` conflicted, as it has every round. Resolved as the UNION: main's note is kept whole (it carries #60's +11 tests and the #64/#65 rationale) with this branch's entry appended, and the count RE-MEASURED on the merge result rather than either side's number -- `pytest --collect-only` reports 403, which is main's 402 plus this branch's one test. No ceiling moved; nothing new is skipped. Co-Authored-By: Claude Opus 5 <noreply@anthropic.com>
THE DEFECT. main has no branch protection, and `gh pr checks` lists what DID report -- so a check that never started is not red, it is missing, and a PR with no Gate reads exactly like a PR whose Gate passed. Silence indistinguishable from success: this repo's founding defect, twice over. 2026-08-23, five python-ci jobs died at a shared install step and #61/#64/#65 merged with all five red. 2026-08-24, #90's Gate run was held at `action_required` with ZERO jobs and merged with no lint, no format and no typecheck, landing six F821s found only because somebody ran ruff by hand. THE CAUSE WILL BE DIFFERENT NEXT TIME, so nothing here models holds. A check can vanish to a hold, a cancellation, a deleted or renamed workflow, a rate limit, a mistaken path filter or a GitHub incident. All present identically to whoever is merging. scripts/check_checks_reported.py asks only: did every check that NORMALLY reports also report here. TWO SIMPLER DESIGNS WERE TRIED AND REJECTED BY REAL DATA, both recorded in the file because the next person will reach for them: * one reference PR -- failed on the actual incident. #90 had no Gate, and the newest merged PR (#93) had no Gate checks either, so #90 was declared healthy. The hold had already swallowed the yardstick. * the union across recent merges -- caught #90 (21 absent) but reported 25-26 absences on entirely healthy PRs, sweeping in event-driven checks. A test that cries wolf 25 times gets waived. Frequency (>=75% of 12 merged PRs) discriminates: #90 exit 1 with 10 absent, be a second copy of the CI topology. NOT BRANCH PROTECTION, deliberately. A required check that is HELD never reports, so the PR could never merge -- the clear path blocked by the very thing the gate measures. On a solo-maintained repo "unverified but movable" beats "permanently stuck". pr-00-gate.yml also gains `push: [main]`. A held PR run cannot be fixed from inside CI, but the silence AFTER the merge can: #90's F821s would have gone red on main within one run instead of never. Fail toward noise. docs/ABSENT_CHECK_LANE_WIRING.md carries the one step this repo cannot land -- the closer's pre-merge call and the prerun `--sweep` line, since the lane TOMLs live outside any repository. Same shape as docs/MIRROR_SYNC_PATCH.md. Already found a live one: --sweep flags open PR #91 with 10 absent checks, which I would have merged on a green-looking list. Verified: 446 passed, 0 failed, 0 skipped, floor 446, 85/85 selftests, 5 of 5 gates, mypy Success, ruff and black clean. Break -> revert: removing the max(2, ...) threshold floor fails test_the_threshold_never_falls_to_one; reverted byte-identical. Co-Authored-By: Claude Opus 5 <noreply@anthropic.com>
* feat(ci): an absent check is a red, whatever made it absent THE DEFECT. main has no branch protection, and `gh pr checks` lists what DID report -- so a check that never started is not red, it is missing, and a PR with no Gate reads exactly like a PR whose Gate passed. Silence indistinguishable from success: this repo's founding defect, twice over. 2026-08-23, five python-ci jobs died at a shared install step and #61/#64/#65 merged with all five red. 2026-08-24, #90's Gate run was held at `action_required` with ZERO jobs and merged with no lint, no format and no typecheck, landing six F821s found only because somebody ran ruff by hand. THE CAUSE WILL BE DIFFERENT NEXT TIME, so nothing here models holds. A check can vanish to a hold, a cancellation, a deleted or renamed workflow, a rate limit, a mistaken path filter or a GitHub incident. All present identically to whoever is merging. scripts/check_checks_reported.py asks only: did every check that NORMALLY reports also report here. TWO SIMPLER DESIGNS WERE TRIED AND REJECTED BY REAL DATA, both recorded in the file because the next person will reach for them: * one reference PR -- failed on the actual incident. #90 had no Gate, and the newest merged PR (#93) had no Gate checks either, so #90 was declared healthy. The hold had already swallowed the yardstick. * the union across recent merges -- caught #90 (21 absent) but reported 25-26 absences on entirely healthy PRs, sweeping in event-driven checks. A test that cries wolf 25 times gets waived. Frequency (>=75% of 12 merged PRs) discriminates: #90 exit 1 with 10 absent, be a second copy of the CI topology. NOT BRANCH PROTECTION, deliberately. A required check that is HELD never reports, so the PR could never merge -- the clear path blocked by the very thing the gate measures. On a solo-maintained repo "unverified but movable" beats "permanently stuck". pr-00-gate.yml also gains `push: [main]`. A held PR run cannot be fixed from inside CI, but the silence AFTER the merge can: #90's F821s would have gone red on main within one run instead of never. Fail toward noise. docs/ABSENT_CHECK_LANE_WIRING.md carries the one step this repo cannot land -- the closer's pre-merge call and the prerun `--sweep` line, since the lane TOMLs live outside any repository. Same shape as docs/MIRROR_SYNC_PATCH.md. Already found a live one: --sweep flags open PR #91 with 10 absent checks, which I would have merged on a green-looking list. Verified: 446 passed, 0 failed, 0 skipped, floor 446, 85/85 selftests, 5 of 5 gates, mypy Success, ruff and black clean. Break -> revert: removing the max(2, ...) threshold floor fails test_the_threshold_never_falls_to_one; reverted byte-identical. Co-Authored-By: Claude Opus 5 <noreply@anthropic.com> * fix(ci): degrade the absent-check sweep gracefully on a rate limit The sweep runs every lane round from handoff-prerun.sh. Dumping GitHub's full rate-limit paragraph hourly would train the reader to skip the whole section, which is how a health report stops being read. One line instead, and it says UNKNOWN rather than implying clean -- a reporter that cannot report must not read as a clean bill of health. Found by testing the prerun block while genuinely rate-limited, which is the degraded path I would otherwise have had to simulate. Co-Authored-By: Claude Opus 5 <noreply@anthropic.com> * fix(ci): ratchet the expected-check set so a sustained outage cannot erode it FOUND BY DOGFOODING, not by review. Running --pr 91 before merging it, the check PASSED -- and should not have. While pr-00-gate.yml sat held, every newly merged PR merged WITHOUT the Gate, so after twelve such merges the Gate's checks no longer appeared on 75% of the reference window, stopped counting as "normally reporting", and their absence stopped being flagged. The expected set fell 23 -> 14 names and #91 was pronounced clean by the tool written to catch exactly that. A SUSTAINED outage is the case that matters most, and it was the one case the frequency rule could not see. The erosion test already in this file covered only PRs that reported NOTHING; a PR reporting some checks but not the Gate's slid straight through. config/expected-checks.json is now the high-water mark, seeded from PRs #87/#89 whose Gate demonstrably ran (33 names, 8 of them python-ci). A name that has ever been expected stays expected until somebody DELETES ITS LINE -- a visible act in a diff. Same shape as pyproject.toml's mypy exempt ratchet, and the same reason: an automatic downward move is indistinguishable from the defect. --update-ratchet raises it and never lowers it. With the ratchet, PR 91's head correctly reports 19 absences including the whole python-ci set. PR 89 (Gate ran) reports one, "guard", which is a TRUE positive: agents-guard.yml was already held by then. Break -> revert: returning the observed set instead of its union with ratchet_names() fails test_the_ratchet_is_wired_into_the_expected_set; byte-identical after. 448 passed, floor 448, 85/85 selftests, 5 of 5 gates, mypy Success, ruff and black clean. Co-Authored-By: Claude Opus 5 <noreply@anthropic.com> --------- Co-authored-by: Tim Stranske <tim@stranskemo.com> Co-authored-by: Claude Opus 5 <noreply@anthropic.com>
Stacked on #64 (base is
claude/propensity-defect-finds; retarget as #61 and #64 merge).Why
The loop has exactly two actions — promote (widen a binding) and demote (narrow one) — and
therefore cannot represent "this capability is worth having and is broken." The only available
response to a broken capability was to stop offering it, which silences the thing that should be
fixed and loses a capability worth keeping.
The live case.
repo-playbooksits at 0.500 with one useful and one not-useful verdict, and theFine-Art-Archive audit documented why: its useful content is gated behind
task_type: implement/testgen/mechanical, so areviewconsult receives 308 characters, oneclause of which is factually wrong — it tells auditors a repository's default branch is something
it is not. Demotion silences that. A repair proposal names it, with the words attached.
Two inputs
not_usefulverdicts, with their evidence carried forward. That is the whole differencebetween a flag and a repair: "0.5, one bad verdict" is a number; "308 characters, one clause
factually wrong about the default branch" is an action.
decline_kind_repairable:wrong_match(the matchermay be wrong) and
precondition_unmet. Explicitly notno_landing_zone: nobody's fault, thematch was correct, the capability is working, and proposing a repair there asserts a defect that
does not exist.
scope_too_smallis also excluded — for an arithmetic reason rather than ajudgement: its fix is narrowing the declaration, which is the demotion path, and one decline
must not argue for unbinding and rebuilding at once.
repairableis a second property of the kind, declared once besidedemotableand read by onelookup. They are independent questions, and the pair that proves it is
precondition_unmet: notdemotable, is repairable. Before this channel existed that kind had no action at all — 11 of them
on the live ledger, recorded and inert forever.
Latched-gate check, in writing
record_repair— a named mechanism writing a durable marker with the fixand its artifact, after which a proposal counts only defect evidence newer than that marker.
Not "time passes", not "someone notices". Window expiry is a second drain on the same
WINDOW_DAYSconstant. The first draft had no marker at all: defect evidence stayed in the90-day window, so fixing the capability did not clear its proposal for three months. That is the
latch, and asking question 1 — not testing — is what caught it.
record_repairrequires nothing a standing proposal forbids, and a proposal is report-only on both sides: it
never withholds the capability from
rank(), never lowers its propensity, never blocks aconsult. The capability keeps being offered and keeps earning verdicts while the proposal stands.
WINDOW_DAYS,the one constant
usefulness(),propensity()andsurface_decline_counts()already share,bounds both the defect evidence counted and the repair markers that clear it. One name; a matching
pair of literals would drift.
Runtime rule: every proposal carries
defect_evidence_total(measuring),defect_evidence_since_repair(blocking) andrepairs_recorded(drainable), andreport()carriesrepairs_recordedeven when the proposal list is empty — an empty list cannot say whetheranything is accumulating.
And the tie-break fails toward motion: ledger timestamps are second-granular, so freshness
compares
>=, not>. A defect recorded in the same second as a repair is unorderable and mustre-open the proposal (one report line) rather than vanish (the finding). Asserted directly, on a
capability created for that assertion alone.
Human involvement
Zero. Report-only, never applied, and it queues nothing for anyone (
CLAUDE.md§3). Proposalsare a field in a report the cadence step already writes: nothing waits on a human, nothing expires
against a human, no human action can fall behind. The live ledger produces 13 rows requiring
zero actions, expiring on their own with
WINDOW_DAYS— 0 minutes/week. Every proposalcarries
auto_applied: falseandqueued_for_owner: false, asserted.What this deliberately does NOT do
propose_repair.worth_havingis reported, never a filter. Filtering on it would hide thebroken-and-unwanted population, which is the demotion path's business; both readings must stay
visible because they imply different work.
Live output
Break → revert (six, each confirmed to discriminate on a named assertion)
no_landing_zonemade repairable"correct-match" not in props— a correct match gets proposedprecondition_unmetmade non-repairable"precondition-only" in props— the kind with no other channel goes silent again"worth-fixing" not in drained— the latchrecord_repairartifact guard droppedfactually wrongevidence assertion — the proposal becomes a flagworth_havingused as a filter"precondition-only" in propsVerification
python3 capability_propensity.py --selftestpasses (seven selftests, including the new_selftest_repair).python3 verify.pyre-run on the stacked tree;bash -n orchestrate.shclean.No new pytest tests, so
.verify-floor.jsonis unchanged.Docs updated in the same change per the
ARCHITECTURE.mdcontract, plusREADME.mdand theorchestrate.shcadence step (which now prints the provenance mix and the repair/find counts).Dedup finding recorded in the
capability-propensityledgernotes.🤖 Generated with Claude Code