feat(propensity): make a reasoned DECLINE learnable — a third state, with a kind - #45
Conversation
|
Warning Review limit reachedYou’ve reached a temporary PR review limit under our Fair Usage Limits Policy. Next review available in: 46 minutes Limit details: You’ve used the included review currently available. Your 70 included PR review attempts over the past 7 days set your current allowance at 1 review per hour. Your organization has reached its usage spending cap. Adjust your spending cap in the billing tab. How can I continue?Wait for the limit to reset, then comment An organization admin can change what happens after included review limits in Billing. How do review limits work?CodeRabbit enforces per-developer PR review limits within each organization. For paid Pro and Pro+ reviews, CodeRabbit uses a developer's included PR review attempts over the past 7 days to set the current hourly allowance. At typical activity levels, the full plan allowance applies. Higher sustained activity can lower the allowance until earlier attempts leave the 7-day window. Please refer docs for additional details. Review details⚙️ Run configurationConfiguration used: Path: .coderabbit.yaml Review profile: ASSERTIVE Plan: Pro Run ID: 📒 Files selected for processing (5)
Comment |
… with a kind Two independent audit rounds on 2026-08-23 reached the same conclusion: propensity carried information exactly once, because most candidates sat at the uninformative prior. The missing input was not more consults but DECLINES -- a capability declined on repo-specific grounds looked IDENTICAL in the ledger to one nobody ever considered (`trig 0, use 0, no 0`). Four of nine decisions in one round and 21 of the offers in another were reasoned rejections and none was learnable. A DECLINE IS A THIRD STATE, NOT A NEGATIVE OUTCOME, and the separation is structural rather than conventional. A decline means the capability did NOT run, so recording it as an `outcome` would bucket it into `not_useful` -- asserting we tried it and it did not help, about something that never executed, corrupting the one signal declines exist to sharpen. So a decline rides on a `match` event (it genuinely WAS offered, which is the only claim the event type makes) tagged `source=capability_decline`, and `usefulness()` reads `outcome` events only. There is no code path from a decline to the posterior. No new event type, no new store: `record_promotion` already carries a non-match fact on `match` distinguished by `metadata.source`. `triggered` / `declined` / `not_triggered_silently` now partition the candidate set, so "rejected on stated grounds", "offered and ignored" and "never considered" are three different findings. `propensity()` reports the decline count BESIDE the posterior -- "prior, no evidence" and "prior, no evidence, four reasoned rejections" were previously identical readings. AND A DECLINE HAS A KIND, because one undifferentiated count licenses the wrong correction. A third audit round declined 25 offers across six classes with opposite fixes: `testgen-lane` matched CORRECTLY three times and was structurally impossible (a read-only audit has no commit target) -- fix: nothing; `offload` was declined at nine surfaces because it is declared surface-wide and a one-subsystem audit has nothing big enough to hand off -- fix: a precondition or a narrower declaration; and `frontend-verifier`, declined on two frontend-less repos, then produced the second-strongest finding of an audit on a repo that DOES have a display surface -- fix: evaluate the condition, do not weaken the binding. `demotable` is therefore a property of the KIND (`DECLINE_KINDS`), declared once: only `wrong_match` and `scope_too_small` can propose a demotion. An unknown kind is refused rather than coerced. Demotion consumes declines as a second, much lower-floor rule, and the two rules read DISJOINT populations: `never_triggered` counts only offers where nothing was said. The first draft counted every offer, so eight honest `no_landing_zone` declines tripped the silent-non-use rule and demoted a correct match through the back door -- the exact wrong correction the taxonomy exists to prevent, arriving via the other rule. Attribution is now on the event: the advisor records `surface` on each `match`. It recorded only `skill`, and the CLI has no `--skill` flag, so every `--surface` consult wrote `skill: null` and the entire control arm of those audit rounds was unattributable to the surface that produced it. Reachable from bash (`decline --capability --experiment --reason --kind --surface`) and from MCP (`capability_decline`), because the surfaces that make these decisions are skills and lane automations, not Python. The MCP guard is AST-based: a substring check over the handler body was written first and did NOT discriminate -- deleting `surface=` from the call left the word in the response dict and the break stayed green. Break->revert demonstrations, all confirmed to fail without the fix: decline-as-outcome moves the posterior; a blank reason is accepted; demotion ignores declines; a triggered capability is still demoted; surface attribution dropped; the partition collapses; every kind treated as demotable; the silent rule counting declines; an unknown kind coerced; `no_landing_zone` and `precondition_unmet` declared demotable; `surface` advertised but not forwarded; `reason` dropped from the schema; `capability_id` hardcoded. Pre-existing on this machine and unrelated: 3 pytest failures for the `evidence-acquisition` ledger row (machine-local, 0 references in committed capabilities.py), confirmed identical with the change stashed. Co-Authored-By: Claude Opus 5 <noreply@anthropic.com>
…inding `detect` enumerated `SURFACE_RECORD_GLOBS | SURFACE_BINDINGS` — the DECLARED keys. But `repo-audit:dimension-1` has no table entry of its own; it inherits `offload` surface-wide. So when three independent 2026-08-23 audit rounds each declined `offload` at that surface with a stated reason, all three were written to the ledger and none was ever read. A drain that cannot see a surface cannot drain it, and the surfaces most likely to be OVER-BOUND are exactly the ones that only inherit. `observed_surfaces()` derives the set from the trials themselves rather than adding a second list, which would drift from the surfaces that exist and would still omit the inheriting ones. The first version of the selftest asserted `observed_surfaces()` and stayed GREEN when `detect()` was reverted to the declared-keys-only enumeration — a test of the helper instead of the answer, which is this project's most-repeated testing mistake. It now asserts what the CALLER receives, and the break fails as required. Effect on the live ledger, report-only as always: demotion proposals go 2 -> 3, the new one being `repo-audit:dimension-1 -> offload`, declined in 3 of 3 offers across three audits, all three attributable to the binding. Co-Authored-By: Claude Opus 5 <noreply@anthropic.com>
806a2fd to
6379950
Compare
The finding this closes
Three independent audit rounds on 2026-08-23 (
Workflows,Trend_Model_Project,Fine-Art-Archive) reached the same conclusion: propensity carried information exactly once, because most candidates sat at the uninformative prior. The missing input was not more consults — it was declines. A capability declined on repo-specific grounds looked byte-identical in the ledger to one nobody ever considered (trig 0, use 0, no 0).A decline is a THIRD state, not a negative outcome
This is the whole difficulty. A decline means the capability did not run, so recording it as an
outcomewould bucket it intonot_useful— asserting we tried it and it did not help, about something that never executed, corrupting the one signal declines exist to sharpen.The separation is structural, not conventional: a decline rides on a
matchevent (it genuinely was offered — the only claim the event type makes) taggedsource=capability_decline, andusefulness()readsoutcomeevents only. There is no code path from a decline to the posterior. No new event type and no new store:record_promotionalready carries a non-match fact onmatchdistinguished bymetadata.source.triggered/declined/not_triggered_silentlynow partition the candidate set, so "rejected on stated grounds", "offered and ignored" and "never considered" are three different findings.propensity()reports the decline count beside the posterior — "prior, no evidence" and "prior, no evidence, four reasoned rejections" were previously identical readings.And a decline has a KIND
One undifferentiated count licenses the wrong correction. The third round declined 25 offers across six classes with opposite fixes:
testgen-lanematched correctly 3× in a read-only audit with no commit targetno_landing_zoneoffloaddeclined at 9 surfaces — declared surface-wide, one-subsystem auditscope_too_smallfrontend-verifierdeclined on two frontend-less repos, then produced the second-strongest finding of an audit on a repo that does have a display surfaceprecondition_unmetwrong_matchgated_offdeferreddemotableis a property of the kind, declared once inDECLINE_KINDS. An unknown kind is refused rather than coerced — a typo silently becomingunspecifiedwould discard the classification the caller believed it made.Demotion consumes declines as a second, much lower-floor rule, and the two rules read disjoint populations:
never_triggeredcounts only offers where nothing was said. The first draft counted every offer, so eight honestno_landing_zonedeclines tripped the silent-non-use rule and demoted a correct match through the back door — the exact wrong correction the taxonomy exists to prevent, arriving via the other rule.Two attribution defects found on the way
skillon eachmatch, and the CLI has no--skillflag — so every--surfaceconsult wroteskill: nulland the entire control arm of all three audit rounds was unattributable to the surface that produced it.surfaceis now on the event.detect()enumerated only the declared surface keys.repo-audit:dimension-1has no table entry — it inheritsoffloadsurface-wide — so three audits decliningoffloadthere were recorded and never read. A drain that cannot see a surface cannot drain it, and the surfaces most likely to be over-bound are exactly the ones that only inherit.Reachable from the surfaces that actually decide
CLI
decline --capability --experiment --reason --kind --surface, and MCPcapability_decline— the callers are skills and bash automations, not Python. The MCP guard is AST-based: a substring check over the handler body was written first and did not discriminate (deletingsurface=from the call left the word in the response dict and the break stayed green).Break→revert, all confirmed to fail without the fix
decline-as-outcome moves the posterior · a blank reason accepted · demotion ignores declines · a triggered capability still demoted · surface attribution dropped · the partition collapses · every kind treated as demotable · the silent rule counting declines · an unknown kind coerced ·
no_landing_zonedemotable ·precondition_unmetdemotable ·surfaceadvertised but not forwarded ·reasondropped from the schema ·capability_idhardcoded ·detect()enumerating declared keys only.Two of these did not discriminate on the first attempt and were rewritten; both are noted in the code where they live.
Local verdict
python3 verify.py: 365 passed, 82/82 selftests ran, 0 skipped, 4 of 5 gates ok. The 3 pytest failures and the 2 gate failures are pre-existing and machine-local — all five are theevidence-acquisitionledger row (0 references in committedcapabilities.py; another session's uncommitted module). Confirmed identical with the change stashed. CI is the verdict.Contract docs updated (
ARCHITECTURE.md,README.md). No diagram change: no stage, component, rail/role classification or role-registry entry moved.