review: design notes for family suppression and a first-review volume cap - #343
review: design notes for family suppression and a first-review volume cap#343jwbron wants to merge 1 commit into
Conversation
…on and a first-review volume cap
🦋 Changeset detectedLatest commit: 2931b33 The changes in this PR will be included in the next version bump. This PR includes changesets to release 0 packagesWhen changesets are added to this PR, you'll see the packages that this PR includes changesets for and the associated semver types Not sure what this means? Click here to learn what changesets are. Click here if you're a maintainer who wants to add another changeset to this PR |
Review GuidanceCommon patterns2 files: New design-note markdown docs following the same structural template: a - (no prior file)
+ # Design note: <topic>
+
+ Status: design only. ...
+
+ ## The measured problem
+ ...
+ ## [Invariants / Fail-closed rules that carry over unchanged]
+ ...
+ ## [Eval plan / Measurement before default] |
There was a problem hiding this comment.
Approved — no blocking issues found.
Note: test-adequacy not assessed this run (test-adequacy output unavailable).
Note: completeness not assessed this run (completeness output unavailable).
Note: holistic not assessed this run (holistic output unavailable).
Note: first-principles not assessed this run (first-principles output unavailable).
Note: conventions not assessed this run (conventions output unavailable).
Note: documentation not assessed this run (documentation output unavailable).
What
Two design notes, docs only, in a new
workflows/review/design/:family-suppression.md: mechanism-level cross-file suppression, the follow-up review: human-resolved bot threads become an adjudicated suppression corpus #332 names. Records the measured problem (webapp#41290 carries five bot threads on one mechanism, the tmv2 CEDAR side effect on flagged turns, acrosstmv2.goandSPEC.md; one variant was downvoted with reasonduplicate), why the review: human-resolved bot threads become an adjudicated suppression corpus #332 corpus misses it (the matcher keys on path plus the similarity floors, so a cross-file variant fails before similarity is computed), three candidate designs (a reconciler judgment pass, cross-file similarity without the path key, finder-minted mechanism fingerprints) with the false-suppression risk each carries, the fail-closed rules that carry over unchanged (blocking never suppressed, human adjudication required, fail toward posting), and the eval plan: corpus cases from the 41290 family with hard negatives, precision gates and recall ranks, before any code.volume-cap.md: a roster-wide non-blocking cap for first reviews. Records the measured problem (webapp#41440: 15 inline comments in one first review, with overlap clusters the path-keyed clusterer cannot merge), the prior art to generalize (the documentation reviewer's ranked caps, review: a blocking-only re-review modifier, so repeat reviews surface only blockers #329's collapse rendering, review: damp the recap's repeat non-blocking entries to label + link #334's damping), the open design questions (ranking signal, cap value, collapse versus drop, interaction with clustering), the invariants (verdict computed from all validated claims; caps only move findings off the inline surface; the review body discloses how many findings the cap collapsed; the dispatch gate extends), and the measurement bar: priced via the live A/B, shipped opt-in viaROUTING.Why
Both are follow-ups the post-deploy webapp data (2026-08-11 through 2026-08-12) shows are the remaining gaps after #312/#332/#333/#334: cross-file re-derivation and first-review volume. Writing the plan down before code is the repo's pattern (
eval/live-ab-plan.md); both notes are explicit that nothing lands before the eval work they describe.Tests
Docs plus an empty changeset (the
live-ab-plan.mdprecedent).vitest run workflows/review(1326 passed),pnpm lint, andpnpm typecheckall green.