Repository navigation
review: drop a subject that restates the discussion; stage body-size stats #365
New issue
Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.
By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.
Already on GitHub? Sign in to your account
Changes from 1 commit
0116e7d
36d3da7
fab4730
3a2547d
e746282
542d051
3da9c08
68591d0
File filter
Filter by extension
Conversations
Jump to
Diff view
Diff view
There are no files selected for viewing
| Original file line number | Diff line number | Diff line change |
|---|---|---|
| @@ -0,0 +1,9 @@ | ||
| --- | ||
| "review": minor | ||
| --- | ||
|
|
||
| Stop posting a subject line that restates the discussion, and stage body-size stats with every submission plan. | ||
|
|
||
| The 2026-08-20 by-version audit of webapp reviews found a prose repetition cluster in the v1.11.0-v1.13.0 windows (5 of 29 blind-judged bodies restated one fact two to four times, vs 1 of 60 before), and the root cause is mechanical: `joinProse` concatenates the label contract's `subject` and `discussion` verbatim, producers routinely emit a subject that restates a discussion claim, and the v1.8.0 task-mode removal (#284/#288/#289) deleted the orchestrator rewrite pass that used to absorb the overlap. `joinProse` now drops a subject whose folded tokens are all contained in a single discussion sentence (stopwords ignored on the subject side, light inflection folding on both sides, markdown and trailing punctuation stripped); a subject carrying any token no single sentence holds is kept whole, so the check can only remove a duplicate sentence, never content. `buildClaims`' first-sentence split then recovers the discussion's own opening claim as the subject, so no downstream field goes empty. | ||
|
|
||
| The same audit found the +60% median-body step (557 to 889 chars across webapp's v1.7.0 to v1.11.0 bump) shipped with no changeset naming it, and a by-version audit four versions later was the detection mechanism. `submission-plan.json` now carries `bodyStats` (comment count, median/p90/max/total rendered chars over the final comment bodies, footer rides included, plus the review body's length), echoed in the CLI's run-log summary, so the next render-path regression is visible in the first runs' artifacts. Artifact-only; nothing gates on it. The README's versioning section now requires render-path changesets to state their expected output-shape effect. | ||
| Original file line number | Diff line number | Diff line change |
|---|---|---|
|
|
@@ -87,18 +87,133 @@ export type Candidate = { | |
| authorDispute?: string; | ||
| }; | ||
|
|
||
| /** | ||
| * Fold a prose token toward its stem so an inflection difference does not | ||
| * defeat the restatement check below ("drops" vs "dropped", "cache" vs | ||
| * "caches"). Deliberately crude: strip one of ing/ed/es/s, collapse a | ||
| * doubled final consonant ("dropped" -> "dropp" -> "drop"), then strip a | ||
| * trailing "e" so "caches" -> "cach" meets "cache" -> "cach". Both sides | ||
| * of every comparison fold identically, and a miss is safe — the subject | ||
| * is kept and the body merely stays as long as it is today. | ||
| */ | ||
| const foldToken = (token: string): string => { | ||
|
Contributor
There was a problem hiding this comment. Choose a reason for hiding this commentThe reason will be displayed to describe this comment to others. Learn more. suggestion (non-blocking): ~110 lines of stemming/stopword machinery for a mode whose common case is a verbatim prefix. Three of the four positive tests are near-verbatim restatements; only one needs inflection folding. The repo already has a normalize-and-compare-prefix idiom for precisely "does this field restate that one" in A sketch, not a committable replacement: review detailsfound by first-principles |
||
| if (token.length <= 3) { | ||
| return token; | ||
| } | ||
| let folded = token; | ||
| for (const suffix of ["ing", "ed", "es", "s"]) { | ||
| if (folded.endsWith(suffix) && folded.length - suffix.length >= 3) { | ||
| folded = folded.slice(0, folded.length - suffix.length); | ||
| break; | ||
| } | ||
| } | ||
| if (/([b-df-hj-np-tv-z])\1$/.test(folded)) { | ||
| folded = folded.slice(0, -1); | ||
| } | ||
| return folded.length >= 4 && folded.endsWith("e") | ||
| ? folded.slice(0, -1) | ||
| : folded; | ||
| }; | ||
|
|
||
| /** | ||
| * Function words that carry no claim content; ignored on the SUBJECT side | ||
| * of the restatement check so "turns are dropped" still matches "drops | ||
| * turns" (the sentence has no "are"). Never filtered from the sentence | ||
| * side — there they can only help containment, not hurt it. | ||
| */ | ||
| const STOPWORDS = new Set([ | ||
|
khan-actions-bot marked this conversation as resolved.
Resolved
|
||
| "a", | ||
| "an", | ||
| "the", | ||
| "is", | ||
| "are", | ||
| "was", | ||
| "were", | ||
| "be", | ||
| "been", | ||
| "being", | ||
| "it", | ||
| "its", | ||
| "this", | ||
| "that", | ||
| "these", | ||
| "those", | ||
| "and", | ||
| "or", | ||
| "of", | ||
| "to", | ||
| "in", | ||
| "on", | ||
| "by", | ||
| "for", | ||
| "with", | ||
| "as", | ||
| "at", | ||
| "so", | ||
| ]); | ||
|
|
||
| /** | ||
| * The comparable word tokens of a prose fragment: markdown emphasis | ||
| * stripped, lowercased, internal punctuation kept (`counts.go` is one | ||
| * token) but trailing punctuation shed ("turns." and "turns" are the same | ||
| * word). | ||
| */ | ||
| const proseTokens = (text: string): string[] => | ||
| ( | ||
| text | ||
| .toLowerCase() | ||
| .replace(/[`*_]/g, "") | ||
| .match(/[a-z0-9][a-z0-9./:-]*/g) ?? [] | ||
| ).map((token) => token.replace(/[./:-]+$/, "")); | ||
|
|
||
| /** | ||
| * Whether the subject merely restates one sentence of the discussion: | ||
| * every folded subject token already appears in a single discussion | ||
| * sentence, so prepending the subject adds repetition and no vocabulary. | ||
| * This is the W4-W5 prose failure mode the 2026-08-20 version audit | ||
| * measured (5 of 29 sampled bodies restated one fact two to four times, vs | ||
| * 1 of 60 in W0-W3): producers routinely emit a `subject` that restates a | ||
| * `discussion` claim, and the v1.8.0 task-mode removal deleted the | ||
| * orchestrator rewrite pass that used to absorb the overlap (PRA-46). | ||
| * Per-sentence containment is the conservative direction — a subject | ||
| * summarizing ACROSS sentences (tokens no single sentence holds) is a | ||
| * genuine lede and is kept whole, as is one carrying any token the | ||
| * discussion lacks. | ||
| */ | ||
| export const subjectRestatesDiscussion = ( | ||
|
khan-actions-bot marked this conversation as resolved.
Resolved
|
||
| subject: string, | ||
| discussion: string, | ||
| ): boolean => { | ||
| const subjectTokens = proseTokens(subject) | ||
| .filter((token) => !STOPWORDS.has(token)) | ||
| .map(foldToken); | ||
| if (subjectTokens.length === 0) { | ||
| return false; | ||
| } | ||
| return discussion.split(/(?<=[.!?])\s/).some((sentence) => { | ||
|
khan-actions-bot marked this conversation as resolved.
Resolved
|
||
| const sentenceTokens = new Set(proseTokens(sentence).map(foldToken)); | ||
| return subjectTokens.every((token) => sentenceTokens.has(token)); | ||
|
khan-actions-bot marked this conversation as resolved.
Resolved
|
||
| }); | ||
| }; | ||
|
|
||
| /** | ||
| * Join the label contract's `subject` and `discussion` into one prose block. | ||
| * A subject with no terminal punctuation gets a sentence break, not a bare | ||
| * space (run 29897276810 posted "...memory Both TestExpiration..."); the | ||
| * break also keeps `buildClaims`' first-sentence split recovering the | ||
| * subject. | ||
| * | ||
| * A subject that restates the discussion's opening sentence | ||
|
khan-actions-bot marked this conversation as resolved.
Resolved
|
||
| * ({@link subjectRestatesDiscussion}) is dropped instead of joined: the | ||
| * posted body then opens with the discussion's own first claim, and | ||
| * `buildClaims`' first-sentence split recovers that as the subject, so | ||
| * nothing downstream loses a field — only the duplicate sentence. | ||
| */ | ||
| export const joinProse = (subject: string, discussion: string): string => { | ||
| if (discussion === "") { | ||
| return subject.trim(); | ||
| } | ||
| if (subject === "") { | ||
| if (subject === "" || subjectRestatesDiscussion(subject, discussion)) { | ||
|
khan-actions-bot marked this conversation as resolved.
Resolved
khan-actions-bot marked this conversation as resolved.
Resolved
khan-actions-bot marked this conversation as resolved.
Resolved
|
||
| return discussion.trim(); | ||
| } | ||
| const trimmed = subject.trimEnd(); | ||
|
|
||
Uh oh!
There was an error while loading. Please reload this page.