Skip to content

docs(GLOSSARY): Candidate-count Goodhart entry + batched minimal tick row (durable-home for already-named rule) #723

New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

Sign up for GitHub

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Merged
AceHack merged 2 commits into from
Apr 30, 2026
Merged

docs(GLOSSARY): Candidate-count Goodhart entry + batched minimal tick row (durable-home for already-named rule) #723

Changes from all commits
Commits
Show all changes
2 commits
Select commit Hold shift + click to select a range
496ab1d
docs(GLOSSARY): land Candidate-count Goodhart entry (already-named rule)
AceHack Apr 30, 2026
75c97db
docs(GLOSSARY): fix Markdown hyphen-wrap rendering on Candidate-count…
AceHack Apr 30, 2026
File filter

Filter by extension

Filter by extension
Conversations
Failed to load comments.
Loading
Jump to
Jump to file
Failed to load files.
Loading
Diff view
Diff view
27 changes: 27 additions & 0 deletions docs/GLOSSARY.md
View file
Open in desktop
Original file line number Diff line number Diff line change
Expand Up @@ -856,6 +856,33 @@ Each entry names the authoritative source of the
definition — this glossary's job is pointer-plus-gist, not
canonical definition.

### Candidate-count Goodhart

**Plain:** Raw search hits are not violation counts. When
auditing for a forbidden pattern, count matches to *find* work,
but *classify* the context of each match to *decide* whether
work is needed. A scanner that reports "12 references to X" has
located 12 candidates; only context classification
(rule-definition / sample / live-code / disclosure / etc.)
determines
Comment on lines +865 to +867

Copilot AI Apr 30, 2026

Copy link

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

The hard-wrapped paragraph leaves “determines” on its own line, which reads like an accidental wrap and makes the entry harder to scan. Reflow this sentence so the line breaks don’t strand a single word.

Suggested change
located 12 candidates; only context classification
(rule-definition / sample / live-code / disclosure / etc.)
determines
located 12 candidates; only context classification determines

Copilot uses AI. Check for mistakes.
how many — if any — actually need rewriting. Treating the
candidate count as the violation count is a Goodhart-class
failure: it optimizes for the visible metric (count) at the
expense of the intended target (real risk).
**Technical:** A specialization of Goodhart's Law applied to
factory audit patterns. Codified after a 2026-04-28
ServiceTitan-name audit found 12 raw matches → 0 active
rewrites required after context classification (KEEP-NAME for
factory-funding-chain disclosure / HISTORICAL-POINTER for
research history / GENERICIZE for reusable code samples). The
canonical operational rule: *"Count matches to find work.
Classify context to decide work."* Fifth member of the
Goodhart catch family in this factory (Catches #1-4 are
substrate-IS-amortized-precision, commit-count vs tree-numstat,
sample-classification ≠ clearance, tree-diff vs content-loss
surface). Authoritative source:
Comment on lines +879 to +883

Copilot AI Apr 30, 2026

Copy link

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

This entry introduces “Goodhart catch family” and references Catches #1–4, but the shorthand labels used here don’t appear elsewhere in the glossary and aren’t linked to their authoritative sources. To keep the glossary navigable, consider either (a) adding explicit pointers for catches #1–4 (memory files or glossary anchors) or (b) removing the numbered list and keeping the Candidate-count Goodhart rule self-contained.

Suggested change
Classify context to decide work."* Fifth member of the
Goodhart catch family in this factory (Catches #1-4 are
substrate-IS-amortized-precision, commit-count vs tree-numstat,
sample-classification ≠ clearance, tree-diff vs content-loss
surface). Authoritative source:
Classify context to decide work."* Part of the factory's
broader Goodhart-pattern guidance: optimizing for the raw hit
count is a measurement trap unless context classification
decides whether any rewrite is actually required. Authoritative
source:

Copilot uses AI. Check for mistakes.
`memory/feedback_candidate_count_goodhart_raw_hits_are_not_violations_aaron_amara_2026_04_28.md`.
Comment on lines +883 to +884

Copilot AI Apr 30, 2026

Copy link

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

docs/GLOSSARY.md is a current-state surface that should avoid personal-name attribution/mentions (per .github/copilot-instructions.md:305-346). This entry’s “Authoritative source” path includes personal names in the filename (..._aaron_amara_...). Consider adding a name-free alias/redirect memory file (or another name-free canonical pointer) and citing that path here instead.

Suggested change
surface). Authoritative source:
`memory/feedback_candidate_count_goodhart_raw_hits_are_not_violations_aaron_amara_2026_04_28.md`.
surface). Authoritative source: the 2026-04-28 memory entry
on candidate-count Goodhart classification.

Copilot uses AI. Check for mistakes.

### Harmonious Division

**Plain:** The maintainer's name for the meta-algorithm that
Expand Down
Loading