Repository navigation
Record equivalent mutants by what they sit inside, and catch drift on every commit #2037
New issue
Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.
By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.
Already on GitHub? Sign in to your account
Changes from 13 commits
768ec33
7e24f3a
ba9e5d3
36509a2
3397c3c
91492fa
2b01588
0b4616c
754f365
583ed06
37f70db
7bbd57f
a2ac3fd
f132c51
ba55141
74f7940
a82c6a1
368b203
cb189da
17c85b5
478e085
1d6ac5c
57a0b86
3b6977a
c6276ce
a1d33ae
b4b0b63
9184de7
9ea6e2a
c37fae9
50ab94b
de83423
28749a4
c0d11af
227c086
233d93d
a9f9fbd
f1182e6
File filter
Filter by extension
Conversations
Jump to
Diff view
Diff view
There are no files selected for viewing
| Original file line number | Diff line number | Diff line change |
|---|---|---|
| @@ -0,0 +1,24 @@ | ||
| #!/usr/bin/env -S deno run -A | ||
| /** Command line over `checkEquivalentMutants`, run by `deno task check:equivalents`. */ | ||
|
|
||
| import { | ||
| checkEquivalentMutants, | ||
| DEFAULT_REGISTRY_DIR, | ||
| } from "#scripts/mutation/check-equivalents.ts"; | ||
| import { projectRoot } from "#scripts/project-root.ts"; | ||
|
|
||
| const problems = await checkEquivalentMutants({ | ||
| registryDir: DEFAULT_REGISTRY_DIR, | ||
| root: projectRoot, | ||
| }); | ||
| if (problems.length === 0) { | ||
| console.log("Every equivalent-mutant entry still points at a real mutant."); | ||
| Deno.exit(0); | ||
| } | ||
| console.error(problems.join("\n")); | ||
| console.error( | ||
| `\n${problems.length} equivalent-mutant entries need attention. Each names the` + | ||
| "\nthing it sits inside; re-record it against where that code lives now, or" + | ||
| "\nremove it if the expression is gone.", | ||
| ); | ||
| Deno.exit(1); |
| Original file line number | Diff line number | Diff line change |
|---|---|---|
| @@ -0,0 +1,191 @@ | ||
| /** | ||
| * Where a mutant sits, said so it means the same thing tomorrow. | ||
| * | ||
| * An anchor names the thing a mutant sits inside — the function, method, or | ||
| * value it belongs to — and fingerprints the expression it mutates. Both parts | ||
| * are derived from the code, so resolving an anchor is the same act as | ||
| * verifying it: an anchor that resolves has found the expression it was | ||
| * recorded against, not merely something in the right place. | ||
| * | ||
| * An anchor holds only `A-Z a-z 0-9 _ $ - . % ~ @`, so it can never contain a | ||
| * space or the `#` that starts a registry line's comment. Any other character | ||
| * in a name is percent-encoded. | ||
| */ | ||
|
|
||
| /** Characters an anchor segment may hold unencoded: the ones an identifier or | ||
| * a kebab-case key is made of. Everything else — including the `.` that joins | ||
| * segments, and anything that would end a registry line early — is | ||
| * percent-encoded. */ | ||
| const SAFE_IN_NAME = /[A-Za-z0-9_$-]/; | ||
|
|
||
| const encodeName = (name: string): string => | ||
| [...name] | ||
| .map((char) => | ||
| SAFE_IN_NAME.test(char) | ||
| ? char | ||
| : [...new TextEncoder().encode(char)] | ||
| .map((byte) => `%${byte.toString(16).padStart(2, "0")}`) | ||
| .join(""), | ||
| ) | ||
| .join(""); | ||
|
|
||
| /** FNV-1a over the expression's text. Short and stable is all this needs to | ||
| * be: it tells two expressions under one name apart, and changes when the | ||
| * expression it was recorded against changes. */ | ||
| const fingerprint = (text: string): string => { | ||
| let hash = 0x811c9dc5; | ||
| for (const char of text) { | ||
| hash ^= char.codePointAt(0)!; | ||
| hash = Math.imul(hash, 0x01000193) >>> 0; | ||
| } | ||
| return hash.toString(36).padStart(7, "0").slice(-7); | ||
| }; | ||
|
|
||
| interface Span { | ||
| end: number; | ||
| start: number; | ||
| } | ||
|
|
||
| /** A node that might lend its name to whatever sits inside it. */ | ||
| interface NamedNode { | ||
| id?: { name?: string } | null; | ||
| key?: { name?: string; value?: unknown } | null; | ||
| type?: string; | ||
| } | ||
|
|
||
| /** The name a declaration contributes, or nothing when it is anonymous. An | ||
| * anonymous arrow inside a call takes its name from whatever it is assigned | ||
| * to instead. */ | ||
| const nameOf = (node: NamedNode): string | null => { | ||
| const named = node.id?.name ?? node.key?.name; | ||
| if (typeof named === "string" && named !== "") return named; | ||
| const literalKey = node.key?.value; | ||
| return typeof literalKey === "string" && literalKey !== "" | ||
| ? literalKey | ||
| : null; | ||
| }; | ||
|
|
||
| /** Declarations whose name is worth carrying into the path. A block or an `if` | ||
| * contributes nothing: naming by them would move the anchor when surrounding | ||
| * code is merely re-nested. */ | ||
| const NAMING_TYPES = new Set([ | ||
|
There was a problem hiding this comment. Choose a reason for hiding this commentThe reason will be displayed to describe this comment to others. Learn more.
Fresh evidence beyond the resolved property/enum/name threads is that Useful? React with 👍 / 👎.
Member
Author
There was a problem hiding this comment. Choose a reason for hiding this commentThe reason will be displayed to describe this comment to others. Learn more. Fixed in 9184de7. The regression test is your example: one function, two arms whose bodies are character-identical Every case label in Generated by Claude Code There was a problem hiding this comment. Choose a reason for hiding this commentThe reason will be displayed to describe this comment to others. Learn more.
When anonymous callbacks are handed directly to different calls, such as Useful? React with 👍 / 👎.
Member
Author
There was a problem hiding this comment. Choose a reason for hiding this commentThe reason will be displayed to describe this comment to others. Learn more. Confirmed the collapse ( A callee genuinely is a name the code writes, unlike a ternary arm or a guessed discriminant. What sinks it is that it does not do the job. I ran the case that matters: Naming by callee turns those into Every other naming rule accepted here removes the ordinal outright, because a key, a prop, a case label and a parameter are each unique among their siblings by construction. A callee is not, and that difference is the whole argument. The measurement is in the ternary thread: 3 of 540 entries carry an ordinal, none in a call argument. The boundary and its reasoning are now at Generated by Claude Code |
||
| "ClassDeclaration", | ||
| "FunctionDeclaration", | ||
|
There was a problem hiding this comment. Choose a reason for hiding this commentThe reason will be displayed to describe this comment to others. Learn more.
When named function or class expressions are used somewhere that has no property or variable name of its own, such as array entries or call arguments, this list does not collect their Useful? React with 👍 / 👎.
Member
Author
There was a problem hiding this comment. Choose a reason for hiding this commentThe reason will be displayed to describe this comment to others. Learn more. Correct. Both are in the test: Worth naming the pattern the last three of these share: a node carries a name in a position where nothing around it does. Generated by Claude Code |
||
| "MethodDefinition", | ||
| "PropertyDefinition", | ||
| "TSEnumDeclaration", | ||
|
There was a problem hiding this comment. Choose a reason for hiding this commentThe reason will be displayed to describe this comment to others. Learn more.
When a TypeScript enum has repeated mutable initializers, this collects only the enum name, not the member name, so Useful? React with 👍 / 👎.
Member
Author
There was a problem hiding this comment. Choose a reason for hiding this commentThe reason will be displayed to describe this comment to others. Learn more. Right, same shape as the function-expression one. Test: Generated by Claude Code |
||
| "TSInterfaceDeclaration", | ||
| "TSModuleDeclaration", | ||
| "TSTypeAliasDeclaration", | ||
| "VariableDeclarator", | ||
|
There was a problem hiding this comment. Choose a reason for hiding this commentThe reason will be displayed to describe this comment to others. Learn more.
When a function has two default parameter initializers with identical mutable text, such as a string-typed Useful? React with 👍 / 👎.
Member
Author
There was a problem hiding this comment. Choose a reason for hiding this commentThe reason will be displayed to describe this comment to others. Learn more. Fixed in 28749a4. A defaulted parameter now lends its name — a default is code of its own, and two parameters of one function can default to the same words — so your case gives This was live in the tree, and more so than any earlier naming finding: thirteen entries were sitting on the enclosing function alone, told apart only by order. Each moved onto the parameter it always sat in — Every one kept its fingerprint, and every new name extends the old one rather than replacing it — that pair of facts is what makes the re-anchoring checkable rather than a guess. The binding-pattern line still holds: a parameter that destructures ( Generated by Claude Code |
||
| ]); | ||
|
|
||
| const isSpan = (value: unknown): value is Span => | ||
| typeof value === "object" && | ||
| value !== null && | ||
| typeof (value as Span).start === "number" && | ||
| typeof (value as Span).end === "number"; | ||
|
|
||
| interface Descent { | ||
| /** The span whose text is fingerprinted. */ | ||
| context: Span; | ||
| names: string[]; | ||
| } | ||
|
|
||
| /** | ||
| * Walk down to a mutant's span, collecting the names enclosing it and the | ||
| * smallest node that strictly contains it. Follows only the child holding the | ||
| * span, so the cost is the tree's depth, not its size. | ||
| * | ||
| * Strictly containing is what makes the fingerprint identify this mutant and | ||
| * nothing else. A swapped operator sits in the gap inside its own expression, | ||
| * so that expression is chosen — `a ?? 0`, not the array it shares with its | ||
| * neighbours. A replaced literal *is* a node, so the node one step out is | ||
| * chosen: the expression giving that literal its meaning. | ||
| */ | ||
| const descendTo = (program: object, mutant: Span): Descent => { | ||
| const offset = mutant.start; | ||
| const width = mutant.end - mutant.start; | ||
| const names: string[] = []; | ||
| const spans: Span[] = []; | ||
| let node: object = program; | ||
| for (;;) { | ||
| const record = node as Record<string, unknown>; | ||
| if (typeof record.type === "string" && NAMING_TYPES.has(record.type)) { | ||
| const named = nameOf(record as NamedNode); | ||
| if (named !== null) names.push(named); | ||
| } | ||
| if (isSpan(record)) spans.push(record); | ||
| const next = Object.values(record) | ||
| .flatMap((value) => (Array.isArray(value) ? value : [value])) | ||
| .find( | ||
| (value) => isSpan(value) && value.start <= offset && offset < value.end, | ||
| ); | ||
| if (!next) break; | ||
| node = next as object; | ||
| } | ||
| // The program is itself a span holding every offset, so there is always at | ||
| // least one to fall back to. | ||
| const containing = spans.filter((span) => span.end - span.start > width); | ||
| return { context: containing.at(-1) ?? spans.at(-1)!, names }; | ||
|
There was a problem hiding this comment. Choose a reason for hiding this commentThe reason will be displayed to describe this comment to others. Learn more.
Fresh evidence beyond the earlier ordinal issue is the new Useful? React with 👍 / 👎.
Member
Author
There was a problem hiding this comment. Choose a reason for hiding this commentThe reason will be displayed to describe this comment to others. Learn more. Confirmed, and worse than you described. Fixed in ba55141. I reproduced both halves before touching anything. At file scope, The fingerprint now stops at the mutant's own statement. It still climbs to the expression that gives a swapped operator or a replaced literal its meaning — Three tests lock it: Five registry entries were anchored under the old rule and are re-recorded, including two Generated by Claude Code |
||
| }; | ||
|
|
||
| export interface AnchoredMutant { | ||
|
There was a problem hiding this comment. Choose a reason for hiding this commentThe reason will be displayed to describe this comment to others. Learn more.
A repo-wide AGENTS.md reference: AGENTS.md:L81-L81 Useful? React with 👍 / 👎. |
||
| /** `name~fingerprint`, plus `@n` when two mutants are indistinguishable. */ | ||
| anchor: string; | ||
| } | ||
|
|
||
| interface HasLocation { | ||
| end: number; | ||
| newOperator: string; | ||
| operator: string; | ||
| start: number; | ||
| } | ||
|
|
||
| /** Every character an anchor can hold, for callers validating one. */ | ||
| export const ANCHOR_PATTERN = /^[A-Za-z0-9_$\-.%~@]+$/; | ||
|
There was a problem hiding this comment. Choose a reason for hiding this commentThe reason will be displayed to describe this comment to others. Learn more.
AGENTS.md reference: AGENTS.md:L81-L81 Useful? React with 👍 / 👎.
Member
Author
There was a problem hiding this comment. Choose a reason for hiding this commentThe reason will be displayed to describe this comment to others. Learn more. Correct. The assertion it backed — "every anchor holds only anchor characters" — was already redundant: the round-trip test writes each key to a registry line, reads it back, and asserts it is identical, which fails on any character that would not survive the line. That is the property that matters, tested through the behaviour rather than by inspecting the representation. I exported it to make a test possible, which is the thing the rule is about. Generated by Claude Code |
||
|
|
||
| /** | ||
| * Give every mutant its anchor. | ||
| * | ||
| * Two mutants collide only when they share an enclosing name, a `from → to`, | ||
| * and character-identical expression text — genuinely indistinguishable, so | ||
| * they take `@1`, `@2` in source order. Anything that merely moves code leaves | ||
| * every anchor alone. | ||
| * | ||
| * Top-level code belonging to no declaration anchors on the file itself, | ||
| * written `%3cfile%3e`. | ||
| */ | ||
| export const anchorMutants = <M extends HasLocation>( | ||
| program: object, | ||
| content: string, | ||
| mutants: readonly M[], | ||
| ): (M & AnchoredMutant)[] => { | ||
| const described = mutants.map((mutant) => { | ||
| const { context, names } = descendTo(program, mutant); | ||
| const name = (names.length === 0 ? ["<file>"] : names) | ||
| .map(encodeName) | ||
| .join("."); | ||
| const text = content.slice(context.start, context.end); | ||
| return { | ||
| ...mutant, | ||
| base: `${name}~${fingerprint(text)}`, | ||
| }; | ||
| }); | ||
| const totals = new Map<string, number>(); | ||
| for (const mutant of described) { | ||
| const key = `${mutant.base} ${mutant.operator}→${mutant.newOperator}`; | ||
| totals.set(key, (totals.get(key) ?? 0) + 1); | ||
| } | ||
| const seen = new Map<string, number>(); | ||
| return described.map((mutant) => { | ||
| const key = `${mutant.base} ${mutant.operator}→${mutant.newOperator}`; | ||
| const nth = (seen.get(key) ?? 0) + 1; | ||
| seen.set(key, nth); | ||
| const { base, ...rest } = mutant; | ||
| return { | ||
| ...(rest as unknown as M), | ||
| anchor: totals.get(key)! > 1 ? `${base}@${nth}` : base, | ||
|
There was a problem hiding this comment. Choose a reason for hiding this commentThe reason will be displayed to describe this comment to others. Learn more.
When character-identical expressions occur under the same collected name but in different lexical contexts, they are not necessarily equivalent: for example, two object callbacks can both contain Useful? React with 👍 / 👎.
Member
Author
There was a problem hiding this comment. Choose a reason for hiding this commentThe reason will be displayed to describe this comment to others. Learn more. Half fixed, half irreducible — worth separating the two. Fixed in ba55141. Your concrete case was right and is gone: It re-anchored 124 of the 519 entries. Every one was carried across mechanically by matching its old anchor to the mutant it named and writing the new one — 0 unmatched, 0 ambiguous — and The residual What I have done instead is shrink the surface and stop hiding it: 4 of 519 entries carry an ordinal, and both the module doc and the registry README now say plainly that the ordinal is the one part of an anchor a reordering can move, and that an entry carrying one is worth re-checking when its neighbours change. Generated by Claude Code |
||
| }; | ||
| }); | ||
| }; | ||
| Original file line number | Diff line number | Diff line change |
|---|---|---|
| @@ -0,0 +1,125 @@ | ||
| /** | ||
| * Check that every known-equivalent mutant entry still points at a real mutant. | ||
| * | ||
| * | ||
| * Resolution only: no lint, no type-check, no tests, so it is fast enough to | ||
| * run on every commit. Re-proving that an entry is still *equivalent* applies | ||
| * each mutant through the static gates and stays `mutation:audit-equivalents`. | ||
| * | ||
| * An entry fails here when it is a duplicate, when its path is not the | ||
| * canonical project-relative one the mutation runner keys by, or when nothing | ||
| * it could suppress exists any more. | ||
| */ | ||
|
|
||
| import { isAbsolute, relative, resolve, SEPARATOR } from "@std/path"; | ||
| import { generateMutants } from "./generate.ts"; | ||
| import { | ||
| EQUIVALENT_MUTANTS_DIR, | ||
| listRegistryFiles, | ||
| mutantKeyForPath, | ||
| parseRegistryText, | ||
| registryFilePath, | ||
| } from "./ignore.ts"; | ||
|
|
||
| /** Where the checked-in registry lives. */ | ||
| export const DEFAULT_REGISTRY_DIR: URL = EQUIVALENT_MUTANTS_DIR; | ||
|
There was a problem hiding this comment. Choose a reason for hiding this commentThe reason will be displayed to describe this comment to others. Learn more.
AGENTS.md reference: AGENTS.md:L79-L79 Useful? React with 👍 / 👎.
Member
Author
There was a problem hiding this comment. Choose a reason for hiding this commentThe reason will be displayed to describe this comment to others. Learn more. Right. Fixed in ba55141 — It was a leftover from splitting the checker's rules out of its CLI: the caller needed the constant, and instead of importing it I re-exported it under a new name in the file it had just moved out of. No default, no transformation, no guard — the alias layer the rule is about. Generated by Claude Code |
||
|
|
||
| export interface EquivalentCheckOptions { | ||
|
There was a problem hiding this comment. Choose a reason for hiding this commentThe reason will be displayed to describe this comment to others. Learn more.
A repo-wide AGENTS.md reference: AGENTS.md:L81-L81 Useful? React with 👍 / 👎. |
||
| /** Where the registry files live. */ | ||
| registryDir: string | URL; | ||
| /** The project root every entry's path is written relative to. */ | ||
| root: string; | ||
| } | ||
|
|
||
| interface Entry { | ||
| key: string; | ||
| registry: string; | ||
| sourcePath: string; | ||
| } | ||
|
|
||
| /** | ||
| * Why a path cannot be used as written, or nothing when it can. | ||
| * | ||
| * The runner keys a mutant by its canonical project-relative path, so any | ||
| * other spelling of the same file — absolute, or reaching out and back — | ||
| * resolves here while never matching what the runner suppresses. | ||
| */ | ||
| const whyPathUnusable = (sourcePath: string, root: string): string | null => { | ||
| if (isAbsolute(sourcePath)) return "path must be relative to the project"; | ||
| const resolved = resolve(root, sourcePath); | ||
| const canonical = relative(root, resolved); | ||
| if (canonical === ".." || canonical.startsWith(`..${SEPARATOR}`)) { | ||
| return "path escapes the project"; | ||
| } | ||
| return canonical === sourcePath | ||
| ? null | ||
| : `path must be written as "${canonical}"`; | ||
| }; | ||
|
|
||
| const readEntries = async ( | ||
| registryDir: string | URL, | ||
| root: string, | ||
| ): Promise<Entry[]> => { | ||
| const entries: Entry[] = []; | ||
| for (const file of await listRegistryFiles(registryDir)) { | ||
| const registry = relative(root, registryFilePath(file)); | ||
| for (const parsed of parseRegistryText( | ||
| await Deno.readTextFile(file), | ||
| file, | ||
| )) { | ||
| entries.push({ | ||
| key: parsed.key, | ||
| registry, | ||
| sourcePath: parsed.sourcePath, | ||
| }); | ||
| } | ||
| } | ||
| return entries; | ||
| }; | ||
|
|
||
| /** Every mutant key a source file can produce, generated exhaustively so an | ||
| * entry for an exhaustive-only replacement still resolves. */ | ||
| const keysFor = async ( | ||
| sourcePath: string, | ||
| root: string, | ||
| ): Promise<Set<string>> => { | ||
| const file = resolve(root, sourcePath); | ||
| const content = await Deno.readTextFile(file); | ||
|
There was a problem hiding this comment. Choose a reason for hiding this commentThe reason will be displayed to describe this comment to others. Learn more.
When a branch deletes or renames a source file that still has a registry entry, this read throws Useful? React with 👍 / 👎. |
||
| return new Set( | ||
| generateMutants(content, file, true).map((mutant) => | ||
| mutantKeyForPath(sourcePath, mutant), | ||
| ), | ||
| ); | ||
| }; | ||
|
|
||
| export const checkEquivalentMutants = async ( | ||
| options: EquivalentCheckOptions, | ||
| ): Promise<string[]> => { | ||
| const { root } = options; | ||
| const entries = await readEntries(options.registryDir, root); | ||
| const problems: string[] = []; | ||
| const seen = new Set<string>(); | ||
|
|
||
| const usable = entries.filter((entry) => { | ||
| const why = whyPathUnusable(entry.sourcePath, root); | ||
| if (why) problems.push(`${why} (${entry.registry}): ${entry.key}`); | ||
| return why === null; | ||
| }); | ||
|
|
||
| const byPath = new Map<string, Set<string>>(); | ||
| for (const sourcePath of new Set(usable.map((e) => e.sourcePath))) { | ||
| byPath.set(sourcePath, await keysFor(sourcePath, root)); | ||
| } | ||
| for (const entry of usable) { | ||
| if (seen.has(entry.key)) { | ||
| problems.push(`duplicate (${entry.registry}): ${entry.key}`); | ||
| } | ||
| seen.add(entry.key); | ||
| if (!byPath.get(entry.sourcePath)!.has(entry.key)) { | ||
|
coderabbitai[bot] marked this conversation as resolved.
Resolved
|
||
| problems.push( | ||
| `stale (nothing to suppress — did it move or get renamed?) (${entry.registry}): ${entry.key}`, | ||
| ); | ||
| } | ||
| } | ||
| return problems; | ||
| }; | ||
There was a problem hiding this comment.
Choose a reason for hiding this comment
The reason will be displayed to describe this comment to others. Learn more.
When object members use legal numeric keys, Oxc exposes
key.valueas a number, so this string-only branch contributes no property name. Two character-identical callbacks under keys1and2therefore share the enclosing name and fingerprint and receive source-order@n; reordering those properties can make an entry audited for one callback suppress the other callback, whose contextual type and equivalence may differ. Fresh evidence beyond the resolved named-property thread is that numeric literal keys remain unnamed; encode legal non-string literal keys before anchoring.Useful? React with 👍 / 👎.
There was a problem hiding this comment.
Choose a reason for hiding this comment
The reason will be displayed to describe this comment to others. Learn more.
Correct —
nameOfonly accepted a stringkey.value, so{ 1: … }contributed nothing. Fixed: a numeric key now names its member, soconst o = { 7: (x) => x ?? 0 }anchors ono.7.Test:
names an object member written as a number, plus a{ 1: (x) => x ?? "", 2: (x) => x ?? "" }case in the round-trip corpus, which is exactly your two-identical-callbacks shape — both keys come back distinct with no ordinal.No registry entry moved, so nothing here rested on a numeric key today. That is the point of catching it now rather than the first time one appears.
Generated by Claude Code