feat(testgen): the ranker that orders test-writing work now has a caller - #161
Merged
Merged
Conversation
`escaped_defect_priority` merged in #153 and nothing invoked it. In this repository that is the documented failure mode rather than an oversight, so this is wiring, not building: the ranker existed, the lane existed, the edge between them did not. `testgen_lane --rank-sources N` chooses the lane's `--source` values by measured priority — escaped defects, then churn, then uncovered mass — and puts the reason for each file in the prompt. The order is stated as a priority, not a mandate, with an instruction to say so and take the next file rather than write a smoke test to clear one that cannot be tested. The substance is `rank_status`, and it is about the latch rather than the ranking. Both git-backed tiers go through a helper that returns "" on a non-zero exit, so a directory that is not a git repository produces exactly the empty ranking a pristine one does — a caller picking work would read "no file needs tests" off a failed subprocess. The probe is asked before ranking rather than inferred from an empty result, and ok / no_signal / unavailable stay three findings, not one. Every fallback fails toward motion. Switch off, ranker unimportable, repo unreadable, nothing scored: all fall back to hand-named sources and say which in a note that is always printed. Only with nothing to fall back on does the CLI exit 2, and then the note names what was missing. Two defects in my own draft, both found by testing rather than by reading it: - `coverage run --source=src/mod.py` measures nothing and exits 0 — verified, and already documented at testgen_gate.py:44-47. Ranked file paths become importable names, dropping the leading component only when it is a source root, detected by a missing `__init__.py` because `src` is a root here and a package elsewhere. - The rationale was unfalsifiable: it read `escaped`/`churn`/`uncovered` with a default of 0 while `as_dict` emits `tier1_escaped_defects` and friends, so every file's stated reason was "escaped 0, churn 0, uncovered 0" under a correct ordering. Absent keys now render `?`. A rendered `?` is a visible defect; a rendered 0 is a lie that reads as good news. The heartbeat is filed under `testgen-lane` rather than as a new capability row. The ranker is a rail the lane consults — no model call, no dispatch, no work a caller could be offered instead of the lane — and a second lifecycle record for an implementation detail is how this project loses track of features. +14 pytest tests, eleven of them on the fallbacks. Floor 515 -> 529 measured on this merge result, rationale appended to the note. Three deliberate breaks, each reverted: removing the git probe (2 fail), restoring the `.get(key, 0)` rationale (1 fail), hardcoding the source-root rule (1 fail). verify.py from the checkout: 529 passed, 0 failed, 0/26 max skipped, 88/88 selftests, 5 gates. Co-Authored-By: Claude Opus 5 <noreply@anthropic.com>
|
Warning Review limit reachedNext included review available in 6 minutes. View limit detailsLimit details: You’ve used all 2 included reviews currently available. Your 55 included PR review attempts over the past 7 days set your current allowance at 2 reviews per hour. Your organization has reached its usage spending cap. Adjust your spending cap in the billing tab. Review configuration: ⚙️ Run configurationConfiguration used: Path: .coderabbit.yaml Review profile: ASSERTIVE Plan: Pro Run ID: 📒 Files selected for processing (4)
Comment |
stranske
temporarily deployed
to
agent-standard
August 30, 2026 02:27 — with 
GitHub Actions
Inactive
GitHub Actions
Inactive
Contributor
Workflow source neededPR #161 needs either a linked GitHub issue or one valid non-issue Workflow Source before PR metadata automation can manage it safely. Please do one of:
Once a valid source is present, this warning will not be reposted. |
Automated Status SummaryHead SHA: d325972
Coverage Overview
Coverage Trend
Top Coverage Hotspots (lowest coverage)
Low Coverage Files (<50.0%)
Updated automatically; will refresh on subsequent CI/Docker completions. Keepalive checklistScopeNo scope information available Tasks
Acceptance criteria
|
Contributor
|
Workflow state fingerprint for Agents Gate Followups. Do not edit. |
This file contains hidden or bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
Sign up for free
to join this conversation on GitHub.
Already have an account?
Sign in to comment
1 participant
Add this suggestion to a batch that can be applied as a single commit.This suggestion is invalid because no changes were made to the code.Suggestions cannot be applied while the pull request is closed.Suggestions cannot be applied while viewing a subset of changes.Only one suggestion per line can be applied in a batch.Add this suggestion to a batch that can be applied as a single commit.Applying suggestions on deleted lines is not supported.You must change the existing code in this line in order to create a valid suggestion.Outdated suggestions cannot be applied.This suggestion has been applied or marked resolved.Suggestions cannot be applied from pending reviews.Suggestions cannot be applied on multi-line comments.Suggestions cannot be applied while the pull request is queued to merge.Suggestion cannot be applied right now. Please check back later.
The ranker had no caller
escaped_defect_prioritymerged in #153 and nothing invoked it. This repository's dominant defect class is built-and-forgotten features, so that is the documented failure mode rather than an oversight — and it means this PR is wiring, not building. The ranker existed, the lane existed, the edge between them did not.Dedup finding, before writing anything: grepped for
rank/priority/churn/escaped; searched the improvement log fortestgen(44 hits) andescaped(4); checked open PRs. Both halves present, edge absent. Recorded on improvement-log item 2, wheretestgen_lane's build is recorded.What it does
testgen_lane --rank-sources Npicks the lane's--sourcevalues by measured priority — escaped defects, then churn, then uncovered mass — and puts each file's reason in the prompt:followed by an explicit statement that the order is a priority, not a mandate, with an instruction to say so and take the next file rather than write a smoke test to clear one that cannot be tested meaningfully.
The substance is
rank_status, and it is about the latchBoth git-backed tiers go through
_git, which returns""on a non-zero exit. A directory that is not a git repository therefore produces exactly the empty ranking a pristine one does — so a caller choosing test-writing work would read "no file needs tests" off a failed subprocess. That is this workspace's most repeated defect in its test-writing costume: one value meaning both measured zero and could not measure, where only the first is good news.rev-parse --git-diris asked before ranking rather than inferred from an empty result, andok/no_signal/unavailablestay three findings. Theno_signalreason states what was not read, so a drained result and an unread one never render alike.Every fallback fails toward motion. Switch off, ranker unimportable, repo unreadable, nothing scored — all fall back to sources named on the command line and say which, in a note that is always printed (a lane that quietly fell back would be indistinguishable from one that ranked). Only with nothing to fall back on does the CLI exit 2, and then the note names what was missing.
Two defects in my own draft, both found by testing rather than by reading it
coverage run --source=src/mod.pymeasures nothing and exits 0. Verified empirically, and already documented attestgen_gate.py:44-47as one of two directions this goes wrong. So ranked file paths become importable names, dropping the leading component only when it is a source root — detected by a missing__init__.py, becausesrcis a root here and a package elsewhere. A wrong guess is not silent:testgen_gate.unmeasured_sourcesalready fails the gate on a source no measured file belongs to.The rationale was unfalsifiable. The prompt read
escaped/churn/uncoveredwith a default of0whileas_dictemitstier1_escaped_defectsand friends. Every key missed, so every file's stated reason was "escaped 0, churn 0, uncovered 0" — a correct ordering under a reason no input could ever contradict. Absent keys now render?. A rendered?is a visible defect; a rendered0is a lie that reads as good news.Why no new ledger row
The heartbeat is filed under
testgen-lane. The ranker is a rail the lane consults — no model call, no dispatch, no work a caller could be offered instead of the lane — so it is one capability with two code paths that can prove it ran. A second lifecycle record for an implementation detail is how this project loses track of features.Verification
python3 src/verify.pyfrom the checkout: 529 passed, 0 failed, 0/26 max skipped, 88/88 selftests, 5/5 gates. Floor 515 → 529 measured on this merge result, rationale appended to the note rather than replacing it.+14 pytest tests — eleven on the fallbacks, three on the ranking, because the fallbacks are where an empty answer gets mistaken for good news. Three deliberate breaks, each reverted:
.get(key, 0)rationale restoredOne note for the reviewer: the admission gate goes red on the owner's machine right now, and it is not this change. The shared machine-local ledger acquired findability-exemption declarations for 12 capabilities absent from
KNOWN_DECLARATIONSwhile I worked. Confirmed pre-existing by running the same test on cleanorigin/mainin a scratch worktree, and confirmed absent under freshORCH_STATE_DIR+ORCH_LOCAL_RUNTIME(10 passed, 3 skipped) — which is what CI runs. A concurrent session's rows.🤖 Generated with Claude Code