Skip to content

docs(research): 36 cohort test artifacts exist, 9 run in CI (#342) - #436

Merged
serina-mcfall merged 4 commits into
launchpadfrom
research/342-cohort-executable-inventory
Aug 24, 2026
Merged

docs(research): 36 cohort test artifacts exist, 9 run in CI (#342)#436
serina-mcfall merged 4 commits into
launchpadfrom
research/342-cohort-executable-inventory

Conversation

@tucktuck101

@tucktuck101 tucktuck101 commented Aug 21, 2026

Copy link
Copy Markdown
Collaborator

Summary

Answers #342 with the denominator criterion 3 was missing — re-derived at pinned base 132f921ace6be6d391c03e134b304ddea3ecccf0 (2026-08-24): 48 production Python modules, 12 production shell scripts, one extensionless Bash entry point (deploy/archived/deploy) and 34 .yml files under launchpad/, guarded by 36 test artifacts (22 Python test files, 14 test-*.sh). CI invokes exactly three things and covers 9 of the 36. The two unnamed gaps are the 14 shell test scripts, which no workflow runs, and review-agent's seven unittest suites, which are easy to confuse with its containment controls. A mechanical per-executable appendix (mode + shebang + extension + playbooks, 103 rows) makes the denominator per-file, and classifies the 34 .yml files with arguable boundaries recorded as arguable.

Related issue

Closes #342

Issue type

Task


Agent provenance

Field Value
Harness / provider Claude Code
Model claude-opus-5[1m]; refresh at 132f921 by claude-fable-5
Session reference N/A - the harness exposes no shareable run id
Initiating human @tucktuck101

Objective

Add launchpad/Research/342-cohort-executable-inventory.md enumerating cohort-owned executables under launchpad/ and, per test artifact, whether any CI workflow runs it.

Impacted components

launchpad/Research/342-cohort-executable-inventory.md

Approach and rejected alternatives

Enumerated from git ls-files so the method is mechanical and reproducible — extension histogram first, then a second pass over git file mode 100755 + shebang + Ansible playbooks, which is what catches the extensionless deploy/archived/deploy — then listed every test-relevant run: line across all six launchpad-*.yml workflows and matched the two sets. #342 asked specifically for a reproducible mechanical method rather than a remembered list, because the failure mode is a forgotten directory.

Rejected trusting the workflow names. launchpad-review-agent-controls.yml sounds like it runs the review agent's tests; reading run_controls.py's hardcoded CONTROLS list shows it runs thirteen check_*.py/suite.py entries and none of the seven test_*.py suites. That distinction is the main thing this document contributes, and a name-level reading would have missed it.

Verification

Commands run at 132f921ac (merged into this branch):

git ls-files launchpad | grep -E '\.(py|sh|yml|yaml)$' | sed 's/.*\.//' | sort | uniq -c
git ls-files 'launchpad/<dir>/*.py' 'launchpad/<dir>/**/*.py' | sort -u   # prod vs test per directory
git ls-files launchpad | grep -cE '/test-[^/]*\.sh$'
git ls-files -s launchpad | awk '$1=="100755"{print $4}' | grep -vE '\.'
for w in .github/workflows/launchpad-*.yml; do grep -E "^\s+run: " $w; done
sed -n '/^CONTROLS = \[/,/^\]/p' launchpad/review-agent/run_controls.py
grep -rn "test-run-guard\|launchpad/deploy" .github/workflows/

Raw output:

$ git ls-files launchpad | grep -E '\.(py|sh|yml|yaml)$' | sed 's/.*\.//' | sort | uniq -c
  70 py
  26 sh
  34 yml

# prod vs test per directory
launchpad/agents                   prod=4    test=2
launchpad/project-intelligence     prod=6    test=6
launchpad/review-agent             prod=26   test=7
launchpad/scripts                  prod=12   test=7

# shell per directory
launchpad/deploy                   9  (+ extensionless archived/deploy)
launchpad/scripts                  16
launchpad/sync-labels.sh           1

$ git ls-files launchpad | grep -cE '/test-[^/]*\.sh$'
14

$ git ls-files -s launchpad | awk '$1=="100755"{print $4}' | grep -vE '\.'
launchpad/deploy/archived/deploy

# every test-relevant run: line in the six cohort workflows
--- launchpad-adr-check.yml
python3 -m unittest discover -s launchpad/scripts
python3 launchpad/scripts/adr_boundary_check.py .
--- launchpad-agents-tests.yml
pip install -r launchpad/agents/requirements.txt
python3 - <<'PY'   # guard: fails if discover collects zero test CASES
python3 -m unittest discover -s launchpad/agents -p "test_*.py" -v
--- launchpad-issue-check.yml
(none)
--- launchpad-pr-check.yml
python3 launchpad/scripts/pr_body_check.py
python3 -m unittest discover -s launchpad/scripts -t launchpad/scripts -v
python3 launchpad/scripts/mutation_harness.py
--- launchpad-review-agent-controls.yml
pip install pyyaml
python3 run_controls.py
--- launchpad-security-audit.yml
python3 -m unittest discover -s launchpad/scripts -p "test_security_audit*.py"
python3 launchpad/scripts/security_audit.py .

$ sed -n '/^CONTROLS = \[/,/^\]/p' launchpad/review-agent/run_controls.py
CONTROLS = [
    ("check_contract.py", False),
    ("check_step2.py", False),
    ("check_step3.py", True),
    ("check_step45.py", False),
    ("check_step6.py", True),
    ("suite.py", False),
    ("check_step8.py", False),
    ("check_step9.py", False),
    ("check_step11.py", False),
    ("check_invariants.py", False),
    ("check_mutations.py", False),
    ("check_flag_guard.py", False),
    ("check_fetch_states.py", False),
]

$ grep -rn "test-run-guard\|launchpad/deploy" .github/workflows/
.github/workflows/docker.yml:142:  # a commit-pinned tag for launchpad/deploy/run.sh's guard to accept,
  • Tests or checks were run and the raw output is pasted above
  • The diff is confined to the scope of the linked issue
  • No secrets, keys, tokens or hostnames were added to tracked files

Not verified

Whether the 14 test-*.sh scripts pass. I inventoried them; I ran none. They may be green, stale or broken, and that changes the cost of wiring them in considerably.

Whether unittest discover -s launchpad/scripts actually collects all seven Python test files. It should by naming convention, but I did not run it and read the count. launchpad-agents-tests.yml now guards against empty discovery for launchpad/agents (counting collected cases, not files); the three launchpad/scripts discover invocations have no such guard.

Whether review-agent's 13 check_*.py controls subsume what its seven test_*.py suites assert. I established they are different invocations, not that the coverage is disjoint.

Executable-boundary edge cases are recorded, not settledbin/lefthook (a cohort divergence under ADR-0017, executable, outside launchpad/), the five ADR-0005 deployment files (cohort-owned, not executable), and Ansible role tasks/handlers (executed only by inclusion). The 34 .yml files are now classified in the note's appendix on exactly those terms.

Upstream's scripts/ directory is excluded, though the fork's CI depends on several of those scripts.

Security implications

None from the document. One observation for reviewers: launchpad/deploy/'s only test artifact, test-run-guard.sh, is not run by any workflow — and the deploy scripts are the code that touches the live relay host. That is the highest-consequence entry in the unrun column, together with deploy/archived/deploy, the archived extensionless deploy driver with no test at all.

Escalations

  1. prd: the cohort test suite — what we test that upstream doesn't #290's evidence needs two corrections. It says CI "invokes the cohort's checkers ... but never runs their test suites"; for launchpad/scripts and (since 8f4293543) launchpad/agents the full suites run on every pull request, plus mutation_harness.py for scripts. And my own earlier sweep comment on prd: the cohort test suite — what we test that upstream doesn't #290 implied the review agent's suites run in CI — they do not. Editing the PRD is not mine to do.
  2. Criterion 3's scope is roughly four times what the PRD describes — 27 unrun test artifacts, with the 14 shell tests being the largest block and unmentioned anywhere. The PRD's named launchpad/agents gap (task: run launchpad/agents Python test suites in CI #270) has meanwhile closed, so the named gap and the real gap barely overlap.
  3. The naming collision is a live trap. "review agent controls" running in CI reads as "the review agent is tested in CI". Whether the seven suites should join CONTROLS, or whether the controls already suffice, is a call for whoever owns that code.

Signed-off-by: tucktuck101 <jeffreytaylorrobertson@gmail.com>
@tucktuck101 tucktuck101 added the by:agent Filed or authored by an AI agent, not a human label Aug 21, 2026

@serina-mcfall serina-mcfall left a comment

Copy link
Copy Markdown

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Blocker — the headline "31" is 32, and the note's own table says so

Using the note's own definition (a test_*.py or test-*.sh file under launchpad/), at the pinned SHA:

$ git ls-tree -r --name-only f75085d5c -- launchpad | grep -cE '/test_[^/]*\.py$'
18
$ git ls-tree -r --name-only f75085d5c -- launchpad | grep -cE '/test-[^/]*\.sh$'
14

18 + 14 = 32. The note's own coverage table sums the same way — 6 + 13 + 5 + 6 + 1 + 1 = 32. deploy/test-run-guard.sh appears as a row in the table and is then dropped from the total.

The same off-by-one surfaces twice more from the same root:

  • line 49 — "26 shell scripts of which 13 are tests." Fourteen match test-*.sh; 13 is the count for launchpad/scripts alone.
  • line 13 — "13 production shell scripts." 26 − 14 = 12.

Why this blocks rather than files: 31 is the PR title, the frontmatter description, and the denominator of "6 of 31" (line 90), "25 unrun test artifacts" (line 128), and the criterion-8 phrasing. Those become 6 of 32, 26 unrun, 14 shell test scripts. This is exactly the figure that gets lifted into #290 and never re-derived — and #290's criterion 8 is being written from it.

Low — .yaml vs .yml

Lines 13 and 163 say "34 YAML files". The count is right; every one of them is .yml. The label will send the next person grepping for the wrong glob.

What is right — and it is nearly all of it

The number that actually matters, "6 run in CI", is correct, and so is essentially every supporting figure. Verified independently:

  • Extension totals reproduce exactly: 62 py / 26 sh / 34 yml.
  • Production Python by directory reproduces exactly: scripts 11, review-agent 24 (21 + 3 in dimensions/), project-intelligence 6, agents 3 (1 + 2 in the-professor/tools/) = 44; 44 + 18 = 62. The launchpad/scripts shell figure of 16 is right too (14 in the directory + 2 in testdata/).
  • The complete run: inventory across the five launchpad-*.yml workflows reproduces line for line — including the detail that the three discover invocations differ (adr-check bare at :63, pr-check with -t … -v at :100, security-audit narrowed with -p "test_security_audit*.py" at :56). That is the kind of distinction most inventories flatten.
  • No workflow references launchpad/project-intelligence, launchpad/agents, or any launchpad/scripts/test-*.sh. The single launchpad/deploy mention repo-wide is docker.yml:142, and it is a comment — exactly as claimed.
  • The CONTROLS list is verbatim: 13 entries, none a test module.
  • suite.py references none of the five test_*.py modules.

Your "Not checked" list is honest and includes the sharpest question in the whole area — whether discover actually collects all six, given that empty discovery reads as a pass. That one is worth its own issue whatever happens to the count.

The fix here is arithmetic, not research. The research is sound.


Reviewed at head f75085d5c. The recount was run by me directly at that SHA.

🤖 Review drafted by Claude Code (claude-opus-5) for @serina-mcfall.

Signed-off-by: tucktuck101 <jeffreytaylorrobertson@gmail.com>
@tucktuck101 tucktuck101 changed the title docs(research): 31 cohort test artifacts exist, 6 run in CI (#342) docs(research): 32 cohort test artifacts exist, 6 run in CI (#342) Aug 23, 2026
@tucktuck101

Copy link
Copy Markdown
Collaborator Author

Review panel — two independent reviewers (Fable, gpt-5.6-sol xhigh), consolidated

Summary: Both reviewers independently reached the same verdict: changes needed. The research method is sound, the prior 31→32 headline fix was real, and the note's central contribution (the controls-vs-suites distinction in run_controls.py) is accurate — but the base branch moved on 2026-08-24 in a way that falsifies the headline before merge, one line the prior review explicitly flagged survived unfixed, and the note carries no SHA pin, so its counts read as current claims. (Posted as a comment — author is tucktuck101, so this panel cannot formally approve or request changes.)

Blocking

  1. The headline is already wrong on today's base — and the note pins no snapshot. (both reviewers, independently recounted) Merge 8f4293543 (2026-08-24) landed a sixth workflow, .github/workflows/launchpad-agents-tests.yml, which discovers and runs launchpad/agents/test_*.py — so the coverage-table row "launchpad/agents/test_*.py | 1 | no (task: run launchpad/agents Python test suites in CI #270)" is now affirmatively false, launchpad/agents has 2 test files, launchpad/review-agent has 7 (not 5; test_run_adjudication.py, test_verdicts.py), and both reviewers' recounts land at 35 test artifacts, 8 run in CI (unrun: 27 by one recount — recount at the rebase point). "All five launchpad-*.yml workflows" and "CI invokes exactly two things" are likewise stale. Required: refresh every count against the rebase point and add an explicit "as of commit <sha>" pin with the real, un-elided derivation commands — the note's method section currently replaces its central pipeline with ... (:31-35), which defeats the note's own reproducibility pitch. This document is the denominator prd: the cohort test suite — what we test that upstream doesn't #290 will lift; it must be dated evidence, not a silently rotting claim.
  2. The line the prior review explicitly flagged was not fixed. 342-cohort-executable-inventory.md:49 still says "26 shell scripts of which 13 are tests" (it's 14 — git ls-files launchpad | grep -cE 'test-[^/]*\.sh$'; both reviewers re-verified), now contradicting the note's own corrected line 13 (26 − 12 production = 14). The same off-by-one survives in the criterion-8 sentence (:142-144: "13 shell + 12 Python = 25" against the same section's "26 unrun") and in "Not checked" (:154). The dropped artifact is launchpad/deploy/test-run-guard.sh — the entry the note's own security paragraph calls the highest-consequence unrun item, and the criterion-8 sentence is the one proposed for verbatim lift into prd: the cohort test suite — what we test that upstream doesn't #290.

High

  1. The inventory doesn't yet meet task: find out the complete inventory of cohort-owned executables and which have tests in CI #342's per-executable requirement. (codex) Counts are aggregated by directory and the coverage table lists groups, so a reader cannot determine which individual production executables have tests; the 34 Ansible/YAML files are left unclassified. Also, the enumeration misses launchpad/deploy/archived/deploy — a tracked 100755 extensionless Bash entry point — while counting the same archive's .sh and Ansible files; either enumerate it as archived or state and apply an explicit exclusion rule.

Medium

  1. PR body and the standing task: find out the complete inventory of cohort-owned executables and which have tests in CI #342 comment carry the superseded figures (31/25/13; both reviewers). The issue's DoD is the comment; post a correction alongside the refreshed note.

Low

  • :119 pastes 34 yaml output against .yml prose with the command elided — paste the real command and real output. (Fable)
  • :239 "Whether the 13 test-*.sh scripts pass": scope to launchpad/scripts/ or say 14. (Fable)

What looks right (independently verified by both reviewers)

  • The CONTROLS list in launchpad/review-agent/run_controls.py is verbatim, 13 entries, no test module — the controls-vs-suites distinction is the note's real contribution and remains accurate; the new agents workflow doesn't touch it.
  • The six launchpad/scripts/test_*.py files are run by PR CI (.github/workflows/launchpad-pr-check.yml:99-108); no workflow anywhere runs a launchpad/ shell test; the only launchpad/deploy reference in any workflow is a comment in docker.yml:142.
  • The "Not checked" section is honest and unusually sharp (empty-discovery hazard, controls-subsumption question, YAML classification boundary).

Panel: fable + gpt-5.6-sol (codex, xhigh), independent reviews consolidated by the lane; the reviewers converged on the blocking findings from independent recounts. Author is tucktuck101, so this is a comment, not a formal review — needs an external reviewer once addressed. Posted under Jeff's standing authorization.

@serina-mcfall serina-mcfall left a comment

Copy link
Copy Markdown

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Re-review — the 31→32 fix landed; two blockers stand, one of them new

Agent-authored under @serina-mcfall's instruction. Per launchpad/AGENTS.md:165
"Draft everything. Approve nothing." — this raises concerns and clears none.

The original blocker is genuinely fixed. The headline, frontmatter and coverage table all read
32 now, and the frontmatter says "14 shell test scripts". That was the arithmetic, and it is right.

Blocker 1 — the line the previous review named explicitly is still unfixed, and now contradicts the note's own frontmatter

The previous review listed three occurrences of the off-by-one and quoted this one by line number.
It is unchanged:

$ git show origin/research/342-cohort-executable-inventory:launchpad/Research/342-cohort-executable-inventory.md | sed -n '49p'
**Totals: 44 production Python modules, 18 Python test files, 26 shell scripts of which 13 are

My own recount at this PR's head:

$ git ls-tree -r --name-only origin/research/342-cohort-executable-inventory -- launchpad | grep -cE '/test-[^/]*\.sh$'
14
$ ... | sed 's|/[^/]*$||' | sort | uniq -c
      1 launchpad/deploy
     13 launchpad/scripts

14, not 13. The dropped artifact is launchpad/deploy/test-run-guard.sh — the entry the note's own
security paragraph calls the highest-consequence unrun item. Line 49 now says 13 while line 2 says
14, in one document. :154 carries the same unscoped 13 ("the 13 test-*.sh scripts"); :20 is
fine because it scopes to launchpad/scripts/.

Blocker 2 — the note pins no revision, and its counts are already wrong on today's launchpad

The note states its figures in the present tense — "32 test artifacts exist under launchpad/.
Six of them run in CI"
— and I can find no SHA pin or as-of date anywhere in it. Measured at
origin/launchpad today:

$ git ls-tree -r --name-only origin/launchpad -- .github/workflows | grep launchpad
.github/workflows/launchpad-adr-check.yml
.github/workflows/launchpad-agents-tests.yml        <- sixth workflow, not in this branch
.github/workflows/launchpad-issue-check.yml
.github/workflows/launchpad-pr-check.yml
.github/workflows/launchpad-review-agent-controls.yml
.github/workflows/launchpad-security-audit.yml

$ git ls-tree -r --name-only origin/launchpad -- launchpad/review-agent | grep -cE '/test_[^/]*\.py$'
7
$ git ls-tree -r --name-only origin/launchpad -- launchpad/agents | grep -E 'test_.*\.py$'
launchpad/agents/test_goose_config.py
launchpad/agents/test_project_pack.py

$ git ls-tree -r --name-only origin/launchpad -- launchpad | grep -cE '/test_[^/]*\.py$'   # 22
$ git ls-tree -r --name-only origin/launchpad -- launchpad | grep -cE '/test-[^/]*\.sh$'   # 14

So on trunk today it is 36 test artifacts, not 32; launchpad/review-agent has 7 suites,
not 5; launchpad/agents has 2 test files, not 1 — and the coverage-table row
`launchpad/agents/test_*.py` | 1 | **no** | no workflow mentions the directory (#270) is
affirmatively false, because launchpad-agents-tests.yml discovers and runs exactly that directory
(:71 suite = unittest.defaultTestLoader.discover("launchpad/agents", pattern="test_*.py")).

This blocks because the note is the denominator #290's criterion 3 will lift, and it would land
stating a coverage gap that has since been closed. Either fix closes it: add an explicit
"as of commit <sha>" pin and leave the figures as dated evidence, or rebase and re-derive against
the new base. The pin is the cheaper and more durable of the two, and it is the convention the
sibling notes on #405/#413/#374 already adopted.

What is right

The controls-vs-suites distinction in launchpad/review-agent/run_controls.py is the note's real
contribution and it is unaffected by either blocker — the CONTROLS list is verbatim, 13 entries,
none of them a test module. The extension totals, the per-directory production Python breakdown,
the complete run: inventory across the workflows including the three differing discover
invocations, and the observation that no workflow runs any launchpad/ shell test all still hold.
The "Not checked" section remains the sharpest in this batch — the empty-discovery hazard it names
is precisely what the new agents workflow's own guard at :60-71 was written to catch.

The research is sound. It is the pinning discipline and one arithmetic line.


Reviewed at head b3f9475/c812a796e. Every count above was run by me directly at the stated ref, not taken on report.

🤖 Review drafted by Claude Code (claude-opus-5) for @serina-mcfall.

@serina-mcfall

Copy link
Copy Markdown

Correction to the footer of my review just above: the head reviewed is c812a796e. The b3f9475 in that line is a stray from a sibling PR in the same batch (#438) and should be ignored — every command in the review body names its ref explicitly, and those are the ones that were run.

tucktuck101 and others added 2 commits August 24, 2026 13:11
…local

Signed-off-by: tucktuck101 <jeffreytaylorrobertson@gmail.com>
…un in CI (#342)

Re-derives every count at the merged base tip and pins the note to it:
the new launchpad-agents-tests.yml workflow moves the agents suites into
CI, and four new test files land since f75085d. Adds the mechanical
per-executable appendix issue #342's DoD asks for (mode + shebang +
extension + playbooks), which surfaces deploy/archived/deploy; classifies
the 34 .yml files with arguable boundaries recorded as arguable; un-elides
the derivation pipeline; fixes the three surviving 13-vs-14 shell-test
sites and criterion 8 (14 shell + 13 Python = 27 unrun).

Co-Authored-By: Claude Fable 5 <noreply@anthropic.com>
Signed-off-by: tucktuck101 <jeffreytaylorrobertson@gmail.com>
@tucktuck101 tucktuck101 changed the title docs(research): 32 cohort test artifacts exist, 6 run in CI (#342) docs(research): 36 cohort test artifacts exist, 9 run in CI (#342) Aug 24, 2026
@tucktuck101

Copy link
Copy Markdown
Collaborator Author

Re-review requested — pushed afc4dea (base 132f921ac merged in, note fully re-derived). Disposition per finding, covering both @serina-mcfall's review and the panel comment:

Panel blocker 1 (counts stale vs. moved base; no SHA pin; elided pipeline) — fixed, with one correction to the finding itself. The note now opens with an explicit pin to 132f921ace6be6d391c03e134b304ddea3ecccf0 and a history note. The refreshed figures are 36 artifacts / 9 in CI / 27 unrun — not the 35/8/27 in the panel comment, which itself missed scripts/test_pr_review_batch.py (both re-evaluators converged on 36/9 independently). All derived counts refreshed: agents 2 suites now in CI via launchpad-agents-tests.yml, review-agent 7 suites, six workflows, three CI test invocations. The derivation pipeline at the old :32 is now the real command with real output (70 py / 26 sh / 34 yml).

Serina's blocker + panel blocker 2 (surviving 13-vs-14 sites) — fixed. The three sites that survived c812a79 (:49, criterion-8 at :142-143, "Not checked" :154) now read 14 repo-wide / "14 shell + 13 Python = 27". Sorry for the partial first fix — the criterion-8 sentence was exactly the one flagged for verbatim lift into #290.

Panel high (per-executable inventory, extensionless executable) — fixed. New appendix: a 103-row per-file table derived mechanically (extension ∪ git mode 100755 ∪ shebang ∪ Ansible playbook), with test name-match and CI columns — this is what surfaces deploy/archived/deploy (mode 100755, bash shebang, no test, no CI). The 34 .yml files are classified (6 playbooks / 13 tasks-handlers executed-by-inclusion / 15 config-data), with the genuinely arguable boundaries recorded as arguable per #342's own out-of-scope rule rather than ruled on.

Panel medium (PR body + standing #342 comment) — fixed. PR title and body re-derived at the pin (title now "36 … 9"); a superseding correction is posted on #342 (issuecomment-5389629505) with the corrected table and the pinned SHA.

Lows — fixed. The "34 yaml" block was hand-edited output — it's now the real command and real output, and the note states explicitly that all 34 are .yml (zero .yaml exist). The old :154 shell-test bullet now says 14 repo-wide.

One deliberate scope note: the empty-discovery hazard the first revision flagged is now half-closed upstream — launchpad-agents-tests.yml guards by counting collected cases; the note records that the three launchpad/scripts discover invocations still have no such guard.

@serina-mcfall serina-mcfall left a comment

Copy link
Copy Markdown

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Both blockers verified fixed — dismissing the change-requests

Agent-authored under @serina-mcfall's instruction. Per launchpad/AGENTS.md:165"Draft everything. Approve nothing." — this clears the two blockers I raised and does not approve the PR. It still needs an approving review from someone else.

Head reviewed: afc4deaf3.

Blocker 1 — the unscoped 13 — FIXED

The previous review quoted line 49 by number and it was unchanged at that point. It is fixed now:

$ git show pr436chk:launchpad/Research/342-cohort-executable-inventory.md | grep -nE "26 shell scripts of which"
66:**Totals: 48 production Python modules, 22 Python test files, 26 shell scripts of which 14 are

14, not 13. And line 32 now carries the breakdown that makes the number self-checking rather than merely correct:

32:1. **All 14 `test-*.sh` scripts (13 in `launchpad/scripts/`, 1 in `launchpad/deploy/`) are run by

That is the right shape — the 13 was never wrong, it was unscoped, and naming both directories removes the ambiguity instead of just changing a digit.

I also checked the other occurrence the review flagged. No unscoped 13 survives anywhere:

$ git show pr436chk:...342-cohort-executable-inventory.md | grep -nE "13 .test-\*\.sh|which 13"
(no output)

Blocker 2 — no revision pin — FIXED, and the pin holds

Line 8 now states:

All counts in this document are as of commit 132f921ace6be6d391c03e134b304ddea3ecccf0

A pin is only worth having if its numbers reconcile at it, so I checked that rather than the pin's presence:

$ git cat-file -t 132f921ace6be6d391c03e134b304ddea3ecccf0
commit

$ git ls-tree -r --name-only 132f921ac -- launchpad | grep -cE '/test_[^/]*\.py$'
22
$ git ls-tree -r --name-only 132f921ac -- launchpad | grep -cE '/test-[^/]*\.sh$'
14

22 + 14 = 36, exactly what the note claims. The commit exists, and every headline figure derives from it. A pin whose counts did not reconcile would have been worse than no pin at all — this one is sound.

What is genuinely good about this revision

The fix went further than the finding required. The original blocker was an arithmetic error — 31 where the note's own table summed to 32. Rather than patch the digit, this revision re-derived every count at a pinned commit, which is why the figures moved again (32 → 36) and why they now reconcile. That is the difference between correcting a number and fixing the reason it was wrong.

The launchpad/deploy/test-run-guard.sh entry — the one dropped from the total, and the one the note's own security section calls the highest-consequence unrun item — is now counted and named.

What I have not checked

I verified the two blockers and the arithmetic behind them. I did not re-review the rest of the note at this head: the CI-invocation inventory, the coverage table's per-row claims, or the appendix classification. The previous review confirmed those at an earlier head, and the recount has since moved several figures, so a fresh pass over the unchanged sections is not something I can claim to have done.

Also unchecked, and still the sharpest open question in the note's own "not checked" list: whether discover actually collects all the suites it is pointed at — empty discovery reads as a pass.


Dismissing review 5003891553 and 4998595585. This is not an approval — reviewDecision returns to REVIEW_REQUIRED and a human still has to sign off.

🤖 Review drafted by Claude Code (claude-opus-5) for @serina-mcfall.

@serina-mcfall
serina-mcfall dismissed stale reviews from themself August 24, 2026 02:29

Both blockers verified fixed at head afc4dea. The unscoped 13 is now 14 with a 13-in-scripts/1-in-deploy breakdown at line 32, and a revision pin exists at line 8 whose counts reconcile: 22 test_.py + 14 test-.sh = 36 at commit 132f921. Evidence in the review comment. Not an approval.

@serina-mcfall serina-mcfall left a comment

Copy link
Copy Markdown

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Approved. Both blockers verified fixed; counts reconcile at the pinned commit.

@serina-mcfall
serina-mcfall merged commit abb3c4b into launchpad Aug 24, 2026
24 checks passed
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

by:agent Filed or authored by an AI agent, not a human

Projects

None yet

Development

Successfully merging this pull request may close these issues.

task: find out the complete inventory of cohort-owned executables and which have tests in CI

2 participants