compass(gates): join the README's exclude-list counts to the list

[jgong5]

Closes #213

What changed

tests/compass/test_cpu_gate_exclude.py gets one test, test_the_readme_states_the_counts_the_list_holds, parametrised as [total], [generated] and [manual]. Each case reads its number from the cpu_gate_exclude.txt row of scripts/compass/README.md and asserts that it equals the count derived from the list. On a disagreement the message names the README, the stated number, the list and the entries it holds.

A missing row or a reworded number also fails by name. It does not pass vacuously.

Nothing under scripts/compass/ changes. The directory's tree object is ba78568c8 at both the tip and the head, so the gate below is the same instrument on both sides.

Lines: 0 production, 28 test (+28/-0). The estimate was ~35.

Staleness check at f84b4d778

• The hold this issue described has lifted: gate_gpu.sh resolves aiter by importing it, which hangs on a wedged driver #166 and A gate piped anywhere loses its own exit code; a run that exited 98 reports rc=0 #191 landed as compass(gates): locate aiter for gate_gpu.sh without importing it #271 and compass(gates): gate_cpu.sh verdict line carries its reason through a pipe #276.
• scripts/compass/README.md:20 still restates 29 / 28 GENERATED / 1 MANUAL, and nothing joins it to the list.
• All three numbers are still right. The list has 29 path lines: 28 between the GENERATED markers and 1 between the MANUAL markers.

The choice: assert the README against the list, rather than stop restating

gate_gpu.sh:111-122 settles the shape. It keeps BASE_FAILED=5 as a stated number, counts gpu_gate_known_failures.txt, and on a disagreement refuses, naming both. The README's own row for that file describes the same shape: "two statements of one fact; gate_gpu.sh refuses to run if they disagree". This PR copies that shape instead of inventing a second one.

On simplicity: dropping the numbers from the README would be fewer lines. But the row's 29 is the size of the CPU tier's blind spot, and that figure is the reason a reader opens the row. That is also why BASE_FAILED is kept beside its list rather than removed.

The join also costs nothing the gate executes. The alternative, editing the README, would have moved the scripts/compass tree object, and that object is the instrument every open gate delta is measured with.

The existing MANUAL pin (== 1) is kept as it is. It is the in-file statement that separates a chosen empty section from one that arrived, and the join does not replace it.

Named result: node 18, xiaobizh_n18_cpu, one module per tree

Each mutation is a line-count-preserving edit built with git commit-tree, so every run names a commit. atom.__file__ resolved under each staged root in every run.

tree	edit	result
tip f84b4d778 (null control)	none	70 passed, rc=0
head 3fb992d2c (null control)	none	73 passed, rc=0; the 3 new ids pass
678c85a2a = tip + manual entry	cpu_gate_exclude.txt:49, a comment line becomes tests/test_gc_utils.py	1 failed: test_the_manual_section_holds_the_entries_it_is_pinned_to_hold. The README stays wrong in silence.
c17909d47 = head + the same entry	same	3 failed: the pin, test_the_readme_states_the_counts_the_list_holds[total] and test_the_readme_states_the_counts_the_list_holds[manual]
197c307ec = tip + README says 2 MANUAL	README.md:20, **1 MANUAL** becomes **2 MANUAL**	70 passed, rc=0. Silent.
038cee2c3 = head + README says 2 MANUAL	same	1 failed: test_the_readme_states_the_counts_the_list_holds[manual]

The join's assertion text at c17909d47, [manual]:

AssertionError: README.md states **1 MANUAL** but cpu_gate_exclude.txt holds 2: ['tests/test_gc_utils.py', 'tests/test_lmcache_offload_disk_integration.py']. Update the README row to match the list.

The added entry was chosen to break nothing else. tests/test_gc_utils.py exists, sorts before the existing entry, keeps a comment line directly above it, and is not in the GENERATED section. Only the counts move.

I also fired the two refusal paths on the head tree, both with line-count-preserving README edits:

• **29** excluded reworded to 29 excluded: 1 failed, [total], with "README.md's cpu_gate_exclude.txt row no longer states /…/".
• The row indented so it no longer starts with the list's cell: 3 failed, all three ids, each with "README.md has 0 rows for cpu_gate_exclude.txt".

One note from the brief, stated rather than left to happen quietly: this PR does not raise the MANUAL count, so test_the_manual_section_is_sorted_and_unique is still trivially true at n=1. At n=2 in the mutation above it had signal and passed, because the added entry was placed in sorted order.

Gate 1: ATOM's suite, unmodified, as a delta against a measured control

Node 18, xiaobizh_n18_cpu. Each tree was staged with git archive and docker exec -i … tar -x into /tmp/i213gates/<side>/ATOM, with .compass-commit / .compass-changed written from the same sha. Content digests matched on both ends. Each tree ran its own scripts/compass/gate_cpu.sh, unpiped, under timeout -k 10, one at a time.

side	commit stamp	result	GATE_CPU_RC
control f84b4d778	f84b4d778 (stamp), gpu: not required	5214 passed, 155 skipped, 3 xfailed	0
head 3fb992d2c	3fb992d2c (stamp), gpu: not required	5217 passed, 155 skipped, 3 xfailed	0

Node-id delta (from --junitxml): 0 ids only in the control, and 3 only in the head, all passed:

• tests/compass/test_cpu_gate_exclude.py::test_the_readme_states_the_counts_the_list_holds[total]
• …[generated]
• …[manual]

No flake-listed test moved.

git merge-tree --write-tree f84b4d778 3fb992d2c gives 65f9f1fedbcd067c9f7d375cabc8820f3aa8434a, which equals 3fb992d2c^{tree}.

Gate 2: CPU-only tests

The three ids above run in the CPU tier. They exercise scripts/compass/README.md and scripts/compass/cpu_gate_exclude.txt, neither of which this PR adds or edits. ruff check and ruff format --check on the file: clean.

Not done, and left for a successor (filed as #311)

• scripts/compass/README.md restates other counts with no join: 30 trigger paths on line 21 (still right at the tip), and 4779 passed / 5 failed plus "five" known failures on lines 15 and 22.
  • These are outside this brief's file set, so they are not joined here.
  • The trigger count carries the same risk as this issue: regen_gpu_gate_triggers.sh rewrites the file wholesale, and the README would not follow.
• atom/compass/design/16_execution_plan.md:171 also says "29 entries are 28 GENERATED + 1 MANUAL". It is a dated record of what one commit produced ("From this commit…"), not a live statement, so it is not joined.

🤖 Generated with Claude Code

[jgong5]

Non-blocking (principles 3 and 8). The PR body says the MANUAL pin (test_the_manual_section_holds_the_entries_it_is_pinned_to_hold, L147-168) is kept because "the join does not replace it". I measured that claim, and it does not hold. The pin's stated job is to stop an emptied MANUAL section from passing silently, with its parametrisation collecting nothing. This join does that job already.

tree	edit (line count preserved)	result, tests/compass/test_cpu_gate_exclude.py only
a821914ad = head + MANUAL emptied	cpu_gate_exclude.txt:51 commented out	3 failed, 68 passed, 1 skipped: the pin, [total], [manual]
a833627b9 = the same + pin neutralised	also assert len(entries) == 1 -> >= 0	2 failed, 69 passed, 1 skipped: test_the_readme_states_the_counts_the_list_holds[total] and [manual] (states **1 MANUAL** but ... holds 0: [])

After this PR, the MANUAL count is stated twice by hand: **1 MANUAL** in the README and == 1 in the pin. Both are joined to the list, and every edit that fails one fails the other. Reaching zero on purpose now takes two edits instead of one.

Either choice is fine:

• Delete the pin (about -24 lines). Then point the control test's docstring (L172) and the module docstring (L28-30) at this join.
• Keep the pin with a reason that holds. The only real difference is locality: the pin's number sits in the file the list is tested from. The PR body's sentence should then say that, not that the join cannot replace it.

[jgong5]

Non-blocking (principle 6). This sentence picks a side, which the comment at L206 says the join does not do ("a disagreement names both rather than picking one"). gate_gpu.sh's message is the model this test copies, and it only states the two numbers: BASE_FAILED=%s but %s names %s node-id(s).

The side it picks is wrong in exactly the case the MANUAL pin exists for. Emptying the MANUAL section by accident (mutation a821914ad: cpu_gate_exclude.txt:51 commented out) fails [manual] with README.md states **1 MANUAL** but cpu_gate_exclude.txt holds 0: []. Update the README row to match the list. Here the list is the wrong side, and the message tells the reader to make the README agree with it. For GENERATED the list usually is right, because a regenerator wrote it. For MANUAL it is a hand edit, and either side can be the mistake.

Suggested fix: end the message at {entries}., or replace the last sentence with a neutral one such as One of the two is wrong; fix that one. This matters more if the pin goes (see the comment on L202), because the pin's "Add or remove one deliberately" wording is then no longer printed beside it.

[jgong5]

This review is agent-authored. I read the eight design principles in atom/compass/design/README.md and atom/compass/AI_DEV_RULES.md before reading the diff.

Verdict: APPROVE 3fb992d2ce15c7cb78f2a3972c54ea4de18dad16. There are no blocking issues.

• The named result reproduces by node id.
• Every refusal path fires.
• Two mechanisms of my own also redden the join. One of them, a stray entry outside the markers, is caught only by [total].
• The tree that will land gates green on node 18.

There are two inline comments, both non-blocking:

• L202: the MANUAL pin is now redundant, measured. The PR body's claim that "the join does not replace it" does not hold.
• L214: the message's last sentence picks the list's side, so it points the wrong way when a MANUAL entry is lost by accident.

Rulings

1. Assert rather than remove (principle 3): keeping the numbers and pinning them is the right call.

• The brief allowed either option and said to match gate_gpu.sh's shape. That shape keeps the stated number and refuses on a disagreement (gate_gpu.sh:111-122).
• The numbers are why a reader opens the row: 29 files the CPU tier cannot see, split by provenance.
• The cost is 28 test lines and no new statement of the fact. The test reads the README's number; it adds no literal of its own.
• The join does more than restate the pin:
  • [generated] reddens on a regenerator shrinking the list (f9ec8f500). Nothing else in the tree catches that.
  • [total] catches entries outside both markers (60fbd4cdf). Nothing else in the tree catches those either.
• So "stop restating" would have been fewer lines, and it would have given up the only check on two of the three numbers.
• The developer's second reason is secondary but true: a README edit moves scripts/compass off tree ba78568c8.

3. Parser robustness (principle 6): it refuses on every format change I tried, and never falls back.

format change	result
number reworded	[total] fails: "no longer states /…/"
row indented (no longer found)	all three fail: "has 0 rows"
row duplicated	all three fail: "has 2 rows"

• A spelled-out number cannot match \d+, so it refuses too.
• The one soft spot is not worth a fix: re.search takes the first match within the row. That only matters if the row ever carries two **N MANUAL** phrases.
• The row key is the table cell's exact prefix. The other rows that mention the file do not start with it.

Named result and mutations

Node 18, xiaobizh_n18_cpu. Each tree is a git commit-tree object built from a line-count-preserving edit (git diff --numstat 1/1 on each file). Each was staged with git archive + docker exec -i … tar -x into /tmp/pr310rv/<name>/ATOM, and the tarball md5 matched on both ends. atom.__file__ resolved under each staged root.

Each run covered tests/compass/test_cpu_gate_exclude.py only, under timeout -k 10 600. Node ids are under that file; [x] is short for test_the_readme_states_the_counts_the_list_holds[x].

#	tree	edit	result
N1	head 3fb992d2c (null)	none	73 passed, rc=0
N2	tip fbfc37ef8 (null)	none	70 passed, rc=0
A	5f37da4ca = tip + manual entry	cpu_gate_exclude.txt:49 becomes tests/test_gc_utils.py	1 failed: test_the_manual_section_holds_the_entries_it_is_pinned_to_hold. The README stays silently stale.
A′	1b5ec0937 = head + the same edit	same	3 failed: the pin, [total] (states **29** excluded test files but … holds 30) and [manual] (states **1 MANUAL** but … holds 2: ['tests/test_gc_utils.py', 'tests/test_lmcache_offload_disk_integration.py'])
B	6d4f571ba = tip + README "2 MANUAL"	README.md:20	70 passed. Silent.
B′	60f00f51c = head + the same edit	same	1 failed: [manual] (states **2 MANUAL** but … holds 1)
R1	deee19dc1 = head, **29** excluded becomes 29 excluded	README.md:20	1 failed: [total], README.md's cpu_gate_exclude.txt row no longer states /\*\*(\d+)\*\* excluded test files/
R2	b76e7f897 = head, row indented two spaces	README.md:20	3 failed: [total], [generated], [manual], each README.md has 0 rows for cpu_gate_exclude.txt
M1	652a62b23 = head, row duplicated over the triggers row	README.md:21	3 failed, each has 2 rows
M2	39550dd98 = tip, first GENERATED entry commented out (a regenerator shrinking the list)	cpu_gate_exclude.txt:18	69 passed. Silent.
M2′	f9ec8f500 = head, the same edit	same	2 failed: [total] (29 vs 28) and [generated] (**28 GENERATED** vs 27). The pin does not move.
M3	f14ac226e = tip, stray entry outside both markers	cpu_gate_exclude.txt:16 (a header comment) becomes tests/test_gc_utils.py	71 passed. Silent. gate_cpu.sh would exclude a 30th file that nothing documents.
M3′	60fbd4cdf = head, the same edit	same	1 failed: [total] alone. So [total] is not the sum of the other two cases. It is the only guard on the file-wide count that gate_cpu.sh actually reads (principle 7).
M4	a821914ad = head, MANUAL emptied	cpu_gate_exclude.txt:51 commented out	3 failed, 68 passed, 1 skipped: the pin, [total], [manual]
M4′	a833627b9 = M4 + the pin neutralised (== 1 becomes >= 0)	also test_cpu_gate_exclude.py:164	2 failed: [total] and [manual]. The join alone catches the emptied section; see the inline comment at L202.

M1-M4 are my own. The developer's table reproduces exactly, with my own commit objects.

Gate: the tree that will land

• The tip moved from f84b4d778 to fbfc37ef8 (compass(tests): hold four refusals the artifact store states and nothing reached #306, which touches only tests/compass/test_artifact_invalidation.py).
• git merge-tree --write-tree fbfc37ef8 3fb992d2c gives 150a2a0e5cf544601997ee511f02296eadd6b797. That is no longer 3fb992d2c^{tree} (65f9f1fed), so the developer's gate does not stand, and I gated this tree once.
• Stamp commit: git commit-tree 150a2a0e5 -p fbfc37ef8 -p 3fb992d2c = 8b8eca71d8bd476fd328ab99e01bb5fb0ebba82e.
• .compass-changed is tests/compass/test_cpu_gate_exclude.py (diff against the tip).
• scripts/compass is tree ba78568c8 at the tip, the head and the merge, so the gate is the same instrument on every side.

tree	atom.__file__	gate_cpu.sh (the tree's own, unpiped, timeout -k 10 1800)	verdict
merged 8b8eca71d	/tmp/pr310rv/merged/ATOM/atom/__init__.py	commit: 8b8eca71d (stamp), gpu: not required, 5225 passed, 155 skipped, 3 xfailed (190 s)	GATE_CPU_RC=0 PASSED

5225 is the number the history predicts:

• 5214 at the developer's control f84b4d778;
• +8 from compass(tests): hold four refusals the artifact store states and nothing reached #306 (its own gate: 5206 to 5214);
• +3 from this PR.

The three new ids are in the merged run's --junitxml. No flake-listed test failed, so no re-run was needed.

ruff check and ruff format --check on the file, in the merged tree on node 18 (ruff 0.16.7): RUFF_RC=0, FMT_RC=0.

Other checks

• Design-doc references: none in tests/compass/test_cpu_gate_exclude.py at the head. I checked the whole file, not only the added lines.
• Gate 2 is met: the new test reads two files this PR does not add or edit.
• One more unjoined restatement of these same three numbers, which neither this PR nor compass(gates): join the README's other restated gate counts (triggers, GPU baseline) to their sources #311 names.
  • scripts/compass/gate_cpu.sh:9-11 says "29 files that reach the driver: 28 at collection time … plus 1". It goes silently stale under mutation A′ exactly as the README did before this PR.
  • It is outside this brief's file set, so it does not block. I am adding it to compass(gates): join the README's other restated gate counts (triggers, GPU baseline) to their sources #311 as a comment.

ponytail-review

tests/compass/test_cpu_gate_exclude.py:L147-168: delete: the MANUAL `== 1` pin; `[manual]` now reddens every edit it does (M4′: 2 failed with the pin neutralised). Point the control's docstring (L172) and the module docstring (L28-30) at the README join instead.

The added 28 lines themselves are the minimum: one parametrised test, a row lookup, a regex, and an assert. The three cases are worth their node ids, since M2′ and M3′ each redden a different subset.

net: -24 lines possible.