compass(tests): delete the tests that assert prose

[jgong5]

Dev record for #490 (implements the ruling in #489: prose is not a test subject).

Deletions only under tests/compass/. No production code, doc or script is touched. Two structural assertions that shared a test with a prose check were split into their own small tests. One source-only test moved out of a deleted file. Every symbol listed in the brief was re-checked at 42e4aa324 before deletion.

What was deleted, per file

File	Deleted	Kept or moved
test_model_runner_citations.py	whole file	nothing (every test parses the design docs' citations)
test_design_rpc_reply_contract.py	whole file: audit, stated, unpack_site, REVERTED, the module-level DOCUMENT read, and every test that runs the paragraph through audit	test_the_two_claims_the_paragraph_makes_without_a_number reads only async_proc.py and engine_core.py. It moves to test_runner_rpc_surface.py as test_the_worker_cannot_break_on_an_unpack_and_the_wait_is_unbounded, with the same assertions
test_tier0_memory_goal_cites_empirical.py	whole file	nothing
test_kv_blob_doc_table.py → test_kv_blob_site.py	stated, _split_table, _doc_rows, assert_the_table_lists_what_the_backend_emits, _derived_keys, DOC, HEADER, _FIELD, _CITE, and the four doc-table tests	test_each_backend_builds_the_blob_at_exactly_one_site and _blob_site. The file is renamed because it no longer has anything to do with a doc table. The line-number citations in _blob_site's docstring are removed too
test_sync_inventory.py	COUNT_TABLES, _stated_counts, test_every_stated_count_matches_the_rows, unused Path import	rest unchanged
test_cpu_gate_exclude.py	README, test_the_readme_states_the_counts_the_list_holds, unused re import	rest unchanged
test_gate_gpu_surplus.py	README, _base, test_the_readme_states_the_baseline_the_gate_holds, test_gate_gpu_states_the_expectation_for_a_tree_without_compass_tests, unused re import	test_gate_gpu_derives_the_surplus_through_the_helper (code structure of gate_gpu.sh)
test_pipeline_minimum.py	DESIGN, test_the_design_document_states_the_table_and_widths_that_were_derived, and the docstring sentence that described it	rest unchanged
test_pp_kv_shard_key.py	REPO, DESIGN, test_the_document_states_the_split_that_was_derived, and the docstring sentence that described it	that test's last assertion checks config data, not prose (num_hidden_layers // full_attention_interval equals the paged count from layer_types). It is kept as test_the_interval_count_agrees_with_the_layer_kinds
test_artifact_invalidation.py	REPO, MATRIX_DOC, MATRIX_HEADING, CELL, test_the_matrix_is_the_documents_table, the docstring paragraph in test_every_cell_of_the_matrix_decides_by_itself that pointed at it, unused pathlib/re/Cell imports	rest unchanged
test_artifact_store.py	REPO, KEY_TABLE_DOC, ROW, words, test_the_six_artifacts_are_the_ones_the_key_table_declares, unused re import	its code-only assertion set(KEY_FIELDS) == set(Kind) is kept as test_every_kind_declares_its_key_fields
test_runner_non_allocating.py	both ast.get_docstring(runner) assertions	the code half of test_the_docstring_names_every_attribute_that_holds_the_ring is kept, renamed test_exactly_two_attributes_of_the_runner_hold_the_ring. test_the_overrides_bind_no_attribute_that_could_hold_a_tensor keeps its binding assertion
test_runner_rpc_surface.py	PACKAGE_DOC, UNWAITED, _refusal_comment, CITATION, COUNT_WORDS, _bullets, _modules, _stated_module_counts, _cite, _imported, _engine_imports, unused re/resolve_name imports, and the eight tests the brief lists by name	test_what_the_comment_says_a_hole_at_exit_loses_is_what_exit_does keeps its ModelRunner.exit AST assertions, renamed test_what_a_hole_at_exit_loses_is_what_exit_does. test_the_unanswered_helper_describes_its_whole_return_and_not_one_half keeps its single-caller and partition assertions, renamed test_the_unanswered_helper_returns_one_list_its_one_caller_partitions
test_memory_compare.py	_TAGS, PACKAGE, test_no_module_in_the_package_carries_a_design_reference, test_the_guard_catches_the_forms_that_were_actually_removed, test_what_the_pattern_matches_beside_its_targets_is_on_record, unused pathlib/re/memory_package imports	rest unchanged

Listed in the brief but kept, and why

• Nothing the brief lists was kept whole. Three listed tests contained assertions about config data or code structure, not prose. Those assertions survive under new names, as the table shows: test_pp_kv_shard_key.py's interval check, test_artifact_store.py's KEY_FIELDS check, and test_design_rpc_reply_contract.py's source-only test.

Missed by the brief

• None deleted. I swept tests/compass/ for other reads of source text, comments and docs. Two judgement calls stay, for the reviewer to confirm:
  • test_cpu_gate_exclude.py::test_every_manual_entry_states_why checks that each MANUAL entry in cpu_gate_exclude.txt has a # line above it. It checks that a reason exists, not what the reason says, in a checked-in data file the gate reads. Kept.
  • test_runner_rpc_surface.py::test_the_profiler_replies_are_forwarded_whole_and_never_unpacked scans source for the string trace_dir, which matches comments too. Its subject is which code produces the reply's keys. Kept.

Named result

(1) The brief's grep at the head (grep -rnE '\.md\b|__doc__|get_docstring' tests/compass) matches three lines:

• test_gate_cpu_pipe_identity.py, module docstring and DOC: the string scripts/compass/README.md is asserted to appear in gate_cpu.sh's rendered failure output. The test runs the gate's printf block through bash and never opens the README. It tests the gate script's output, so it is a behaviour test under docs(compass): prose is not a test subject; no line numbers or item counts; the kind of claim decides a doc-code disagreement #489. It stays.
• test_capture_real_model.py, argparse.ArgumentParser(description=__doc__): a script entry point's help text. Nothing asserts on it. It stays.

(2) scripts/compass/gate_cpu.sh on node 18 (container xiaobizh_n18_cpu). Each side was a git archive snapshot with .compass-commit/.compass-changed stamps. PYTHONPATH pointed at its own root, and atom.__file__ was verified under it. The two sides ran one after the other.

Side	Commit (stamp printed)	Result	rc
control	42e4aa324	5281 passed, 155 skipped, 3 xfailed	0
branch	154a6114c	5207 passed, 155 skipped, 3 xfailed	0

Difference: 74 passed. That matches a collect-only diff of the touched files on the two snapshots: 76 nodes deleted outright and 2 nodes added (the two split-out assertions above), a net of 74. Renamed and moved tests count on both sides and cancel out. Per file: model_runner_citations 16, design_rpc_reply_contract 11 (one of which moved to rpc_surface), tier0 11, kv_blob 8, runner_rpc_surface 9, memory_compare 8, cpu_gate_exclude 4, gate_gpu_surplus 4, sync_inventory 2, pipeline_minimum 1, pp_kv_shard_key 1, artifact_invalidation 1, artifact_store 1. The skipped and xfailed counts are unchanged, and no flake re-run was needed.

ruff check and black --check are clean on every touched file.

Left undone (filed as #491)

Out of this PR's file set: atom/compass/artifacts/matrix.py's module docstring still names the deleted test_the_matrix_is_the_documents_table. Design docs 01 and 03 still say a test holds their tables. sync_scan.category_counts now has no caller.

Closes #490

🤖 Generated with Claude Code

[jgong5]

This review is agent-authored.

Review cycle 1

Verdict: APPROVE

Head covered: 154a6114c42c82a370aba9721ef6139c2ff81702 (base feature/atomcompass_new at 42e4aa324, which is an ancestor of the head).

Did any deletion remove a behaviour or code-structure assertion?

No. I read every deleted and edited test at 42e4aa324 and sorted each assertion into prose (reads a .md, a README, a docstring or a comment to check its text) or code (runs code, walks an AST, reads a data file or script output). Every code assertion still holds at the head:

Code assertion inside a deleted test (base)	Where it holds at the head
test_design_rpc_reply_contract.py::unpack_site: capture_cudagraph is unpacked with one arity, at one EngineCore site	test_runner_rpc_surface.py::test_capture_cudagraph_answers_the_arity_its_call_sites_unpack (arities == {3}, two sites in the tree)
test_design_rpc_reply_contract.py::test_the_two_claims_the_paragraph_makes_without_a_number	moved without changes to test_runner_rpc_surface.py::test_the_worker_cannot_break_on_an_unpack_and_the_wait_is_unbounded
test_runner_rpc_surface.py: no site of an unwaited name passes wait_out=True; UNWAITED is non-empty and is not the whole table	test_each_name_waits_exactly_as_its_own_call_sites_say (every name, both directions) and test_the_two_names_no_caller_waits_for_and_what_replying_costs (the exact set)
test_runner_rpc_surface.py: ModelRunner.exit AST checks; unanswered_rpc_names has one caller, and that caller partitions its result	kept, renamed test_what_a_hole_at_exit_loses_is_what_exit_does and test_the_unanswered_helper_returns_one_list_its_one_caller_partitions
test_runner_non_allocating.py: the two ring holders, and the three override bindings	kept (the first renamed test_exactly_two_attributes_of_the_runner_hold_the_ring)
test_artifact_invalidation.py::test_the_matrix_is_the_documents_table: implied that MATRIX has every Row × Axis cell	test_every_cell_of_the_matrix_decides_by_itself is parametrised over Row × Axis and indexes MATRIX[row][axis], so a missing cell raises KeyError
test_artifact_store.py: set(KEY_FIELDS) == set(Kind)	salvaged as test_every_kind_declares_its_key_fields
test_pp_kv_shard_key.py: num_hidden_layers // full_attention_interval equals the paged count	salvaged as test_the_interval_count_agrees_with_the_layer_kinds
test_kv_blob_doc_table.py: one blob site per connector	kept in test_kv_blob_site.py

Each other deleted assertion compares code or data against prose: a doc table, a README row, a gate comment, a docstring, or a citation's line span. The full list of deleted node ids is at the end.

Two deletions are worth recording. Neither blocks:

• test_kv_blob_doc_table.py::_derived_keys asserted that each blob is a dict literal with string-literal keys. It existed only to derive the field list that was then compared with the doc table (that file's own docstring: "This guards the document, not the connector"). Nothing in Compass reads the blob's field set from source, so no behaviour is lost.
• test_memory_compare.py's design-reference guard scanned code, including runtime refusal strings, rather than prose. The brief deletes it on purpose: AI_DEV_RULES.md ("No design-doc references in code or runtime output ... Check at the head") makes this a review check. The PR body lists the deletion without giving that reason, but the reason is in compass(tests): delete the tests that assert prose (#489) #490. I ran the guard's own pattern over atom/compass/memory/*.py at the head, as that rule asks, and it finds nothing.

Salvaged and renamed pins: mutation evidence

Each mutation changes one line and keeps the line count, and the runs were serial against the detached worktree. Every file was restored afterwards, and git status --porcelain came back empty. import atom resolved to the worktree. Published: 139 passed across the five affected files.

Mutation (line count kept)	Result
null control: # null appended to each mutated line in moriio, _fv_ring, busy_loop and exit	2 / 31 / 62 / 62 passed (all green)
moriio: a second seq.kv_transfer_params_output = None in place of the comment above the site	1 failed / 1 passed: test_each_backend_builds_the_blob_at_exactly_one_site[moriio] ("at [977, 983]")
moriio: the site's attribute renamed	1 failed / 1 passed, same node ("at []")
ModelRunner: a third ring holder (self._fv_rinG = ...)	1 failed / 30 passed: test_exactly_two_attributes_of_the_runner_hold_the_ring
NonAllocatingRunner: an extra binding (self.kv_pool_sizinG)	1 failed / 30 passed: test_the_overrides_bind_no_attribute_that_could_hold_a_tensor
busy_loop: out, = func(*args)	1 failed / 61 passed: test_the_worker_cannot_break_on_an_unpack_and_the_wait_is_unbounded
call_func: outputs_queue.get(timeout=60)	2 failed / 60 passed: the moved test and test_the_caller_that_waits_has_no_timeout_to_rescue_it
engine_core: AsyncIOProcManagerX(	1 failed / 61 passed: the moved test, on its third assertion
ModelRunner.exit: empty_cache() → synchronize() / del self.model → self.model = None / KV if hasattr(...) → if True:	1 failed / 61 passed each: test_what_a_hole_at_exit_loses_is_what_exit_does
composed runner: _UNREAD's not dropped	2 failed / 60 passed: test_the_unanswered_helper_returns_one_list_its_one_caller_partitions and test_the_binding_module_refuses_rather_than_composing_a_hole
KEY_FIELDS: MACHINE_SPEC, SHAPE_POPULATION or REGION_TERMS commented out	1 failed / 39 passed each: only test_every_kind_declares_its_key_fields
config fixture: full_attention_interval 4 → 3	1 failed / 3 passed: test_the_interval_count_agrees_with_the_layer_kinds (64 // 3 == 16)

One mutant was thrown away: commenting out Kind.OP_GRAPH (and PRICE_LIST) broke collection with a KeyError from graph_key at module scope in test_artifact_store.py. That is red for the wrong reason, so I used the three kinds that reach the test instead. None of the pins is inert.

Other checks

• File set: git diff --stat 42e4aa324 154a6114c touches 15 files, all under tests/compass/.
• Dead helpers, doc-reading constants: none left. ruff --select F is clean on every touched file. The ARG hits match the base exactly, and nothing in the touched files still opens a .md at import. No deleted test name or file is listed in scripts/compass/*.txt|*.sh. The prose that still names the deleted tests (matrix.py, docs 01 and 03) and the helper sync_scan.category_counts, which no longer has a caller, are filed as compass: cite code by symbol, not line number, and drop the prose and helper left over from the deleted prose tests #491.
• Brief grep: the only matches are test_gate_cpu_pipe_identity.py (asserts the gate script's rendered output, which is behaviour) and argparse(description=__doc__) (nothing asserts on it). I agree both stay.
• Gate numbers: I did not re-run the gate. --collect-only over the touched files gives 453 nodes at the base and 379 at the head, net 74. That is 82 ids gone and 8 new, where 6 of the new ones are renames or the move and 2 are the salvages. 5281 − 74 = 5207, which matches the PR body.

The two borderline tests flagged in the PR

• test_cpu_gate_exclude.py::test_every_manual_entry_states_why is not a prose test. It checks a format rule of a checked-in data file that the gate reads: each MANUAL entry has a # line above it. It never reads what the reason says, and no doc edit can turn it red. Keep it.
• test_runner_rpc_surface.py::test_the_profiler_replies_are_forwarded_whole_and_never_unpacked (the trace_dir scan) is not a prose test. Its subject is code: which modules touch the reply's keys. It asserts nothing about what any comment says. The one catch is that it is a raw text match, so a comment mentioning trace_dir in model_engine/ or compass/runner/ would turn it red. test_kv_simulated_connector.py's scan for forbidden clock names works the same way. Keep both. If either ever goes red on a comment, the fix is to narrow it to AST nodes, not to reword the comment.

Non-blocking

• test_runner_rpc_surface.py still cites file.py:NNN line numbers in docstrings the PR did not touch: engine_core.py:749, pp_engine_core.py:118, :379, llm_engine.py:300, block_manager.py:116-121, scheduler.py:1364-1376. They were there before this PR, and compass(tests): delete the tests that assert prose (#489) #490 leaves prose cleanup to follow compass(design): revise the design documents to the PDES time model #443. They could be added to compass: cite code by symbol, not line number, and drop the prose and helper left over from the deleted prose tests #491.
• The PR body's per-file counts add up to 77, not 76, because the design_rpc_reply_contract figure (11) includes the test that moved. The net of 74 is correct.

ponytail-review

The diff is 116 insertions and 1572 deletions. Every added line is a surviving test or a rename. There are no new helpers or abstractions.

Lean already. Ship.

Deleted node ids (base collect-only minus head, renamed ones excluded)

model_runner_citations 16, design_rpc_reply_contract 10 (plus 1 moved), tier0_memory_goal 11, kv_blob_doc_table 8, runner_rpc_surface 9, memory_compare 8, cpu_gate_exclude 4, gate_gpu_surplus 4, sync_inventory 2, pipeline_minimum 1, pp_kv_shard_key 1, artifact_invalidation 1, artifact_store 1. Total 76 deleted outright, 2 salvaged.