compass(memory): drive two spellings of one phase at both shape checks - #298
Conversation
Shape's docstring says phases compare exactly, so two spellings of one phase refuse each other. The only test of a phase difference drove decode against prefill, which differ in content, so a case-insensitive phase comparison at either shape check in compare() left the suite green. Parametrize that test over a second phase pair, "decode" against "Decode", at both the outright refusal and the per-term refusal. compare.py is unchanged: phases stay case-sensitive. Closes #296 Co-Authored-By: Claude Opus 5.5 (1M context) <noreply@anthropic.com>
| def test_a_decode_side_refuses_a_prefill_side_at_the_same_token_count(at_shape): | ||
| @pytest.mark.parametrize( | ||
| "mine, theirs", | ||
| [("prefill", "decode"), ("decode", "Decode")], |
There was a problem hiding this comment.
Non-blocking (principle 8, evidence matches the claim). The dev record's evidence for two spellings is ATOM's SequenceType, whose .name is "DECODE" (all upper case, atom/model_engine/sequence.py, members are auto(), so .value is an int and .name is the only string). This pair uses "Decode", which nothing in the tree produces. Every case-insensitive mechanism I tried catches both, so this does not weaken the pin, but ("decode", "DECODE") would drive the exact pair the record names as plausible. Optional.
| [("prefill", "decode"), ("decode", "Decode")], | ||
| ids=["prefill-decode", "two-spellings"], | ||
| ) | ||
| def test_a_decode_side_refuses_a_prefill_side_at_the_same_token_count( |
There was a problem hiding this comment.
Non-blocking (principle 4). The test name now overstates two of its four cases: [two-spellings-*] has no prefill side, and the recorded side's run= label and the local name decode still say "decode step" when theirs is the variable. The added comment line explains it, so this reads fine; a rename (for example ..._refuses_another_phase_at_the_same_token_count) would rename the node ids again, and nothing in the tree selects them. Optional.
| @pytest.mark.parametrize( | ||
| "mine, theirs", | ||
| [("prefill", "decode"), ("decode", "Decode")], | ||
| ids=["prefill-decode", "two-spellings"], |
There was a problem hiding this comment.
ponytail-review — L646-650: shrink: drop ids=; pytest's generated ids for these tuples are prefill-decode and decode-Decode, and without it the decorator fits one 87-column line: @pytest.mark.parametrize("mine, theirs", [("prefill", "decode"), ("decode", "Decode")]). -4 lines. Non-blocking: it trades the word two-spellings for the literal spellings in the node id.
|
Review 1 — PR #298 (issue #296), head Agent-authored review. Read the eight design principles in Verdict: APPROVE at 1. Design ruling (principle 6): case-sensitive is the right direction
2. Named result, reproduced (principle 8)Setup:
Failing node ids:
This matches the dev record. My independent mechanism, normalising at construction, is caught at both sites. The pin is live, not inert. 3. Renamed node ids
No other open PR touches 4.
|
Closes #296
What changed
test_a_decode_side_refuses_a_prefill_side_at_the_same_token_countgains a second parameter axis, the phase pair:("prefill", "decode"), as before, and("decode", "Decode"), two spellings of one phase. Both run against both shape checks incompare(): the outright refusal (at_shapeempty) and the per-term refusal (at_shapenamesactivations).atom/compass/memory/compare.pyis unchanged.Lines: production 0; test +13 / -5, all in
tests/compass/test_memory_compare.py. The estimate was about 8.Decision: phases are case-sensitive by design
Shapesays: "Equality is exact, so two spellings of one phase refuse each other, which is the safe direction."Shapeis a frozen dataclass compared with==at both sites (predicted.shape != recorded.shapefor the outright refusal,shapes_agree = predicted.shape == recorded.shapefor the per-term one).__post_init__checks only that the phase is non-blank. Nothing normalises it.memory.Shape. Every phase value in the tree is a literal intests/compass/test_memory_compare.py:"prefill"and"decode", all lower case. (atom/compass/audit/sync_scan.pyhas its own, unrelatedShapeclass.) The phase comes from callers, so this module cannot know which spellings they mean.SequenceTypemembers arePREFILLandDECODE. A caller that writes.namegets"DECODE", and one that writes a literal gets"decode".So this PR takes the "yes" branch of the brief: it adds a test and leaves the docstring and code alone.
Named result
Each mutation is line-count-preserving (847 lines before and after), applied to a copy of each staged tree on node 18 (
xiaobizh_n18_cpu). Onlytests/compass/test_memory_compare.pywas run.6a83b56bcad9a1c5e0(tokens, phase.lower())on both sides of!=[two-spellings-outright],DID NOT RAISE MemoryRefusal(tokens, phase.lower())on both sides of==[two-spellings-per-term],assert set() == {'activations'}Failing node ids:
tests/compass/test_memory_compare.py::test_a_decode_side_refuses_a_prefill_side_at_the_same_token_count[two-spellings-outright]tests/compass/test_memory_compare.py::test_a_decode_side_refuses_a_prefill_side_at_the_same_token_count[two-spellings-per-term]Gates
Both runs were on node 18 (
xiaobizh_n18_cpu) using the tree's ownscripts/compass/gate_cpu.sh. Each tree was agit archivewith.compass-commitand.compass-changedstamps, staged at/tmp/i296gates/{tip,head}/ATOM.atom.__file__resolved under each staged root.6a83b56bc(control)ad9a1c5e0GATE_CPU_RCThe node-id delta from junit comes to 5324 on the tip and 5326 at the head. Every changed id is in this test:
[outright],[per-term][prefill-decode-outright],[prefill-decode-per-term],[two-spellings-outright],[two-spellings-per-term]No other test changed status.
The merged tree:
git merge-tree --write-tree 6a83b56bc ad9a1c5e0gives81414803cbbd3c9944c3f95a76c405cdbba3c139, which equalsad9a1c5e0^{tree}.ruff format --checkandruff checkpass on the test file.Left undone
Nothing. The two existing parameter ids were renamed, from
[outright]and[per-term]to[prefill-decode-*]. Anything that selects them by id needs the new names. Nothing in the tree does.🤖 Generated with Claude Code