Skip to content

compass(docs): pin the CPU gate readings in scripts/compass/README.md to their commits, and 08/16 leftovers - #351

Merged
jgong5 merged 2 commits into
feature/atomcompass_newfrom
compass/issue-350-pin-cpu-gate-readings
Sep 23, 2026
Merged

jgong5 merged 2 commits into
feature/atomcompass_newfrom
compass/issue-350-pin-cpu-gate-readings

Conversation

@jgong5

@jgong5 jgong5 commented Sep 23, 2026

Copy link
Copy Markdown
Owner

Closes #350

Pins the CPU gate readings in scripts/compass/README.md to their commits, following #350's body and its last comment (the #349 review's N1 and inline r4087445424, r4087445580, r4087445720). It also fixes 08:34, 08:39, 08:65 and 16:126-129. The wording mirrors #349 (cb684287f).

Change: 3 files, +17 / −16 (README +9/−8, 08 +5/−4, 16 +3/−4). Production lines 0, test lines 0. The brief estimated 10-20.

Dev record

Citations

Claim Commit Evidence
4030 / 0 / 149 / 3, rc=0 105ca4197 GET pulls/6: head.sha=105ca4197…, merged=true, merge_commit_sha=4c16792d9…. rev-parse ^{tree} gives fbcb03674 for both. #6's "Gates" table has 4030 / 0 / 149 / 3 and GATE_CPU_RC 0 in the branch column
3956 is #6's control, with #6's scripts/compass/ copied in 042aad97d #6's body: "Control is fork/feature/atomcompass_new at 042aad97d with only scripts/compass/ copied in". The control column reads 3956 / 0 / 149 / 3, rc 0. git ls-tree 042aad97d scripts/ shows no scripts/compass
Clock 25.6-30.8 s / 31.6-36.9 s over six runs at d737f15e7 and 7ff80cc4b 8c0ee374f The message says "six runs, three at d737f15 and three at 7ff80cc … the same 4030 / 0 / 149 / 3 in all six", and the diff carries both ranges. 8c0ee374f is an ancestor of 105ca4197. Neither run commit is, because they predate the restack. All three resolve on the fork (GET commits/<sha>)
3925, 32 exclusions, no tests/compass 83daf636d 9c8df1328 (on fork/compass/p0.2-baselines, parent 83daf636d): "Measured at 83daf63 on node 18 … a 125-file CPU gate … 3925 passed, 0 failed". At 83daf636d, ls-tree gives 187 test files, 30 in tests/plugin, and nothing under tests/compass: 157 − 32 = 125
3956 = 3925 + 31 83daf636d → 042aad97d git diff --stat 83daf636d 042aad97d -- tests is empty. cpu_gate_exclude.txt at eb1d6d4fa (32 entries, P0.1's first cut) against 105ca4197 (29) differs by exactly test_dp_metadata.py, test_dp_sync_layout.py and test_forward_mode.py. I measured those three at 83daf636d in xiaobizh_n18_cpu: 31 passed in 0.23s, rc=0
4005, tests/compass at 49 3afcb4880 b78e0bc97: "CPU row is the measured 4005 with its 3956 ATOM + 49 tests/compass decomposition … at 3afcb48 on node 18". At 3afcb4880: 1 tests/compass file, 13 triggers, 29 exclusions
4022, tests/compass at 66 71d2a1ac2 236abfd9a (parent 71d2a1ac2) writes the trigger header "Measured at 71d2a1a… 188 files = 30 + 29 + 129 … 4022 passed + 128 skipped + 3 xfailed", and 5ca21de2b removes it. f3f584ea4, an ancestor of 71d2a1ac2, says "4005 -> 4022 (3956 + 66)". At 71d2a1ac2: 188 / 30 / 29 exclusions, 1 tests/compass file, 30 triggers (+17 over 13)
Plugin collection: 153 tests collected, 7 errors, rc=2 fada7424e I measured it in xiaobizh_n18_cpu: rc=2, 153 tests collected, 7 errors in 1.41s
The 2 tier files reach rocminfo alone fada7424e I measured it: test_postprocess_width.py rc=2, 1 error in 0.82s; test_v4_checkpoint_slot_copy.py rc=2, 1 error in 0.88s; both Get GPU arch from rocminfo failed. Neither is in fada7424e's cpu_gate_exclude.txt
test_prefill_prefix_vs_native.py: 4 passed in the GPU container cb684287f I measured it (below)

08:65: GPU measurement

xiaobizh_n18 on node 18, 2026-09-24. I staged a git archive of cb684287f into /tmp/i350gates/tip/ATOM, and the tar md5 matched at both ends. atom.__file__ = /tmp/i350gates/tip/ATOM/atom/__init__.py, which I asserted before the run.

Card: HIP_VISIBLE_DEVICES=3. I chose it with rocm-smi --showmeminfo vram (card 3 held 298 MB). Cards 0 and 1 held about 171 GB each at 0% use.

torch 2.10.0+rocm7.2.4.git3d3aa833, hip 7.2.53211. The run was under timeout -k 10 900, unpiped:

PASSED tests/test_prefill_prefix_vs_native.py::test_the_split_only_reassociates_it_does_not_bias[1]
PASSED tests/test_prefill_prefix_vs_native.py::test_the_split_only_reassociates_it_does_not_bias[4]
PASSED tests/test_prefill_prefix_vs_native.py::test_the_split_only_reassociates_it_does_not_bias[7]
PASSED tests/test_prefill_prefix_vs_native.py::test_all_keys_in_prefix_equals_all_keys_in_extend
4 passed in 8.67s   PYTEST_RC=0

Named result

  • grep -n "commit not recorded" scripts/compass/README.md finds nothing (rc=1).
  • Every CPU-gate total in the README now names its commit. 3925 → 83daf636d, 3956 → 042aad97d, 4005 → 3afcb4880, 4022 → 71d2a1ac2, 4030 → 105ca4197; 4380 → 68ef4f329 and 4501 → 186d12829 were already pinned. L64's Baseline is 4030 passed is the output of 105ca4197's own gate_cpu.sh, named at L30.
  • Every changed line in 08 and 16 names its commit.

Gate 1: CPU tier, delta against a measured control

Node 18, xiaobizh_n18_cpu, 2026-09-24, load average about 17-20 with other tenants' gates running.

  • Each side is a git archive with stamps, staged with docker exec -i … tar -x into /tmp/i350gates/{ctl,br}/ATOM. The tar md5 matched at both ends.
  • The branch stamp is git commit-tree 5dc3962ab -p cb684287f -p 3d504c255 = bbb9eefeb. No ref was created.
  • atom.__file__ was asserted under each root before the run.
  • Each side ran the tree's own scripts/compass/gate_cpu.sh --junitxml=… under timeout -k 10 2400, sequentially and unpiped.
control cb684287f (tip) branch (merged tree 5dc3962ab)
printed stamp commit: cb684287f (stamp) commit: bbb9eefeb (stamp)
gpu: not required not required (3 doc files changed)
result 5258 passed, 155 skipped, 3 xfailed, 186.65 s 5258 passed, 155 skipped, 3 xfailed, 189.10 s
GATE_CPU_RC 0 0
junit test cases 5416 5416
  • Node-id delta: none. No test case is on only one side, and none changed status between the two junit files.
  • No timing class failed, so none needed a re-run: TestTheRegionIsNotCopiedPerChunk, TestNoSizeAtWhichACallStopsBeingOne::…[minimax] and test_gc_utils.py::test_freezing_twice_is_additive_and_harmless.
  • Lines changed: production 0, test 0, docs +17 / −16.
  • git merge-tree --write-tree cb684287f 3d504c255 = 5dc3962ab4ee27b41f1f2b08604584567072e80b. That equals 3d504c255^{tree}, and the tip was read again just before this was written.

Gate 2 does not apply, because the change is docs only. Gate 4 is the coordinator's reviewer.

🤖 Generated with Claude Code

jgong5 and others added 2 commits September 23, 2026 21:33
…o their commits; fix 08:34, 08:39, 16:126

scripts/compass/README.md: the 4030 row names 105ca41 (PR #6's gate
table), its clock is the six runs at d737f15 and 7ff80cc recorded
in 8c0ee37, and the 3956 half says its control had PR #6's
scripts/compass/ copied in. The readings paragraph pins 3925 at
83daf63, 4005 at 3afcb48, 4022 at 71d2a1a and 4030 at 105ca41,
and drops 3988, which no record ties to a commit.

08: the plugin collection no longer attaches one time to two commits,
and the floor counts the 2 tier files that reach rocminfo alone (34).
16: two sentences that both said the 3956 had no tests/compass are one.

Closes #350

Co-Authored-By: Claude Opus 5.5 (1M context) <noreply@anthropic.com>
test_prefill_prefix_vs_native.py measured at cb68428 in xiaobizh_n18
on node 18, HIP_VISIBLE_DEVICES=3 (card 3 holding 0.3 GB before the
run): 4 passed in 8.67s, rc=0.

Co-Authored-By: Claude Opus 5.5 (1M context) <noreply@anthropic.com>
@jgong5

jgong5 commented Sep 23, 2026

Copy link
Copy Markdown
Owner Author

Review cycle 1 — PR #351 (issue #350), head 3d504c2556f64d518940cd58c5d7acfcf0407362

Agent-authored review. I read the eight Design principles in atom/compass/design/README.md and atom/compass/AI_DEV_RULES.md at tip fb07aba32 first.

Verdict: APPROVE 3d504c2556f64d518940cd58c5d7acfcf0407362. No findings block. Every citation in the claim table checks out against git and GitHub, and I re-measured the four readings the developer took. The two notes below are non-blocking. Both are on lines outside the diff hunks, so they have no inline comment.

Citations checked (principle 8: "Every claim carries its measurement")

Claim Commit What I checked True?
4030 / 0 / 149 / 3, rc=0 105ca4197 GET pulls/6: head.sha=105ca4197…, merged=true, merge_commit_sha=4c16792d9…. Both have ^{tree} = fbcb03674. PR #6's gate table: branch 4030 / 0 / 149 / 3, GATE_CPU_RC 0 true
3956, with PR #6's scripts/compass/ copied in 042aad97d PR #6's body, L136: "Control is … 042aad97d with only scripts/compass/ copied in". Its control column reads 3956. git ls-tree 042aad97d scripts/ has no compass true
Clock 25.6-30.8 s / 31.6-36.9 s, six runs at d737f15e7 and 7ff80cc4b 8c0ee374f Its message says "six runs, three at d737f15 and three at 7ff80cc … the same 4030 / 0 / 149 / 3 in all six", and its diff carries both ranges. 8c0ee374f is an ancestor of 105ca4197, and the two run commits are not. All three resolve on the fork true
3925 at 83daf636d, 32 exclusions, no tests/compass 83daf636d 9c8df1328 (parent 83daf636d): "a 125-file CPU gate … 3925 passed". 83daf636d has 187 test files, 30 of them plugin and 0 under tests/compass. The 32 exclusions at eb1d6d4fa and at 9120a96a3 are the same list, so 187 − 30 − 32 = 125 true
3956 = 3925 + 31 83daf636d → 042aad97d git diff 83daf636d 042aad97d touches only CLAUDE.md, AI_DEV_RULES.md and design/*.md, so there is no test or code change. The exclusion list at eb1d6d4fa against 105ca4197 differs by exactly the three dp/forward files. Re-measured on node 18 in xiaobizh_n18_cpu, from a git archive of 83daf636d with atom.__file__ asserted under the staged root: 31 passed in 0.23s, rc=0. Run one at a time they give 4 + 5 + 22, with 0 skipped true
4005 at 3afcb4880, tests/compass at 49 3afcb4880 b78e0bc97's message: "the measured 4005 with its 3956 ATOM + 49 tests/compass decomposition … at 3afcb48 on node 18" true
4022 at 71d2a1ac2, tests/compass at 66 71d2a1ac2 236abfd9a (parent 71d2a1ac2) adds "Measured at 71d2a1a… 4022 passed + 128 skipped + 3 xfailed", and 5ca21de2b removes that line. f3f584ea4 is an ancestor of 71d2a1ac2 and says "4005 -> 4022 (3956 + 66)". 71d2a1ac2 has 30 triggers against 13 at 3afcb4880, which is +17 true
3988 has no commit, so it is dropped — git log --all -S3988: among Compass commits it first appears in 3afcb4880's gate_cpu.sh ("Baseline is 3988 passed, 0 failed at 29"). --grep=3988 finds only 71197264e, which has no commit. This is stronger than the PR claims: 71197264e says 3988 had 29 exclusions and tests/compass at 32. No committed tree has that pair: eb1d6d4fa and f0096fda7 have 32 exclusions, and 3afcb4880 has 29 but 49 cases. So 3988 was read on an uncommitted tree and cannot be pinned. REST search finds it only in #68, #82, #345, #350 and #351. git grep 3988 at the head is empty, so the drop leaves no orphan true
9120a96a3 not cited — GET commits/9120a96a3 returns 422 ("No commit found"). Locally it exists and prints "Baseline at 83daf63 is 3925" true
L64's Baseline is 4030 passed 105ca4197 That string is at 105ca4197:scripts/compass/gate_cpu.sh:169, and 105ca4197's scripts/compass is tree ddb69e7aa, as L30 says true

The 08 floor of 34

29 excluded, plus 2 tier files, plus 3 plugin rocminfo files, is 34. That is exactly the count measured to reach the driver. The reason given now matches it: 23 plugin files remain unmeasured, and 30 − 7 = 23. Re-measured at fada7424e in xiaobizh_n18_cpu:

  • test_postprocess_width.py: rc=2, 1 error in 0.90s, Get GPU arch from rocminfo failed.
  • test_v4_checkpoint_slot_copy.py: rc=2, 1 error in 0.95s, same cause.
  • Neither is in fada7424e's cpu_gate_exclude.txt: grep -c gives 0.
  • tests/plugin --collect-only: rc=2, 153 tests collected, 7 errors in 1.58s. That is 3 rocminfo, 1 sglang and 3 fused_gdn_gating, the same decomposition as the text.

My plugin collection took 1.58 s. The developer read 1.41 s and the #349 reviewer 1.41 s, so dropping the shared 1.45s from 08:34 is right.

The 08:65 GPU reading

Re-run in xiaobizh_n18 at cb684287f from a git archive stage, with atom.__file__ = /tmp/pr351r1-gates/cb68/ATOM/atom/__init__.py. I picked the card with rocm-smi --showmeminfo vram: cards 0 and 1 held 171 GB each, and cards 2-7 held 298 MB each. I used HIP_VISIBLE_DEVICES=4, torch 2.10.0+rocm7.2.4.git3d3aa833, hip 7.2.53211, under timeout -k 10 900, unpiped: 4 passed in 8.94s, PYTEST_RC=0, with the same four node-ids as the PR body. The claim is true.

README L99, outside the brief's sections

"The CPU row below does not name the commit it ran on" would be false once the row names 105ca4197. "Once named no commit" is true of the pre-PR row, which read "commit not recorded". The edit is needed, and it is the smallest one that keeps the sentence true.

Named result, and a sweep of the README

  • grep -n "commit not recorded" scripts/compass/README.md at the head gives rc=1 and no output.
  • I swept every passed or timing line in the README at the head. Each CPU-gate total or clock in the two sections of the brief names its commit: 4030/105ca4197, 3956/042aad97d, 3925/83daf636d, 4005/3afcb4880, 4022/71d2a1ac2, 4380/68ef4f329, 4501/186d12829. The clock is pinned by 8c0ee374f. L37-39, L203 and L218 were already pinned.

Non-blocking

  • N1 (principle 8, "A number without a source is a defect"), at 08_validation_protocol.md:51 in the tier table, outside the hunk and the file set. It reads "no tests collected, 1 error in 0.78s, rc=2, each", which is one time for two runs. That is the defect this PR fixes at 08:34. The same two files read 0.82 / 0.88 s for the developer and 0.90 / 0.95 s for me, all at fada7424e. A follow-up can drop the time, as 08:34 now does.
  • N2 (principle 8, "Every claim carries its measurement"), at scripts/compass/README.md:191, outside the brief's two sections. The flake-rate table says "n=21 … one branch" and does not name the commit. It records run outcomes rather than a pass count, so the named result does not cover it. It is still the one unpinned reading left in the file.

ponytail-review (over the diff)

Docs only, +17/−16. The 16 merge already takes the −1 the brief asked for. The README row's note about the copied control and the "3925 + 31" in the readings paragraph are both required by the brief. I found no delete:, stdlib:, native:, yagni: or shrink: finding that would not remove a pin the brief asked for.

Lean already. Ship.

Gate 1: CPU tier on the tree that will land

  • The tip was read again as fb07aba32fa0c7fb273d84cc0193b29cfbca0d9c. It moved past cb684287f by compass(ir): refuse a str by the container test alone, and reach the replicated KV refusal #347, so the gated tree differs from the head's.
  • git merge-tree --write-tree fb07aba32 3d504c255 = bf134c5b0c8eddf92bf9c22a64d03320966c5e8f, which is not 3d504c255^{tree} (5dc3962ab). The merge is clean.
  • The stamp is git commit-tree bf134c5b0 -p fb07aba32 -p 3d504c255 = 0913aa243161a0a3e4869cb90c3b88f554a48e9f. No ref was created.
  • .compass-changed holds the three doc files. gate_cpu.sh printed commit: 0913aa243 (stamp) and gpu: not required.
  • Staging: git archive of the stamp, then docker exec -i … tar -x into /tmp/pr351r1-gates/merged/ATOM in xiaobizh_n18_cpu. The tar md5 matched at both ends (1a87abf6…), and atom.__file__ = /tmp/pr351r1-gates/merged/ATOM/atom/__init__.py was asserted.
  • The run used the tree's own scripts/compass/gate_cpu.sh --junitxml, under timeout -k 10 2400, unpiped, and alone after the other tenant's gate had finished. Load average was 28.
  • Result: 5260 passed, 155 skipped, 3 xfailed, 191.90 s, GATE_CPU_RC=0. Junit has 5418 test cases, 0 failures and 0 errors.
  • Delta check: the developer's control at cb684287f read 5258 / 155 / 3, with 5416 cases. compass(ir): refuse a str by the container test alone, and reach the replicated KV refusal #347 adds exactly +2 collected cases: --collect-only on tests/compass/test_backend_kv_geometry.py and test_ir_data_model.py gives 182 at cb684287f and 184 at the stamp. 5258 + 2 = 5260 and 5416 + 2 = 5418, so the docs diff contributes zero.
  • The timing classes did not fail: TestTheRegionIsNotCopiedPerChunk (4 cases), TestNoSizeAtWhichACallStopsBeingOne (12) and test_freezing_twice_is_additive_and_harmless (1). None needed a re-run.

No blocking issues.

@jgong5
jgong5 marked this pull request as ready for review September 23, 2026 22:05
@jgong5
jgong5 merged commit be86326 into feature/atomcompass_new Sep 23, 2026
jgong5 added a commit that referenced this pull request Sep 23, 2026
… 19 of the flake table's 21 runs (#354) (#356)

08:51 gave one time, 0.78s, for two files run alone. It came in with
4c16792 (#6), the only commit git log -S finds for it, and that commit
states no per-file figure, so there is nothing to split. The time is
dropped, as #351 did at 08:34. The outcome, the rc and the commit stay.

The flake-rate paragraph in scripts/compass/README.md said "one branch".
#93, the brief #99 copied it from, names no commit either. PR #79's review
(issue comment 5765186351) records 19 runs at b58a48c (7 gate + 12 direct):
17 nominal, 1 skip-variant (4476 / 150) and the 1.89x hard failure. The
paragraph now names that commit for those 19 and says the other 2 name none.

Co-authored-by: Claude Opus 5.5 (1M context) <noreply@anthropic.com>
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

None yet

Projects

None yet

Development

Successfully merging this pull request may close these issues.

1 participant