Skip to content

compass(docs): stop restating the CPU gate census in 08, and pin scripts/compass/README.md:14 to a commit - #345

Merged
jgong5 merged 2 commits into
feature/atomcompass_newfrom
compass/issue-342
Sep 23, 2026
Merged

jgong5 merged 2 commits into
feature/atomcompass_newfrom
compass/issue-342

Conversation

@jgong5

@jgong5 jgong5 commented Sep 23, 2026

Copy link
Copy Markdown
Owner

Closes #342

Developer round 1, head 1b901f085. Docs only, with no production or test lines.

What changed

atom/compass/design/08_validation_protocol.md (commit 936b98d21)

  • The CPU tier row's What runs now says "Every test file outside tests/plugin/ and scripts/compass/cpu_gate_exclude.txt", which is compass(docs): stop restating the CPU gate census in CLAUDE.md, and pin two scripts/compass figures #339's wording. It drops "(30 files)", "the 29 driver-dependent files" and "= 130 of 189 files". "among the 130 handed" becomes "both in the tier". The existing rocminfo caveat, pinned at 236abfd9a, is kept.
  • The row's Measured result is now: "Green, GATE_CPU_RC=0. The counts are not restated here, because they move with every test the tier gains or loses: gate_cpu.sh prints them under the commit: line of the tree it ran, and scripts/compass/README.md says what the tier runs." This removes "4030 passed", the 2026-09-21 timing spread and the 3956 + 74 decomposition.
  • The "Six different totals" paragraph (3925/3956/3988/4005/4022/4030) becomes: "A total for this tier is a reading of one tree, not a property of the suite. Each total belongs to one exclusion list and one tests/compass, both of which change, so two totals compare only when each names its commit. test_dp_metadata.py, test_dp_sync_layout.py and test_forward_mode.py were once excluded and were re-measured CPU-green (16); prose saying those three need the driver is stale."

Which option I chose: no number. The row's bar is green, so no count is needed to make its point. The GPU row keeps its pinned 4779 / 5 at fe9ea043c, because a delta needs a baseline.

scripts/compass/README.md:14 (same commit): "(tests/test_postprocess_width.py does, through atom.model_engine.model_runner's from aiter import …; measured on node 18 at bdd244c57)". That is the wording r4086819090 suggested, with the commit I measured.

atom/compass/design/README.md, rows 529 and 593 (commit 1b901f085). This is outside the briefed file set, and it is a separate commit so it can be dropped. Both rows restated 08's census ("130 of 189 files", "4030 passed"), so they would contradict 08 after the change, and #342's named result greps all of atom/compass/design/. Both now carry 08's wording. Row 593's "4779 / 5" gained "at fe9ea043c", the commit row 529 and gate_gpu.sh's BASE_COMMIT already give it.

The README.md:14 example, measured

On node 18, xiaobizh_n18_cpu, on a git archive of bdd244c57. atom.__file__ = /tmp/i342gates/control/ATOM/atom/__init__.py. python -m pytest tests/test_postprocess_width.py run alone gives rc=2 and 1 error in 0.82s. The route is tests/test_postprocess_width.py:24 pytest.importorskip( → atom/model_engine/model_runner.py:19 from aiter import destroy_dist_env, init_dist_env → CalledProcessError on /opt/rocm-7.2.4/bin/rocminfo → RuntimeError: Get GPU arch from rocminfo failed. In the gate's junit, the same file is skipped: collection skipped.

Named result: met in part, and the rest is escalated

Gate 1: ATOM's suite, unmodified, as a delta against a control

Node 18, xiaobizh_n18_cpu, one run at a time. Each tree was staged by git archive, streamed with docker exec -i into /tmp/i342gates/{control,branch}/ATOM, and md5-matched on both ends. Each ran the tree's own scripts/compass/gate_cpu.sh --junitxml=… under timeout -k 10 1800, not piped. atom.__file__ was asserted under each staged root before the run. .compass-changed holds this PR's 3 files on both sides.

tree stamp commit: printed result GATE_CPU_RC
control tip bdd244c57 bdd244c57 bdd244c57 (stamp) 5253 passed, 155 skipped, 3 xfailed, 185.8 s 0
branch merged 08edfc00e 7820c28cd (git commit-tree, parents tip + head) 7820c28cd (stamp) 5253 passed, 155 skipped, 3 xfailed, 191.6 s 0
  • Both printed gate: 29 files excluded + tests/plugin and gpu: not required (.compass-changed stamp).
  • Node-id delta from the two junit files: 5411 ids on each side, none only on one side, and no outcome changed.
  • The timing classes all passed on both sides, so none needed a re-run: TestTheRegionIsNotCopiedPerChunk, TestNoSizeAtWhichACallStopsBeingOne (including [minimax]) and test_gc_utils.py::test_freezing_twice_is_additive_and_harmless.
  • git merge-tree --write-tree bdd244c57 1b901f085 = 08edfc00eeab46c604d5b10a363d23644f3e8481, which equals 1b901f085^{tree}. The tip had not moved at push time.
  • An earlier gate ran on the head before a one-phrase fix to row 593, stamp ba4d3cd15. It gave the same counts, GATE_CPU_RC=0, and a zero node-id delta. It is superseded by the row above.
  • Lines: production 0, tests 0, docs +9 / −16 (08 +6/−13, design/README.md +2/−2, scripts/compass/README.md +1/−1).

No test opens 08_validation_protocol.md. The README rows that test_cpu_gate_exclude.py and test_gate_gpu_surplus.py join are the cpu_gate_exclude.txt, gpu_gate_triggers.txt and gate_gpu.sh rows, and this PR leaves them untouched.

Not done

🤖 Generated with Claude Code

jgong5 and others added 2 commits September 23, 2026 20:16
…ME.md:14 to a commit

08_validation_protocol.md's CPU tier row and the paragraph under it still
stated "130 of 189 files", "4030 passed" and a 2026-09-21 timing spread, none
joined to the commit they were measured at. The row now says what the tier is,
in the wording #339 settled (every test file outside tests/plugin/ and
cpu_gate_exclude.txt, no driver as a batch), and sends the reader to
gate_cpu.sh's own output and scripts/compass/README.md for the counts. The
six-totals paragraph becomes two sentences: a total is a reading of one tree.

scripts/compass/README.md:14 anchored its rocminfo example with a minute-precision
date. Re-measured on node 18 at bdd244c: tests/test_postprocess_width.py run
alone gives rc=2, "1 error in 0.82s", RuntimeError from rocminfo, reached through
its importorskip of atom.model_engine.model_runner, whose line 19 imports aiter.
The line now names that route and the commit.

Refs #342

Co-Authored-By: Claude Opus 5.5 (1M context) <noreply@anthropic.com>
…for 08

design/README.md restated 08's CPU tier figures in its document index row and
in the D43.1 decision-log row ("130 of 189 files", "4030 passed"). After the
previous commit they contradict 08, and #342's named result greps all of
atom/compass/design/ for both strings. Both rows now carry 08's wording. The
D43.1 row's GPU baseline, 4779 / 5, now names fe9ea04, the commit the index
row above it and gate_gpu.sh's BASE_COMMIT already give it.

Refs #342

Co-Authored-By: Claude Opus 5.5 (1M context) <noreply@anthropic.com>
@jgong5

jgong5 commented Sep 23, 2026

Copy link
Copy Markdown
Owner Author

Coordinator ruling on the named result. #342's grep criterion is narrowed to exclude atom/compass/design/16_execution_plan.md. Its three 4030 lines (117, 123, 125) are the execution plan's record of the tier's first measurement. Rewriting them is outside this file set and past twice the estimate, so they are tracked in #344, together with the "Revised 2026-09-20" paragraph in 08 (lines 22–40).

The separate commit 1b901f085, which edits atom/compass/design/README.md rows 529 and 593, is accepted into scope. Without it those rows would contradict 08. The reviewer should check it like any other change.

| Tier | What runs | Measured result |
|---|---|---|
| **CPU tier** — per task, and green is the bar | `tests/` minus `tests/plugin/` (30 files) minus the 29 driver-dependent files in `scripts/compass/cpu_gate_exclude.txt` = **130 of 189 files**. Driven by `scripts/compass/gate_cpu.sh`; no driver is touched **as a batch**. That is the weaker and correct claim: `test_postprocess_width.py` and `test_v4_checkpoint_slot_copy.py` are among the 130 handed, and each reaches `rocminfo` when run *alone* (`no tests collected, 1 error in 0.78s`, rc=2, each, measured at `236abfd9a` in `xiaobizh_n18_cpu`); inside the gate they module-skip on the mock an earlier file installs. Driver-freedom here is a property of the batch, not of every file in it. | **4030 passed, 0 failed**, 149 skipped, 3 xfailed, rc=0 — identical in every run taken on 2026-09-21, where the clock read **25.4-31.7 s of pytest inside 31.1-37.8 s of wall (`time` real)**, a measured spread rather than a bound: it tracks what else is on the node. Decomposition (principle 7): **3956 ATOM + 74 `tests/compass` = 4030**. Measured 2026-09-21 in container `xiaobizh_n18_cpu` on hjbog-srdc-18, against a `git archive` snapshot with `PYTHONPATH` asserted to resolve `atom` under that root and pytest's own exit status captured before any pipe. |
| **CPU tier** — per task, and green is the bar | Every test file outside `tests/plugin/` and `scripts/compass/cpu_gate_exclude.txt`. Driven by `scripts/compass/gate_cpu.sh`; no driver is touched **as a batch**. That is the weaker and correct claim: `test_postprocess_width.py` and `test_v4_checkpoint_slot_copy.py` are both in the tier, and each reaches `rocminfo` when run *alone* (`no tests collected, 1 error in 0.78s`, rc=2, each, measured at `236abfd9a` in `xiaobizh_n18_cpu`); inside the gate they module-skip on the mock an earlier file installs. Driver-freedom here is a property of the batch, not of every file in it. | Green, `GATE_CPU_RC=0`. The counts are not restated here, because they move with every test the tier gains or loses: `gate_cpu.sh` prints them under the `commit:` line of the tree it ran, and `scripts/compass/README.md` says what the tier runs. |

Copy link
Copy Markdown
Owner Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Non-blocking (principle 8: "Every claim carries its measurement. A number without a source is a defect.")

The Measured result cell now reads "Green, GATE_CPU_RC=0." with no tree attached. The GPU row beside it names fe9ea043c, the node, the container and the stack. This cell names nothing, so a reader cannot tell which tree the 0 was read on. The issue's named result is still met, because an rc is not a file count, pass count or timing. But this is the one reading left in a column headed "Measured".

Suggested fix, same line count: "Green, GATE_CPU_RC=0, at bdd244c57 in xiaobizh_n18_cpu. The counts are not restated here …". That anchors it without restating any count. I measured it at the merged tree 08edfc00e (stamp 847f646a6): GATE_CPU_RC=0. You measured it at bdd244c57 itself.

because `test_dp_metadata.py`, `test_dp_sync_layout.py` and `test_forward_mode.py` were
re-measured and are CPU-green. Prose saying the list is 32, or that those three need the
driver, is stale.
**A total for this tier is a reading of one tree, not a property of the suite.** Each total

Copy link
Copy Markdown
Owner Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Non-blocking. ponytail-review:

shrink: the first two sentences here ("A total for this tier is a reading of one tree … compare only when each names its commit") restate the cell above ("they move with every test the tier gains or loses … prints them under the commit: line"). The paragraph can be only its last sentence, the test_dp_* / test_forward_mode note, which is the one fact the cell lacks. That saves about 2 lines.

That last sentence checks out: none of the three is in cpu_gate_exclude.txt at the head. In the junit of the gate I ran on the merged tree, they give 4 + 5 + 22 = 31 passed, which matches 16:108's "31 passed".

@jgong5

jgong5 commented Sep 23, 2026

Copy link
Copy Markdown
Owner Author

Review cycle 1: APPROVE at head 1b901f085ec41977479c7ddedcee88dd6a34f9b3.

There are no blocking findings. Two inline comments are non-blocking: r4087085010 (08:49, principle 8) and r4087085189 (08:52, ponytail shrink:). I read the eight design principles in atom/compass/design/README.md and AI_DEV_RULES.md, both at bdd244c57, before reviewing.

Truth checks (principle 8: "Every claim carries its measurement.")

  1. scripts/compass/README.md:14 is true. I checked it on node 18, xiaobizh_n18_cpu, on a git archive of the merged tree, with atom.__file__ = /tmp/pr345r1-gates/merged/ATOM/atom/__init__.py. That tree is identical to bdd244c57 outside the 3 doc files.
    • python -m pytest tests/test_postprocess_width.py run alone gives PYTEST_RC=2 and 1 error in 0.98s.
    • The route is tests/test_postprocess_width.py pytest.importorskip("atom.model_engine.model_runner", exc_type=ImportError) → atom/model_engine/model_runner.py:19 from aiter import destroy_dist_env, init_dist_env → CalledProcessError on /opt/rocm-7.2.4/bin/rocminfo → RuntimeError: Get GPU arch from rocminfo failed.
    • That RuntimeError is not an ImportError, so importorskip does not catch it, and collection errors.
    • test_v4_checkpoint_slot_copy.py alone gives the same: rc=2.
  2. The "Six different totals" rewrite is true.
    • test_dp_metadata.py, test_dp_sync_layout.py and test_forward_mode.py are absent from cpu_gate_exclude.txt at the head.
    • In the gate's junit they give 4 + 5 + 22 = 31 passed, which matches the (16) pointer (16_execution_plan.md:108, "31 passed").
    • "Each total belongs to one exclusion list and one tests/compass": ATOM's own non-plugin, non-compass test files are 157 at fada7424e, 236abfd9a and bdd244c57, so those two really are the only parts that moved.
  3. "4779 / 5 at fe9ea043c" matches gate_gpu.sh at the head: BASE_PASSED=4779, BASE_FAILED=5, BASE_COMMIT=fe9ea043c.
  4. The rewritten 08:49 row drops no fact the doc needs.
    • The rocminfo caveat is kept verbatim with its 236abfd9a pin.
    • "both in the tier" holds: neither file is in the exclusion list.
    • The junit shows both as skipped message="collection skipped" inside the gate, which confirms "module-skip on the mock an earlier file installs".
    • "(30 files)" survives in the "Revised" paragraph, which compass(docs): 16_execution_plan.md still restates the CPU gate census (4030, 130 of 189) #344 tracks.
    • The measurement method the old cell described (git archive, PYTHONPATH asserted, rc captured before any pipe) is now enforced by gate_cpu.sh itself (compass_require_tree, finish).
    • The only gap is that the Measured result cell names no tree (r4087085010, non-blocking).
    • "gate_cpu.sh prints them under the commit: line" is true. The log reads commit: 847f646a6 (stamp) at line 4, and the pytest summary follows.

Named result (as ruled: 16_execution_plan.md excluded)

  • git grep -nE "130 of 189" 1b901f085 -- atom/compass/design/ finds 0 hits.
  • git grep -n 4030 1b901f085 -- atom/compass/design/ finds 3 hits, all in 16_execution_plan.md (117, 123, 125), which the ruling excludes and compass(docs): 16_execution_plan.md still restates the CPU gate census (4030, 130 of 189) #344 tracks.
  • I swept the changed regions of all 3 files for any count, pass count or timing without a commit. There is none. What is left is 0.78s/rc=2 at 236abfd9a, bdd244c57, and 4779 / 5 at fe9ea043c.

Out-of-scope commit 1b901f085 (accepted by the coordinator's ruling)

Rows 529 and 593 of atom/compass/design/README.md now use 08's own wording: every test file outside tests/plugin/ and cpu_gate_exclude.txt, driver-free as a batch, held to green. They make no new claim. "4779 / 5 at fe9ea043c" in row 593 agrees with row 529 and gate_gpu.sh's BASE_COMMIT. After the change, no design row restates the CPU census.

ponytail-review (AI_DEV_RULES gate 4)

  • 08_validation_protocol.md:52-54: shrink: the paragraph's first two sentences restate the cell at :49. Keep only the test_dp_* / test_forward_mode sentence.

net: -2 lines possible.

Gate 1: merged tree, node 18

  • I re-read the tip: feature/atomcompass_new = bdd244c57, and the PR head is 1b901f085.
  • git merge-tree --write-tree bdd244c57 1b901f085 = 08edfc00eeab46c604d5b10a363d23644f3e8481, which is 1b901f085^{tree}.
  • The stamp was made with git commit-tree (parents: tip and head) as 847f646a6. .compass-changed holds the 3 doc files.
  • The tree was staged by git archive and docker exec -i … tar -x into /tmp/pr345r1-gates/merged/ATOM, md5-matched on both ends. I asserted atom.__file__ under that root.
  • I ran the tree's own scripts/compass/gate_cpu.sh --junitxml once, under timeout -k 10 1800, not piped.
  • The gate printed commit: 847f646a6 (stamp), gate: 29 files excluded + tests/plugin and gpu: not required (.compass-changed stamp).
  • Result: 5253 passed, 155 skipped, 3 xfailed, 186.43 s; GATE_CPU_RC=0. The junit holds 5411 ids and 0 failures. This matches the developer's control and branch counts exactly.
  • No timing-class failures, so none needed a re-run.

@jgong5
jgong5 marked this pull request as ready for review September 23, 2026 20:45
@jgong5
jgong5 merged commit d96da10 into feature/atomcompass_new Sep 23, 2026
jgong5 added a commit that referenced this pull request Sep 23, 2026
…paragraph in 08 to their commits (#349)

16_execution_plan.md, P0.1's per-task CPU-tier bullet:
- The 130 of 189 files and 4030 / 0 / 149 / 3, rc=0, are pinned to
  105ca41, P0.1's head. That is the branch column of PR #6's gate
  table, and 105ca41^{tree} == 4c16792^{tree} (fbcb036).
- The clock spread is now the one the record pins: six runs at d737f15
  and 7ff80cc (commit 8c0ee37). The wider 25.4-31.7 s range came from
  seventeen runs across ten commits that the record does not name.
- 3956 is pinned to the base 042aad9, the control column of the same table.
- 3925 and 4022 were never recorded against a commit, so they are dropped.
  4005 at 3afcb48 stays. The 49 -> 74 growth is stated as +17 and +8.
- "Green is not exercised": the 22 files are pinned at 105ca41. The
  skip decomposition is pinned to b963c94, where P0.2 measured it, and
  carried forward because tests/ and atom/ differ between the two commits
  only by the two tests/compass files, which skip nothing.
- The trigger-file header counts are pinned at fada742, the commit the
  header itself names.

08_validation_protocol.md:
- In the Revised paragraph, 189 / 29 of 159 / 30 are pinned at fada742.
  The driver claim was re-measured there on node 18, xiaobizh_n18_cpu, one
  file per process. All 28 GENERATED entries fail collection with "Get GPU
  arch from rocminfo failed". test_lmcache_offload_disk_integration.py
  fails 2 tests with "hipHostMalloc failed: 100" and then does not exit.
  tests/plugin collected alone gives the same "153 tests collected,
  7 errors in 1.45s", rc=2, as at 236abfd.
- The Measured result cell names bdd244c, where #345 read
  GATE_CPU_RC=0 (r4087085010).
- The paragraph under the table keeps only its test_dp_* /
  test_forward_mode sentence (r4087085189, net -2).

Closes #344

Co-authored-by: Claude Opus 5.5 (1M context) <noreply@anthropic.com>
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

None yet

Projects

None yet

Development

Successfully merging this pull request may close these issues.

1 participant