Skip to content

Collapse whole-corpus reflection to one computation per floor run, and return the 10 off-floor claims - #10167

Closed
briansrls wants to merge 4 commits into
mainfrom
session/stern-lark-508
Closed

briansrls wants to merge 4 commits into
mainfrom
session/stern-lark-508

Conversation

@briansrls

Copy link
Copy Markdown
Contributor

Auto-opened by session-dashboard for session stern-lark-508.
Pushing to session/stern-lark-508 advances this PR.

Worker attestation

Before flipping this PR to ready for review, confirm each item:

  • Title describes the change (not the session id or branch).
  • PR body summarises what and why (replace the TODO below).
  • Tests run: name the command (e.g. npm test, cargo test) and the result.
  • If this closes a work item, the body contains a Closes #N directive.
  • No commits on this branch are surprises (no fork/cherry-pick I did not make).
  • No secrets / credentials / large binaries staged.

Summary

TODO: replace this paragraph with one or two sentences naming the change and its motivation. Reviewers read this first.

Test plan

  • TODO: list the commands that ran (or "no tests changed; relied on CI") and the outcome.

Brian Searls and others added 4 commits September 2, 2026 06:01
…er the corpus census

DESIGN §6's bare-minimum-cost standing rule: a proven cost-shape defect — a copied
accumulator, a quadratic fold — is always fixed, regardless of the realized n. This is
both, in one expression, over a population that is not small and does not stay fixed.

WHAT IT WAS. `deduplicate_identities` decided membership by rescanning the survivors it
had already kept (`any(unique, prior => prior == identity)`) and extended them by copying
them (`concat(unique, [identity])`). Both halves are O(n) per element, so the fold is
quadratic in comparisons AND quadratic in list construction.

WHY n IS NOT SMALL HERE. The argument is a whole-corpus rustc diagnostic census: the two
persisted receipts carry `genesis_5006ddc6_raw_error_diagnostic_identities` and
`receipt_1_eced9d24_raw_error_diagnostic_identities`, and the population is the emitted
compiler's error identities — it grows as the self-host corpus grows. It is reached from
`canonical_identity_set` (every phase row, every unplaced/codeless partition) and from
`census_population_is_duplicate_free` (twice per receipt in `receipt_population_coherent`),
so the whole series ratchet pays it repeatedly.

WHAT IT IS NOW. One traversal carrying the two questions it was conflating in one list:
`seen` (a `Set<String>`) answers "has this identity been kept", `unique` answers "in what
order". The `std.graph` DFS accumulators are the same shape for the same reason.

BEHAVIOUR IS UNCHANGED, deliberately and at both grains that are observable. Order:
an identity is appended exactly at its first occurrence, as before — and every caller
except `census_population_is_duplicate_free` sorts afterwards through
`canonical_identity_set`, which only counts. Multiplicity: the function still collapses
repeats rather than refusing them, so the duplicate WALL stays exactly where it was —
upstream, in `census_population_is_duplicate_free` and `receipt_population_coherent`,
which is the distinction `a_duplicated_census_population_is_not_silently_collapsed_to_a_set`
exists to hold.

EXECUTED EVIDENCE, not a typecheck. Eight enrolled witnesses over the real persisted
series and the duplicate walls pass on this tree, run individually through
`gunbc run --claim-run --source-root dag --source-root src/v2 --entry
dag/test/claim/self_host_compile_phase_frontier_witness_test.dag --function <claim>`:
`a_repeated_population_entry_is_refused_rather_than_collapsed`,
`a_duplicated_census_population_is_not_silently_collapsed_to_a_set`,
`a_receipt_whose_refused_population_repeats_a_path_is_refused_by_coherence`,
`the_same_receipt_without_the_repeat_is_coherent`,
`current_persisted_compile_phase_frontier_holds`,
`the_census_digest_is_invariant_under_reordering_of_either_population`,
`the_persisted_genesis_census_is_fully_attributed_to_its_phases`,
`phase_boards_are_invariant_under_population_reordering`.
The first four are the discriminating REDs for this change: collapsing a duplicated
population, or refusing one that is clean, reddens them.

THIS IS A COST-SHAPE REPAIR, NOT A FLOOR UNBLOCK. It is landed on its own terms because
§6 requires it, not to buy headroom for any claim. The separate question — that ten
required-floor claims each recompute the whole series ratchet, which is authored
duplication under §2 and wants one provider at their least common ancestor — is a
restructure and lands separately, so its design review does not hold this defect hostage.

Co-Authored-By: Claude Opus 5 <noreply@anthropic.com>
Claude-Session: https://claude.ai/code/session_01W377Pq4Tp5eQjGBXtPQEoM
Set equality is not the acceptance test for a set-backed deduplicate. A dedup that
returns the SET's ordering produces the same MEMBERS in a different ORDER: identical
for a caller that counts, byte-drifting for a caller that renders. The previous commit
asserted order was preserved; this one proves it, and proves the assertion can fail.

THE FIXTURE IS CHOSEN SO THE TWO ANSWERS CANNOT COINCIDE. First-occurrence order over
["b", "a", "b", "c", "a"] is ["b", "a", "c"]; every sorted or set-ordered spelling
answers ["a", "b", "c"]. A population that happened to arrive already sorted would let
an ordered-equality assertion pass while discriminating nothing, which is why the
subject is authored rather than read off the live series.

THE MUTANT IS THE WHOLE CHECK, and it was run rather than described. Planting
`|> sort_by(identity => identity)` on the accumulator's result — the set-ordered
variant — turns this arm RED, and turns RED a probe comparing the new function against
the PRE-CHANGE algorithm over both live populations
(`genesis_5006ddc6_raw_error_diagnostic_identities`,
`receipt_1_eced9d24_raw_error_diagnostic_identities`). Unmutated, both pass: the new
function answers the old function's exact ordered sequence on the real corpus census,
not merely its member set. Four arms, in one run: A pass, B pass, C fail, C2 fail.

WHERE THE ORDER CAN AND CANNOT ESCAPE. `deduplicate_identities` has exactly two
consumers in the corpus: `canonical_identity_set`, which sorts the result and therefore
erases order, and `census_population_is_duplicate_free`, which counts it. The projection
consumer — `gunbc.design_ledgers compile_phase_frontier_blocks`, which renders committed
bytes into docs/design-ledgers.md — reaches it only through the sorting one, and that
order-independence is separately executed by
`the_census_digest_is_invariant_under_reordering_of_either_population` and
`phase_boards_are_invariant_under_population_reordering`, both passing here. So the
projected bytes cannot move under this change by construction, which is a stronger
statement than one byte comparison over one population would have been.

The arm is therefore not a belt on either current caller. It holds the contract for the
NEXT consumer — the one that would not survive a silent reordering, and that would
discover it as generated-artifact drift at a distance from the diff that caused it.

Co-Authored-By: Claude Opus 5 <noreply@anthropic.com>
Claude-Session: https://claude.ai/code/session_01W377Pq4Tp5eQjGBXtPQEoM
… two compiler_tests failures on this PR's run are main's, inherited through a merge ref pinned at 06:47Z against bb96afa

The merge ref CI built for f3839b2 was pinned at push time against main as it
then stood (bb96afa), which carried #9886's stale mirror. That is the exact
subject #10017 repaired at 06:56Z, nine minutes after this PR's run started.

Evidence, checked rather than assumed:
  - main at bb96afa, run 33596615712: 'test result: FAILED. 642 passed; 2
    failed' with render_rust_applied_type_routes_qualified_base_through_leaf_name
    and shell_service_unmodeled_output_key_refuses -- byte-identical counts and
    identities to this PR's rust-unit-tests failure.
  - #10017's own body names 'regen FAIL generated surface drift:
    compiler_tests.rs, std_realization_schedule.rs' reproduced on 4059156 and
    bb96afa -- the same two files this PR's build job reported.
  - This branch changes two .dag files and no Rust: git diff origin/main...HEAD
    over compiler_tests.rs and std_realization_schedule.rs is empty.

Merging rather than rebasing per the squash-merge policy, and pushing after
main's last move so the merge ref is recomputed against the repaired base.

Co-Authored-By: Claude Opus 5 <noreply@anthropic.com>
Claude-Session: https://claude.ai/code/session_01W377Pq4Tp5eQjGBXtPQEoM
…control it is missing

main landed a competing repair for the same defect while this PR waited for a floor
slot (`|> sort_by` then compare against the last kept element). Mine used a Set for
membership and `list_push` for the accumulator. Both fix the quadratic scan; I argued
mine additionally fixed a copied accumulator, and I measured that argument rather than
asserting it.

THE MEASUREMENT REFUTES MY PATCH. Three variants over an amplified probe — ~720
identities with every one duplicated, ten dedup passes per run, two rounds, same host:
mine 112.1s / 110.3s, main's 110.6s / 127.3s, main's-with-`list_push` 110.9s / 137.6s.
Round one spans 1.5% and round two is dominated by host noise in the wrong direction.
`concat(unique, [identity])` is not a measurable O(n) copy in this interpreter, so §6's
bare-minimum-cost argument does not distinguish the two implementations. Forking a
landed fix on an unmeasured preference is the thing this project keeps catching, so
main's implementation is taken whole and `dag/gunbc/self_host_compile_phase_frontier.dag`
is byte-identical to `origin/main` in this commit.

WHAT SURVIVES IS A GAP IN THAT FIX, NOT A COMPETING ONE. main's fold compares only
against the LAST KEPT element, which is sound exactly because the input was sorted first
— equal identities are adjacent. The sort is therefore a correctness precondition, not a
presentation choice, and nothing in the corpus pins it: delete `|> sort_by(identity =>
identity)` and the fold silently stops collapsing duplicates separated by any other
identity while still collapsing adjacent ones.

MEASURED, NOT ASSERTED, INCLUDING THE PART THAT DECIDES WHETHER IT IS WORTH LANDING:
  A  new witness against main's implementation                      PASS
  B  same witness, `sort_by` deleted                                FAIL
  C  the four pre-existing duplicate-wall witnesses vs that mutant  ALL PASS

C is the finding. `a_repeated_population_entry_is_refused_rather_than_collapsed`,
`a_duplicated_census_population_is_not_silently_collapsed_to_a_set`,
`a_receipt_whose_refused_population_repeats_a_path_is_refused_by_coherence` and
`current_persisted_compile_phase_frontier_holds` all stay green with the sort removed, so
this arm is the only thing in the corpus that reddens on it.

THE SUBJECT IS CHOSEN SO THE MUTANT CANNOT PASS: `["b", "a", "b"]` holds a duplicate that
is not adjacent in encounter order — 2 under the sorted fold, 3 without the sort — and the
all-adjacent cases sit beside it as the positive control the mutant still passes, which is
what makes the first arm discriminating rather than the set being merely two examples.

IT ASSERTS DUPLICATE SURVIVAL RATHER THAN OUTPUT ORDER, deliberately. Encounter order is
unobservable through both consumers (`canonical_identity_set` sorts its result,
`census_population_is_duplicate_free` reads only the count), so an order assertion would
pin a property no caller can distinguish — and the previous revision of this branch did
exactly that, which main's change would have reddened.

Co-Authored-By: Claude Opus 5 <noreply@anthropic.com>
Claude-Session: https://claude.ai/code/session_01W377Pq4Tp5eQjGBXtPQEoM
@briansrls
briansrls marked this pull request as ready for review September 3, 2026 01:25
@chatgpt-codex-connector

chatgpt-codex-connector Bot commented Sep 3, 2026 •

Copy link
Copy Markdown

Codex Review Summary

This comment shows the latest Codex review activity on this pull request.

Review Status Commit Review trigger
📝 Code Review ✅ Completed 2026-09-03T01:27:02.743127Z 48bf811 Draft marked ready
ℹ️ About Codex in GitHub

Your team has set up Codex to review pull requests in this repo. Reviews are triggered when you

  • Open a pull request for review
  • Mark a draft as ready
  • Comment "@codex review" or "@codex security review".

Codex reacts with 👀 while any review is running, comments if it has suggestions, and reacts with 👍 once all reviews finish with no findings.

@gunbai-bot

gunbai-bot Bot commented Sep 3, 2026

Copy link
Copy Markdown
Contributor

Closing as superseded — this PR has no remaining content, and fixing its CI would be work on an artifact that should not land.

Its one contribution is already in main. The 28-line sort-precondition control for deduplicate_identities merged as d6a7a0e7 (#10012); main carries the identical assertions.

GitHub's diff view is misleading here, which is why this is worth spelling out. The three-dot diff against the merge base shows "1 file changed, 28 insertions" — a clean, reviewable-looking addition. But the merge base predates d6a7a0e7, so that view is showing content main has since landed independently. The two-dot diff against main's tip tells the real story: 345 files, 3739 insertions, 39051 deletions — the branch is far behind main, not ahead of it.

And the file itself is stale in a way that matters. Main's copy of self_host_compile_phase_frontier_witness_test imports current_compile_phase_frontier_standing and asks through the standing; this branch's copy still imports phase_board_series_ratchet and asks through the ratchet. That is precisely the collapse that landed from another lane — which stern-lark-508 assessed itself, concluding that the other shape is better than its own for the specific reason it had already measured in its own implementation (a _from_verdict helper plus a current_* composition turned one memoized call into three, the middle one taking the whole receipt series as an argument, ~2-6% relative cost on both rewired modules across two floor runs).

So there is nothing to rescue. #10012 merged, #10032 was closed as superseded with an empty diff, and this is a third PR auto-opened on the session branch after the lane was closed out.

One method note kept from that lane, because it generalizes: comparing per-claim floor cpu_ms across two runs is unreadable raw — the host moved 8-13% between runs and flipped the sign of the result twice. Pairing touched identities against untouched controls measured in the same two runs is what made the signal readable, and checking only one of two rewired modules is what produced a false exculpation in between.

— sent from crisp-ibex-710

@gunbai-bot gunbai-bot Bot closed this Sep 3, 2026
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

None yet

Projects

None yet

Development

Successfully merging this pull request may close these issues.

1 participant