Repository navigation
Evidence doc: why CI performance is tanking (2026-07-10/11 main-red incident, argued serially with receipts) - #6469
Merged
Merged
Conversation
…(increment 1) The Rc-safe half of run ≜ realize ∘ materialize ∘ dependency_view: compose the two catamorphisms that need no interpreter thread-safety. - src/v2/std/spine.dag: level-profile fold over the DependencyView DAG → critical_path_depth (longest dependency chain / the reduce spine) + independence_width (max nodes at one level / what parallelizes to hw width). spine_receipt emits the §9 numbers by COMPUTATION, not assertion, joining materialize's content-hash Share/dedup counts. - Discriminating RED (§9): single chain A→B→C → depth 3 / width 1 (serial); root over N independent leaves → depth 2 / width N. - Witnesses 9/9 (spine) green by execution; materialize 6/6 unchanged. The RUNNER (wall-clock parallel execution, §8 steps 3-4) stays increment 2+, gated on interpreter Rc→Arc (v1 interp is !Send). This increment measures the achievable width/critical-path floor the runner targets; it does not fabricate wall-clock N×. Co-Authored-By: Claude Opus 4.8 (1M context) <noreply@anthropic.com>
…e PR CI root cause: the recompute-trace extension edited v1_interpreter.rs, which hosts the atom_identity_hash/hash_combine fnv1a64 intrinsics that dag_collect_fingerprint_witness executes. That correctly invalidated the re-verify 'assumed-green node-frontier' SKIP of that witness (cone-touch, fail-closed) — but the witness is a pre-existing 806s (13min) fnv1a64 whole-corpus walk, so un-skipping it blew the 30-min floor budget. Main stays green only because its interpreter cone is untouched (witness stays skipped). The interpreter extension is a separable, env-gated (GUNBC_RECOMPUTE_TRACE=1), READ-mode diagnostic — not a dependency of the spine/materialize analysis substrate (pure .dag, content_hash-based). Reverting v1_interpreter.rs to main restores byte-identical interpreter content → the fingerprint witness returns to assumed-green SKIP → floor budget restored. The recompute-trace extension lands as its own follow-up PR where the fingerprint-witness re-run is expected. Co-Authored-By: Claude Opus 4.8 (1M context) <noreply@anthropic.com>
…sses
The operator's 4-rule business logic generalized on one axis: the decision is
a function of WHEN the redundancy is knowable and WHETHER what was knowable
was prepared for. Errors fire only on knowable-but-unprepared; genuine
emergence and declared triviality are typed acceptances, never silence.
std.materialization_ladder (dag/std):
- Frame/FrameKind: SharedStateFrame | IsolatedChildrenFrame |
ReplayedFrame{attempts} | UnboundedSiblingsFrame — the nested-scope axis;
a 'run' at any layer is a frame, never a different kind of thing.
- DemandNature: Pure | IdempotentEffect | WorldRead{envelope_declared} |
FreshEffect — the D3 effect gate; TTL = unmodeled dependency XOR declared
staleness, never an eviction knob.
- CacheProvider: keying (ContentKeyed|ExistenceKeyed) + EvictionPolicy
(ScopeExit|SpacePacked) REQUIRED by construction — rule 3 unwritable-bad.
- LadderVerdict (total): AuthoredDuplication | Discharged | RefusedNoProvider
| RefusedScopeTooNarrow | RefusedExistenceKeyed (the build-if-absent #6352
wall) | RefusedUnmodeledWorldRead | ExemptFreshEffect |
AcceptedBelowCostFloor | AcceptedSingleRecompute.
- Declared-emergent frames obligate UP FRONT: retry frames derive
checkpointing, unbounded-sibling frames (server loops, CI-runs-over-time)
derive persistent caches — prepare-before-demand.
12/12 witnesses green by execution; each test is one cell of the table,
fixtures mirror the live worked example (workflow/jobs/steps, retry, fleet).
Co-Authored-By: Claude Opus 4.8 (1M context) <noreply@anthropic.com>
…④ + the state x decision table - roadmap_authority ④ (ROADMAP.md regenerated via main_wet, drift-consistent): the nested-frame law, plurality cells, declared-emergent prepare-up-front, keying/staleness walls, the caching-completion un-shelve, the ①-vs-§2 cross-run-caching contradiction subsumed into per-node derived verdicts, and the v1.compiler.ownership §3-convergence row. - duplicate-work design doc: the state x decision table as the qualifier's final verdict logic + the ownership consolidation subsection. Co-Authored-By: Claude Opus 4.8 (1M context) <noreply@anthropic.com>
… law Operator synthesis (2026-07-09): Materialization = the verdict vocabulary; ownership = the verdict computation at the eval frame; realization = provider selection discharging the verdict at each frame's carrier. Share's handler is layer-aware — the cheapest cache is a REFERENCE (same program + ref-capable carrier + readonly => point-at-it, no copy, no destroyed memory), demoting through HAMT structural share / process memo / artifact / CAS, and demotion must be priced, never silent (#6249 clone-fallback is the receipt). Census of every hand-rolled instance found in-tree (16 rows): each named with its frame, ladder cell, action, and dissolution trigger — ownership (consolidated), Rc/HAMT (reference tier), clone-fallback (demotion wall), ParseTable + cached_stage (C5 grounding), M1 memo / intern / assumed-green / sccache / cargo-cache (C2 provider rows), PROCESS_RESOLVE_STORE (C3 = the 9GB scope fix), build-if-absent (landed as the keying wall), resolve-cache #5789 (rule-4 requirements), recompute-trace (state-4 finding source). Peers kept distinct: affected-set, Independence/Placement, mutable state. Sequence C1-C5 with C1 = this PR (ladder + live CI gate). Co-Authored-By: Claude Opus 4.8 (1M context) <noreply@anthropic.com>
…providers
First consolidation increment (operator-directed): v1.compiler.ownership's
decision IS provider selection at the value grain, now expressed in the ladder
and proven by mirror witnesses.
- ProviderTier: Reference | Copy | Memo{keying} | Artifact{keying} | Cas{keying}
— keying folded INTO store tiers, so an existence-keyed reference is
unwritable BY SHAPE (§5 construction); frame obligations at isolation LCAs
are dischargeable only by store tiers (a reference cannot cross an isolation
boundary — witnessed).
- Value tier: AccessMode (Read|Consume|Carry|Project), ValueUsage,
value_materialization -> ValueDead | ValueMoved | ValueSharedByReference |
ValueDemotedToCopy (PRICED, requires a declared CopyTier provider — the
#6249 silent clone-fallback made refusable) | ValueRefusedAffine |
ValueRefusedNoCarrierProvider.
- Faithful to v1's THREE distinct plurality readings, kept separate:
take_count = semantic_consumer_count (Consumed only, affine axis);
value_access_plurality = binding_fan_out (Carry excluded, reference axis);
borrow_count = whole_value_borrow_count (Read+Carry not Project, movability).
- Fold-accumulator case mirrored at conservative pre-FoldAccUnwrap semantics:
carries excluded from plurality but blocking the move -> demoted-to-copy.
23/23 witnesses green (14 frame cells + 9 value cells). Corpus-level
equivalence against live v1 ownership folds = the C4 receipt.
Co-Authored-By: Claude Opus 4.8 (1M context) <noreply@anthropic.com>
…ist + natures pin 19)
gunbai-bot
Bot
changed the title
Contributor
Author
|
The ci red on Structural relief is #6472 (operator-directed serial job split, currently 2 approvals + CI pending): it removes the 12m25s emit-determinism gate from this exact 60-minute window and moves the release build to its own budget. Plan: once #6472 merges, refresh this branch from main so its CI runs the split workflow, and re-run. Content here is final — no changes planned. — sent from merry-moth-539 |
This was referenced Jul 11, 2026
Merged
briansrls
added a commit
that referenced
this pull request
Jul 13, 2026
…or peak (review corrections) The routed review (sharp-bee-290) caught the doc grading the wrong axis: the adaptive governor packs width to the memory.high line BY CONSTRUCTION (ci_floor_measurement.dag's own text), so floor_peak_post ~15 GiB is a design constant under queue pressure — a successful retention cut raises width and drops wall-clock while the peak stays at the line. Amended: (1) §0 states the failure axis precisely (retention -> forced_serial -> step-cap time death); (2) M1/M2/exit done-bars now completion-within-step-budget at width > 1, with the isolated probe peak as the retention measure; (3) M0 widened with the dial-side baseline (width/wall/outcome paired with retention) so the mechanism is tested, not assumed; (4) M2 go/no-go reworded off the peak; (5) eval-memo (#6441/#6469) scope boundary stated: frame-exit drains it per witness (claim_batch.rs:458/638, claim_executor.rs:623/691) — its OOM class is the single giant witness, owned by the long/ lane. Co-Authored-By: Claude Fable 5 <noreply@anthropic.com>
briansrls
added a commit
that referenced
this pull request
Jul 13, 2026
…ial receipt (resolve peak 5.27→1.48 GB; whole-corpus run 120m-killed → 8m green at width 9) (#6528) * v1 run-stability throughline: decision tree + milestones for the corpus-OOM lane (measurement-first, construction fixes, cache-grain retention strip) Operator-requested alignment doc before any cut lands. Grounded in today's receipts: 11.1 MB corpus -> 5.28 GB single strict resolve (typecheck envs = 87%), floor censored-pinned at the 15 GiB memory.high clamp, per-module retention growing with closure size (superlinear total). Locates the three construction sites (union base-choice, merge_inductive_fields fold-from-empty, fresh cycle sets), clears func_env, and stages M0 denominate -> M1 construction fixes -> M2 roster-aware env strip (the tripped #5892 flip-trigger's action, corrected to the grain that frees bytes) -> M3 reference to S2a Inc-B. Non-goals pinned: no envelope raises, no absorbing fallbacks, no v1 investment beyond the bridge; terminal dissolution = Wave 3+ native-artifact cutover. Co-Authored-By: Claude Fable 5 <noreply@anthropic.com> * WIP: corpus OOM * throughline: grade throughput/completion, not the governor-pinned floor peak (review corrections) The routed review (sharp-bee-290) caught the doc grading the wrong axis: the adaptive governor packs width to the memory.high line BY CONSTRUCTION (ci_floor_measurement.dag's own text), so floor_peak_post ~15 GiB is a design constant under queue pressure — a successful retention cut raises width and drops wall-clock while the peak stays at the line. Amended: (1) §0 states the failure axis precisely (retention -> forced_serial -> step-cap time death); (2) M1/M2/exit done-bars now completion-within-step-budget at width > 1, with the isolated probe peak as the retention measure; (3) M0 widened with the dial-side baseline (width/wall/outcome paired with retention) so the mechanism is tested, not assumed; (4) M2 go/no-go reworded off the peak; (5) eval-memo (#6441/#6469) scope boundary stated: frame-exit drains it per witness (claim_batch.rs:458/638, claim_executor.rs:623/691) — its OOM class is the single giant witness, owned by the long/ lane. Co-Authored-By: Claude Fable 5 <noreply@anthropic.com> * M0 receipt: inductive-field list mass = 409M entries (400x all map entries) — M1b is the first cut --ancestry-report on today's tree (1,101 modules): te.inductive_fields list mass 409,240,584 Rc slots (~3.3 GB in Vec slots alone ≈ the whole typecheck bloat); worst single module 22.6M entries at ~1,338 types (~17k duplicates per type) — merge_inductive_fields concat-on-collision compounding multiplicatively along the import DAG. All map-spine dup_factors 1.00 (zero cross-module sharing, as diagnosed) but map entries total only ~1.07M — not the bytes. M1 order re-derived from the receipt: M1b first, M1a second, M1c demoted (cycle sets 24k entries, negligible). Pre-fix oracle baseline pinned: corpus_fingerprint=82e9c68b96617067 emit_graph_fingerprint=cfdc338c2795a035. Co-Authored-By: Claude Fable 5 <noreply@anthropic.com> * WIP: corpus OOM * WIP: corpus OOM * WIP: corpus OOM * WIP: corpus OOM * M1b: regenerated seed + receipts on the carrier (probe 5.27->2.05 GB, fingerprint-identical, battery green) Completes the M1b cut whose .dag authority edits the session checkpoint already pushed (04_env.dag merge_inductive_fields skip-if-equal + insert-not-concat; 04_infer.dag parents-fold init from first parent, both build_type_env paths): the regenerated seed .rs land here so regen --verify holds on the committed pair (verified: regen_divergence_count=0). Receipts (all by execution, controlled pre/post on identical tree+binary shape, recorded in the throughline doc §1): - whole-tree strict resolve peak RSS 5,274,587,136 -> 2,045,423,616 B (-61%) - te.inductive_fields list mass 409,240,584 -> 95,050,553 (-77%); worst module 22.6M -> 4.1M - corpus_fingerprint d34a91a4d8fe8845 pre == post (byte-identical); emit_graph fingerprint identical — the dedupe is invisible to the whole-corpus diagnostics+emit oracle - v1-compiler lib battery 212/0 (incl. module-grain + import-closure equivalence modules); claim_executor memo oracles 3/3 Doc updated: status = executing (M0 DONE, M1b LANDED, M1a next); M0's 'measurement only' framing scoped to the instrument; M1 sign-off provenance recorded; the throughput half of the M1 done-bar (whole-corpus floor at width>1 within step budget) explicitly declared outstanding, not asserted. Addresses cursor REQUEST_CHANGES (review 37480): the PR now carries the M1 scope honestly (title/body updated), the post-cut receipts are on the carrier, and the sign-off gate is documented as discharged. Co-Authored-By: Claude Fable 5 <noreply@anthropic.com> * WIP: corpus OOM * WIP: corpus OOM * WIP: corpus OOM * M1 dial receipt + DESIGN.md open-thread link (fixes this PR's own doc-graph orphan) Whole-corpus falsifier cold run on the M1b-2 tree: EXIT=0 green, all 1,757 witnesses / 0 skipped, ~8m wall, max_width_reached=9, forced_serial=0, hard_backoffs=0, peak 8.83 GB VmHWM (local 33.6 GB budget; fits CI's 16 GB envelope with headroom — CI-slot confirmation declared outstanding). Pre-M1b baseline on the same artifact: forced_serial=1, 474 hard_backoffs, step-cap TIME death. The DESIGN.md open-threads row links the throughline doc, which the restored cold sweep itself flagged as a doc-graph orphan (doc_graph_has_no_orphan_docs red on this tree; green with the link, dangling-links witness green). Also records the three latent corpus reds the revived sweep caught (two pre-existing, fixed in #6530; this orphan the third). Co-Authored-By: Claude Fable 5 <noreply@anthropic.com> * WIP: corpus OOM --------- Co-authored-by: Brian Searls <briansrls@gunb.ai> Co-authored-by: Claude Fable 5 <noreply@anthropic.com>
briansrls
added a commit
that referenced
this pull request
Jul 13, 2026
…larger spine (probe 1.48→1.40 GB, fingerprints byte-identical) + inductive-field base-case guard (#6534) * v1 run-stability throughline: decision tree + milestones for the corpus-OOM lane (measurement-first, construction fixes, cache-grain retention strip) Operator-requested alignment doc before any cut lands. Grounded in today's receipts: 11.1 MB corpus -> 5.28 GB single strict resolve (typecheck envs = 87%), floor censored-pinned at the 15 GiB memory.high clamp, per-module retention growing with closure size (superlinear total). Locates the three construction sites (union base-choice, merge_inductive_fields fold-from-empty, fresh cycle sets), clears func_env, and stages M0 denominate -> M1 construction fixes -> M2 roster-aware env strip (the tripped #5892 flip-trigger's action, corrected to the grain that frees bytes) -> M3 reference to S2a Inc-B. Non-goals pinned: no envelope raises, no absorbing fallbacks, no v1 investment beyond the bridge; terminal dissolution = Wave 3+ native-artifact cutover. Co-Authored-By: Claude Fable 5 <noreply@anthropic.com> * WIP: corpus OOM * throughline: grade throughput/completion, not the governor-pinned floor peak (review corrections) The routed review (sharp-bee-290) caught the doc grading the wrong axis: the adaptive governor packs width to the memory.high line BY CONSTRUCTION (ci_floor_measurement.dag's own text), so floor_peak_post ~15 GiB is a design constant under queue pressure — a successful retention cut raises width and drops wall-clock while the peak stays at the line. Amended: (1) §0 states the failure axis precisely (retention -> forced_serial -> step-cap time death); (2) M1/M2/exit done-bars now completion-within-step-budget at width > 1, with the isolated probe peak as the retention measure; (3) M0 widened with the dial-side baseline (width/wall/outcome paired with retention) so the mechanism is tested, not assumed; (4) M2 go/no-go reworded off the peak; (5) eval-memo (#6441/#6469) scope boundary stated: frame-exit drains it per witness (claim_batch.rs:458/638, claim_executor.rs:623/691) — its OOM class is the single giant witness, owned by the long/ lane. Co-Authored-By: Claude Fable 5 <noreply@anthropic.com> * M0 receipt: inductive-field list mass = 409M entries (400x all map entries) — M1b is the first cut --ancestry-report on today's tree (1,101 modules): te.inductive_fields list mass 409,240,584 Rc slots (~3.3 GB in Vec slots alone ≈ the whole typecheck bloat); worst single module 22.6M entries at ~1,338 types (~17k duplicates per type) — merge_inductive_fields concat-on-collision compounding multiplicatively along the import DAG. All map-spine dup_factors 1.00 (zero cross-module sharing, as diagnosed) but map entries total only ~1.07M — not the bytes. M1 order re-derived from the receipt: M1b first, M1a second, M1c demoted (cycle sets 24k entries, negligible). Pre-fix oracle baseline pinned: corpus_fingerprint=82e9c68b96617067 emit_graph_fingerprint=cfdc338c2795a035. Co-Authored-By: Claude Fable 5 <noreply@anthropic.com> * WIP: corpus OOM * WIP: corpus OOM * WIP: corpus OOM * WIP: corpus OOM * M1b: regenerated seed + receipts on the carrier (probe 5.27->2.05 GB, fingerprint-identical, battery green) Completes the M1b cut whose .dag authority edits the session checkpoint already pushed (04_env.dag merge_inductive_fields skip-if-equal + insert-not-concat; 04_infer.dag parents-fold init from first parent, both build_type_env paths): the regenerated seed .rs land here so regen --verify holds on the committed pair (verified: regen_divergence_count=0). Receipts (all by execution, controlled pre/post on identical tree+binary shape, recorded in the throughline doc §1): - whole-tree strict resolve peak RSS 5,274,587,136 -> 2,045,423,616 B (-61%) - te.inductive_fields list mass 409,240,584 -> 95,050,553 (-77%); worst module 22.6M -> 4.1M - corpus_fingerprint d34a91a4d8fe8845 pre == post (byte-identical); emit_graph fingerprint identical — the dedupe is invisible to the whole-corpus diagnostics+emit oracle - v1-compiler lib battery 212/0 (incl. module-grain + import-closure equivalence modules); claim_executor memo oracles 3/3 Doc updated: status = executing (M0 DONE, M1b LANDED, M1a next); M0's 'measurement only' framing scoped to the instrument; M1 sign-off provenance recorded; the throughput half of the M1 done-bar (whole-corpus floor at width>1 within step budget) explicitly declared outstanding, not asserted. Addresses cursor REQUEST_CHANGES (review 37480): the PR now carries the M1 scope honestly (title/body updated), the post-cut receipts are on the carrier, and the sign-off gate is documented as discharged. Co-Authored-By: Claude Fable 5 <noreply@anthropic.com> * WIP: corpus OOM * WIP: corpus OOM * WIP: corpus OOM * M1 dial receipt + DESIGN.md open-thread link (fixes this PR's own doc-graph orphan) Whole-corpus falsifier cold run on the M1b-2 tree: EXIT=0 green, all 1,757 witnesses / 0 skipped, ~8m wall, max_width_reached=9, forced_serial=0, hard_backoffs=0, peak 8.83 GB VmHWM (local 33.6 GB budget; fits CI's 16 GB envelope with headroom — CI-slot confirmation declared outstanding). Pre-M1b baseline on the same artifact: forced_serial=1, 474 hard_backoffs, step-cap TIME death. The DESIGN.md open-threads row links the throughline doc, which the restored cold sweep itself flagged as a doc-graph orphan (doc_graph_has_no_orphan_docs red on this tree; green with the link, dangling-links witness green). Also records the three latent corpus reds the revived sweep caught (two pre-existing, fixed in #6530; this orphan the third). Co-Authored-By: Claude Fable 5 <noreply@anthropic.com> * WIP: corpus OOM * WIP: corpus OOM * WIP: corpus OOM * M1a receipt: sized union dispatch controlled pair (probe 1.481→1.401 GB, fingerprints byte-identical), battery green Co-Authored-By: Claude Fable 5 <noreply@anthropic.com> --------- Co-authored-by: Brian Searls <briansrls@gunb.ai> Co-authored-by: Claude Fable 5 <noreply@anthropic.com>
This file contains hidden or bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
Sign up for free
to join this conversation on GitHub.
Already have an account?
Sign in to comment
1 participant
Add this suggestion to a batch that can be applied as a single commit.This suggestion is invalid because no changes were made to the code.Suggestions cannot be applied while the pull request is closed.Suggestions cannot be applied while viewing a subset of changes.Only one suggestion per line can be applied in a batch.Add this suggestion to a batch that can be applied as a single commit.Applying suggestions on deleted lines is not supported.You must change the existing code in this line in order to create a valid suggestion.Outdated suggestions cannot be applied.This suggestion has been applied or marked resolved.Suggestions cannot be applied from pending reviews.Suggestions cannot be applied on multi-line comments.Suggestions cannot be applied while the pull request is queued to merge.Suggestion cannot be applied right now. Please check back later.
Operator-requested evidence document: the coherent end-to-end argument for the current main-red, every claim carrying its receipt (run id + step timing, floor receipt line, or controlled A/B). Emitted at
docs/plans/ci-performance-tanking-evidence.mdfrom the.dagauthority (dag/gunbc/plans/ci_performance_tanking_evidence.dag), registered in plan batch g.The argument in one breath: green baseline (floor 16–18 min, witness batch 46s, emit-determinism gate the real wall) → #6441 did NOT slow the floor (16m04s; the +30min was a cold release build from the .rs change — a standing ~33min term on every .rs merge) → the affected-set flip changed enrollment 8→72+ entries and (a) exposed a latent-red witness serially, (b) multiplied eval demand ×12 (27.9M keyed calls, 53min duplicated-demand CPU disclosed), while (c) resolve stayed 22 SECONDS pooled — refuting the resolve-is-the-CI-wall narrative → the #6441 identity tax (controlled 3.1×) multiplies exactly that corpus eval → the retention defect was a separate memory class, already fixed (#6456, independent receipt) → since 23:11 every red is a timeout: build + corpus-eval + fleet saturation stacking inside 60/45-minute budgets. Three distinct "witness slowness" pains are disambiguated (CI-red acute / dev-loop request-major resolve / offline recipes), the plan is ordered P1–P4 with owners and acceptance receipts, and open unknowns are typed so they can't harden into claims.
Doc-only PR (one
.dagauthority + registry line + emitted.md). Dissolution trigger declared: retires when main is green with the flip on and the two pinning receipts (P1 red enumeration, P2 corpus A/B) have landed.🤖 Generated with Claude Code