Repository navigation
Un-dark the affected-set falsifier: a status note in ci_layer_roots was deciding corpus-wide selection - #7988
Merged
Merged
Conversation
…as deciding corpus-wide selection The scheduled affected-set falsifier has failed on every run since 2026-08-06 ~17:00Z, and it failed at its BUILD GATE — the `Affected-set selection control (floor_skip_discovery_witness)` step — so the falsifier job never started and the cold control produced zero corpus evidence. Issue #7737 has been open on it since 2026-08-03; this is the current cause. ROOT CAUSE, measured. #7898 (Lane D) added import v2.compiler.self_host.wet_receipt_enrollment { ... } to `gunbc.ci_layer_roots`, used in exactly one place: the prose status projection `dispatch_bijection_real_roster_red_falsifier_self_host_wet_dark_note()`. Two of the three imported symbols were used only there; the third (`falsifier_self_host_wet_template_entries`) was never used at all. `gunbc.ci_spec` imports `gunbc.ci_layer_roots`, and nearly every witness entry imports `ci_spec`, so that one edge pulled the `v2.compiler.self_host` subtree — and `v2.lens.module_graph` through `self_host.frontier_probe_survey -> compiler.name_resolve -> lens.reference_deps` — into a large share of the corpus's import closures. Both are declared live-read carrier homes (`LIVE_READ_CARRIER_HOME_MODULES_V0`), and affected-set selection's axis (iv), `runtime_data_dependency_touched_via_carrier_closure`, is deliberately coarse: closure reaches a carrier home AND any path is touched counts as a hit, without checking which path. So every entry the edge reached stopped being skippable on EVERY diff. MEASURED, by walking the import graph at the last-good falsifier head (bb4048a) and the first-failing one (7afea5f): affected_set_floor_runner_test.dag closure 388 -> 435 modules carrier homes it reaches none -> v2.compiler.self_host, v2.lens.module_graph *_test.dag entries reaching a carrier home 144/1200 (12%) -> 210/1219 (17%) That entry is the selection control's cross-entry specimen, so `entry_file_helper_fn_edit_scopes_runs_to_touched_entry_only` — which asserts a helper-fn edit in one entry SKIPS witnesses in unrelated entries — measured passed=2/skipped=0 instead of 1/1 and panicked. The mechanism was answering correctly for the closure it was given; the closure was wrong. Verified against the arms directly (probe over the real module-graph facts): for `affected_set_floor_runner_test.dag`, import_closure=false, axis=Absent, path_literals=false, and `runtime_data_dependency=true` — axis (iv) alone was blocking the skip. THE FIX. The note joins two authorities — this module's waiting-pattern projection and wet_receipt_enrollment's cadence/enrollment facts — so it belongs with the consumer that already reaches both, which is its ONLY consumer: `v2.test.claim.witness_exclusion_reconciliation_test`. Re-homed there; the import is dropped from `ci_layer_roots`. No behavior of the note changes and its executing consumer is unchanged. Removing the edge restores, measured on this tree: affected_set_floor_runner_test.dag closure 435 -> 392, no carrier home reached *_test.dag entries reaching a carrier home 210/1219 -> 147/1222 (12%) so 63 witness entries regain affected-set skippability alongside the control. NOT FIXED HERE, and reported rather than improvised: (1) axis (iv)'s over-approximation is what converts any closure widening into corpus-wide run-all; making it path-precise is the named G2/G3 work in `live-read-witness-classification-design.md`, not a change to make under this brief. (2) `floor_skip_discovery_witness` is BUILT in per-PR CI but never RUN there — it executes only in the scheduled falsifier — which is why this merged green and surfaced ~8h later as total falsifier darkness rather than as a PR failure. Enrolling it per-PR is a cost decision for the operator. Co-Authored-By: Claude Opus 5 (1M context) <noreply@anthropic.com> Claude-Session: https://claude.ai/code/session_012azzL2u4Mn8HkAehT7dqcv
This file contains hidden or bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
Sign up for free
to join this conversation on GitHub.
Already have an account?
Sign in to comment
Add this suggestion to a batch that can be applied as a single commit.This suggestion is invalid because no changes were made to the code.Suggestions cannot be applied while the pull request is closed.Suggestions cannot be applied while viewing a subset of changes.Only one suggestion per line can be applied in a batch.Add this suggestion to a batch that can be applied as a single commit.Applying suggestions on deleted lines is not supported.You must change the existing code in this line in order to create a valid suggestion.Outdated suggestions cannot be applied.This suggestion has been applied or marked resolved.Suggestions cannot be applied from pending reviews.Suggestions cannot be applied on multi-line comments.Suggestions cannot be applied while the pull request is queued to merge.Suggestion cannot be applied right now. Please check back later.
The class, first: a prose status note created a real import edge, and the edge disabled affected-set selection for 63 witness entries
This is the durable finding, and it generalizes past the one edge it broke.
Measured (symbol-level, over the file):
gunbc.ci_layer_rootsimportedv2.compiler.self_host.wet_receipt_enrollmentwith three symbols. Two of them have exactly one call site each, both inside the status stringdispatch_bijection_real_roster_red_falsifier_self_host_wet_dark_note(). The third,falsifier_self_host_wet_template_entries, has no code reference anywhere in the file — it occurs in the import line and thereafter only inside prose (thereason:strings anddata … : Stringnotes).Inference (stated as such): that the edge exists in order to render status. Nothing records the author's intent; what is measured is that rendering the note is the only thing the edge is used for, and that removing the note removes every use of the edge.
DESIGN §4c says a status, receipt, or ruling belongs in a typed carrier and that plain prose is modeling debt. Here the debt was not cosmetic. It was paid in the selection mechanism: describing status required an import, the import was a dependency edge, and the edge propagated through
gunbc.ci_specinto a large share of the corpus.The generalized rule this suggests: any module whose only reason to import something is to describe status is a candidate for the same defect. A status projection must not decide selection for the corpus.
How the edge became corpus-wide selection loss
gunbc.ci_specimportsgunbc.ci_layer_roots, and nearly every witness entry importsci_spec. So the one edge pulled thev2.compiler.self_hostsubtree — andv2.lens.module_graphthroughself_host.frontier_probe_survey → compiler.name_resolve → lens.reference_deps— into those entries' import closures.Both are declared live-read carrier homes (
LIVE_READ_CARRIER_HOME_MODULES_V0), and affected-set selection's axis (iv),runtime_data_dependency_touched_via_carrier_closure, is deliberately coarse: closure reaches a carrier home AND any path is touched counts as a hit, without checking which path. So every entry the edge reached stopped being skippable on every diff.Measured
Walking the import graph at the last-good falsifier head (
bb4048a5f09) and the first-failing one (7afea5fd860):affected_set_floor_runner_test.dagclosureself_host,lens.module_graph*_test.dagentries reaching a carrier homeRe-measured against current main including #7933 (
f07c26d009d): unchanged at 435 / both carrier homes / 211/1222. #7933 does not affect this condition. 63 witness entries regain affected-set skippability.Verified against the arms directly, with a probe over the real module-graph facts. For
affected_set_floor_runner_test.dag:import_closure=false,axis=Absent,path_literals=false,runtime_data_dependency=true. Axis (iv) alone was blocking the skip.Why the falsifier went dark
That entry is the selection control's cross-entry specimen, so
entry_file_helper_fn_edit_scopes_runs_to_touched_entry_only— which asserts a helper-fn edit in one entry SKIPS witnesses in unrelated entries — measuredpassed=2 skipped=0instead of1/1and panicked.That panic is in the
falsifier-buildjob, which gatesfalsifier. So the cold whole-corpus run — the actual backstop — never started, and produced zero corpus evidence.Note the direction: the control failed on over-selection, which is the safe direction. Nothing was silently skipped by this. The harm is second-order and real: the failure gated the job, so the mechanism that catches under-selection was dark. A fix that merely silenced the assertion would restore nothing — the mechanism was answering correctly for the closure it was given, and the closure was wrong.
Outage bound (causal, narrower than "no success since Aug 3"): the build-gate cause began between run
31110312994(2026-08-06T14:20Z, control PASS) and31136192646(2026-08-07T00:53Z, control FAIL). Earlier failures in the no-success window are separate causes — one runner comms loss, and pre-08-06 failures inside thefalsifierjob itself rather than the build gate.Green by execution
The selection control passes on this branch at its real consumer — the
falsifier-buildjob, which is where it failed on main.Deliberately proven on both heads. #7933 lands 205 lines in
cli_run.rson the selection-admission path this control asserts against, so a proof pinned only to the pre-merge head would be stale evidence about the very commit most likely to perturb it.falsifier-buildconcludes success on the current head, so thefalsifierjob is reachable rather than skipped — for the first time since 2026-08-03.Cost note, corrected: the control takes 11m08s to complete. The ~53s figure visible in the failing main logs is time-to-panic on an aborted suite, not time-to-complete — the suite has eighteen scenarios and only ran two before asserting. This is the first completed run, so 11m08s is the first honest number for it. It matters to the per-PR enrollment question in item 2 below: 53s would be a rounding error, 11 minutes is a scheduling decision.
I could not run this witness locally: the session container SIGKILLs it (RSS still climbing at ~2.43 GiB, while the cgroup's own
memory.maxis 31.3 GiB andmemory.eventsreadsmax 0, oom 0— the limiter is outside the cgroup). So the CI run above is the executed evidence, not a local one. A post-merge falsifier run is what proves the corpus half; this run proves the gate.The fix
The note joins two authorities —
ci_layer_roots's waiting-pattern projection andwet_receipt_enrollment's cadence/enrollment facts — so it belongs with the consumer that already reaches both, which is its only consumer:v2.test.claim.witness_exclusion_reconciliation_test. Re-homed there; the import is dropped fromci_layer_roots. The note's behavior and its executing consumer are unchanged — not deleted, not duplicated.Not fixed here, reported rather than improvised
Axis (iv)'s over-approximation is what converts any closure widening into corpus-wide run-all. Today an import added to a widely-imported module can silently disable selection for hundreds of entries, and the only signal is the falsifier control dying in a way that takes the whole falsifier down. Named next-rung trigger: the G2/G3 call-reachability precision work in
docs/plans/live-read-witness-classification-design.md. Deliberately not scoped into this PR.floor_skip_discovery_witnessis BUILT in per-PR CI but never RUN there — it executes only in the scheduled falsifier. That is why this merged green and surfaced ~8h later as total falsifier darkness rather than as a PR failure. Observed runtime in the falsifier log is ~53s (06:03:03→06:03:56); peak memory is unmeasured (see below). Enrolling it per-PR is a cost decision for the operator, not one to make here.Ownership: this PR touches only
dag/gunbc/ci_layer_roots.dagandsrc/v2/test/claim/witness_exclusion_reconciliation_test.dag. It does not touchfalsifier_workflow.dag,claim_executor.rs, orfalsifier-alert.yml, so it does not overlap the live alert lanes (#7935, #7956).🤖 Generated with Claude Code
https://claude.ai/code/session_012azzL2u4Mn8HkAehT7dqcv