diff --git a/DESIGN.md b/DESIGN.md index 6042cf58ade..c87d8b94045 100644 --- a/DESIGN.md +++ b/DESIGN.md @@ -114,7 +114,7 @@ The recursion's payoff is that **language design itself opens up.** It is normal ## Recurring failure modes (instances of §3–§5, kept for pattern-matching) -hollow alias (minimality ≠ grounding) · state-space conflation (an `Option`/`None` meaning >2 things — split into named variants) · absorbing fallback (a failure arm that widens — rerun everything, scan all keys, always run — instead of refusing; ⊤-as-answer conflated with ⊤-as-ignorance; the deficit's frequency zeroed by construction, the corpus-denominated cost breaking the budget later) · cache impurity (key on declared-input content; byte-identical cached-vs-cold is the purity oracle) · reflection evidence ≠ structural proof (prove a read axis by execution, with a no-host-enumeration control) · coercion proven by normalized round-trip, not a golden string · parallel-representation debt (an honestly-marked scaffold duplicating a canonical fact is still a violation) · internal review finds missing tests, external review finds missing checks · unmarked workaround (an author-side absorbing fallback — routing around an obstacle with no dissolution trigger; the line-stop signal, never a landing state) · positional citation (a `file:line` pointer where a symbol was available — see §3). +hollow alias (minimality ≠ grounding) · state-space conflation (an `Option`/`None` meaning >2 things — split into named variants) · absorbing fallback (a failure arm that widens — rerun everything, scan all keys, always run — instead of refusing; ⊤-as-answer conflated with ⊤-as-ignorance; the deficit's frequency zeroed by construction, the corpus-denominated cost breaking the budget later) · **empty-observation narrow** (the absorbing fallback's mirror: ⊥-as-answer conflated with ⊥-as-ignorance — an observation that could not express what changed rendered as the verdict "nothing is affected", so the run narrows to nothing instead of widening. Strictly worse than the widen §5 already forbids: a widen is merely expensive, a narrow is silently uncovered. Live specimen: a push whose baseline ref *is* the pushed ref diffs the commit against itself, and the witness floor reported the resulting zero-path observation as "N unaffected (import-closure, skipped without resolve)" — receipt, two main pushes reporting an identical `4118 unaffected` on different commits, a skip count that cannot be a function of the diff. The tell that one observation is wrong rather than three local choices reasonable: its consumers disagree — compile-clean widened to whole-tree, regen gated on the event name, the witness floor skipped) · cache impurity (key on declared-input content; byte-identical cached-vs-cold is the purity oracle) · reflection evidence ≠ structural proof (prove a read axis by execution, with a no-host-enumeration control) · coercion proven by normalized round-trip, not a golden string · parallel-representation debt (an honestly-marked scaffold duplicating a canonical fact is still a violation) · internal review finds missing tests, external review finds missing checks · unmarked workaround (an author-side absorbing fallback — routing around an obstacle with no dissolution trigger; the line-stop signal, never a landing state) · positional citation (a `file:line` pointer where a symbol was available — see §3). ## Open threads diff --git a/dag/gunbc/design_document.dag b/dag/gunbc/design_document.dag index 88e61ebc7d1..6d2c0a0542f 100644 --- a/dag/gunbc/design_document.dag +++ b/dag/gunbc/design_document.dag @@ -145,7 +145,7 @@ fn section_7_blocks() -> List { fn failure_modes_blocks() -> List { [ h2(text: "Recurring failure modes (instances of §3–§5, kept for pattern-matching)"), - p(text: "hollow alias (minimality ≠ grounding) · state-space conflation (an `Option`/`None` meaning >2 things — split into named variants) · absorbing fallback (a failure arm that widens — rerun everything, scan all keys, always run — instead of refusing; ⊤-as-answer conflated with ⊤-as-ignorance; the deficit's frequency zeroed by construction, the corpus-denominated cost breaking the budget later) · cache impurity (key on declared-input content; byte-identical cached-vs-cold is the purity oracle) · reflection evidence ≠ structural proof (prove a read axis by execution, with a no-host-enumeration control) · coercion proven by normalized round-trip, not a golden string · parallel-representation debt (an honestly-marked scaffold duplicating a canonical fact is still a violation) · internal review finds missing tests, external review finds missing checks · unmarked workaround (an author-side absorbing fallback — routing around an obstacle with no dissolution trigger; the line-stop signal, never a landing state) · positional citation (a `file:line` pointer where a symbol was available — see §3)."), + p(text: "hollow alias (minimality ≠ grounding) · state-space conflation (an `Option`/`None` meaning >2 things — split into named variants) · absorbing fallback (a failure arm that widens — rerun everything, scan all keys, always run — instead of refusing; ⊤-as-answer conflated with ⊤-as-ignorance; the deficit's frequency zeroed by construction, the corpus-denominated cost breaking the budget later) · **empty-observation narrow** (the absorbing fallback's mirror: ⊥-as-answer conflated with ⊥-as-ignorance — an observation that could not express what changed rendered as the verdict \"nothing is affected\", so the run narrows to nothing instead of widening. Strictly worse than the widen §5 already forbids: a widen is merely expensive, a narrow is silently uncovered. Live specimen: a push whose baseline ref *is* the pushed ref diffs the commit against itself, and the witness floor reported the resulting zero-path observation as \"N unaffected (import-closure, skipped without resolve)\" — receipt, two main pushes reporting an identical `4118 unaffected` on different commits, a skip count that cannot be a function of the diff. The tell that one observation is wrong rather than three local choices reasonable: its consumers disagree — compile-clean widened to whole-tree, regen gated on the event name, the witness floor skipped) · cache impurity (key on declared-input content; byte-identical cached-vs-cold is the purity oracle) · reflection evidence ≠ structural proof (prove a read axis by execution, with a no-host-enumeration control) · coercion proven by normalized round-trip, not a golden string · parallel-representation debt (an honestly-marked scaffold duplicating a canonical fact is still a violation) · internal review finds missing tests, external review finds missing checks · unmarked workaround (an author-side absorbing fallback — routing around an obstacle with no dissolution trigger; the line-stop signal, never a landing state) · positional citation (a `file:line` pointer where a symbol was available — see §3)."), ] } diff --git a/dag/gunbc/executor_schedule_retention.dag b/dag/gunbc/executor_schedule_retention.dag index 0be70769eff..e53ad9247b0 100644 --- a/dag/gunbc/executor_schedule_retention.dag +++ b/dag/gunbc/executor_schedule_retention.dag @@ -16,6 +16,14 @@ data schedule_retention_retain_on_unknown: Bool = true data schedule_retention_retain_on_unknown_note: String = "When reachability CANNOT be computed for a cached module (a provenance gap), the safe arm is RETAIN - correctness-fail-closed but memory-fail-open - so it is a typed, per-module, COUNTED RetentionUnknown row, never a silent retain-everything (that absorbing fallback is a T-as-ignorance widen whose frequency the corpus would never see, S5)." +data schedule_retention_release_and_eviction_counted_apart: Bool = true + +data schedule_retention_release_vs_eviction_note: String = "A module is RELEASED when its retention refcount reaches zero; it is EVICTED only when that release actually dropped recorded cache state. The two are different facts and are counted apart (schedule_releases vs schedule_evictions, released_modules vs evicted_modules). They diverge on every run whose arming reaches modules the run never reconciles - the ordinary case, because arming spans the whole DISCOVERED roster while the affected set resolves a subset. Fusing them overstates the mechanism by that entire population: a live CI receipt read schedule_evictions=2200 against modules_refcounted=2200 and typed_cache_peak=883, i.e. it claimed to have evicted every module it had ever heard of, ~2.5x the modules that were ever cached, and a counter that equals the arm count by construction carries no information at all. Rust lockstep: SCHEDULE_RETENTION_RELEASE_AND_EVICTION_COUNTED_APART in cli_run.rs; discriminating RED: unreconciled_module_releases_without_evicting (an armed-but-never-reconciled module must release without evicting - re-fusing the counters reds it)." + +data schedule_retention_entry_line_asserts_execution: Bool = false + +data schedule_retention_entry_line_note: String = "The per-entry drain line witnesses ONE fact: the schedule moved PAST that entry, so its per-module state can no longer be read. It asserts nothing about selection or execution - index_arm_schedule_retention walks every DISCOVERED row before any predict-skip, so the line also appears for entries the affected set skipped without ever resolving them. It was formerly spelled entry='...', which two independent readers hours apart took as proof the entry had been scheduled and run, and that misreading cost hours of misdirected diagnosis on a real selection defect. It is now schedule_passed_entry='...' beside an ARMED-line legend, because a label that asserts a fact its mechanism cannot support is a defect of the same family as a wrong number (DESIGN section 5: a diagnostic must be typed and located, and locating it on the wrong subject is not locating it)." + data schedule_retention_underflow_refuses: Bool = true data schedule_retention_underflow_note: String = "A decrement past zero (a scheduled entry demands state the refcount said was fully consumed) is a schedule-derivation defect and REFUSES, typed and located - the mechanism never serves a wrong verdict against evicted state. Content-key recompute-on-miss is the correctness license; the refusal is the loudness that keeps a derivation bug from hiding." diff --git a/dag/test/claim/executor_schedule_retention_test.dag b/dag/test/claim/executor_schedule_retention_test.dag index fda8b431d9e..b62d3065c72 100644 --- a/dag/test/claim/executor_schedule_retention_test.dag +++ b/dag/test/claim/executor_schedule_retention_test.dag @@ -6,6 +6,8 @@ import gunbc.executor_schedule_retention { schedule_retention_heads_stay_resident, schedule_retention_retain_on_unknown, schedule_retention_underflow_refuses, + schedule_retention_release_and_eviction_counted_apart, + schedule_retention_entry_line_asserts_execution, } data witness_note: String = "In-corpus consumer of the schedule-derived retention policy carrier: asserts the modeled policy facts by execution (green-by-execution). The Rust realization in cli_run.rs mirrors these same facts and a Rust lockstep witness (schedule_retention_policy_matches_modeled_authority) pins the two together, so a policy edit that forgets either side reds." @@ -14,6 +16,14 @@ test fn schedule_retention_exposes_no_tunables() -> Bool { schedule_retention_tunable_count == 0 } +test fn schedule_retention_release_is_not_an_eviction() -> Bool { + schedule_retention_release_and_eviction_counted_apart +} + +test fn schedule_retention_drain_line_claims_no_execution() -> Bool { + schedule_retention_entry_line_asserts_execution == false +} + test fn schedule_retention_policy_is_fail_closed() -> Bool { schedule_retention_grain_is_per_module && schedule_retention_heads_stay_resident diff --git a/src/v1/stage0/src/cli_run.rs b/src/v1/stage0/src/cli_run.rs index 2b16d1f16d7..8c388cab194 100644 --- a/src/v1/stage0/src/cli_run.rs +++ b/src/v1/stage0/src/cli_run.rs @@ -7380,6 +7380,20 @@ fn note_interface_hash( /// Kept in lockstep by `schedule_retention_policy_matches_modeled_authority`. const SCHEDULE_RETENTION_TUNABLE_COUNT: i64 = 0; +/// Mirror of `gunbc.executor_schedule_retention` +/// `schedule_retention_entry_line_asserts_execution` (false): the per-entry drain line +/// witnesses the SCHEDULE PASSING an entry, never that the entry was selected or run — +/// `index_arm_schedule_retention` walks every discovered row before any predict-skip. +/// Kept in lockstep by `schedule_retention_policy_matches_modeled_authority`. +const SCHEDULE_RETENTION_ENTRY_LINE_ASSERTS_EXECUTION: bool = false; + +/// Mirror of `gunbc.executor_schedule_retention` +/// `schedule_retention_release_and_eviction_counted_apart` (true): `schedule_releases` +/// counts refcount-zero releases, `schedule_evictions` counts only the releases that +/// actually dropped cached state. Discriminating RED: +/// `unreconciled_module_releases_without_evicting`. +const RELEASE_AND_EVICTION_COUNTED_APART: bool = true; + /// Measurement control (RED #1): with `GUNBC_SCHEDULE_RETENTION_EVICT=0` the schedule /// still ARMS and COUNTS (refcounts, RetentionUnknown, decrement-underflow refusals) /// but drops nothing — reproducing today's pegged peak so the mechanism's effect on @@ -7413,12 +7427,27 @@ struct ModuleCacheKeys { /// One entry-completion's eviction decision — applied to the index caches by the /// caller (this struct owns no cache handles, so it stays a pure, unit-testable /// bookkeeper; the RED controls exercise it in isolation). +/// +/// RELEASE and EVICTION are separate facts and are counted separately. A module is +/// RELEASED when its refcount reaches zero (no remaining scheduled entry reaches it); +/// it is EVICTED only when that release actually dropped recorded cache state. The two +/// diverge on every run where an armed module was never reconciled — the ordinary case, +/// since arming spans the whole discovered roster while the affected set resolves a +/// subset. Counting the release as an eviction overstated the mechanism by the whole +/// unresolved population (a live receipt read `schedule_evictions=2200` against +/// `modules_refcounted=2200` and `typed_cache_peak=883`: it claimed to have evicted +/// every module it had ever heard of, ~2.5x the modules that were ever cached). #[derive(Default, Debug, PartialEq, Eq)] pub struct ScheduleEvictionBatch { pub typed_keys: Vec, pub raw_files: Vec, - /// The module names dropped this step (for the located receipt line). - pub module_names: Vec, + /// Module names whose refcount reached zero this step — released, whether or not + /// any cache state existed to drop (for the located receipt line). + pub released_modules: Vec, + /// The subset of `released_modules` that actually dropped recorded cache keys — + /// the only names for which state left a cache. Always a subset, never equal by + /// construction. + pub evicted_modules: Vec, } /// Schedule-derived per-module retention bookkeeping (pure). Refcount is keyed by @@ -7441,6 +7470,11 @@ pub struct ScheduleRetention { /// names already counted as RetentionUnknown (cached but reachability-uncomputable) /// — counted once, retained forever (never evicted). unknown_counted: HashSet, + /// Modules whose refcount reached zero (released). NOT an eviction count: a module + /// the run never reconciled has no cache state to drop, so its release frees nothing. + schedule_releases: u64, + /// Modules whose release actually dropped recorded cache keys — the honest measure + /// of state this mechanism removed. schedule_evictions: u64, retention_unknown: u64, /// Assembled `ResolvedGraph`s dropped from `resolved_graph_memo` on entry completion @@ -7481,6 +7515,7 @@ impl ScheduleRetention { entry_closures, cache_keys: std::collections::HashMap::new(), unknown_counted: HashSet::new(), + schedule_releases: 0, schedule_evictions: 0, retention_unknown: 0, resolved_graph_evictions: 0, @@ -7563,18 +7598,35 @@ impl ScheduleRetention { // above and REFUSES, instead of silently reading as "never armed". The // map is bounded by module count, so retaining zeros is free. if self.evict_enabled { + // Release and eviction are counted apart. A module the run never + // reconciled carries no recorded keys, so its release drops nothing + // — counting it as an eviction is a claim about freed state that no + // cache can corroborate. + let mut dropped_state = false; if let Some(keys) = self.cache_keys.remove(&name) { + dropped_state = !keys.typed_keys.is_empty() || !keys.raw_files.is_empty(); batch.typed_keys.extend(keys.typed_keys); batch.raw_files.extend(keys.raw_files); } - batch.module_names.push(name); - self.schedule_evictions += 1; + self.schedule_releases += 1; + if dropped_state { + self.schedule_evictions += 1; + batch.evicted_modules.push(name.clone()); + } + batch.released_modules.push(name); } } } Ok(batch) } + /// Modules released (refcount reached zero). Bounded above by the armed module + /// count and, at run end, equal to it — which is exactly why it is not an + /// eviction measure and never labelled as one. + pub fn schedule_releases(&self) -> u64 { + self.schedule_releases + } + /// Modules whose release actually dropped recorded cache state. pub fn schedule_evictions(&self) -> u64 { self.schedule_evictions } @@ -7670,9 +7722,16 @@ fn index_arm_schedule_retention(index: &MultiEntryIndex, rows: &[DiscoveryRow]) // Unconditional ARMED receipt: proves schedule-derived retention is LIVE on this run, // so a later `schedule_evictions=0` reads as "shared closure held resident" rather // than "never armed" (the private-index serial regime is the only one that arms). + // The legend is part of the receipt because the per-entry lines below are otherwise + // read as an execution log: they are not one, and `discovered_entries` says why — + // arming walks the DISCOVERED roster, before selection decides anything. eprintln!( - "[floor-drain] schedule-retention ARMED: entries={entries} \ - modules_refcounted={} evict_enabled={evict_enabled}", + "[floor-drain] schedule-retention ARMED: discovered_entries={entries} \ + modules_refcounted={} evict_enabled={evict_enabled} \ + (legend: a `schedule_passed_entry=` line below marks the schedule moving past an \ + entry — discovery + arming only; it does NOT mean the entry was selected or run. \ + `released_modules` = refcount reached zero; `evicted_modules` = the subset that \ + actually dropped cached state.)", sr.refcount_len() ); *index.schedule_retention.borrow_mut() = Some(sr); @@ -7919,7 +7978,7 @@ fn index_schedule_entry_completed( .is_some(), _ => false, }; - let (sched_evictions, retention_unknown, graph_evictions) = { + let (sched_releases, sched_evictions, retention_unknown, graph_evictions) = { let mut slot = index.schedule_retention.borrow_mut(); match slot.as_mut() { Some(sr) => { @@ -7927,20 +7986,32 @@ fn index_schedule_entry_completed( sr.note_graph_eviction(); } ( + sr.schedule_releases(), sr.schedule_evictions(), sr.retention_unknown(), sr.resolved_graph_evictions(), ) } - None => (0, 0, 0), + None => (0, 0, 0, 0), } }; // Unconditional (not floor_verbose-gated) so the receipt survives a step-cap timeout. + // + // `schedule_passed_entry=` names what this line actually witnesses: the schedule + // moved PAST that entry, so its per-module state can no longer be read. It asserts + // nothing about selection or execution — arming spans the whole discovered roster + // (`index_arm_schedule_retention` walks every row, before any predict-skip), so this + // line appears for entries the affected set skipped without ever resolving them. The + // former `entry='…'` spelling was read as "this entry was scheduled and ran" by two + // independent readers hours apart, which is a property of the label, not the readers. eprintln!( - "[floor-drain] schedule-retention: entry='{entry}' evicted_modules={} evicted_graph={} \ - schedule_evictions={sched_evictions} graph_evictions={graph_evictions} \ - retention_unknown={retention_unknown} typed_cache={} resolved_graphs={}", - batch.module_names.len(), + "[floor-drain] schedule-retention: schedule_passed_entry='{entry}' \ + released_modules={} evicted_modules={} evicted_graph={} \ + schedule_releases={sched_releases} schedule_evictions={sched_evictions} \ + graph_evictions={graph_evictions} retention_unknown={retention_unknown} \ + typed_cache={} resolved_graphs={}", + batch.released_modules.len(), + batch.evicted_modules.len(), graph_evicted as u8, index.typed_module_cache.borrow().len(), index.resolved_graph_memo.borrow().len(), @@ -7988,7 +8059,10 @@ mod heartbeat_feed_red_controls { #[cfg(test)] mod schedule_retention_red_controls { - use super::{ScheduleRetention, SCHEDULE_RETENTION_TUNABLE_COUNT}; + use super::{ + ScheduleRetention, RELEASE_AND_EVICTION_COUNTED_APART, + SCHEDULE_RETENTION_ENTRY_LINE_ASSERTS_EXECUTION, SCHEDULE_RETENTION_TUNABLE_COUNT, + }; /// Locate the modeled retention-policy carrier by walking up from the crate /// manifest dir until `dag/gunbc/executor_schedule_retention.dag` appears — the @@ -8051,9 +8125,23 @@ mod schedule_retention_red_controls { "schedule_retention_heads_stay_resident", "schedule_retention_retain_on_unknown", "schedule_retention_underflow_refuses", + "schedule_retention_release_and_eviction_counted_apart", ] { assert!(modeled_bool(&dag, flag), "modeled {flag} must be true"); } + assert_eq!( + modeled_bool(&dag, "schedule_retention_entry_line_asserts_execution"), + SCHEDULE_RETENTION_ENTRY_LINE_ASSERTS_EXECUTION, + "the per-entry drain line witnesses the schedule passing an entry, never its \ + execution — arming walks the discovered roster before selection decides \ + anything, so a line that claims execution asserts a fact the mechanism \ + cannot support" + ); + assert!( + RELEASE_AND_EVICTION_COUNTED_APART, + "release (refcount reached zero) and eviction (cached state actually dropped) \ + are counted apart in the realization" + ); } /// GREEN: eviction fires exactly at refcount zero. A shared module survives until @@ -8075,19 +8163,63 @@ mod schedule_retention_red_controls { // Finishing a drops a_only (rc 1→0); shared stays (rc 2→1). let batch_a = sr.entry_completed("a").expect("no underflow"); - assert_eq!(batch_a.module_names, vec!["a_only".to_string()]); + assert_eq!(batch_a.released_modules, vec!["a_only".to_string()]); + assert_eq!(batch_a.evicted_modules, vec!["a_only".to_string()]); assert_eq!(batch_a.typed_keys, vec!["tk_a".to_string()]); + assert_eq!(sr.schedule_releases(), 1); assert_eq!(sr.schedule_evictions(), 1); // Finishing b drops shared (rc 1→0) and b_only. let batch_b = sr.entry_completed("b").expect("no underflow"); - let mut got: Vec = batch_b.module_names.clone(); + let mut got: Vec = batch_b.released_modules.clone(); got.sort(); assert_eq!(got, vec!["b_only".to_string(), "shared".to_string()]); + assert_eq!(sr.schedule_releases(), 3); assert_eq!(sr.schedule_evictions(), 3); assert_eq!(sr.retention_unknown(), 0); } + /// RED for the counter defect this PR fixes: a module the run ARMED but never + /// reconciled has no recorded cache keys, so its refcount reaching zero frees + /// nothing. It must count as a RELEASE and NOT as an eviction. Before the fix both + /// counters were the same increment, so a live receipt read schedule_evictions=2200 + /// against a typed_cache_peak of 883 — a claim of freed state that no cache could + /// corroborate. This asserts the two counters DIVERGE on that population; if a + /// future edit re-fuses them, `schedule_evictions` here goes 1 -> 2 and reds. + #[test] + fn unreconciled_module_releases_without_evicting() { + let mut sr = ScheduleRetention::armed( + vec![( + "a".to_string(), + vec!["reconciled".into(), "never_reconciled".into()], + )], + true, + ); + // Only one of the two armed modules ever reaches reconcile (the ordinary case: + // arming spans the discovered roster, the affected set resolves a subset). + sr.record_module("reconciled", "tk", "f"); + + let batch = sr.entry_completed("a").expect("no underflow"); + let mut released = batch.released_modules.clone(); + released.sort(); + assert_eq!( + released, + vec!["never_reconciled".to_string(), "reconciled".to_string()], + "both armed modules are released — refcount reached zero for each" + ); + assert_eq!( + batch.evicted_modules, + vec!["reconciled".to_string()], + "only the module that had recorded cache state is evicted" + ); + assert_eq!(sr.schedule_releases(), 2); + assert_eq!( + sr.schedule_evictions(), + 1, + "an unreconciled module's release must not be counted as an eviction" + ); + } + /// RED #1 (eviction-disabled control): with `evict_enabled=false` the schedule /// still arms, records, counts, and REFUSES underflow — but drops nothing, so peak /// retention is the pre-M2 process-lifetime hold. This is the control run that @@ -8101,7 +8233,11 @@ mod schedule_retention_red_controls { sr.record_module("m", "tk", "f"); sr.record_module("orphan", "tk_o", "f_o"); // not in any closure → RetentionUnknown let batch = sr.entry_completed("a").expect("no underflow"); - assert!(batch.module_names.is_empty(), "disabled pole drops nothing"); + assert!( + batch.released_modules.is_empty() && batch.evicted_modules.is_empty(), + "disabled pole drops nothing" + ); + assert_eq!(sr.schedule_releases(), 0); assert_eq!(sr.schedule_evictions(), 0); assert_eq!( sr.retention_unknown(), @@ -8148,7 +8284,8 @@ mod schedule_retention_red_controls { assert_eq!(sr.retention_unknown(), 1); // The gap is never decremented/evicted: finishing `a` leaves it retained. let batch = sr.entry_completed("a").expect("no underflow"); - assert!(!batch.module_names.contains(&"gap".to_string())); + assert!(!batch.released_modules.contains(&"gap".to_string())); + assert!(!batch.evicted_modules.contains(&"gap".to_string())); } /// END-TO-END WIRING (real index, real closure computation, real cache eviction): @@ -8358,8 +8495,13 @@ pub struct IndexRetentionSnapshot { pub ownership_diag_cache_entries: usize, pub intern_table_entries: usize, pub typed_cache_evictions: u64, - /// Schedule-derived per-module evictions (v1-run-stability M2): modules dropped - /// because no remaining scheduled entry's closure reached them. + /// Schedule-derived per-module RELEASES (v1-run-stability M2): modules whose + /// refcount reached zero because no remaining scheduled entry's closure reaches + /// them. At run end this equals the armed module count by construction, so it is + /// a progress measure, not an eviction measure — see `schedule_evictions`. + pub schedule_releases: u64, + /// The releases that actually dropped cached state — the honest count of modules + /// this mechanism removed from a cache. pub schedule_evictions: u64, /// Counted `RetentionUnknown` rows: modules cached but reachability-uncomputable, /// retained (correctness-fail-closed) — visible and prioritizable, never absorbed. @@ -8452,6 +8594,12 @@ pub fn index_retention_snapshot(index: &MultiEntryIndex) -> IndexRetentionSnapsh ownership_diag_cache_entries: index.ownership_diag_cache.borrow().len(), intern_table_entries: index.intern_table.borrow().index.len(), typed_cache_evictions: index.typed_cache_evictions.get(), + schedule_releases: index + .schedule_retention + .borrow() + .as_ref() + .map(|s| s.schedule_releases()) + .unwrap_or(0), schedule_evictions: index .schedule_retention .borrow() @@ -8496,6 +8644,7 @@ fn retention_snapshot_peak( .max(b.ownership_diag_cache_entries), intern_table_entries: a.intern_table_entries.max(b.intern_table_entries), typed_cache_evictions: a.typed_cache_evictions.max(b.typed_cache_evictions), + schedule_releases: a.schedule_releases.max(b.schedule_releases), schedule_evictions: a.schedule_evictions.max(b.schedule_evictions), retention_unknown: a.retention_unknown.max(b.retention_unknown), peak_rss_bytes: match (a.peak_rss_bytes, b.peak_rss_bytes) { @@ -8540,13 +8689,14 @@ fn emit_floor_drain_receipt( eprintln!( "[floor-drain] receipt: groups={total_groups} \ typed_cache_peak={} parse_cache_peak={} resolved_memo_peak={} \ - intern_table_peak={} evictions={} schedule_evictions={} retention_unknown={} \ - peak_rss={} cap_entries={}", + intern_table_peak={} evictions={} schedule_releases={} schedule_evictions={} \ + retention_unknown={} peak_rss={} cap_entries={}", peaks.typed_module_cache_entries, peaks.parse_cache_entries, peaks.resolved_graph_memo_entries, peaks.intern_table_entries, peaks.typed_cache_evictions, + peaks.schedule_releases, peaks.schedule_evictions, peaks.retention_unknown, peaks @@ -16976,6 +17126,56 @@ fn floor_git_diff_range() -> Result { } } +/// Read back the baseline the affected-set diff was taken against — a PROJECTION of +/// `resolve_diff_baseline` (`v2.workflow.floor_diff_observe` +/// `floor_observe_diff_baseline_readout_for_ci`), never a second derivation. Used only +/// to LOCATE the no-observed-change diagnostic: a receipt that says "zero changed paths" +/// without naming the base it compared against cannot be acted on, because the +/// interesting case is precisely a base that names the head commit itself. Failure to +/// read it is not fatal here — the diagnostic degrades to an unnamed baseline and says +/// so, rather than suppressing the state. +fn floor_diff_baseline_readout() -> Result<(String, String), String> { + use v1_interpreter::Value; + let roots = default_source_roots(); + let entry = "src/v2/workflow/floor_diff_observe.dag"; + let (graph, indices) = resolve_entry_graph_shared(&roots, entry) + .map_err(|e| format!("floor_diff_observe resolve: {e}"))?; + let ctx = make_eval_context(&graph, indices, v1_interpreter::ExecutionMode::Wet); + let result = + v1_interpreter::run_in_context(&ctx, "floor_observe_diff_baseline_readout_for_ci", false) + .map_err(|e| format!("floor_observe_diff_baseline_readout_for_ci: {e}"))?; + match &result { + Value::Variant { + variant_name, + fields, + .. + } if ctx.sym_eq(*variant_name, "BaselineReadout") => { + let base = match ctx.field(fields, "ref") { + Some(Value::Str(s)) => s.clone(), + _ => return Err("BaselineReadout missing `ref`".to_string()), + }; + let event = match ctx.field(fields, "event_name") { + Some(Value::Str(s)) => s.clone(), + _ => return Err("BaselineReadout missing `event_name`".to_string()), + }; + Ok((base, event)) + } + Value::Variant { + variant_name, + fields, + .. + } if ctx.sym_eq(*variant_name, "BaselineReadoutRefused") => match ctx.field(fields, "reason") + { + Some(Value::Str(r)) => Err(r.clone()), + _ => Err("baseline readout refused (no reason)".to_string()), + }, + other => Err(format!( + "floor_observe_diff_baseline_readout_for_ci returned `{}`, expected FloorDiffBaselineReadout", + ctx.format_value(other) + )), + } +} + fn string_list_from_value(val: &v1_interpreter::Value, field: &str) -> Result, String> { use v1_interpreter::Value; match val { @@ -19199,6 +19399,7 @@ fn run_discovery_corpus_with_options_inner( &rows, &index, &diff_edits, + &changed_paths, ); // Derive every leg for the WHOLE roster here, above the width dispatch, while this // thread's shared index is warm. At width > 1 the pool hands each worker its own chunk @@ -19685,6 +19886,7 @@ fn eprintln_affected_set_categorization( rows: &[DiscoveryRow], index: &MultiEntryIndex, diff_edits: &FloorDiffEdits, + changed_paths: &[String], ) { let total = rows.len(); let entries = rows @@ -19707,30 +19909,94 @@ fn eprintln_affected_set_categorization( NodeFrontierSelectionMode::Applied => { let declared_paths = index.module_graph_facts.declared_repo_paths(); let touched: Vec = diff_edits.touched_entry_files.iter().cloned().collect(); - match discovery_entry_fast_skip_without_resolve( + let skipped = discovery_entry_fast_skip_without_resolve( rows, &index.module_graph_facts, &declared_paths, &touched, diff_edits, - ) { - Ok(fast) => { - let skipped = rows.iter().filter(|r| fast.contains(&r.entry)).count(); - let candidates = total - skipped; - eprintln!( - "{ts} [affected-set] {total} witness(es) across {entries} entr(y/ies) · {skipped} unaffected (import-closure, skipped without resolve) · {candidates} in the affected closure (resolving to decide node-frontier)" - ); - } - Err(e) => { - eprintln!( - "{ts} [affected-set] {total} witness(es) across {entries} entr(y/ies) · upfront import-closure categorization unavailable ({e}); per-shard selection is authoritative" - ); - } - } + ) + .map(|fast| rows.iter().filter(|r| fast.contains(&r.entry)).count()); + // The baseline is read back ONLY in the state that needs locating, so the + // ordinary path pays nothing for it. + let located = changed_paths + .is_empty() + .then(|| match floor_diff_baseline_readout() { + Ok((base, event)) => format!("baseline='{base}' event='{event}'"), + Err(e) => format!("baseline=UNREADABLE ({e})"), + }); + eprintln!( + "{ts} {}", + affected_set_applied_report_line(total, entries, skipped, located.as_deref()) + ); } } } +/// The Applied-mode report, rendered purely so both of its states are assertable. +/// +/// Its other state — `located` is `Some` — names the case in which the observation +/// carried NO changed path at all. +/// +/// A zero-changed-path observation and "the diff proves these rows unaffected" are +/// different states, and the ordinary categorization line renders the first as the +/// second — every row lands in the `unaffected (import-closure, skipped without +/// resolve)` bucket, which reads as a computed verdict. That is the exact mirror of +/// DESIGN §5's absorbing fallback: there, a mechanism that cannot compute the affected +/// set substitutes the SUPERSET and conflates ⊤-as-answer with ⊤-as-ignorance; here it +/// substitutes the EMPTY set and conflates ⊥-as-answer with ⊥-as-ignorance. The narrow +/// is the worse of the two, because a widen is merely expensive while a narrow is +/// silently uncovered. +/// +/// It is not hypothetical: a push whose baseline ref IS the pushed ref compares the +/// commit to itself, so a merge to the default branch observes nothing and every entry +/// the merge itself modified skips, with no line naming it (skips are counted, not +/// narrated). Receipt: main-push runs 30774223741 and 30773369033 report an identical +/// `4118 unaffected` on two different commits — a skip count that cannot be a function +/// of a diff it does not depend on. +/// +/// This is REPORTING ONLY. The run disposition is untouched, deliberately: the empty +/// diff has no special run arm (operator ruling 2026-07-05, PR-A — it was dissolved +/// into the general disposition and must not be resurrected as one). What was missing +/// was not an arm but the state's visibility; the baseline itself is fixed separately. +fn affected_set_applied_report_line( + total: usize, + entries: usize, + skipped: Result, + located: Option<&str>, +) -> String { + let head = format!("[affected-set] {total} witness(es) across {entries} entr(y/ies)"); + match (located, skipped) { + (Some(located), skipped) => { + let under = match skipped { + Ok(n) => format!("{n} of {total}"), + Err(_) => "an uncounted number of".to_string(), + }; + format!( + "{head} · NO OBSERVED CHANGE ({located}): the diff observation returned ZERO changed paths, \ + so no row can be proven affected and {under} witness(es) skip under it. This is NOT the \ + verdict 'nothing is affected' — a baseline that names the head commit itself (a push whose \ + base ref is the pushed ref) produces exactly this observation, and that is ignorance about \ + what changed, not a finding that nothing did. Read every `skipped` count in this run's \ + receipts as skipped-under-no-observation, never as skipped-because-unaffected. Run \ + disposition is unchanged by this line (the empty diff has no special run arm — operator \ + ruling 2026-07-05); it makes the state countable, it neither widens nor narrows the run." + ) + } + (None, Ok(skipped)) => { + let candidates = total.saturating_sub(skipped); + format!( + "{head} · {skipped} unaffected (import-closure, skipped without resolve) · \ + {candidates} in the affected closure (resolving to decide node-frontier)" + ) + } + (None, Err(e)) => format!( + "{head} · upfront import-closure categorization unavailable ({e}); per-shard \ + selection is authoritative" + ), + } +} + /// Live realization view: stream affected witnesses to stderr as they finish, one colored /// line per shard, so a run reads as "the affected set unrolling in real time" rather than a /// silent wait then a summary. On by default (opt out with `GUNBC_FLOOR_QUIET=1`); color @@ -31778,6 +32044,91 @@ mod witness_layer_roots_compile_clean_tests { } } + /// The affected-set report must not render a zero-changed-path OBSERVATION as the + /// verdict "these rows are unaffected". Both directions are asserted against the + /// SAME counts, so the discriminator is the observation state and nothing else: a + /// renderer that dropped the distinction fails one arm or the other. + #[test] + fn no_observed_change_is_reported_as_its_own_state_not_as_unaffected() { + let observed = super::affected_set_applied_report_line(6374, 886, Ok(4104), None); + assert!( + observed.contains("4104 unaffected (import-closure, skipped without resolve)") + && observed.contains("2270 in the affected closure"), + "a real observation still reports the computed categorization: {observed}" + ); + assert!( + !observed.contains("NO OBSERVED CHANGE"), + "a real observation must NOT claim the no-observation state: {observed}" + ); + + let unobserved = super::affected_set_applied_report_line( + 6374, + 886, + Ok(4104), + Some("baseline='origin/main' event='push'"), + ); + assert!( + unobserved.contains("NO OBSERVED CHANGE") + && unobserved.contains("baseline='origin/main'") + && unobserved.contains("event='push'"), + "the no-observation state must be named AND located — an unlocated 'zero \ + changed paths' cannot be acted on: {unobserved}" + ); + assert!( + !unobserved.contains("unaffected (import-closure, skipped without resolve)"), + "the no-observation state must not borrow the computed-verdict phrasing — \ + that conflation is the defect: {unobserved}" + ); + assert!( + unobserved.contains("4104 of 6374"), + "the state must carry its own frequency (rows skipped under it): {unobserved}" + ); + + // A receipt whose whole job is to stop a misreading has to be readable. Source + // indentation leaks into a multi-line `format!` unless every continuation is a + // real `\`-newline — it did, and review caught runs of ~18 spaces mid-sentence. + // Asserted rather than fixed-once, so the next edit cannot silently reintroduce it. + for line in [ + &observed, + &unobserved, + &super::affected_set_applied_report_line(1, 1, Err("boom".into()), None), + ] { + assert!( + !line.contains(" "), + "report line carries a run of literal spaces from source indentation: {line}" + ); + } + } + + /// The baseline read-back is a live consumer of the modeled projection + /// (`v2.workflow.floor_diff_observe.floor_observe_diff_baseline_readout_for_ci`), + /// executed here rather than asserted: it must ANSWER for a push-shaped event and + /// must REFUSE — carrying the model's reason — when the event names a merge target + /// it cannot derive. The refusal arm is the discriminating one: a readout that + /// fabricated a base would answer both times. + #[test] + fn diff_baseline_readout_answers_or_refuses_but_never_fabricates() { + with_env_test_lock(|| { + with_workspace_cwd(|| { + let _ovr = EnvGuard::remove("GUNBC_CI_DIFF_BASE"); + let _event = EnvGuard::set("GITHUB_EVENT_NAME", "push"); + let (base, event) = super::floor_diff_baseline_readout() + .expect("a push event resolves to a baseline"); + assert!(!base.is_empty(), "the answering arm must name a base ref"); + assert_eq!(event, "push"); + + let _pr = EnvGuard::set("GITHUB_EVENT_NAME", "pull_request"); + let _no_base = EnvGuard::remove("GITHUB_BASE_REF"); + let refusal = super::floor_diff_baseline_readout() + .expect_err("a pull_request with no base ref must refuse, not fabricate"); + assert!( + refusal.contains("GITHUB_BASE_REF"), + "the refusal must carry the modeled reason, got: {refusal}" + ); + }); + }); + } + /// Hand-Rust receipt: the emit leg is a strict superset of resolve for the same sources. #[test] fn emit_success_implies_resolve_success_on_live_witness_roots() { diff --git a/src/v2/workflow/floor_diff_observe.dag b/src/v2/workflow/floor_diff_observe.dag index 5746854a0f0..60874f4e588 100644 --- a/src/v2/workflow/floor_diff_observe.dag +++ b/src/v2/workflow/floor_diff_observe.dag @@ -114,6 +114,27 @@ fn floor_observe_git_diff_unified_for_ci() -> FloorUnifiedDiffResult { floor_observe_git_diff_unified(policy: gunbc_ci_spec.diff_policy) } +data floor_diff_baseline_readout_note: String = "A PROJECTION of the already-resolved baseline for diagnostics only — it observes no git, decides nothing, and adds no second baseline authority (resolve_diff_baseline stays the single one, DESIGN section 3). Its consumer is the affected-set no-observed-change diagnostic in the seed executor: when the observation returns zero changed paths, the receipt must be able to NAME the baseline the comparison was made against, because 'nothing changed' and 'the baseline could not express what changed' are different states with different remedies and the second is what a self-referential baseline produces (a push whose base ref is the pushed ref itself). Locating the state requires naming its base and event, so the diagnostic reads them from here rather than re-deriving them." + +type FloorDiffBaselineReadout + = BaselineReadout { ref: String, event_name: String } + | BaselineReadoutRefused { reason: String } + +fn floor_observe_diff_baseline_readout(policy: DiffPolicy) -> FloorDiffBaselineReadout { + match floor_resolve_baseline(policy: policy) { + BaselineRefused { reason: reason } => BaselineReadoutRefused { reason: reason } + BaselineSelected { baseline: baseline } => + BaselineReadout { + ref: diff_baseline_ref(b: baseline) as String, + event_name: floor_observe_ci_diff_event().event_name + } + } +} + +fn floor_observe_diff_baseline_readout_for_ci() -> FloorDiffBaselineReadout { + floor_observe_diff_baseline_readout(policy: gunbc_ci_spec.diff_policy) +} + fn floor_name_status_from_raw(raw: String) -> FloorNameStatusDiffResult { match git_diff_name_status_parse(raw: raw) { GitDiffNameStatusTruncated { pending: _, entries: _ } =>