From d762793f67b3897bb2f684467382aa89ddf5b914 Mon Sep 17 00:00:00 2001 From: Brian Searls Date: Sat, 19 Sep 2026 20:57:11 +0000 Subject: [PATCH 1/6] live_deploy: stop carrying a floor-peak figure this row does not own The passage quoted "22.74 GiB uncensored" as what the 2026-09-12 demand ruling sized the runner slot from. The ruling says the opposite in as many words: it records TWO disagreeing readings, takes the LARGER, and calls sizing to the smaller "the fail-open direction". So this row attributed to the ruling the exact arm the ruling refused, and named the fail-open direction as what the fail-closed decision was. The figure is REMOVED rather than corrected. This passage's own argument is that the floor's demand does not apply to this deployment at all, and that holds at any value -- so nothing downstream of it changes. It is a citation defect, not a wrong decision. Carrying a second copy of a number owned by gunbc.runner_slot_allocation gunbc_runner_slot_memory_max_ruling_note is the DESIGN section 6 defect (name the producer, never transcribe its output), which is also why no newer figure is substituted here even though one now exists. Co-Authored-By: Claude Opus 5 (1M context) --- dag/gunbc/live_deploy/emit.dag | 14 +++++++++++++- 1 file changed, 13 insertions(+), 1 deletion(-) diff --git a/dag/gunbc/live_deploy/emit.dag b/dag/gunbc/live_deploy/emit.dag index c3ea5983659..e518f39943f 100644 --- a/dag/gunbc/live_deploy/emit.dag +++ b/dag/gunbc/live_deploy/emit.dag @@ -426,13 +426,25 @@ fn release_revision_execstart_text(binding: ReleaseRevisionBinding) -> String { // // WHAT FALSIFIED IT: the runner row stopped being an allowance for "a gunbc process". The // 2026-09-12 demand ruling sizes it from the REQUIRED FLOOR'S measured peak -- one prepared subject -// over the whole corpus, 22.74 GiB uncensored -- which is work this deployment never performs. The +// over the whole corpus, a figure gunbc.runner_slot_allocation gunbc_runner_slot_memory_max_ruling_note +// owns and this row deliberately does not restate -- which is work this deployment never performs. The // same 2026-09-04 measurement the old note cited for scale is the refutation at this size: serve // peaked at 3.46 GB and the belt at 0.97 GB. Sharing one number with the floor was a coincidence of // magnitude that read as an identity of workload, and the moment the floor's number moved, the // coincidence ended while the sentence asserting the identity did not. That is the pattern this // corpus keeps finding -- a name that states a coincidence as a law. // +// THE FIGURE THIS PASSAGE USED TO QUOTE WAS THE WRONG ONE, AND IT IS REMOVED RATHER THAN CORRECTED. +// It read "22.74 GiB uncensored". The ruling it attributes that to records TWO disagreeing readings +// and states in as many words that it takes the LARGER and that sizing to the smaller "is the +// fail-open direction" -- so this row named the arm the ruling refused, and named the fail-open +// direction as what the fail-closed decision was. Nothing downstream of THIS passage changes: its +// whole argument is that the floor's demand does not apply to this deployment at all, and that holds +// at any value. It is a citation defect, not a wrong decision. The repair is to stop carrying a +// second copy of a number this row does not own (DESIGN section 6: name the producer, never +// transcribe its output) -- which is also why no newer figure is substituted here, even though one +// now exists. +// // THE VALUES DO NOT CHANGE TODAY, deliberately. They are the bytes this deployment already runs // with, restated as its own fact with its own justification rather than re-derived to something // tighter in the same change that decouples them. Whether 16 GiB is the right bound for a 4.43 GB From 004bc3e428aec79cc45dc35edef99713b5b54ca5 Mon Sep 17 00:00:00 2001 From: Brian Searls Date: Sat, 19 Sep 2026 22:08:14 +0000 Subject: [PATCH 2/6] Floor memory demand: a peak may not travel without its censoring standing THE DEFECT THIS CLOSES. A peak read from a run that was HELD measures what the process was ALLOWED, not what it needs. gunbc.runner_slot_allocation already states that reading -- a peak equal to the throttle line is "the censored lower bound ... the workload being held, not the workload being measured" -- and then, on 2026-09-12, recorded two disagreeing figures for one run and said it did not know which cgroup level each named. Nothing in the corpus made that ambiguity unwritable, so the smaller figure propagated to gunbc.live_deploy.emit as "the required floor's measured peak, uncensored". gunbc.floor_memory_demand derives the standing from the kernel's own counters instead of leaving it to prose. Three arms, and which three is the whole point: DemandObserved (nothing held it -- the only arm citable as a demand), DemandBounded (a lower bound, whether because the process was killed or because it was throttled -- one arm because the consequence for a consumer is identical, with the cause in the fields), and DemandUnreadable, which must never render like either. A supervisor that cannot tell "could not read" from "it fit" reproduces one layer out the conflation this exists to remove. TWO AXES, KEPT APART. Termination (how the process ended, observed from outside) and censoring (whether anything held it) are independent: a run can exit cleanly having been throttled throughout, and a run can be killed having never been throttled. Folding them into one enum loses exactly the pair an operator needs. WHY THE SUBJECT IS THE PROCESS AND THE READING IS TAKEN FROM OUTSIDE. An in-process observer -- a Drop guard, an exit hook, a final log line -- covers ordinary returns and unwinding panics and nothing else. It does not run on panic=abort, process::abort, process::exit or SIGKILL, and Rust's allocation-error handler normally ABORTS. So allocation failure and the OOM kill, the two terminations a memory instrument most exists to report, are precisely the ones no in-process observer can report. The kernel maintains memory.peak and memory.events regardless; a supervisor that outlived the child can still read them. Measured 2026-09-19: a systemd-managed unit REAPS its cgroup on exit (the peak is gone, though Result=oom-kill survives), while a cgroup the supervisor created itself keeps both readable past a SIGKILL. THE DISCRIMINATING PAIR IS TWO REAL RUNS, NOT TWO FIXTURES. Both on srv1 at 56375ec44a, same binary, same tree, back to back, one variable -- the throttle line. Run A uncensored: peak 28962353152, all events zero, qualifies as a demand. Run B at a 16 GiB line: peak 17181028352 with 17466 high events, does not. Four further witnesses cover the cases counters alone would miss: a peak pinned exactly to the line with zero events (the shape every figure behind the old 16/15 row had), a kill far below every limit, an unobserved termination, and an unreadable cgroup. All six green by execution via claim_batch. ALSO: one recurring_failure_mode row, filed from a near-miss I caused taking these measurements. A cgroup placement write failed EPERM under a 2>/dev/null and the workload ran UNCONSTRAINED on a shared host -- and an uncapped run is indistinguishable from a capped one in everything the workload itself emits. The row's general form is that a precondition establishing the ENVIRONMENT must be verified by readback, never inferred from the call's exit status; the repo's own ctrl-build "forwarding env: (none)" warning is the same class on a different knob. Co-Authored-By: Claude Opus 5 (1M context) --- dag/gunbc/floor_memory_demand.dag | 177 ++++++++++++++++++ ...ailure_runs_the_workload_unconstrained.dag | 30 +++ .../floor_memory_demand_witness_test.dag | 133 +++++++++++++ 3 files changed, 340 insertions(+) create mode 100644 dag/gunbc/floor_memory_demand.dag create mode 100644 dag/gunbc/recurring_failure_mode/suppressed_precondition_failure_runs_the_workload_unconstrained.dag create mode 100644 dag/test/claim/floor_memory_demand_witness_test.dag diff --git a/dag/gunbc/floor_memory_demand.dag b/dag/gunbc/floor_memory_demand.dag new file mode 100644 index 00000000000..0cf5594a871 --- /dev/null +++ b/dag/gunbc/floor_memory_demand.dag @@ -0,0 +1,177 @@ +module gunbc.floor_memory_demand + +import std.types { Bool, Int, NonEmptyStr, String } +import std.nat { Nat } +import std.measure { ByteSize, byte_size_count } +import extdeps.linux.cgroup_v2_memory { + CgroupMemoryLimitValue, + CgroupMemoryLimited, + CgroupMemoryUnlimited, + CgroupMemoryLimitUnparseable, +} + +// WHAT THIS MODULE IS FOR, AND WHY IT IS NOT A SECOND HOME FOR A NUMBER. +// +// A peak RSS or cgroup charge read from a run that was HELD measures what the process was +// ALLOWED, not what it needs. `gunbc.runner_slot_allocation` states the reading directly -- a +// peak equal to the throttle line is "the censored lower bound ... the workload being held, not +// the workload being measured" -- and then, on 2026-09-12, recorded two disagreeing figures for +// one run and said in as many words that it did not know which cgroup level each named. +// +// This module makes that ambiguity unwritable rather than re-describing it. A demand figure may +// not travel without its standing: whether anything held the workload, and whether the number is +// a demand or a floor under one. THE JUDGMENT IS DERIVED FROM THE KERNEL'S OWN COUNTERS, never +// authored beside them. +// +// SUBJECT: the measured PROCESS (or its declared process subtree), not any function inside it. +// That distinction is load-bearing and is why the reading is taken by a SUPERVISOR from outside. +// An in-process observer -- a Drop guard, an exit hook, a final log line -- covers ordinary +// returns and unwinding panics and NOTHING ELSE. It does not run on `panic = abort`, on +// `process::abort`, on `process::exit`, or on SIGKILL, and Rust's allocation-error handler +// normally ABORTS rather than unwinds. So allocation failure and the OOM kill -- the two +// terminations a memory instrument most exists to report -- are exactly the ones no in-process +// observer can report on. The kernel maintains `memory.peak` and `memory.events` regardless, and +// a supervisor that outlived the child can still read them, so the supervisor's post-mortem read +// is the load-bearing evidence and any in-process emission is a convenience beside it. + +// HOW THE MEASURED PROCESS ENDED, observed by the supervisor from OUTSIDE it (a wait status, or +// the service manager's own record). This is an INDEPENDENT AXIS from whether the reading was +// censored: a run can terminate cleanly and still have been throttled the whole way, and a run +// can be killed without ever having been throttled. Folding them into one enum would lose +// exactly the pair of facts the operator needs to tell "it fits" from "it was held". +type SupervisedTermination + = TerminationExited { code: Int } + | TerminationSignalled { signal: Int } + | TerminationUnobserved { cause: NonEmptyStr } + +// The cumulative counters from `memory.events`, read AFTER the child is gone. `oom_kills` is the +// kernel's own statement that it killed something in this cgroup; it is not inferred from the +// wait status, because the two can disagree and the disagreement is informative. +type CgroupMemoryEvents { + high_events: Nat, + max_events: Nat, + oom_kills: Nat, +} + +// WHY A READ CAN REFUSE, each arm naming a distinct thing that went wrong. None of these may be +// rendered as a zero or as "it fit": a supervisor that cannot distinguish "could not read" from +// "the workload was within its limit" reproduces one layer out the very conflation this module +// exists to remove. +type DemandReadRefusalCause + = CgroupDirectoryAbsent { path: String } + | CgroupFileUnreadable { path: String, detail: String } + | CgroupValueUnparseable { file: NonEmptyStr, body: String } + | SupervisorDidNotOwnTheCgroup { detail: String } + +// THE THREE ARMS AN OPERATOR READS, and they are three because two of the pairings are the ones +// that have actually caused harm here. +// +// `DemandObserved` is the only arm whose peak may be cited as a DEMAND. `DemandBounded` carries a +// peak that is a LOWER BOUND on demand -- the workload wanted at least this much and we cannot say +// how much more -- and it covers both ways that happens, because the consequence for a consumer is +// identical: the number may not be used to size anything downward. Which way it happened is +// carried in the fields rather than in a fourth arm, so a consumer asking "may I treat this as a +// demand?" gets a two-valued answer and a consumer asking "why not?" still gets the cause. +// +// `DemandUnreadable` is separate from both and must never render like either. +// +// WHAT `DemandObserved` DOES NOT SAY, because the gap is easy to cross by accident: it says +// NOTHING HELD THE MEMORY. It does not say the workload was COMPLETE. A run that refused half its +// witnesses and exited early was not throttled, so it qualifies here, and its peak is still a +// truthful statement about what that run demanded -- but it is not a statement about what a +// complete run would demand. Workload completeness is the caller's subject and is carried by the +// caller's own verdict, never inferred from this one. +type FloorMemoryQualification + = DemandObserved { + peak: ByteSize, + limit_max: CgroupMemoryLimitValue, + limit_high: CgroupMemoryLimitValue, + events: CgroupMemoryEvents, + termination: SupervisedTermination, + } + | DemandBounded { + peak_lower_bound: ByteSize, + limit_max: CgroupMemoryLimitValue, + limit_high: CgroupMemoryLimitValue, + events: CgroupMemoryEvents, + termination: SupervisedTermination, + } + | DemandUnreadable { cause: DemandReadRefusalCause } + +// A LIMIT THAT IS ABSENT CANNOT HOLD ANYTHING, and an unparseable one cannot be reasoned about at +// all. `CgroupMemoryUnlimited` is the `high=max` case that makes an uncensored measurement +// possible in the first place, so it must read as "this line did not bind" rather than as a very +// large number. +fn peak_reached_limit(peak: ByteSize, limit: CgroupMemoryLimitValue) -> Bool { + match limit { + CgroupMemoryLimited { bytes: b } => byte_size_count(b: peak) >= byte_size_count(b: b) + CgroupMemoryUnlimited => false + CgroupMemoryLimitUnparseable { body: _ } => false + } +} + +fn any_events(events: CgroupMemoryEvents) -> Bool { + events.high_events > 0 || events.max_events > 0 || events.oom_kills > 0 +} + +// A TERMINATION THAT WAS NOT AN ORDINARY EXIT WITHHELD THE REST OF THE WORKLOAD, so whatever the +// peak reached, the process had not finished wanting memory. An UNOBSERVED termination counts the +// same way and deliberately so: not knowing how a process ended is not evidence that it ended +// well, and the fail-closed reading of an unobserved end is that the figure is a lower bound. +fn termination_withheld_completion(termination: SupervisedTermination) -> Bool { + match termination { + TerminationExited { code: _ } => false + TerminationSignalled { signal: _ } => true + TerminationUnobserved { cause: _ } => true + } +} + +// THE JUDGMENT. Every input is a kernel counter or a wait status; nothing here is authored. +// +// The order of the tests is the order of severity, and each one independently establishes that the +// peak is a lower bound rather than a demand: +// a kill -- the workload was stopped before it finished wanting memory; +// an event -- reclaim or the OOM killer acted, so the workload was held below what it asked for; +// a pin -- the peak sits AT a limit, which is the censored signature even when the counters +// have not yet ticked, and is what a throttled run looks like from the outside. +// Only a run that survived all three may be read as a demand. +fn qualify_floor_memory_demand( + peak: ByteSize, + limit_max: CgroupMemoryLimitValue, + limit_high: CgroupMemoryLimitValue, + events: CgroupMemoryEvents, + termination: SupervisedTermination, +) -> FloorMemoryQualification { + let killed = termination_withheld_completion(termination: termination) + let pinned = + peak_reached_limit(peak: peak, limit: limit_high) + || peak_reached_limit(peak: peak, limit: limit_max) + if killed || any_events(events: events) || pinned { + DemandBounded { + peak_lower_bound: peak, + limit_max: limit_max, + limit_high: limit_high, + events: events, + termination: termination, + } + } else { + DemandObserved { + peak: peak, + limit_max: limit_max, + limit_high: limit_high, + events: events, + termination: termination, + } + } +} + +// THE ONE QUESTION A SIZING CONSUMER MAY ASK, so that no consumer re-derives the judgment and +// gets a different answer (DESIGN section 3). A bounded or unreadable qualification is never a +// demand; only `DemandObserved` is. +fn qualification_is_a_demand(q: FloorMemoryQualification) -> Bool { + match q { + DemandObserved { peak: _, limit_max: _, limit_high: _, events: _, termination: _ } => true + DemandBounded { peak_lower_bound: _, limit_max: _, limit_high: _, events: _, termination: _ } => false + DemandUnreadable { cause: _ } => false + } +} diff --git a/dag/gunbc/recurring_failure_mode/suppressed_precondition_failure_runs_the_workload_unconstrained.dag b/dag/gunbc/recurring_failure_mode/suppressed_precondition_failure_runs_the_workload_unconstrained.dag new file mode 100644 index 00000000000..3f0a6b3556c --- /dev/null +++ b/dag/gunbc/recurring_failure_mode/suppressed_precondition_failure_runs_the_workload_unconstrained.dag @@ -0,0 +1,30 @@ +module gunbc.recurring_failure_mode.suppressed_precondition_failure_runs_the_workload_unconstrained + +import std.types { NonEmptyStr } +import gunbc.recurring_failure_mode { RecurringFailureMode } + +data suppressed_precondition_failure_runs_the_workload_unconstrained: RecurringFailureMode = RecurringFailureMode { + identity: "suppressed_precondition_failure_runs_the_workload_unconstrained" as NonEmptyStr, + + receipts: [ + "**a precondition that establishes the ENVIRONMENT a workload runs in fails, its diagnostic is discarded, and the workload proceeds in the unconstrained world while every observable it produces is identical to the constrained one** (INVALID STATE: a run reported as bounded -- capped, sandboxed, flag-carrying, quota-bearing -- when the operation that was to bound it refused. The workload's own output cannot discriminate, because a limit that is never approached and a limit that was never applied produce the same bytes, the same exit code and the same duration.)", + + "WHY IT IS NOT `control_plane_acknowledgement_minted_as_effect`, whose DISTINGUISHING FACTS exclude it in as many words: that class requires that every precondition genuinely HELD and that no diagnostic fires, and calls the defect a category substitution rather than a missing validation. Here the precondition did NOT hold and the diagnostic DID fire -- it was thrown away. Nor is it `absorbing_fallback`: nothing widened and no plausible value was substituted. The failure arm was not taken at all, so there is no arm to inspect.", + + "WHY IT SURVIVES: the suppression is usually incidental rather than considered -- a `2>/dev/null` reaching one command further than intended, a `|| true`, an ignored non-zero in the middle of a pipeline -- so there is no moment at which an author decides to proceed unbounded. And the next N steps all succeed, loudly and correctly, which reads as confirmation.", + + "THE DISCRIMINATOR IS OBSERVABILITY, NOT CARE, and it is what makes readback mandatory rather than prudent. Ask: if this precondition had failed, would anything in the subject's own output differ? Where the answer is no, the exit status of the setting call is a CONTROL-side fact and the only subject-side fact is a read of the realized environment -- the kernel's membership, the receiving process's view of the flag, the mount's actual options. Reading back the DESIRED value is not enough and not what is meant: `cgroup.procs` containing the pid is the kernel's own answer about membership, whereas the write returning 0 is the request.", + + "EVIDENCE, EXECUTED, 2026-09-19 on srv1 (a shared 502 GiB host carrying six live CI runner slots and 400+ sessions). A probe intended to verify that a cgroup memory receipt survives an OOM kill placed its child with `echo $PID > $CG/cgroup.procs` inside a block carrying `2>/dev/null`. The write failed EPERM -- a pid cannot be moved ACROSS a delegation boundary, from `session-N.scope` into a cgroup under `user@1000.service` -- so the 256 MiB cap was never applied and an unbounded allocation loop ran free. The shape was repeated once before it was noticed. The host went from roughly 315 GiB available to 29 GiB. Measured aftermath: no collateral damage -- the only OOM kill in the window was a correctly-capped earlier probe with `constraint=CONSTRAINT_MEMCG`, and all six runner slots stayed loaded/active/running -- but the absence of harm was luck, not construction.", + + "THE SAME CLASS IS ALREADY NAMED ELSEWHERE IN THIS REPOSITORY'S OPERATING GUIDANCE, which is the argument that it generalises past cgroups: `ctrl-build` prints `forwarding env: ...` on every remote dispatch precisely because a forwarded build flag that never reached rustc produces a GREEN BUILD THAT PROVES NOTHING, and the standing instruction is to STOP if it says `(none)`. Same invalid state, different environment knob: the subject cannot tell you whether the flag arrived, so the harness must report the realized environment separately.", + + "RUNG: found BELOW the ladder -- nothing refuses, nothing counts, and the run reports success. CEILING 2, mechanically preventable, at the harness rather than in the substrate: a helper that establishes an environment can REFUSE to start the workload until it has read the realized environment back, which makes the unconstrained run unreachable through that helper. It cannot climb to 3 or 4 for hand-written shell, because a shell author can always write the unverified form directly.", + + "NEXT-RUNG TRIGGER: a single modelled supervisor that owns environment establishment for measurement runs -- create the cgroup, SPAWN the child into it (never move it), read back membership and the realized limits, and only then exec the workload -- so that measurement runs reach the environment through a route that cannot skip the readback. The floor-memory qualification supervisor is the first consumer; until it exists this row is consumed by author diligence only, which this receipt is the evidence is insufficient.", + + "REVIEW TELL: `2>/dev/null`, `|| true`, or an unchecked exit status on any call that sets a limit, joins a cgroup, forwards a variable, mounts, or drops a privilege; a measurement whose write-up states the cap it intended rather than the cap it read back; and a harness that reports what it REQUESTED of the environment in the same breath as what it MEASURED of the subject.", + ], + + evidence: [], +} diff --git a/dag/test/claim/floor_memory_demand_witness_test.dag b/dag/test/claim/floor_memory_demand_witness_test.dag new file mode 100644 index 00000000000..526d30f432b --- /dev/null +++ b/dag/test/claim/floor_memory_demand_witness_test.dag @@ -0,0 +1,133 @@ +module test.claim.floor_memory_demand_witness + +import std.types { Bool } +import std.measure { byte_size } +import extdeps.linux.cgroup_v2_memory { + CgroupMemoryLimited, + CgroupMemoryUnlimited, +} +import gunbc.floor_memory_demand { + CgroupMemoryEvents, + TerminationExited, + TerminationSignalled, + TerminationUnobserved, + DemandUnreadable, + CgroupDirectoryAbsent, + qualify_floor_memory_demand, + qualification_is_a_demand, +} + +// THE DISCRIMINATING PAIR IS TWO REAL RUNS, NOT TWO FIXTURES. Both were taken on srv1 at +// 56375ec44a from the same binary and the same tree, back to back, with ONE variable changed: +// the throttle line. Every number below is a kernel counter read from those runs, so if the +// judgment stops separating them, it has stopped separating the two states the corpus has +// actually observed rather than two states someone imagined. +// +// Run A, uncensored: MemoryMax=64G, MemoryHigh=infinity. peak 28962353152, all events zero. +// Run B, 16 GiB throttle line: MemoryHigh=17179869184. peak 17181028352, high events 17466. +// +// Run B's peak sits 1158168 bytes ABOVE its throttle line, which is the reclaim clamp's overshoot +// signature rather than a contradiction: allocations cross the line in a burst, direct reclaim +// drags usage back, and memory.peak records the crossing. + +data run_a_events: CgroupMemoryEvents = CgroupMemoryEvents { + high_events: 0, + max_events: 0, + oom_kills: 0, +} + +data run_b_events: CgroupMemoryEvents = CgroupMemoryEvents { + high_events: 17466, + max_events: 0, + oom_kills: 0, +} + +data no_events: CgroupMemoryEvents = CgroupMemoryEvents { + high_events: 0, + max_events: 0, + oom_kills: 0, +} + +// GREEN: the uncensored run is a demand. An exit code of 1 does not withhold completion -- the +// floor refused two of its own witnesses, which is a statement about the workload and not about +// whether memory was held. +test fn the_uncensored_run_qualifies_as_a_demand_holds() -> Bool { + qualification_is_a_demand( + q: qualify_floor_memory_demand( + peak: byte_size(count: 28962353152), + limit_max: CgroupMemoryLimited { bytes: byte_size(count: 68719476736) }, + limit_high: CgroupMemoryUnlimited, + events: run_a_events, + termination: TerminationExited { code: 1 }, + ), + ) +} + +// RED SIDE OF THE SAME PAIR: the censored run must NOT qualify, and it is the same binary on the +// same tree. This is the assertion that would have stopped the 2026-09-12 reading from travelling +// as a demand. +test fn the_throttled_run_does_not_qualify_as_a_demand_holds() -> Bool { + qualification_is_a_demand( + q: qualify_floor_memory_demand( + peak: byte_size(count: 17181028352), + limit_max: CgroupMemoryLimited { bytes: byte_size(count: 68719476736) }, + limit_high: CgroupMemoryLimited { bytes: byte_size(count: 17179869184) }, + events: run_b_events, + termination: TerminationExited { code: 1 }, + ), + ) == false +} + +// THE CASE EVENT COUNTERS ALONE WOULD MISS, and the one the corpus has been burned by: a peak +// sitting EXACTLY on the throttle line with the counters still reading zero. Every figure behind +// the old 16/15 row had this shape. If the judgment consulted only memory.events this would read +// as a demand. +test fn a_peak_pinned_to_the_line_with_zero_events_is_not_a_demand_holds() -> Bool { + qualification_is_a_demand( + q: qualify_floor_memory_demand( + peak: byte_size(count: 16106127360), + limit_max: CgroupMemoryLimited { bytes: byte_size(count: 17179869184) }, + limit_high: CgroupMemoryLimited { bytes: byte_size(count: 16106127360) }, + events: no_events, + termination: TerminationExited { code: 0 }, + ), + ) == false +} + +// A KILLED PROCESS NEVER FINISHED WANTING MEMORY, so its peak is a lower bound even when it sits +// far below every limit and no counter ticked -- which is what an abort or an external SIGKILL +// looks like from the supervisor. +test fn a_killed_run_is_not_a_demand_even_far_below_its_limits_holds() -> Bool { + qualification_is_a_demand( + q: qualify_floor_memory_demand( + peak: byte_size(count: 1073741824), + limit_max: CgroupMemoryLimited { bytes: byte_size(count: 68719476736) }, + limit_high: CgroupMemoryUnlimited, + events: no_events, + termination: TerminationSignalled { signal: 9 }, + ), + ) == false +} + +// NOT KNOWING HOW A PROCESS ENDED IS NOT EVIDENCE THAT IT ENDED WELL. The fail-closed reading of +// an unobserved termination is a lower bound. +test fn an_unobserved_termination_is_not_a_demand_holds() -> Bool { + qualification_is_a_demand( + q: qualify_floor_memory_demand( + peak: byte_size(count: 1073741824), + limit_max: CgroupMemoryLimited { bytes: byte_size(count: 68719476736) }, + limit_high: CgroupMemoryUnlimited, + events: no_events, + termination: TerminationUnobserved { cause: "supervisor lost the child" }, + ), + ) == false +} + +// THE ARM THE WHOLE INSTRUMENT EXISTS TO KEEP DISTINCT: could-not-read must not render as it-fit. +test fn an_unreadable_cgroup_is_not_a_demand_holds() -> Bool { + qualification_is_a_demand( + q: DemandUnreadable { + cause: CgroupDirectoryAbsent { path: "/sys/fs/cgroup/nonexistent" }, + }, + ) == false +} From 03eca325cdf40d904faa6f4ff3fe98956c38f691 Mon Sep 17 00:00:00 2001 From: Brian Searls Date: Sun, 20 Sep 2026 03:37:09 +0000 Subject: [PATCH 3/6] WIP: floor-memory-qualification instrument rows and the supervisor (host arm pending) Co-Authored-By: Claude Opus 5 (1M context) --- dag/gunbc/instrument_targets.dag | 41 +++- dag/gunbc/target_binding.dag | 6 + src/v1/stage0/src/cli_run.rs | 2 + .../src/cli_run/floor_memory_supervisor.rs | 211 ++++++++++++++++++ 4 files changed, 258 insertions(+), 2 deletions(-) create mode 100644 src/v1/stage0/src/cli_run/floor_memory_supervisor.rs diff --git a/dag/gunbc/instrument_targets.dag b/dag/gunbc/instrument_targets.dag index 9ba161db894..40a410d441f 100644 --- a/dag/gunbc/instrument_targets.dag +++ b/dag/gunbc/instrument_targets.dag @@ -7,6 +7,7 @@ import gunbc.target_binding { BehavioralReceiptPlanProducer, BehavioralReceiptCensusProducer, BehavioralReceiptSelftestProducer, HeadsReadingDifferentialProducer, + FloorMemoryQualificationProducer, CompileCleanDiagnosticCensusProducer, EvaluationStoreAddressExactHeadProducer, CompileCleanDiagnosticCensusStanding, @@ -77,6 +78,17 @@ fn heads_reading_differential_label() -> Label { } } +// THE FLOOR'S MEMORY DEMAND, QUALIFIED. The subject is the required-floor PROCESS -- not any +// function inside it -- which is why the reading is taken by a supervisor from outside rather than +// by anything the floor itself emits. See `gunbc.floor_memory_demand` for why an in-process +// observer cannot report the two terminations that matter most. +fn floor_memory_qualification_label() -> Label { + Label { + package: instruments_package() + target: TargetName { name: "floor-memory-qualification" } + } +} + fn behavioral_receipt_plan_label() -> Label { Label { package: instruments_package() target: TargetName { name: "behavioral-receipt-plan" } } } @@ -164,6 +176,12 @@ fn heads_reading_differential_target() -> BuildTarget { } } +// NOT EVERY `*_target()` BELOW IS CONSUMED. `evaluation_store_address_exact_head_target` and +// `v2_native_cli_target` are each referenced only by their own definition; `instrument_targets()` +// folds LABELS and `instrument_bindings()` folds BINDINGS, and neither reads a `BuildTarget`. That +// is a DESIGN section 3c dangling set, recorded here rather than resolved because resolving it is +// a separate change -- and recorded so the next author adding an instrument does not copy the +// pattern. The floor-memory row deliberately adds no `*_target()` for that reason. fn instrument_test_target(label: Label) -> BuildTarget { BuildTarget { label: label kind: TestRule {} dependencies: [] } } @@ -231,6 +249,13 @@ fn evaluation_store_address_exact_head_binding() -> TargetBinding { TargetBinding { target: evaluation_store_address_exact_head_label() producer: EvaluationStoreAddressExactHeadProducer {} } } +fn floor_memory_qualification_binding() -> TargetBinding { + TargetBinding { + target: floor_memory_qualification_label() + producer: FloorMemoryQualificationProducer {} + } +} + // THE SUBJECT IS THE INSTRUMENT'S OWN FACT, NOT AN ARGUMENT THE CALLER SUPPLIES. // // `gunbc test //gunbc/instruments:heads-reading-differential` takes no source-root option, and @@ -254,6 +279,18 @@ fn v2_native_cli_source_roots() -> List { ["dag", "src/v2"] } +// The floor-memory instrument's own subject, on the same rule its siblings follow: WHICH floor run +// is being qualified is a property of this instrument, not an invocation option, so a run cannot +// quietly measure a different lane while reporting this target's standing. These are the roots and +// the lane the required check ran when this instrument was written. +fn floor_memory_qualification_source_roots() -> List { + ["dag", "src/v2"] +} + +fn floor_memory_qualification_lane() -> String { + "witnesses" +} + fn self_host_source_roots() -> List { ["dag", "src/v2"] } @@ -292,7 +329,7 @@ fn instrument_targets() -> List