Repository navigation
Step 4A: a host-bound session-demand instrument, with its discriminating pair taken from srv1 and srv2 - #9321
Conversation
…first session-demand carrier WHAT THIS ANSWERS, AND WHAT IT DELIBERATELY DOES NOT. One semantic question: can the fleet produce a complete, host-bound concurrent-session-demand observation. It answers nothing about what the session capacity should be -- no MemoryHigh, no MemoryMax, no concurrency policy, no operator margin, and no replacement of the authored 20 GiB aggregate reservation. THE METRIC SEMANTICS ARE THE KERNEL'S, NOT OURS. cgroup-v2 already documents each memory interface file with a distinct temporal meaning, so extdeps.linux.cgroup_v2_memory grounds the three kinds there rather than coining them downstream: Instantaneous, LifetimeEpochScoped (an extremum since cgroup creation or reset -- NOT the maximum during an observation window, which is why the epoch is a field), and CumulativeCounter (only a start/end pair yields a delta). TWO PARSE FACTS THAT A BARE ByteSize FIELD CANNOT CARRY, both taken from the authority: memory.max and memory.high hold either a byte count or the literal "max", so a limit read has FOUR outcomes -- rendering the sentinel as a huge number would fabricate a limit the host does not have. Count files have no sentinel, so they have two, and "max" in a count file is MALFORMED. A negative body is malformed rather than a magnitude; nat_magnitude is applied only after the sign is excluded, where it is the identity, because using it as the crossing would silently turn "-1" into 1. IDENTITIES FIRST, COUNTS PROJECTED. #9280's SliceMembershipSignal carries two Nats, which is sufficient for its classification kernel and NOT sufficient as a measurement denominator: two counts cannot say which containers they counted, so an unclassifiable container has nowhere to go except into one of the counts or out of the population. Both fabricate. The census therefore REFUSES to project when any identity is unclassifiable, and carries the offenders so the remedy names containers rather than a number. A COUNTED ZERO IS NOT AN ABSENCE OF EVIDENCE. NoConcurrentSessionDemandObserved (the read ran and found none) stays distinct from SessionDemandObservationUnavailable (the producer did not run, or the denominator could not be established). They grant capacity on opposite grounds. VERIFIED BY EXECUTION, not by typecheck. All three modules compile with 0 blocking. Eight witnesses PASS under claim_batch --hermetic. The wall is mutation-tested rather than assumed: making the count parser accept the "max" sentinel turns count_parse_has_no_sentinel_arm RED while its sibling limit_parse_discriminates stays GREEN, so the witness catches that exact defect and is not a blanket break. The -type d child enumeration is also execution-grounded: a plain listing of a real cgroup returns 92 entries, ALL of them interface files and NONE a child, so a name filter would have been wrong there and -type d correctly returns 0. NOT YET PRESENT, and this is why no PR is opened: the live host receipt. There is no route to a session host from a session container (ssh refused, no docker context), and the classifier's cgroup naming authority is unobservable from inside a namespaced cgroup. Inventing it would be fabrication. Sequencing is with the operator. Co-Authored-By: Claude Opus 5 <noreply@anthropic.com>
gunbai-bot
Bot
changed the title
…here it was found, so no naming authority is needed
THE BLOCKER I REPORTED DISSOLVED ON INSPECTION, and the reason is worth recording because it
changes what the cut needs. I had said the classifier was unwritable without observing real
cgroup naming from a host I cannot reach. That is true of only ONE HALF of it.
MANAGED a child of sessions.slice IS a managed session BY VIRTUE OF WHERE IT WAS FOUND.
The cgroup hierarchy is the classifier. No docker-scope naming convention has to be
invented, guessed, or observed to read it.
LEGACY a flat scope under system.slice may be an agent session or an unrelated container,
and cgroup structure alone cannot tell them apart.
So the legacy arm is SessionPlacementUnclassifiable rather than pattern-matched, which makes the
census REFUSE on a host carrying flat scopes instead of reporting a population it guessed. On a
fully-converged host the legacy set is empty and the census completes. Inventing a naming rule to
make it complete on a mixed host is exactly the fabrication the carrier exists to prevent, and it
is now recorded as a note rather than left as a silent gap.
AN EMPTY LISTING IS A COUNTED ZERO, NOT A FAILURE, and the distinction is load-bearing at the
read boundary: find exits 0 with empty stdout on a leaf cgroup, so success-with-no-lines reaches
the caller as an observed empty population. A FAILED read becomes Unavailable instead. Collapsing
those would be the empty-observation narrow at the one place it grants capacity.
A COUNTER THAT COULD NOT BE READ IS NOT A ZERO. Each per-session read carries its own parse
outcome through to the receipt rather than defaulting, because an unreadable counter and an idle
container are the same number and opposite facts -- and an aggregate absorbing the first would
read as smaller, more attractive demand than the host actually has.
VALIDATION, and why it is claim_batch rather than a full entry compile. 12 witnesses PASS under
claim_batch --hermetic, which resolves and typechecks session_demand_observe and, through its
imports, the two new HostOperation variants in host_operation_exec. The full `compile --entry`
emit was attempted three times and its gunbc process was KILLED mid-emit every time while this
container's own cgroup reported oom_kill=0 at 19.3 GiB against a 31.27 GiB cap -- the kill comes
from the parent slice, which is invisible from inside. Resolve+typecheck is the check that
validates .dag correctness; the Rust emit adds nothing to it here and cannot complete in this
container. That failure is not incidental to this branch: it is the instrument's own
justification, observed three times while building the instrument.
STILL MISSING: the live session-host receipt. Per the operator ruling the route is to add a
read-only mode to the EXISTING fleet-converge workflow -- which already has workflow_dispatch,
already accepts an exact host, and already pins its second job to that host's runner label -- and
dispatch it AT THIS BRANCH REF before merge. Every actuating step in that job is individually
mode-gated, so a new read-only mode runs none of them.
Co-Authored-By: Claude Opus 5 <noreply@anthropic.com>
…w, so the live receipt can be taken at this branch ref before merge WHY THIS WORKFLOW RATHER THAN A NEW ONE. GitHub only accepts a workflow_dispatch event for a workflow present on the DEFAULT branch, so a new observation workflow authored on this branch could never be dispatched and the receipt could not exist until after merge. It does, however, run the version of an ALREADY-PUBLISHED workflow from whatever ref is dispatched. fleet-converge is already published, already has workflow_dispatch, already takes an exact host, and already pins its second job to that host's runner label -- so extending it is the only route that gets a real session-host receipt BEFORE merge, which is what the vertical-cut rule requires. WHY THIS CANNOT ACTUATE ANYTHING. Every actuating step in that job is individually mode-gated (plan, apply, spark_linger, spark_grants, spark_observe, spark_reboot, spark_durability), so dispatching session_demand_observe executes NONE of them. The step itself runs on a runner pinned to the dispatched host and reads that host's own cgroups over LocalShell: find and cat under /sys/fs/cgroup, no ssh, no fleet key, no new credential surface. It mutates nothing on the host and writes only its own receipt inside the job workspace. THE HOST IS OBSERVED, NOT SUPPLIED. The receipt binds to the hostname the machine itself answers via the already-modeled HostnameReadShort, not to the dispatch input -- otherwise a receipt could name a machine it did not read. THE BACKSTOP TIMEOUT NOTE IS UPDATED BECAUSE IT SAYS TO BE. gunbc_ci_fleet_job_backstop_timeout_note declares itself "the single authority to update alongside any change to that job's step composition, if_condition gating, or mode set", and this is a mode-set change. The worst case does NOT move: this is one aux-shaped step under a spark-tier budget, strictly below the spark_grants aux+spark 65m that sets the bound, so the 105m single-run total stands. ONE WITNESS LEGITIMATELY CAUGHT THIS. workflow_dispatch_choice_input_projects_options_list pins the mode options at seven and re-enumerates them, so the eighth mode turned it RED -- which is the witness doing its job. Updated to eight. Worth recording without acting on it here: that enumeration is a second representation of fleet_converge_mode_options and could be DERIVED from that authority, which would make it self-maintaining and a genuine projection-fidelity check rather than a list needing an edit per mode. That is a change to the epistemics of another lane's witness and does not belong under this title. ONE PIECE OF DRIFT DELIBERATELY NOT ABSORBED. The regen also wanted to add `src/v1/stage0/src/partition_crate_boundary_host.rs !merge` to .gitattributes. That is MAIN's drift, not this branch's -- the .rs has existed since #9244, main's committed .gitattributes lacks the row, and nothing here touches stage0. It is unguarded because the generated-artifact drift gates are on DESIGN's post-floor-cut unguarded list. Reverted rather than swept in under an unrelated title; it wants its own cut off main. VALIDATION: 4/4 workflow-dispatch witnesses PASS and the producer entry resolves and typechecks under claim_batch --hermetic. The regen ran clean at exit 0 with ROADMAP.md untouched, which is the projected-arm precondition established before letting a wet actuator write. Co-Authored-By: Claude Opus 5 <noreply@anthropic.com>
…ot a path: give ReadInterfaceFile one composed input
FOUND BY EXECUTION ON A REAL SESSION HOST, which is the whole point of taking the receipt before
merge. Dispatched at this branch ref against srv1; the job landed on srv1-07 and the producer ran:
Clock.Now done in 1ms
os.Hostname.ReadShort done in 1ms
linux.CgroupV2.ListChildCgroups done in 2ms (managed slice)
linux.CgroupV2.ListChildCgroups done in 3ms (system.slice)
error: TypeError { msg: "unsupported operator Div on String" }
THE DISCRIMINATOR IS IN THE PAIR. ListChildCgroups interpolates ONE value into its argv and
executed twice without complaint. ReadInterfaceFile carried TWO interpolations separated by a
literal slash -- argv: ["cat", "{cgroup_path}/{interface_file}"] -- and it is the only operation
that failed. The interpolation grammar reads ACROSS the slash rather than stopping at the closing
brace, so `/` is taken as the division operator applied to two Strings. It typechecks, because
both sides really are Strings; it only dies at evaluation.
THE FIX IS TO COMPOSE THE PATH WHERE THE TYPE SYSTEM CAN SEE IT. ReadInterfaceFile now takes ONE
file_path input and interpolates once; the caller builds the path with cgroup_v2_child_path, which
already exists and is already the single authority for joining a parent to a child. That is
strictly better than escaping the slash: the composition moves out of a transport-DSL string and
into a modeled function, so there is no second place where a path gets assembled.
NOT CLAIMED: that the interpolation grammar is defective. It may be behaving as specified. What is
established by execution is that an argv of the form "{a}/{b}" does not produce a joined path, and
that a single interpolation does work -- so the remedy above holds either way.
WHAT THIS RUN ALREADY PROVED, independent of the defect. The read-only mode is real: the job ran
on a session host, EVERY actuating step was skipped by its own mode gate (plan, apply, spark_*),
and only the observation step executed. The two new cgroup operations are wired end to end through
LocalShell and reached the host's own hierarchy. The host answered its own name, so the receipt
binds to the machine that was read rather than to the dispatch input.
Main's .gitattributes drift was again offered by the regen and again reverted; it is not this
branch's and wants its own cut.
Co-Authored-By: Claude Opus 5 <noreply@anthropic.com>
…at all: the live srv1 receipt supplied the naming I refused to invent THE RUN THAT PRODUCED THIS IS THE CITATION, and the figures below are quoted only because this message's subject IS the measurement. The instrument is gunbc.session_demand_observe session_demand_observe_ci_wet, dispatched as fleet-converge.yml -f host=srv1 -f mode=session_demand_observe at this branch ref; run 32953395735 on srv1-07. Re-derive rather than trust this paragraph. WHAT THE FIRST GREEN RUN ESTABLISHED. The producer executed on a real session host and the whole chain worked: Clock.Now, os.Hostname.ReadShort (so the receipt binds to the machine that answered, not to the dispatch input), both ListChildCgroups, and every ReadInterfaceFile. Every actuating step in that job was skipped by its own mode gate. 23 managed sessions enumerated with per-session memory.current, plus the parent slice's own current, peak and max. THE NAMING IS NOW OBSERVED RATHER THAN INFERRED. I refused to assert docker-<id>.scope from my own hostname being a 12-hex short id, and said the first run would supply it. It did: all 23 children of sessions.slice are docker-<64 hex>.scope. AND THE RUN IMMEDIATELY FALSIFIED MY LEGACY ARM, which is the part worth recording. system.slice on srv1 has 37 children and ZERO of them are .scope of any kind -- they are cron.service, containerd.service, systemd-networkd.service and the rest. My producer fed EVERY system.slice child into legacy_placements, so it would have marked 37 ordinary systemd units as unclassifiable SESSIONS and REFUSED THE CENSUS OF A PERFECTLY CONVERGED HOST. That is the not-applicable-rendered-as-malformed failure DESIGN names, in its exact shape: one symbol carrying both "this container cannot be attributed to a session" and "this is not a container at all", which have nothing to do with each other and opposite remedies. Nothing in the carriers could see it, because both states really are absent-of-classification; only running it against a real system.slice showed that the population was wrong rather than the classifier. THE FIX SPLITS THEM AT THE POPULATION BOUNDARY. A candidate flat session is a docker scope; everything else is not in the session population at all and never reaches the classifier. A docker scope found FLAT under system.slice stays Unclassifiable and refuses the census, which is the arm that matters and which measured ZERO on srv1 because that host is fully converged. THE NEW WITNESS IS BUILT FROM THE RECEIPT, not from invented names: the four unit names it asserts against are ones srv1 actually reported, and the docker scope it accepts is one of the 23 real identities. A second witness caught my own change while I made it -- the existing fixture passed "z" to legacy_placements and expected one placement, which correctly became zero once the filter landed. Grounded it in a real scope shape rather than weakening the assertion. NO RECEIPT FILE IS COMMITTED, deliberately. DESIGN's measurement bankruptcy ruling is that a figure copied into prose is debt whether or not a producer exists for it, and that a measurement is cited by naming the producer that re-derives it. The producer serializes its own receipt and the workflow prints it; the run is addressable. Transcribing it into docs/ would recreate exactly the corpus that was deleted. Co-Authored-By: Claude Opus 5 <noreply@anthropic.com>
…uce visibly different verdicts from one code path The entry enumerated and stopped, which left the census and the membership gate unexecuted on real hosts -- so the one thing this cut claims, that a COMPLETE observation can be established, rested on fixtures while the live runs only proved the reads worked. It now runs placements -> census -> membership signal -> gate and prints the verdict. WHY THIS MATTERS RIGHT NOW RATHER THAN AS POLISH: the two hosts already disagree. srv1 carries 23 managed sessions and ZERO flat scopes, so its census completes and the aggregate may be established. srv2 carries 26 managed sessions AND ONE FLAT DOCKER SCOPE under system.slice, so its census refuses -- one running session container is outside the cgroup oomd evicts within, and any aggregate over srv2's sessions.slice is measuring 26 of 27 while naming the whole. That is the live RED for the incomplete arm, on a real host, without having to construct one. It is also the exact defect #9280's four-way classification was built to catch, found in production rather than in a fixture. inventory_identities keeps the two absences apart: an inventory that could not be READ contributes no identities and is refused separately, rather than rendering as an empty population that would read as a clean host. NOT CLAIMED: any capacity conclusion. The per-host caps differ (srv1 sessions.slice MemoryMax is twice srv2's) and the authored aggregate reservation does not match either host's measured current, but establishing what the reservation SHOULD be is 4B and the operator's ruling, not this cut. Co-Authored-By: Claude Opus 5 <noreply@anthropic.com>
gunbai-bot
Bot
changed the title
…host nobody observed could report AllSessionsManaged and exit success review 56217 finding 1 is correct and it is the serious one. inventory_identities answered `[]` for an inventory that COULD NOT BE READ, which is indistinguishable from one that was read and found empty. So a failed system.slice enumeration contributed no flat candidates, the census completed, and the entry reported "AllSessionsManaged; an aggregate may be established" and exited SUCCESS over a host whose flat population was never observed. THE DIRECTION IS WHAT MAKES IT SEVERE. The flat side is the side that REFUSES the aggregate, so losing it silently is precisely what turns an unobserved host into a green one. That is the empty-observation narrow -- bottom-as-answer conflated with bottom-as-ignorance -- and DESIGN already records it as strictly worse than the widen it forbids, because a widen is merely expensive and a narrow is silently uncovered. HOW IT SURVIVED TWELVE GREEN WITNESSES, which is the part worth keeping: `[]` is a PERFECTLY VALID population. The census logic below it was correct, the carriers were correct, and every assertion stayed green. The wrong thing was the DENOMINATOR, one layer above anything being asserted about. No amount of testing the classifier could have found it. AND MY OWN ANNOTATION HELPED IT SURVIVE. The deleted function carried a comment asserting that "the entry refuses on it separately". That was true of the managed arm and FALSE of the legacy one, and the sentence bought exactly the reviewer confidence it had not earned. An annotation claiming a protection is worse than no annotation when the protection is partial. REPAIRED BY CONSTRUCTION, NOT BY A SECOND CHECK. inventory_identities is DELETED. There is now no expression anywhere that turns an unavailable inventory into a population: session_population takes both inventories and either establishes one or returns SessionPopulationUnavailable naming which read failed. The verdict line and the exit status are both derived from that ONE value, so they cannot disagree about whether the population was established. FINDING 2 IS ALSO CORRECT. observation_completed sat in the header, which is bound BEFORE the per-session memory.current reads, so the receipt named a completion time that did not bound the observation it described. The header is now assembled after every read and after the verdict, so the clock is consulted last. Verified by let-ordering: the reads are lines 292-295 and the stamp is line 355. THE NEW CONTROL IS MUTATION-TESTED, not merely added. Restoring the old behaviour -- an unreadable system.slice yielding an established population from the managed ids alone -- turns unreadable_inventory_never_establishes_a_population RED while systemd_units_are_not_session_candidates stays GREEN. It asserts BOTH directions, so it cannot pass by guarding only the arm that happens to be checked, and it asserts the readable case still establishes. FINDING 3 IS DECLINED, for the second time and on the same grounds. Measured against current main's DESIGN.md (25bf535) with a known-positive control proving the grep works -- "absorbing fallback" 5 hits, "not-applicable rendered as malformed" 1, "construction-before-validation" 2 -- the claimed gate returns ZERO across five phrasings: walker, mechanical predicate, dissolution case, predicate-dissolution, canonical fold/query. A Bool PROJECTED from a coproduct that remains the single authority is the sanctioned shape, stated outright in gunbc.session_residency session_residency_is_projected_note, and it appears 340 times in dag/gunbc. Co-Authored-By: Claude Opus 5 <noreply@anthropic.com>
|
All three verified against the code. Findings 1 and 2 are correct and fixed in 5bca9e4. Finding 3 is declined, with the measurement below. Finding 1 — correct, and it was the serious oneYou are right, and the direction is what makes it severe. The flat side is the side that refuses the aggregate. Losing it silently is exactly what turns an unobserved host into a green one — the empty-observation narrow, which DESIGN records as strictly worse than the widen it forbids, because a widen is merely expensive and a narrow is silently uncovered. Worth stating how it survived twelve green witnesses: And my own annotation helped it survive. The deleted function carried a comment asserting "the entry refuses on it separately" — true of the managed arm, false of the legacy one. That sentence bought precisely the reviewer confidence it had not earned, which is worse than no annotation at all. Thank you for reading past it. Repaired by construction, not a second check. The new control is mutation-tested, not merely added: restoring the old behaviour turns Finding 2 — correct, fixed
Finding 3 — declined, second time, same groundsI checked against current main's There is no predicate/walker-dissolution gate in DESIGN.md under any of those phrasings, and I could not locate it under others. What DESIGN.md and the corpus do say points the other way. A I'm not resting on prevalence — you're right that prevalence discharges nothing. The discharge is that If the gate does exist and I'm grepping for the wrong words, quote the sentence and I'll fix it in this PR rather than argue — that offer is genuine, and it's the same one I made when this was raised on #9275. — sent from warm-tern-755 |
…ption meaning two things -- plus the in-body annotations that broke the build lane
review 56218's three findings are all correct. The fourth item here is mine: the same push broke
required CI, and the reason is worth more than the fix.
FIRST, THE BUILD-LANE BREAK. The v2-emission phase refused with 9 blocking diagnostics, every one
"source annotation sits inside a declaration body". I had put // comments inside function bodies,
which DESIGN 4c admits only at module-item grain. THE REASON I DID NOT CATCH IT IS THE REASON IT
IS WORTH RECORDING: claim_batch is LENIENT about annotation grain, so all thirteen witnesses ran
green over source that strict preparation refuses. My entire local validation loop is blind to
this class by construction. The cheap local check is structural -- an in-body annotation is an
INDENTED // -- and it is now run over every changed .dag before pushing rather than discovered by
a 16-minute CI lane.
The prose is not deleted; it is hoisted into the enclosing declaration's leading annotation, which
is where 4c puts it.
FINDING 2 IS THE STRONGEST AND IS FIXED AT THE RIGHT LAYER. read_memory_count manufactured a body
-- "<read failed: memory.current>" -- and handed it to the PARSER, which classified this
repository's own invented string as CgroupMemoryCountUnparseable. So "we never obtained a body"
and "the kernel wrote bytes we cannot parse" shared one arm, and the `body` field held something
no kernel ever produced. That is fabricated plausible output feeding a state-space conflation, and
it is the not-applicable-rendered-as-malformed class this module's own commentary cites as its bar
-- committed in the same diff that cited it.
THE SPLIT IS PLACED WHERE THE KNOWLEDGE IS. extdeps.linux.cgroup_v2_memory keeps its two arms
UNCHANGED, because it is an authority on parsing a body it was GIVEN; whether a body was obtained
at all is an observation fact, so the third state is added by the layer that performs reads.
read_interface_file now answers CgroupInterfaceBody | CgroupInterfaceReadFailed, and
ObservedMemoryCount / ObservedMemoryLimit carry Read / Malformed / Unread as distinct arms. The
receipt says UNREAD(cause) where it used to say UNREADABLE(<a string we made up>).
FINDING 1: the filter now precedes the cast. It used to cast every split element to NonEmptyStr
and drop empties afterwards. Worth stating plainly because it changes how the old code should be
read: its safety DEPENDED ON THE CAST NOT BEING ENFORCED. The ragged-input witness passes today
precisely because `"" as NonEmptyStr` does not refuse -- so that code was resting on a substrate
gap and would have begun KILLING THE OBSERVATION on the day that wall landed. Filtering first
makes the empty case unreachable at the cast instead of tolerated past it.
FINDING 3: session_demand_membership_gate returned SessionDemandObservation?, where Absent meant
"the gate admits, proceed" and Present meant "the gate produced a terminal verdict" -- one Option
carrying two unrelated meanings. Nothing about "admits" is an ABSENCE; it is a positive finding
with its own payload, and it was the arm a caller most needed to recognise. It now answers
SessionDemandMembershipGate = MembershipAdmitsAggregate { managed_count } |
MembershipRefusesAggregate { observation }.
THE TYPE CHANGE IMMEDIATELY EARNED ITSELF: three witness matches failed to compile until they
handled both arms. The Option form let those same call sites keep compiling while treating a
missing case as "proceed", which is exactly the defect being removed.
13/13 witnesses PASS. Zero in-body annotations across every .dag this branch touches.
Co-Authored-By: Claude Opus 5 <noreply@anthropic.com>
|
All three findings from review 56218 are correct and are fixed at The build-lane break was real. The v2-emission phase refused with 9 blocking §4c diagnostics — "source annotation sits inside a declaration body". I had Finding 2 — fabricated body fed to the parser. The split is placed where the knowledge is. Finding 1 — cast before filter. Fixed by filtering empties before the cast. Worth stating plainly because it changes how the old code reads: its safety depended on the cast not being enforced. The ragged-input witness passes today precisely because Finding 3 — one 13/13 witnesses PASS; zero in-body annotations across every — sent from warm-tern-755 |
|
Live receipts re-taken at srv1 — run srv2 — run Three things worth separating, because they are easy to read as one. The pair is discriminating, which is what the receipts are for. One host completes and one refuses, on the same code, from a difference in the observed world rather than in the observer. A receipt family where every host answers COMPLETE would establish nothing about whether the refusal arm can fire.
srv2's INCOMPLETE is a real fleet fact, not an artifact. One session container runs flat under
Nothing here touches MemoryHigh, MemoryMax, the 20 GiB reservation, an operator margin, or ctrl actuation. — sent from warm-tern-755 |
|
Status at This PR is not ready to merge, and the reason is not about this PR. Both scheduled reviews on the current head — 56222 (claude) and 56223 (codex) — are So the readiness summary reads This is not new to this head. Review 56216, on I am not re-pushing to trigger a retry, and I am not treating green checks plus a stranded approval count as merge-ready. Flagging for whoever owns the reviewer harness on srv1. — sent from warm-tern-755 |
…that "could not be enumerated"
THE LIVE FLEET FALSIFIED THE CARRIER, WHICH IS WHAT THE LIVE RUNS ARE FOR. Having taken srv1 and
srv2, I dispatched srv3 and srv4 to complete the census the operator's capacity policy needs. Both
answered:
census UNAVAILABLE the managed slice could not be enumerated: cgroup child enumeration
exited unsuccessfully for /sys/fs/cgroup/sessions.slice
The host log says what actually happened:
find: '/sys/fs/cgroup/sessions.slice': No such file or directory
There is no sessions.slice on srv3 or srv4. They run CI runners and host no sessions. Nothing was
wrong with those hosts and nothing was wrong with the read -- and the observer reported blindness.
THIS IS THE not-applicable-rendered-as-malformed CLASS, in the carrier whose whole purpose is to
not do that, one layer out from where review 56218 caught its sibling. DESIGN's recognition rule
fires exactly: the arm sat downstream of an enumeration that found nothing to enumerate. Two
distinct consequences were sharing one symbol.
ABSENT this host contributes no session demand and never will -- a positive, structural
fact about the fleet that a capacity policy may rely on
UNAVAILABLE we are blind, and blindness must never be spent as though it were a zero
Reported as UNAVAILABLE, half the fleet looked broken. Had the arms been folded the other way,
a genuinely unreadable slice on a real session host would have read as "not a session host" and
silently left its sessions out of the aggregate. The second direction is the dangerous one, and
it is why this is a fix rather than a cosmetic relabel.
EXIT STATUS WAS ALSO WRONG. Both runs FAILED. A runner-only host is not a failure to observe.
WHY THE ENUMERATION COULD NOT ANSWER IT, AND WHY A PROBE IS NOT A SECOND CHECK. `find` on a
missing path and `find` on an unreadable one both exit nonzero. An arm keyed on exit status alone
is structurally incapable of separating them, so no amount of care inside observe_cgroup_children
could have fixed this -- the information was never in its hands. extdeps.linux.cgroup_v2 gains
PathIsDirectory (`test -d`, total: 0 for a directory, 1 for anything else, no output to parse), and
its result SELECTS WHICH INVENTORY ARM EXISTS rather than validating one already built.
THE PROBE ITSELF HAS THREE OUTCOMES, NOT TWO. My first cut returned Bool with
HostOperationRefused mapping to false -- which rebuilds this very defect one level further down: a
probe the TRANSPORT refused would answer "the slice is absent", turning a refusal into a structural
claim about the fleet. CgroupPathProbe is Dir | NotDir | Refused, and refusal routes to Unavailable.
THE NEW ARM IS NOT ALLOWED TO BECOME A SECOND ABSORBING ANSWER. An absent system.slice on a host
that HAS a managed slice is not "not a session host" -- every Linux host has system.slice, so its
absence means the probe is lying and the flat side is unknown. That refuses. The witness asserts
it, so NO_SUCH_SLICE cannot quietly admit whatever it is handed.
The witness drives absence and unreadability through the SAME managed input, so it cannot pass by
keying on anything but the arm: fold the two back together and it goes red.
14/14 PASS. Zero in-body annotations -- the §4c structural check ran before the push this time and
caught 23 of them in these very files, which is the local-loop hole the previous commit declared.
Co-Authored-By: Claude Opus 5 <noreply@anthropic.com>
|
The live fleet falsified the carrier, which is what the live runs are for. Having srv1 and srv2, I dispatched srv3 and srv4 to complete the census the operator's capacity policy needs. Both answered: The host log says what actually happened: There is no This is the not-applicable-rendered-as-malformed class, in the carrier whose entire purpose is to not do that, one layer out from where review 56218 caught its sibling. DESIGN's recognition rule fires exactly: the arm sat downstream of an enumeration that found nothing to enumerate. Two consequences were sharing one symbol:
Reported as Why the enumeration could never have answered it. The probe has three outcomes, not two. My first cut returned The new arm is not allowed to become a second absorbing answer. An absent 14/14 witnesses PASS. Zero in-body annotations — the §4c structural check ran before this push and caught 23 of them in these very files, which is precisely the local-loop hole the previous commit declared. srv3/srv4 re-dispatched at the branch ref to confirm on the hosts that found it. — sent from warm-tern-755 |
…nd record that it WAS resolving review 56247 is right that CgroupPathIsDirectory was used in session_demand_observe and absent from its import list. Fixed by adding it. THE STATED MECHANISM IS REFUTED BY EXECUTION, AND THAT IS THE PART WORTH KEEPING. The review says "Name resolution will fail". It did not fail. Two independent executions at commit 3ae20e0 -- the commit WITHOUT the import -- ran the code: * 14/14 witnesses PASS under claim_batch --hermetic, which resolves the whole closure * fleet-converge run 32969294443, headSha 3ae20e0, on srv3, printed census NOT_A_SESSION_HOST -- a verdict reachable ONLY by constructing CgroupPathIsDirectory, dispatching it, and matching the CgroupPathIsNotDir arm A verdict that requires the variant to have been constructed is not compatible with the variant having failed to resolve. The compiler resolved it through the bare-reference closure -- the mechanism the required run reports as load_bare_reference_closure / bare_eligible -- which searches corpus-wide for a name no import binds. WHY IT IS STILL A DEFECT, ON DIFFERENT GROUNDS THAN THE REVIEW GAVE. Bare resolution is corpus-wide and therefore not stable under other people's edits: the name binds to whatever the corpus happens to declare, so a second module declaring CgroupPathIsDirectory would silently retype this call site with nothing local changing and no diagnostic anywhere. An explicit import makes the referent a property of THIS file. That is a real fragility and the fix is the same fix -- but it is a hygiene repair against silent future retyping, NOT the hard resolution error the review described. The distinction is not pedantry. Landing this as "fixed a resolution error" would put a false claim about the compiler into the PR record, and the next reader would reasonably infer that an unimported variant refuses -- and design a check, or trust a green run, on that inference. It does not refuse, which is exactly why the fragility is worth closing by hand. Everything else in review 56247 is an endorsement, and I have not treated it as evidence of anything beyond no-blocking-defect-found. 14/14 PASS. Zero in-body annotations. Co-Authored-By: Claude Opus 5 <noreply@anthropic.com>
|
review 56247 is right that The stated mechanism is refuted by execution, and that is the part worth keeping. The review says "Name resolution will fail". It did not fail. Two independent executions at commit
A verdict that requires the variant to have been constructed is not compatible with the variant having failed to resolve. The compiler resolved it through the bare-reference closure — the mechanism the required run reports as It is still a defect, on different grounds than the review gave. Bare resolution is corpus-wide and therefore not stable under other people's edits: the name binds to whatever the corpus happens to declare, so a second module declaring The distinction is not pedantry. Landing this as "fixed a resolution error" would put a false claim about the compiler into the PR record, and the next reader would reasonably infer that an unimported variant refuses — and then design a check, or trust a green run, on that inference. It does not refuse, which is precisely why the fragility is worth closing by hand rather than relying on the compiler to catch it. The rest of review 56247 is endorsement, which I am treating as no blocking defect found and not as evidence the cut does what was asked. Live receipts now cover all four hosts, and the fourth arm was authored because the fleet produced it rather than because a witness wanted it:
Four hosts, four different arms, each reached from real host state. That is the discriminating evidence the observation cut was asked for. — sent from warm-tern-755 |
…de a verdict that said so srv4's confirming run is the receipt AND the finding. The census reads correctly -- NOT_A_SESSION_HOST, exit success -- and three rows above it on the same receipt, from the same run, read: parent_memory_current UNREAD(cat exited unsuccessfully for memory.current) parent_memory_peak UNREAD(cat exited unsuccessfully for memory.peak) parent_memory_max UNREAD(cat exited unsuccessfully for memory.max) `cat` did exit unsuccessfully. It exited unsuccessfully BECAUSE THE PATH IS NOT THERE, which the counters have no way to say, so absence was rendered as a failed read -- the same not-applicable-rendered-as-malformed class, one layer below where it had just been repaired, on the same host, in the run that was meant to confirm the repair. That is the more useful half of this commit. The class does not live at one site; it lives wherever a failure channel is asked to carry a not-applicable answer, and fixing the loudest instance leaves the quieter ones reading as evidence. A reader who trusts the counters would conclude srv4's slice exists and is unreadable, which is the opposite of what the verdict beside it says. THE FIX IS CONSTRUCTION, NOT A THIRD COUNTER WORD (DESIGN 5). Adding a NO_SUCH_SLICE spelling to ObservedMemoryCount would leave the bad state writable and merely relabel it. ParentSliceObservation wraps the record instead, so THERE IS NO EXPRESSION that produces per-counter values for a cgroup that does not exist -- the reads are not attempted, so they cannot report anything about not having happened. The absent arm prints one line and names no read at all. A REFUSED PROBE STILL READS, AND THE IDENTICAL ARM IS DELIBERATE. If the probe could not run we do not know whether the slice is there, so the honest move is to attempt the reads and let the counters report UNREAD on their own evidence -- a true statement about a read that was tried. Answering ParentSliceAbsent there would convert not-knowing into a structural claim about the fleet, which is the direction this carrier exists to refuse. The two arms are written separately rather than merged so a future third probe outcome cannot inherit either behaviour silently. THE WITNESS IS A PAIR, BECAUSE THE HALF ALONE IS PASSABLE BY A WEAKER CHANGE. An absent slice must mention no read, AND an observed slice whose counters genuinely could not be read must still say UNREAD. Asserting only the first would go green against a projection that had simply stopped printing UNREAD anywhere, which would delete the honest signal instead of the dishonest one. 15/15 PASS. Zero in-body annotations. Co-Authored-By: Claude Opus 5 <noreply@anthropic.com>
|
srv4's confirming run is both the receipt and the next finding, and the second is worth more. The census reads correctly —
That is the general point. The class does not live at one site; it lives wherever a failure channel is asked to carry a not-applicable answer. Fixing the loudest instance leaves the quieter ones still reading as evidence — and here a reader who trusts the counters would conclude srv4's slice exists and is unreadable, which is the opposite of what the verdict beside them says. Two rows of one receipt contradicting each other, both produced by code that was working as written. The fix is construction, not a third counter word. Adding a A refused probe still reads, and that identical arm is deliberate. If the probe could not run we do not know whether the slice is there, so the honest move is to attempt the reads and let the counters say The witness is a pair, because either half alone is passable by a weaker change. An absent slice must mention no read, and an observed slice whose counters genuinely could not be read must still say 15/15 PASS, zero in-body annotations, pushed at — sent from warm-tern-755 |
|
srv4 confirms the repair on the host, and what confirms it is the absence of rows. Run There are no Full fleet, four hosts, four arms, each reached from real host state:
On merge readiness, one correction to the summary rather than to the code. Not merging, and not re-pushing to nudge anything. — sent from warm-tern-755 |
… be UNREAD and the aggregate was still granted (#9347) * The verdict was a function of identities alone, so every metric could be UNREAD and the aggregate was still granted A side-chat review raised six findings against #9321. I never saw it -- the dashboard was down for hours and the PR merged in the meantime -- so these are follow-ups on main rather than pre-merge repairs. Three are verified here by reading the merged code; the worst is a fail-open. THE FAIL-OPEN. session_placement_census filters placements on CLASSIFICATION and the verdict is a function of its result alone. The per-session memory.current reads and the parent slice's three counters were rendered into the receipt and never consulted. So a host on which EVERY read failed still printed census COMPLETE AllSessionsManaged over N session(s); an aggregate may be established and exited success. I repaired the DENOMINATOR narrow -- an unreadable inventory cannot establish a population -- and left the NUMERATOR narrow standing, in the one arm that GRANTS the aggregate. WHY NO WITNESS CAUGHT IT, which is the part worth carrying forward: all fifteen asserted about POPULATION and none about METRICS. The suite was exhaustive over the axis I was thinking about and silent on the axis beside it. Being total at the level examined is not coverage of the level below. SessionDemandObserved is now constructible only with a complete population AND zero metric deficits. A deficit names its cgroup and its interface file, because a consumer repairing a refusal needs to know WHICH read to fix; a count tells nobody where to look. MemoryLimitUnlimited is deliberately not a deficit -- it is a real parsed kernel answer, and folding it in with unread would make an answered question look unanswered, which is this carrier's own failure class. THE PARALLEL REPRESENTATION. SessionDemandReceipt and SessionDemandSample had NO production constructor anywhere -- they existed only as pattern arms and fixture values -- while session_demand_observe_ci_wet assembled a tab-separated String directly and derived its exit status from SessionPopulation separately. A typed model standing beside an operational serializer: two representations of one fact, in the cut whose entire subject is modeling an observation. The producer now returns SessionDemandObservation; render_session_demand_observation projects the text from it and has no other input; process_exit_for_session_demand_observation derives the status from the same value. The receipt and the exit can no longer disagree because they no longer have two sources. THE UNJOINED SUBJECTS. session_population matched host: _ in every arm, so an srv1-managed plus srv2-legacy pair minted one SessionPopulationEstablished. Production reads both locally under one observed host so this never fired -- but a type that permits cross-subject evidence will eventually be handed it, and the witness now drives exactly that pair to a typed HostMismatch. THE OVERCLAIM, WHICH WAS MINE AND REACHED THE OPERATOR. I wrote that srv2 "runs one session container flat" and that "the aggregate measures 26 of 27". The producer establishes neither. It observes a docker scope under system.slice that it CANNOT attribute to an agent session, which is why it refuses. The scope may well be the missing 27th session; the code declines to assert it and the prose should not have. The receipt now says unattributed, names each identity, and says the denominator is unestablished until each is joined to a session-runtime identity or ruled unrelated. 17/17 PASS. Zero in-body annotations. Not addressed here, deliberately: the read-only mode still inherits WIF auth, fleet-key materialization and ssh-agent from unconditional job steps, and the receipt still carries a schema name rather than the executing revision. Both are real; both are separate cuts. Co-Authored-By: Claude Opus 5 <noreply@anthropic.com> * A duplicate match arm compiles clean, and mine shadowed the variant the cut exists to introduce review 56318 is right on both counts and both were mine. THE SHADOW ARM. an_unread_metric_refuses_a_complete_population matched SessionDemandMetricsIncomplete twice: once binding ds and returning count(ds) == 1, then again with every field wildcarded returning false. The second is unreachable. It made the witness LOOK like it distinguished two cases when only one was live, in the witness whose entire subject is that variant. THE REVIEWER LEFT A CONDITIONAL AND IT RESOLVES THE BAD WAY: "if the .dag compiler doesn't refuse duplicate variant arms". It does not. The witness ran 17/17 PASS with the shadow arm present, so this is confirmed by execution rather than by reading -- a duplicate variant arm is accepted, and the dead one is invisible. That is a real substrate gap and it is exactly the shape the exhaustiveness check is supposed to cover: the compiler REFUSES a missing arm and ACCEPTS a duplicated one, so a match can be verified complete while carrying unreachable code the author believes is live. HOW IT GOT THERE, because the mechanism matters more than the instance. The type change forced exhaustiveness failures at nine existing call sites, and I closed them with a blanket text substitution that appended the four new arms after SessionDemandObserved. My witness had already written those arms by hand, so the substitution appended a second copy. A dedupe pass in the same script was written against one exact ordering and silently did not match this one. Automated editing that cannot fail loudly produced dead code that the compiler then declined to notice -- two silent mechanisms composing, which is why it survived to review. THE DUPLICATE IMPORTS are the same substitution's other residue: ManagedSession, SessionDemandPopulationIncomplete, NoConcurrentSessionDemandObserved, SessionDemandObservationUnavailable and SessionDemandObserved were each imported twice from gunbc.session_demand, my new head-of-block additions colliding with the pre-existing tail. Deduped in the PR that introduced the collision rather than left for a later sweep. 17/17 PASS, unchanged, which is the point: deleting unreachable code changes no verdict. The witness is now what it claimed to be. Co-Authored-By: Claude Opus 5 <noreply@anthropic.com> --------- Co-authored-by: Brian Searls <briansearls1@gmail.com> Co-authored-by: Claude Opus 5 <noreply@anthropic.com>
1 participant
What this answers
One semantic question: can the fleet produce a complete, host-bound concurrent-session-demand observation.
It answers nothing about what the session capacity should be. No
MemoryHigh, noMemoryMax, no concurrency policy, no operator margin, and no replacement of the authored 20 GiB aggregate reservation.The vertical cut, proven on real hosts
The instrument is
gunbc.session_demand_observesession_demand_observe_ci_wet, dispatched asfleet-converge.yml -f host=<h> -f mode=session_demand_observeat this branch ref. Re-derive from the runs rather than trusting this page.The discriminating pair came from production, not fixtures — one code path, two hosts, opposite verdicts:
census COMPLETE— AllSessionsManaged over 23 session(s); an aggregate may be establishedcensus INCOMPLETE— 1 unattributable flat docker scope(s); no aggregate may be established over this hostA live fleet finding, not a constructed control
srv2 currently runs one session container flat under
system.slice, outside the managed slice.oomdcannot evict it, and any aggregate over srv2'ssessions.slicemeasures 26 of 27 while naming the whole. That is exactly the defect #9280's four-way classification was built to catch, found in production. The scope isdocker-a4e17d4a…scope.Also worth flagging for the capacity work: the two hosts'
sessions.sliceMemoryMaxdiffer — srv1137438953472, srv268719476736. A single fleet-wide aggregate reservation cannot be right for both.The metric semantics are the kernel's, not ours
extdeps.linux.cgroup_v2_memorygrounds three kinds in the cgroup-v2 documentation rather than coining them:InstantaneousLifetimeEpochScopedCumulativeCountermemory.max/memory.highhold a count or the literalmax, so a limit read has four outcomes; count files have no sentinel, somaxthere is malformed. A negative body is malformed rather than a magnitude.review 56217 — two real defects, fixed in
5bca9e4851An unreadable
system.slicewas rendering as "no flat sessions".inventory_identitiesanswered[]for an inventory that could not be read, which is indistinguishable from one read and found empty — so a failed enumeration contributed no flat candidates, the census completed, and the entry reportedAllSessionsManaged; an aggregate may be establishedand exited success over a host whose flat population was never observed.The direction is what makes it severe: the flat side is the side that refuses the aggregate, so losing it silently turns an unobserved host into a green one. That is the empty-observation narrow, which DESIGN records as strictly worse than the widen it forbids.
It survived twelve green witnesses because
[]is a perfectly valid population — the census logic below it was correct, the carriers were correct, and the wrong thing was the denominator, one layer above anything being asserted about. It also survived because the deleted function's own annotation asserted "the entry refuses on it separately", which was true of the managed arm and false of the legacy one.Repaired by construction:
inventory_identitiesis deleted, no expression turns an unavailable inventory into a population, andsession_populationeither establishes one or returnsSessionPopulationUnavailablenaming which read failed. Verdict and exit status derive from that one value. The new control is mutation-tested — restoring the old behaviour turns it RED while its sibling stays GREEN.observation_completedwas stamped before the per-session reads. Now assembled after every read and after the verdict (reads at lines 292–295, stamp at 355). Note the two timestamps still read equal on srv1 because the observation completes in under a second — that is consistent with the fix, not evidence for it; the evidence is thelet-ordering.A third finding (coproduct→
Boolpredicates) was declined with a measurement, in this comment.Two defects only a live run could find
1.
{a}/{b}in an argv is a division, not a path. The first dispatch died withTypeError: unsupported operator Div on Stringafter bothListChildCgroupssucceeded. The discriminator is in the pair:ListChildCgroupsinterpolates one value and worked;ReadInterfaceFilecarried two interpolations around a literal slash and was the only failure. It typechecks, because both sides really areString. Fixed by composing the path in.dagand interpolating once.2. A systemd unit is not an unclassifiable session — it is not a session at all. The producer fed every
system.slicechild into the legacy arm. srv1 has 37 such children and zero are containers (cron.service,containerd.service, …), so it would have marked 37 systemd units as unclassifiable sessions and refused the census of a fully converged host.That is DESIGN's not-applicable rendered as malformed, exactly: one symbol carrying both "cannot be attributed to a session" and "is not a container at all", which have opposite remedies. Nothing in the carriers could see it — both states genuinely are absent-of-classification, every witness was green, and the census logic was correct. The population was wrong. Only a real
system.slicecould show that.This is the argument for the vertical-cut rule: a carrier-plus-fixtures PR would have merged green carrying this, and it would have surfaced later as "the instrument refuses on every host" — reading as a broken instrument rather than a wrong denominator.
Identities first, counts projected
#9280's
SliceMembershipSignalcarries twoNats — sufficient for its classification kernel, not sufficient as a measurement denominator, because two counts cannot say which containers they counted. The census refuses to project when any identity is unclassifiable and carries the offenders, so the remedy names containers rather than a number.NoConcurrentSessionDemandObserved(read ran, found none) stays distinct fromSessionDemandObservationUnavailable(producer did not run). They grant capacity on opposite grounds.Why this workflow rather than a new one
GitHub only accepts
workflow_dispatchfor a workflow on the default branch — but it runs the version from whatever--refis dispatched.session_demand_observedoes not exist in main's copy, and GitHub accepted it anyway, which is what made a pre-merge session-host receipt possible at all.It cannot actuate. Every actuating step in that job is individually mode-gated, so this mode runs none of
plan,apply, or anyspark_*. The step reads the host's own cgroups overLocalShell—findandcatunder/sys/fs/cgroup, no ssh, no fleet key, no new credential surface.Verification
claim_batch --hermetic.maxsentinel turnscount_parse_has_no_sentinel_armRED whilelimit_parse_discriminatesstays GREEN.Not absorbed
Regen repeatedly offered
src/v1/stage0/src/partition_crate_boundary_host.rs !mergefor.gitattributes. That is main's drift — the.rshas existed since #9244, main's committed file lacks the row, and nothing here touches stage0. It is unguarded because the generated-artifact drift gates are on DESIGN's post-floor-cut unguarded list. Reverted each time; it wants its own cut.