Skip to content

Serving desired state named both Sparks and the removal remedy was the empty list: a role authority, and retirement given a vocabulary - #9679

Merged
briansrls merged 17 commits into
mainfrom
session/eager-pike-541
Aug 30, 2026
Merged

briansrls merged 17 commits into
mainfrom
session/eager-pike-541

Conversation

@gunbai-bot

@gunbai-bot gunbai-bot Bot commented Aug 29, 2026 •

Copy link
Copy Markdown
Contributor

SPARK-PAIR-0 P0 — role separation, and a retirement the spine could not express

Two facts had no home in this corpus. One of them was load-bearing.

What each Spark is for. spark_serving_desired_hosts was a hand-authored list naming srv5 and srv6. That was harmless while both machines served, and became wrong the moment one was wanted as a trainer: deleting a name from a literal is a change with no reviewable subject, and the intent lived nowhere. gunbc.spark.cell_role now owns it — SparkCellRole = SparkServingCell | SparkTrainingCell, a sum rather than two booleans, so a cell holding both roles has no constructor. The serving roster is a projection of it, so srv6 leaves because its role changed, in one edit, in the authority.

How to retire a service. spark_serving_install_effects_for_remove returned []. That was not an oversight: every one of the thirteen ServingInstallEffect constructors installs something, so there was no value that function could have returned meaning "take it down". Removing a host from desired state planned a removal whose remedy was silence — only the plan projection's non-empty-executions refusal stopped it applying as a successful no-op.

Retirement is a sibling vocabulary rather than more arms on the install sum. realization_install_effects and the privilege count that ranks realizations both presume their effects are the price of standing a service up; a StopUserUnit arm would appear in no realization's list, so every realization would fold over a sum half of whose members can never be selected. The two vocabularies meet at the member action — an add or change installs, a removal retires — so the empty install list is now unspellable here rather than merely wrong.

The retain arms are gone, and that is the substantive correction

The first draft had three effects that mutated nothing — RetainModelArtifactAsCache and friends — minted so every address would have a non-empty remedy. That was the empty list wearing a name. An effect that performs no operation cannot produce a planned execution, and the plan row is non-empty in executions by construction, so honouring them meant weakening a real guarantee to satisfy a cosmetic one.

The actual defect was upstream: the runtime closure, the model blob and account linger were never serving-role facts. spark_serving_address_scope splits the roster — four service addresses retire, three baseline addresses stay desired. The cache survives because nothing asks for its removal, not because a retirement effect politely declined to act. Reaching the retirement map with a baseline address is a typed refusal.

SparkTrainingReadyProof is sole-constructor with one producer. Absence is a positive observation, and what counts as evidence is decided per subject: endpoint absence requires a completed listener enumeration whose target population is empty — no connection failure establishes it — while an unreached probe establishes nothing on any axis. Reading an unreached probe as an absence would hand the trainer a host with a live server still holding the unified pool.

Evidence, at the rung it actually reaches

27 of 27 claims PASS by execution (claim_batch --claim-run, both witness files, per-claim receipts with subject hashes). Four CI checks green.

The discriminating control was run and is reported honestly. Flipping srv6 back to SparkServingCell:

  • Through CI: stayed green. All test.claim.spark.* claims are outside the required floor's roster — the floor's own disposition ledger carries no spark rows, and it reports declined_outside_required_gate=64. This is the standing §4b rung drop "Required gate reduced to the compiler floor", not a defect in these claims. Nothing in this PR is gated by CI, and I am not claiming otherwise.
  • Through the claim executor: exactly four REDs — w_spark_roles_are_disjoint_over_the_assignment, w_serving_desired_hosts_are_the_serving_cells, w_serving_cell_is_not_offerable_to_the_training_fabric, w_a_fully_retired_training_cell_is_ready. Every role-independent claim stayed green. That is discrimination rather than a blanket break.

So the rung is mechanically preventable, not structurally guaranteed: the evidence executes and discriminates, but its enrolment depends on an executor the required gate does not currently run. Next-rung trigger: the required gate's restoration to a roster that includes domain claims.

A wall was checked against its own witness, and failed. The readiness witness could not author its fixtures — sole_constructor refused construction outside the defining module. That is DESIGN's "is the RED authorable?" question answered by the compiler, and the repair is a producer rather than a loosened constructor: the retirement observation is assembled from three probe captures through one decoder, and a pool figure must arrive with the capture that read it. A capture that did not answer carries no byte count, whatever integer is passed beside it.

The plan, not just the vocabulary

P0's bar names srv6's frozen plan containing real retirement effects, so three further claims ask what membership_reconcile actually produces when a host leaves the serving roster — which the vocabulary claims say nothing about. A host whose role changed plans exactly four removals and nothing else; every one realizes to exactly one retirement effect, totalling four (the assertion the old -> [] fails at zero); and the same reconcile over an unchanged roster plans nothing, without which "four removals" could be true of a plan that also proposed work on a converged host. Desired is derived by filtering observed through the scope split rather than listing three members by hand, so this is a claim about the split rather than a restatement of it.

Second control run for these: collapsing every address to baseline scope reds exactly the partition claim and the two plan claims, and leaves the idempotence claim green because it does not depend on the split.

One false-green, found in this PR's own evidence

w_the_retirement_sequence_is_the_install_order_reversed asserted membership and cardinality and never looked at position — its first conjunct was a tautology over any list. Every permutation of the four effects passed, including the two that matter: remove-file before stop leaves a running process the manager cannot name, and reload before removal re-reads a definition that still exists. The name made it worse than a weak check, because it told a reader the order was verified.

It now asserts each position through skip/first, and a third control was executed: swapping the first two elements of the authored retirement sequence reds that claim and nothing else. It is recorded here rather than quietly fixed because of why it survived — the two earlier controls varied the role assignment and the address scope, and neither perturbs effect order. A control only falsifies what it perturbs.

What review found that the claims did not

Four review rounds; the findings that changed the design are recorded here because two of them were defects the evidence above did not catch.

  • A nonzero exit was read as absence (review 57444). spark_serving_absence_from_probe mapped every ProbeFailed to ServingAbsent — a fail-open inside the module whose subject is not fail-opening. A permission denial, a malformed argv, a DNS failure or a timeout would each have proved a retirement that never happened. Which result discriminates is a property of the tool, so it is now cited per tool. (The endpoint's first repair — curl exit 7 — was itself still too strong and was superseded; see below. The axis now enumerates listeners via extdeps/iproute2/ss.dag, beside extdeps/procps/pgrep.dag exit 1 and the existing systemd_unit_load_state_not_found_wire.) The unit axis proves the rule could not have been patched by picking a code — systemctl show exits zero and prints not-found, so "nonzero means gone" was inverted there, not merely broad.
  • Two functions with no consumer. spark_serving_retirement_effect_mutates_host lost its reason when the retain arms were dropped, and spark_serving_planned_effect_is_retirement never had one. Both deleted; the first's claim replaced with one whose subject is the realization's ordered retirement sequence.
  • free_pool was a bare Int (reviews 57426, 57431) on the observation, the proof and the producer. Now std.measure.ByteSize, lifted at the parse boundary so the raw count stays at the single site that reads MemAvailable.
  • is_established -> Bool dissolved into spark_serving_absence_refusal, which returns the cause. Better independently of the rule cited for it: the gate now reports every unsatisfied axis with the cause the absence value already carried, instead of the first axis with a cause re-authored beside it.

Not in this PR, and why

The dry successor, and a correction to this PR's own evidence.

Two things landed here after the first review round, both worth naming because both were defects in what this PR claimed rather than in what it built.

The endpoint axis read curl exit 7 as proof that nothing was listening. curl(1) documents 7 as CURLE_COULDNT_CONNECT, "Failed to connect() to host or proxy" — one code for every reason a connect failed, and curl's own documentation tells callers the numeric code does not carry the complete cause. So a 7 establishes curl did not connect, while SparkTrainingReadyProof needs the intended host reached, the intended port tested directly, no proxy in the path, and no listener present. An unreachable host or an intervening proxy would each have proved a retirement that never happened. The axis now enumerates listening sockets (ss -H -l -t -n sport = :PORT) and reads the completed empty population as the absence; a query that did not complete establishes nothing. extdeps/curl is deleted with its last consumer.

That repair exposed the second: the witness fixture built all three absence axes from ProbeFailed{7} — curl's vocabulary and nothing else's — while pgrep exits 1 to mean nothing matched and the unit axis proves absence by answering not-found. The positive control w_a_fully_retired_training_cell_is_ready was failing while this file was reported green, because it had not been re-run after the decoder was made discriminating. Each axis now receives the evidence its own tool produces; 13/13 pass by execution.

What remains for P0 is no longer only wet. The dry successor is #9699, which found that the four valid removals this PR makes expressible could not reach a frozen schedule — the scheduler refused every plan containing a removal, and its refusal was justified by a premise this PR invalidated. P0-B also carries the production join, the teardown order, and the apply readback.

The wet half. P0's gate additionally requires the post-apply observation on srv6 proving endpoint and process absent, and a second convergence producing zero writes. Those need host access this session does not have (both Sparks answer at the network level; the enrolled key is not present here). They remain unclaimed.

gunbc-ci-auto-heal added 3 commits August 29, 2026 16:03
…e empty list: a role authority, and retirement given a vocabulary

SPARK-PAIR-0 P0. Two facts had no home in this corpus, and one of them was
load-bearing.

The first: what each DGX Spark is FOR. `spark_serving_desired_hosts` was a
hand-authored list naming srv5 and srv6, which was harmless while both served
and became wrong the moment one was wanted as a trainer -- deleting a name from
a literal is a change with no reviewable subject, and the intent lived nowhere.
`gunbc.spark.cell_role` now owns it as a closed sum, so a cell holding both
roles has no constructor, and the serving roster is a projection of it.

The second: how to retire a service. `spark_serving_install_effects_for_remove`
returned `[]`, and that was not an oversight -- every one of the thirteen
`ServingInstallEffect` constructors installs something, so there was no value
that function could have returned meaning "take it down". Removing a host from
desired state planned a removal whose remedy was silence, and only the plan
projection's non-empty-executions refusal stopped it applying as a successful
no-op.

Retirement is a sibling vocabulary rather than more arms on the install sum,
because `realization_install_effects` and the privilege count that ranks
realizations both presume their effects are the price of standing a service UP:
a StopUserUnit arm would appear in no realization's list, so every realization
would fold over a sum half of whose members can never be selected. The two meet
at the member ACTION -- an add or change installs, a removal retires -- so the
empty install list is now unspellable here rather than merely wrong.

THE FIRST DRAFT HAD THREE RETAIN ARMS AND THEY ARE GONE. RetainModelArtifactAsCache
and friends were minted so every address would have a non-empty remedy, which
was the empty list wearing a name: an effect that mutates nothing cannot produce
a planned execution, and the plan row is non-empty in executions by construction,
so honouring them meant weakening a real guarantee to satisfy a cosmetic one.
The actual defect was upstream -- the runtime closure, the model blob and account
linger were never serving-role facts. `spark_serving_address_scope` splits the
roster: four service addresses retire, three baseline addresses stay desired. The
cache survives because nothing asks for its removal, not because a retirement
effect politely declined to act.

`SparkTrainingReadyProof` is sole-constructor with one producer, so a training
offer cannot be written against a Spark whose retirement was never terminally
observed. Absence is a positive observation -- a refused connection is evidence,
a probe that never ran is not, and reading the second as the first would hand the
trainer a host with a live server still holding the unified pool.

WET EVIDENCE IS NOT IN THIS COMMIT. The post-apply observation on srv6 and the
second-converge no-op receipt need host access this session does not have; they
are P0's remaining gate, not a claim made here.
@gunbai-bot gunbai-bot Bot mentioned this pull request Aug 29, 2026
6 tasks
…tructor carriers get producers

The check on the check DESIGN asks for, run and failed. Building
SparkServingRetirementObservation and SparkUnifiedPoolObservation directly in the
witness did not compile -- both are sole_constructor and the compiler refuses
construction outside their defining module. A wall whose RED cannot be expressed
anywhere is a decoration, so the repair is a producer rather than a loosened
constructor: the retirement observation is assembled from three probe captures
through the one decoder, and a pool figure must arrive with the capture that read
it -- a capture that did not answer carries no byte count, whatever integer is
passed beside it.
@gunbai-bot gunbai-bot Bot changed the title spark tuning workflow Serving desired state named both Sparks and the removal remedy was the empty list: a role authority, and retirement given a vocabulary Aug 29, 2026
@gunbai-bot
gunbai-bot Bot marked this pull request as ready for review August 29, 2026 18:59
…td.measure ByteSize

Review 57426 is right, and the tell is that the surrounding paragraphs argued in
prose for exactly the distinction the type erased. The comment says a free
integer is a number somebody typed and the capture is what makes it a
measurement -- and then the carrier accepted any integer at all, so long as the
field name ended in _bytes. That is a second unit system standing beside the one
std.measure already owns, and it undoes the single-producer property the proof
exists to carry.

free_pool is now ByteSize on both the observation and the proof, the producer
takes a Nat and constructs through byte_size, and the positivity check reads
through byte_size_count. The witness fixture moves with them, because a fixture
that could still hand over a flat scalar would prove the type accepts one.
@gunbai-bot

gunbai-bot Bot commented Aug 29, 2026

Copy link
Copy Markdown
Contributor Author

Fixed in f8ce03b. Review 57426's four BLOCKING items were all the same defect and all correct.

The tell the review names is the one that decided it: the paragraphs around that carrier argue in prose that "a free integer is a number somebody typed" and "the capture is what makes it a measurement" — and then the type accepted any integer at all, provided the field name ended in _bytes. The comment was doing work the type had declined to do, which is the parallel unit system DESIGN §3 forbids and is worse on a sole-constructor proof than anywhere else, because the whole point of that carrier is that its existence is the evidence.

  • SparkUnifiedPoolObservation.free_bytes: Int → free_pool: ByteSize
  • SparkTrainingReadyProof.free_pool_bytes: Int → free_pool: ByteSize
  • spark_unified_pool_observation_from_capture takes Nat and constructs through byte_size, so the scalar stops at the producer boundary instead of propagating inward
  • the positivity refusal reads through byte_size_count
  • the witness fixture moved too, for the reason the review gives: a fixture that could still hand over a flat scalar would itself prove the type accepts one

No dissolve-on row — std.measure.ByteSize already exists and is already consumed for storage capacity in extdeps.systems.nvidia, so there was nothing to gate toward.

Re-executed after the change: all 9 readiness claims still PASS via claim_batch --claim-run, and the 12 role/retirement claims are unaffected.

One thing the PR body states and I want to keep visible rather than let the green checks imply otherwise: none of these claims is gated by CI. All test.claim.spark.* subjects sit outside the required floor's roster under the standing §4b rung drop "Required gate reduced to the compiler floor" — I verified that by pushing a control that flips srv6 back to SparkServingCell, which CI passed and the claim executor failed on exactly four role-dependent claims. The evidence executes and discriminates; its enrolment does not.

Review 57431, following its own first ruling through: taking a Nat and wrapping
it in byte_size on the way in put the flat scalar one call deeper than it needed
to go, for nothing. Pairing a ByteSize with the capture forbids fabrication
exactly as well, and the caller that reads MemAvailable is the one place a raw
count legitimately exists -- so the unlifted form now stays at the single site
that parses it. The witness rows construct through byte_size at the row, for the
same reason the carriers moved: a fixture that can still pass a flat scalar
proves the API accepts one.
@gunbai-bot

gunbai-bot Bot commented Aug 29, 2026

Copy link
Copy Markdown
Contributor Author

Fixed in the head commit. Review 57431 is right and is its own first ruling followed through: wrapping a Nat in byte_size inside the producer put the flat scalar one call deeper than it needed to go and bought nothing. Pairing a ByteSize with the capture forbids fabrication exactly as well.

  • spark_unified_pool_observation_from_capture now takes free_pool: ByteSize; std.nat is no longer imported by the module at all, so there is no unlifted count anywhere in it
  • the witness rows construct through byte_size(count: …) at the row — following the reviewer's point that a fixture which can still pass a flat scalar is itself proof the API accepts one
  • the paragraph above the carrier now says where the lift happens and why the caller that parses MemAvailable is the one place a raw count legitimately exists

Re-executed: 9/9 readiness claims PASS via claim_batch --claim-run; the 12 role/retirement claims are untouched by this change. No dissolve-on row, since ByteSize already exists and no gate was needed.

gunbc-ci-auto-heal added 2 commits August 29, 2026 20:00
… a host leaves the roster

Re-reading P0's own bar rather than the diff: it requires srv6's frozen plan to
contain actual stop/disable retirement effects rather than an empty removal. The
claims that existed established that the retirement VOCABULARY exists and that
each address maps to one operation -- neither of which says anything about what
membership_reconcile produces when the serving roster loses a host, which is the
question the empty remedy used to answer wrongly.

Three claims: a host whose role changed plans exactly four removals and nothing
else; every one of those removals realizes to exactly one retirement effect,
totalling four -- the assertion the old '-> []' fails at zero; and the same
reconcile over an unchanged roster plans nothing, without which 'four removals'
could be true of a plan that also proposed work on a converged host.

Desired is derived by FILTERING observed through the scope split rather than
listing three members by hand, which is what makes this a claim about the split
instead of a restatement of it. Discriminating control run: collapsing every
address to baseline scope reds exactly the partition claim and the two plan
claims, and leaves the idempotence claim green because it does not depend on the
split.
…onical accessor, and one nothing called

Review 57441 named three Boolean predicates. Two are defects and are gone.

spark_serving_address_is_role_scoped re-wrapped spark_serving_address_scope,
which is the reviewer's own sharpest point: a second way to ask a question the
canonical accessor already answers. The two filters now match the scope
coproduct directly, so there is one surface.

spark_serving_retirement_effect_mutates_host had no production consumer at all.
It was written to stop a retain arm returning a no-op, and when the retain arms
were removed its reason went with them -- leaving a function that existed only to
be asserted, which is the inert-declaration shape. Its claim is replaced by one
with a real subject: the realization's ordered retirement sequence, where every
adjacent pair is load-bearing.

spark_serving_absence_is_established stays, with the reasoning stated on the PR.
@gunbai-bot

gunbai-bot Bot commented Aug 29, 2026

Copy link
Copy Markdown
Contributor Author

Two of the three items in review 57441 were real defects and are fixed in the head commit. The third I am declining, with the reasoning below, because I could not verify the rule it rests on.

Fixed — spark_serving_address_is_role_scoped. This is the reviewer's sharpest point and it is correct: the predicate re-wrapped spark_serving_address_scope, giving two surfaces for one question. The two filters now match the scope coproduct directly and the wrapper is deleted.

Fixed — spark_serving_retirement_effect_mutates_host. Better than flagged, in fact: it had no production consumer at all. I wrote it to stop a retain arm returning a no-op, and when the retain arms were removed earlier in this PR its reason went with them — leaving a function that existed only to be asserted by its own witness. That is the inert-declaration shape, so it is deleted, and its claim is replaced with one whose subject is real: the realization's ordered retirement sequence (stop → disable → remove-file → reload), where every adjacent pair is load-bearing.

Declining — spark_serving_absence_is_established. Two reasons.

First, I cannot locate the cited rule. grep -i predicate over DESIGN.md returns one hit, in §4b, about not lifting arbitrary predicates to proof — a different subject. docs/design-ledgers.md has no predicate-dissolution row. If the rule lives in the review persona rather than the authority docs, I would rather it be named there so I can apply it consistently; as it stands I would be changing code against a rule I cannot read. There are also 1,872 -> Bool functions under dag/gunbc/ today, so a blanket reading of it would not describe this corpus.

Second, on the merits this one is not a convenience wrapper. It has three production call sites in spark_training_readiness, one per axis, and it is not a projection of a canonical accessor — it is the decision that only ServingAbsent establishes absence, which is the substantive claim of the whole module: ServingAbsenceUnobserved and ServingStillPresent are both non-absent for different reasons, and collapsing them at the three call sites would spell that decision three times. Inlining it would duplicate the rule, not dissolve it.

Happy to revisit if the rule is pointed at a file I can read.

24 of 24 claims still PASS by execution after these changes.

— sent from eager-pike-541

Review 57444's first finding is a real fail-open in the module whose whole
subject is not fail-opening. spark_serving_absence_from_probe mapped EVERY
ProbeFailed to ServingAbsent, on the reasoning that a dial against a dead port
fails. That is right about one exit code and wrong about every other: a
permission denial, a malformed argv, a DNS failure and a timeout each exit
nonzero, and each would have proved a retirement that never happened -- admitting
training onto a host with a live server still holding the unified pool, whose
symptom is an out-of-memory failure inside a training run, attributed to the
recipe.

Which result discriminates is a property of the TOOL, so it is cited rather than
decided here. extdeps.curl.exit_status and extdeps.procps.pgrep carry the two
exit codes with their manual citations; the unit axis consumes systemd's existing
systemd_unit_load_state_not_found_wire. The unit axis is what proves the old rule
could not have been patched with a code: systemctl show exits ZERO and prints
not-found, and systemd's own authority records that a nonzero there means the
MANAGER was unreadable -- so on that axis 'nonzero means gone' was not merely too
broad, it was inverted.

Also from the same review: spark_serving_planned_effect_is_retirement had no
caller anywhere in the tree and is deleted, and spark_serving_absence_is_established
is dissolved into spark_serving_absence_refusal, which returns the refusal cause
instead of a Bool. That was the better shape independently of the rule cited for
it: the gate now reports EVERY unsatisfied axis with the cause the absence value
already carried, rather than the first axis with a cause re-authored beside it.
@gunbai-bot

gunbai-bot Bot commented Aug 29, 2026

Copy link
Copy Markdown
Contributor Author

All three items in review 57444 are fixed in the head commit. The first was a real fail-open and the best finding on this PR so far.

ProbeFailed → ServingAbsent for every nonzero exit. Correct, and it was a fail-open sitting in the module whose entire subject is not fail-opening. My reasoning was right about one exit code and wrong about all the others: a permission denial, a malformed argv, a DNS failure and a timeout each exit nonzero, and each would have proved a retirement that never happened — admitting training onto a host with a live server still holding the unified pool. The symptom would surface as an out-of-memory failure inside a training run and be attributed to the recipe.

Which result discriminates is a property of the tool, so it is now cited rather than decided in this module:

  • extdeps/curl/exit_status.dag — exit 7, "Failed to connect to host", cited to curl(1). 6 (resolve), 22 (HTTP error) and 28 (timeout) are explicitly not absence.
  • extdeps/procps/pgrep.dag — exit 1, "no processes matched", cited to pgrep(1) EXIT STATUS, where 2 and 3 mean the question was never answered.
  • the unit axis consumes the existing systemd_unit_load_state_not_found_wire.

The unit axis is the one that shows this could not have been patched by picking a code. systemctl show --property=LoadState exits zero and prints not-found, and systemd's own authority in this repo already records that a nonzero there means the manager was unreadable. So on that axis "nonzero means gone" was not merely too broad — it was inverted. Two new claims cover it in both directions.

spark_serving_planned_effect_is_retirement. Deleted. The reviewer's whole-tree resolution is right: no caller anywhere. I added it alongside the planned-effect sum and never wired it.

spark_serving_absence_is_established. Dissolved into spark_serving_absence_refusal, which returns the refusal cause rather than a Bool — and I withdraw my earlier defence of it, though not for the reason given. The predicate was the wrong shape on its own merits: a Bool discarded exactly the half of the answer the gate needed, so the readiness function was re-authoring a cause string per axis while the absence value was already carrying a better one. The three per-axis branches are now one fold that reports every unsatisfied axis rather than the first, which is what an operator deciding between re-probing and re-running the retirement actually needs. My earlier reply argued inlining would duplicate the rule; that was true and I had not considered the third option, which was to make the carrier answer for itself.

I would still value a pointer to where the predicate-dissolution rule is written, since two reviews have now cited it and I cannot find it in DESIGN.md or docs/design-ledgers.md. It would have gotten me to this shape one round sooner.

27 of 27 claims PASS by execution after these changes (15 role/retirement, 12 readiness — four of them new for the exit-code discrimination).

Found in the side thread, and it is my own false-green rather than a review
finding: w_the_retirement_sequence_is_the_install_order_reversed asserted
membership and cardinality and never once looked at position. Its first
conjunct -- all(seq, e => include(seq, e)) -- is a tautology over any list at
all, and the four that follow ask only that each effect appears somewhere. Every
permutation of the four passed, including the two that are actually dangerous:
remove-file before stop leaves a running process the manager can no longer name,
and reload before removal re-reads a definition that still exists.

That is section 4b's decoration cited as coverage, made worse by a name that
told a reader the order was checked. The claim now asserts each position through
skip/first.

Discriminating control executed: swapping the first two elements of
realization_retirement_effects reds this claim and NOTHING else -- the other
fourteen are order-independent and stay green -- and the restored order passes.
@gunbai-bot

gunbai-bot Bot commented Aug 29, 2026

Copy link
Copy Markdown
Contributor Author

Pushed a fix for a false-green I introduced, found while reviewing this PR's own evidence rather than by a review.

w_the_retirement_sequence_is_the_install_order_reversed asserted membership and cardinality and never looked at position. Its first conjunct — all(seq, e => spark_serving_retirement_effects_include(effects: seq, needle: e)) — is a tautology over any list, and the four that followed only asked that each effect appear somewhere. Every permutation of the four passed, including the two that are actually dangerous: remove-file before stop leaves a running process the manager can no longer name, and reload before removal re-reads a definition that still exists.

The name made it worse than a weak check: it told a reader the order was verified. That is §4b's decoration-cited-as-coverage, and it would have been cited as evidence that the ordering argument in realization_retirement_effects holds.

The claim now asserts each position through skip/first. Discriminating control executed: swapping the first two elements of realization_retirement_effects reds this claim and nothing else — the other fourteen are order-independent and correctly stay green — and the restored order passes.

Worth noting for anyone reading the evidence section of this PR: this one slipped past two controls and four review rounds, because both controls varied the role assignment and the address scope, and neither varies effect order. A control only falsifies what it perturbs.

gunbai-bot Bot pushed a commit that referenced this pull request Aug 29, 2026
…eclared rather than reversed

P0-B, and the premise it repairs is one my own last PR invalidated.

spark_serving_schedule_rows refused every plan containing a MemberRemoved, and
its comment justified that with 'No Spark serving plan produces one today,
which is precisely why the arm must refuse instead of being written
speculatively.' #9679 made that false. So the route was: four valid removals ->
four valid retirement effects -> removal count > 0 -> scheduler refusal -> no
frozen schedule -> no scheduled typed actions -> no applicable plan artifact.
Expressible and unroutable, and no host access repairs it.

THE REFUSAL'S OTHER CLAIM WAS THAT TEARDOWN RUNS THE PREREQUISITE GRAPH BACKWARD,
AND THAT IS WRONG HERE. Reversing the resource edges gives enablement and
activity first, then registration, then the file -- an order that cannot execute,
because systemctl disable reads the unit file's [Install] section to know which
symlinks to remove, so the FILE MUST STILL EXIST WHEN DISABLE RUNS. Install never
had to say so: writing the file always precedes enabling it, so the dependency
was satisfied incidentally by the forward order and was never modeled. There is
nothing to reverse into it. Registration is wrong in the reversed order for a
second reason -- its teardown remedy is a reload that can only follow the file's
deletion, which this module's own vocabulary already calls a barrier, and
barriers do not reverse.

So the teardown chain is declared address by address, each edge carrying its
reason: activity -> enablement -> content -> registration. A mixed plan still
refuses, because composing the forward and teardown chains over the same four
addresses is a third relation nobody has modeled.

18 claims pass by execution, including the frozen order asserted positionally.
gunbai-bot Bot pushed a commit that referenced this pull request Aug 29, 2026
…h retired anything

The second half of the P0-B gap. The role authority landed in #9679 and
spark_serving_full_membership_plan never consulted it: it constructed the full
seven-member serving desired state for any host handed to it. Two production
paths existed and neither retires:

  iterate spark_serving_desired_hosts -> srv6 is absent from the loop -> no
  retirement is ever planned

  call the planner for srv6 directly -> full serving desired state -> serving is
  retained or restored on the machine that was supposed to stop

Every claim before this one built its plan by filtering a fixture through the
scope split, which proves the split and says nothing about the planner. Desired
state is now scoped by the host's role -- seven members for a serving cell, the
three baseline members for a training cell -- so the four removals the teardown
schedule orders come from the ROLE rather than from a filter the caller
remembered to apply.

A HOST WITH NO SPARK ROLE REFUSES rather than defaulting to serving. This planner
is reachable with any HostIdentity, and answering with full serving desired state
for a machine nobody assigned is how a convergence installs a model server on the
wrong box.

20 claims pass by execution, both counts asserted -- a planner returning baseline
for every host would satisfy a claim that only checked srv6.
gunbc-ci-auto-heal and others added 6 commits August 30, 2026 01:01
…rm: the endpoint axis enumerates listeners

The endpoint axis read curl exit 7 as proof that nothing was listening.
curl(1) documents 7 as CURLE_COULDNT_CONNECT, "Failed to connect() to host or
proxy" -- one code covering every reason a connect failed, with curl's own
documentation telling callers the numeric code does not carry the complete
cause. What a 7 establishes is `curl did not connect`. What
SparkTrainingReadyProof needs is that the intended host was reached, that the
intended port was tested directly, that no proxy intervened, and that no
listener exists.

That gap is the same fail-open class this module already repaired once, one
rung subtler: an unreachable host, an intervening proxy or a bound-elsewhere
service each produce a 7, and each would have proved a retirement that never
happened -- admitting training onto a machine still holding the unified pool.

So the axis enumerates instead of dialing. AbsenceByEmptyEnumeration reads a
COMPLETED `ss -H -l -t -n sport = :PORT` whose population is empty; a query that
did not complete exits nonzero and establishes nothing. The absence is a
population rather than a status, which is the distinction the exit code could
not make. extdeps/curl is deleted with its last consumer.

Beside it, the fixture repair the endpoint change exposed. One helper built all
three axes from ProbeFailed{7} -- curl's vocabulary and nothing else's -- while
pgrep exits 1 to mean nothing matched and the unit query proves absence by
ANSWERING not-found. The positive control w_a_fully_retired_training_cell_is_ready
was therefore FAILING while this file was reported green: it had not been re-run
after the decoder was made discriminating. Each axis is now handed the evidence
its own tool produces.

Claims: 13/13 by execution via claim_batch --claim-run, including the three that
replace the exit-code claims. Every input in
w_a_connect_failure_never_establishes_endpoint_absence was ACCEPTED as absence
by the superseded evidence, 7 listed first because it was the code that used to
prove retirement.

Co-Authored-By: Claude Opus 5 (1M context) <noreply@anthropic.com>
Claude-Session: https://claude.ai/code/session_018dHefbXnLiW1XNs2vVxhwj
…t trimmed its neighbours

ss_query_exit_ok and ss_listening_tcp_port_argv_words had no caller. Only
ss_enumeration_is_empty is read, by training_ready's endpoint decoder. The
module's own authority note says only what this repository consumes is
modelled, so the file contradicted its own declared scope -- the same defect
that was trimmed from the curl and pgrep rows an hour earlier, reintroduced by
me in their replacement.

The invocation stays as cited prose and the success status is not minted at
all: the probe capture site that will issue this command is P0's wet half, and
a row waiting for a consumer that does not exist yet is the artifact-with-no-
final-consumer tell. The reason is now written into the module rather than
remembered.

Co-Authored-By: Claude Opus 5 (1M context) <noreply@anthropic.com>
Claude-Session: https://claude.ai/code/session_018dHefbXnLiW1XNs2vVxhwj
… in as a bare Bool

Two review findings, both taken.

SparkUnifiedPoolObservation carried `observed_after_retirement: Bool`. Its own
comment argued that reading the pool before the retirement "would measure the
wrong machine state and would be indistinguishable in the type" -- and then let
the caller assert exactly that fact as a boolean, with the readiness gate
VALIDATING the assertion it had just been handed. That is validation standing
where construction was available, and it fails §5's own tell: the check is
satisfied by editing the argument while the realization lies.

The reading now carries the SparkServingRetirementObservation it followed, and
the producer refuses unless that retirement is established on all three
subjects. "Measured before the retirement" has no constructor any more: the
value proving the ordering did not exist until the retirement was observed.

Two things dissolved with it. The readiness gate took the retirement and the
pool as separate arguments and checked they agreed -- they cannot disagree now,
so it reads one carrier. And its retirement-not-established arm is gone,
because a pool cannot exist over an unestablished retirement. Per §4b the
production checks retire and their discriminating REDs do not: disabling the
producer's refusal reds three claims -- the unobserved-subject axis, the
still-serving endpoint, and the new pool-over-unestablished-retirement claim --
so the refusals those two arms used to carry are now enforced one stage earlier
rather than deleted.

spark_cell_role_eq matched the coproduct against itself in four arms, a second
enumeration of the variants beside spark_cell_role_wire: adding a role would
require both edited and only one would fail to compile. Equality now derives
from the wire projection, the idiom serving_membership's address equality and
serving_execution_schedule's fact equality already use for this question.

13/13 by execution, positive control included.

Co-Authored-By: Claude Opus 5 (1M context) <noreply@anthropic.com>
Claude-Session: https://claude.ai/code/session_018dHefbXnLiW1XNs2vVxhwj
…ction is evidence

Three module paragraphs survived the endpoint repair unchanged and now
contradicted the code beneath them: the absence sum's summary still said "a
refused connection is evidence", the per-subject list still said a dial exits 7
and only 7 to mean nothing is listening, and the observation producer still
framed the axes as disagreeing about what a refused connection means. Leaving a
superseded summary above a corrected model is worse than leaving neither --
DESIGN's §4c says an annotation must not restate what the declaration says, and
one that MISstates it will be read as the authority by anyone who reads the
prose first.

They are rewritten to the current model rather than annotated as historical: no
connection failure establishes endpoint absence; endpoint absence requires a
completed host-local listener enumeration whose exact target population is
empty; a failed or unreached enumeration establishes nothing.

Beside them, the boundary this repair does NOT close, stated so it cannot be
overgraded. SparkServingProbeCapture carries a completion status and stdout and
nothing about provenance, so nothing in the type says an enumeration ran on the
right host, filtered the right port, ran this module's argv, or ran after the
retirement it is read against. The decoder's semantics are correct; its
producer provenance is unmodelled, and replacing the generic capture with a
typed executed receipt is a P0-WET obligation for all three axes.

Co-Authored-By: Claude Opus 5 (1M context) <noreply@anthropic.com>
Claude-Session: https://claude.ai/code/session_018dHefbXnLiW1XNs2vVxhwj
@gunbai-bot

gunbai-bot Bot commented Aug 30, 2026

Copy link
Copy Markdown
Contributor Author

CI status: the floor failure on this head is not in this diff.

The required-witnesses-floor job reports the claim population clean and fails on a different phase:

required-floor: planned=2904 executed=2904 not_attempted=0 terminal=2904
                passed=2829 known_red_held=26 failed=0
required-ci: FAILED PHASE namespace-wave-admission (0 unadjudicated delta(s), 6 stale admission(s))

failed=0 — no claim failed. The six stale rows are rust-source-prefix-relocation-01..06, against gunbc.stage0_rust_product_reachability::path_is_test_harness and gunbc.stage0_rust_source_lifecycle_scaffold::rust_source_discovery_prefix_hints. This diff touches dag/gunbc/spark/**, three extdeps modules, dag/std/measure.dag and its regenerated std_measure.rs projection — none of those declare or bind the cited symbols.

This is the second distinct stale-admission set to block this branch tonight. The first was six DeclaredCallableIdentity hoist rows in v1.compiler.infer_lookup/infer_sigs, which cleared when #9705 ("Fourth admission shrink") merged and I integrated main. This branch now contains current main (38106f3fbee) and the new set appeared with it.

I have not edited the admission ledger here. Retiring another lane's admission rows inside this PR would be fabricating a fix for someone else's subject, in a ledger a separate branch (unbreak/fourth-admission-shrink) exists to shrink — and a wrong retirement there is a silent safety regression rather than a loud one.

required-witnesses-build (regen) and rust-unit-tests pass on this head. The Spark claim modules are green by execution via claim_batch --claim-run; per the standing "required gate reduced to the compiler floor" drop, none of test.claim.spark.* sit inside the required roster, so that greenness is reported here rather than gated by CI.

— sent from eager-pike-541

@gunbai-bot

gunbai-bot Bot commented Aug 30, 2026

Copy link
Copy Markdown
Contributor Author

Correcting my earlier comment on this PR: I diagnosed the six rust-source-prefix-relocation-01..06 stale admissions as external and said they needed an operator decision. That was wrong, and the reasoning behind it was wrong.

I had checked that my diff does not touch gunbc.stage0_rust_product_reachability or stage0_rust_source_lifecycle_scaffold and concluded the rows could not be mine to fix. That only established I did not author them; it said nothing about why they were firing. The actual cause is that those six rows live in src/v1/stage0/src/namespace_wave_admission.rs and were retired on main by fa03a019395 (#9698). My base was 38106f3fbee, exactly one commit earlier, so my branch still carried them while the deltas they describe were gone. The branch was stale, not the ledger.

What pointed at it: main is green at 38106f3fbee and after, and #9715 fails the same phase on a different stale row (required-ci lane vocabulary … (#9698)) from a newer base. Same mechanism, different rows per base — which is not the shape of an external defect.

Fixed by merging current main. No admission ledger was edited.

— sent from eager-pike-541

@gunbai-bot

gunbai-bot Bot commented Aug 30, 2026

Copy link
Copy Markdown
Contributor Author

Re: review 57493 (REQUEST_CHANGES) — addressed. Recording the verification, since the dashboard still counts this as active pending a re-review at the current head.

The finding: ProbeAnswered accepted a caller-supplied free_pool unchanged and only copied stdout into a prose producer field, so any successful capture could certify an arbitrary capacity.

The fix removes the constructor rather than adding a check. spark_unified_pool_observation_from_capture no longer takes free_pool at all — its signature is now (capture: SparkServingProbeCapture, after: SparkServingRetirementObservation) -> SparkUnifiedPoolReading, and the quantity is parsed out of the capture's own stdout via parse_int. There is no longer a parameter through which an unverified capacity can enter, so the state the review describes has no way to be written.

Three typed refusals guard the parse, each landing as SparkUnifiedPoolUnread with a located cause: a body that is not a decimal MemAvailable count, a non-positive count, and a reading paired with a retirement that is not established. That last one is checked before the capture is examined at all.

Two of the review's neighbours were dissolved the same way in this stack: the caller-asserted observed_after_retirement: Bool became the after: SparkServingRetirementObservation carrier, and the duplicate host field was removed so the identity derives from pool.after.host. Discriminating controls: disabling the producer's refusal reds three claims, and the two claims whose states lost their constructors were deleted rather than left asserting an unwritable red.

The review's alternative — "keep readiness refused until the documented P0-WET capability exists" — is still where this sits for provenance. The capture carries no host, argv, port, or retirement generation, and that boundary is documented in the module rather than papered over. Parsing the number from the receipt fixes the dry construction defect; it does not claim the receipt was produced on the host it describes.

spark_training_ready_witness_test runs 13/13 by execution.

— sent from eager-pike-541

@briansrls
briansrls merged commit 1b59e7d into main Aug 30, 2026
4 checks passed
@briansrls
briansrls deleted the session/eager-pike-541 branch August 30, 2026 05:19
briansrls pushed a commit that referenced this pull request Aug 30, 2026
…clared, and the planner joined to the role authority (#9699)

* Serving desired state named both Sparks and the removal remedy was the empty list: a role authority, and retirement given a vocabulary

SPARK-PAIR-0 P0. Two facts had no home in this corpus, and one of them was
load-bearing.

The first: what each DGX Spark is FOR. `spark_serving_desired_hosts` was a
hand-authored list naming srv5 and srv6, which was harmless while both served
and became wrong the moment one was wanted as a trainer -- deleting a name from
a literal is a change with no reviewable subject, and the intent lived nowhere.
`gunbc.spark.cell_role` now owns it as a closed sum, so a cell holding both
roles has no constructor, and the serving roster is a projection of it.

The second: how to retire a service. `spark_serving_install_effects_for_remove`
returned `[]`, and that was not an oversight -- every one of the thirteen
`ServingInstallEffect` constructors installs something, so there was no value
that function could have returned meaning "take it down". Removing a host from
desired state planned a removal whose remedy was silence, and only the plan
projection's non-empty-executions refusal stopped it applying as a successful
no-op.

Retirement is a sibling vocabulary rather than more arms on the install sum,
because `realization_install_effects` and the privilege count that ranks
realizations both presume their effects are the price of standing a service UP:
a StopUserUnit arm would appear in no realization's list, so every realization
would fold over a sum half of whose members can never be selected. The two meet
at the member ACTION -- an add or change installs, a removal retires -- so the
empty install list is now unspellable here rather than merely wrong.

THE FIRST DRAFT HAD THREE RETAIN ARMS AND THEY ARE GONE. RetainModelArtifactAsCache
and friends were minted so every address would have a non-empty remedy, which
was the empty list wearing a name: an effect that mutates nothing cannot produce
a planned execution, and the plan row is non-empty in executions by construction,
so honouring them meant weakening a real guarantee to satisfy a cosmetic one.
The actual defect was upstream -- the runtime closure, the model blob and account
linger were never serving-role facts. `spark_serving_address_scope` splits the
roster: four service addresses retire, three baseline addresses stay desired. The
cache survives because nothing asks for its removal, not because a retirement
effect politely declined to act.

`SparkTrainingReadyProof` is sole-constructor with one producer, so a training
offer cannot be written against a Spark whose retirement was never terminally
observed. Absence is a positive observation -- a refused connection is evidence,
a probe that never ran is not, and reading the second as the first would hand the
trainer a host with a live server still holding the unified pool.

WET EVIDENCE IS NOT IN THIS COMMIT. The post-apply observation on srv6 and the
second-converge no-op receipt need host access this session does not have; they
are P0's remaining gate, not a claim made here.

* Repair the three parse and resolution errors the floor named

* An 'as' cast does not parse in an if condition: host identity gets an equality beside its type

* The readiness witness could not author its own fixture: the sole-constructor carriers get producers

The check on the check DESIGN asks for, run and failed. Building
SparkServingRetirementObservation and SparkUnifiedPoolObservation directly in the
witness did not compile -- both are sole_constructor and the compiler refuses
construction outside their defining module. A wall whose RED cannot be expressed
anywhere is a decoration, so the repair is a producer rather than a loosened
constructor: the retirement observation is assembled from three probe captures
through the one decoder, and a pool figure must arrive with the capture that read
it -- a capture that did not answer carries no byte count, whatever integer is
passed beside it.

* The free pool was a bare Int on the proof carrier: route it through std.measure ByteSize

Review 57426 is right, and the tell is that the surrounding paragraphs argued in
prose for exactly the distinction the type erased. The comment says a free
integer is a number somebody typed and the capture is what makes it a
measurement -- and then the carrier accepted any integer at all, so long as the
field name ended in _bytes. That is a second unit system standing beside the one
std.measure already owns, and it undoes the single-producer property the proof
exists to carry.

free_pool is now ByteSize on both the observation and the proof, the producer
takes a Nat and constructs through byte_size, and the positivity check reads
through byte_size_count. The witness fixture moves with them, because a fixture
that could still hand over a flat scalar would prove the type accepts one.

* Lift the byte count at the parse boundary, not one call inside it

Review 57431, following its own first ruling through: taking a Nat and wrapping
it in byte_size on the way in put the flat scalar one call deeper than it needed
to go, for nothing. Pairing a ByteSize with the capture forbids fabrication
exactly as well, and the caller that reads MemAvailable is the one place a raw
count legitimately exists -- so the unlifted form now stays at the single site
that parses it. The witness rows construct through byte_size at the row, for the
same reason the carriers moved: a fixture that can still pass a flat scalar
proves the API accepts one.

* The gate names the PLAN, and nothing asked what a reconcile does when a host leaves the roster

Re-reading P0's own bar rather than the diff: it requires srv6's frozen plan to
contain actual stop/disable retirement effects rather than an empty removal. The
claims that existed established that the retirement VOCABULARY exists and that
each address maps to one operation -- neither of which says anything about what
membership_reconcile produces when the serving roster loses a host, which is the
question the empty remedy used to answer wrongly.

Three claims: a host whose role changed plans exactly four removals and nothing
else; every one of those removals realizes to exactly one retirement effect,
totalling four -- the assertion the old '-> []' fails at zero; and the same
reconcile over an unchanged roster plans nothing, without which 'four removals'
could be true of a plan that also proposed work on a converged host.

Desired is derived by FILTERING observed through the scope split rather than
listing three members by hand, which is what makes this a claim about the split
instead of a restatement of it. Discriminating control run: collapsing every
address to baseline scope reds exactly the partition claim and the two plan
claims, and leaves the idempotence claim green because it does not depend on the
split.

* Two of the three flagged predicates were real: a wrapper over the canonical accessor, and one nothing called

Review 57441 named three Boolean predicates. Two are defects and are gone.

spark_serving_address_is_role_scoped re-wrapped spark_serving_address_scope,
which is the reviewer's own sharpest point: a second way to ask a question the
canonical accessor already answers. The two filters now match the scope
coproduct directly, so there is one surface.

spark_serving_retirement_effect_mutates_host had no production consumer at all.
It was written to stop a retain arm returning a no-op, and when the retain arms
were removed its reason went with them -- leaving a function that existed only to
be asserted, which is the inert-declaration shape. Its claim is replaced by one
with a real subject: the realization's ordered retirement sequence, where every
adjacent pair is load-bearing.

spark_serving_absence_is_established stays, with the reasoning stated on the PR.

* A nonzero exit is not absence, and my own decoder said it was

Review 57444's first finding is a real fail-open in the module whose whole
subject is not fail-opening. spark_serving_absence_from_probe mapped EVERY
ProbeFailed to ServingAbsent, on the reasoning that a dial against a dead port
fails. That is right about one exit code and wrong about every other: a
permission denial, a malformed argv, a DNS failure and a timeout each exit
nonzero, and each would have proved a retirement that never happened -- admitting
training onto a host with a live server still holding the unified pool, whose
symptom is an out-of-memory failure inside a training run, attributed to the
recipe.

Which result discriminates is a property of the TOOL, so it is cited rather than
decided here. extdeps.curl.exit_status and extdeps.procps.pgrep carry the two
exit codes with their manual citations; the unit axis consumes systemd's existing
systemd_unit_load_state_not_found_wire. The unit axis is what proves the old rule
could not have been patched with a code: systemctl show exits ZERO and prints
not-found, and systemd's own authority records that a nonzero there means the
MANAGER was unreadable -- so on that axis 'nonzero means gone' was not merely too
broad, it was inverted.

Also from the same review: spark_serving_planned_effect_is_retirement had no
caller anywhere in the tree and is deleted, and spark_serving_absence_is_established
is dissolved into spark_serving_absence_refusal, which returns the refusal cause
instead of a Bool. That was the better shape independently of the rule cited for
it: the gate now reports EVERY unsatisfied axis with the cause the absence value
already carried, rather than the first axis with a cause re-authored beside it.

* A claim named 'order reversed' that any permutation passes

Found in the side thread, and it is my own false-green rather than a review
finding: w_the_retirement_sequence_is_the_install_order_reversed asserted
membership and cardinality and never once looked at position. Its first
conjunct -- all(seq, e => include(seq, e)) -- is a tautology over any list at
all, and the four that follow ask only that each effect appears somewhere. Every
permutation of the four passed, including the two that are actually dangerous:
remove-file before stop leaves a running process the manager can no longer name,
and reload before removal re-reads a definition that still exists.

That is section 4b's decoration cited as coverage, made worse by a name that
told a reader the order was checked. The claim now asserts each position through
skip/first.

Discriminating control executed: swapping the first two elements of
realization_retirement_effects reds this claim and NOTHING else -- the other
fourteen are order-independent and stay green -- and the restored order passes.

* The four valid removals could not reach a schedule: teardown order, declared rather than reversed

P0-B, and the premise it repairs is one my own last PR invalidated.

spark_serving_schedule_rows refused every plan containing a MemberRemoved, and
its comment justified that with 'No Spark serving plan produces one today,
which is precisely why the arm must refuse instead of being written
speculatively.' #9679 made that false. So the route was: four valid removals ->
four valid retirement effects -> removal count > 0 -> scheduler refusal -> no
frozen schedule -> no scheduled typed actions -> no applicable plan artifact.
Expressible and unroutable, and no host access repairs it.

THE REFUSAL'S OTHER CLAIM WAS THAT TEARDOWN RUNS THE PREREQUISITE GRAPH BACKWARD,
AND THAT IS WRONG HERE. Reversing the resource edges gives enablement and
activity first, then registration, then the file -- an order that cannot execute,
because systemctl disable reads the unit file's [Install] section to know which
symlinks to remove, so the FILE MUST STILL EXIST WHEN DISABLE RUNS. Install never
had to say so: writing the file always precedes enabling it, so the dependency
was satisfied incidentally by the forward order and was never modeled. There is
nothing to reverse into it. Registration is wrong in the reversed order for a
second reason -- its teardown remedy is a reload that can only follow the file's
deletion, which this module's own vocabulary already calls a barrier, and
barriers do not reverse.

So the teardown chain is declared address by address, each edge carrying its
reason: activity -> enablement -> content -> registration. A mixed plan still
refuses, because composing the forward and teardown chains over the same four
addresses is a third relation nobody has modeled.

18 claims pass by execution, including the frozen order asserted positionally.

* The planner never asked the role authority, so neither production path retired anything

The second half of the P0-B gap. The role authority landed in #9679 and
spark_serving_full_membership_plan never consulted it: it constructed the full
seven-member serving desired state for any host handed to it. Two production
paths existed and neither retires:

  iterate spark_serving_desired_hosts -> srv6 is absent from the loop -> no
  retirement is ever planned

  call the planner for srv6 directly -> full serving desired state -> serving is
  retained or restored on the machine that was supposed to stop

Every claim before this one built its plan by filtering a fixture through the
scope split, which proves the split and says nothing about the planner. Desired
state is now scoped by the host's role -- seven members for a serving cell, the
three baseline members for a training cell -- so the four removals the teardown
schedule orders come from the ROLE rather than from a filter the caller
remembered to apply.

A HOST WITH NO SPARK ROLE REFUSES rather than defaulting to serving. This planner
is reachable with any HostIdentity, and answering with full serving desired state
for a machine nobody assigned is how a convergence installs a model server on the
wrong box.

20 claims pass by execution, both counts asserted -- a planner returning baseline
for every host would satisfy a claim that only checked srv6.

* The teardown chain declared an edge no host fact grounds, and a plan of refusals froze as "nothing to do"

Review findings on P0-B, taken in full rather than deferred.

The teardown order was authored as a total chain, activity -> enablement ->
content -> registration. Only three of those edges are grounded: stop and
disable each read the unit file, so both precede its removal, and the reload
follows the removal as a barrier. Nothing grounds an edge BETWEEN stop and
disable -- they answer independent final-state obligations and neither reads
the other's result -- so the chain rejected `disable -> stop -> rm -> reload`,
which is equally safe. The relation is now a partial order carrying a kind per
edge, and the positional claim asserts what the facts establish and not one
edge more. Control: swapping the two predecessors keeps the order claim green;
the blind reversal of the install graph still reds it.

realization_retirement_effects is deleted. It authored the same order as a
list, under a comment saying teardown is the install order read backwards --
the claim this PR's own finding falsifies. Two authorities for one order, one
of them known wrong.

The direction sum reaching relation selection had a Mixed arm answering the
empty relation, correct only because the projection had already refused it:
an absorbing fallback one caller rearrangement away from scheduling a mixed
plan in diff order. SparkServingSchedulableDirection has no constructor for it.
Beside it, a plan whose only actions were MemberRemovalRefused or
MemberRosterDuplicateKeyRefused classified as EMPTY and froze a schedule
saying "nothing to do" about a reconcile that had said "I could not answer".
Both refusal arms now refuse before any row is projected.

spark_serving_converge_host_is_spark is deleted. It answered "is this a Spark
cell?" ahead of the role authority; they agree on today's fleet and would
diverge on the next assigned host, the older list rejecting a cell the
authority had admitted.

spark_cell_role_of folded a list and let the last matching row win, in a lookup
two authorization paths consult. It returns Absent | Unique | Ambiguous, and
both consumers refuse the ambiguous arm.

Evidence, and two false greens found while producing it:

The new claims enter through spark_serving_fleet_converge_plan_artifact, the
production root, rather than through the three helpers it composes: a
regression pointing the planner back at the unscoped desired members would
have left every previous claim green. srv6 observed fully serving plans four
retirement rows, srv5 in the same state plans none, no baseline address appears
in the rows, an unassigned host refuses.

The P0-A positive control w_a_fully_retired_training_cell_is_ready was FAILING
on the previous head. Its fixture built all three absence axes from
ProbeFailed{7}, which is correct only for curl -- pgrep proves absence with
exit 1, and the unit axis proves it by ANSWERING not-found. The fixture
predated the fail-open repair and the wrong file was re-run after it.

spark_serving_full_plan_witness_test asserted a seven-member serving install on
srv6, which the role authority now makes a training cell: an existing direct
consumer of the changed production function, stale in a way no new green file
covers. Recentred on srv5. Its untranslatable-member claim went red because the
refusal moved upstream; retargeted to the refusal it now causes, with a note
that the downstream drop-count check keeps an authorable red through the
reorder fold and is not thereby decoration.

24/24 retirement claims and 45/45 full-plan claims pass by execution via
claim_batch --claim-run. None of these are inside the required floor roster,
under the standing "required gate reduced to the compiler floor" drop.

Co-Authored-By: Claude Opus 5 (1M context) <noreply@anthropic.com>
Claude-Session: https://claude.ai/code/session_018dHefbXnLiW1XNs2vVxhwj

* A dial that failed to connect was standing in the proof's good-news arm: the endpoint axis enumerates listeners

The endpoint axis read curl exit 7 as proof that nothing was listening.
curl(1) documents 7 as CURLE_COULDNT_CONNECT, "Failed to connect() to host or
proxy" -- one code covering every reason a connect failed, with curl's own
documentation telling callers the numeric code does not carry the complete
cause. What a 7 establishes is `curl did not connect`. What
SparkTrainingReadyProof needs is that the intended host was reached, that the
intended port was tested directly, that no proxy intervened, and that no
listener exists.

That gap is the same fail-open class this module already repaired once, one
rung subtler: an unreachable host, an intervening proxy or a bound-elsewhere
service each produce a 7, and each would have proved a retirement that never
happened -- admitting training onto a machine still holding the unified pool.

So the axis enumerates instead of dialing. AbsenceByEmptyEnumeration reads a
COMPLETED `ss -H -l -t -n sport = :PORT` whose population is empty; a query that
did not complete exits nonzero and establishes nothing. The absence is a
population rather than a status, which is the distinction the exit code could
not make. extdeps/curl is deleted with its last consumer.

Beside it, the fixture repair the endpoint change exposed. One helper built all
three axes from ProbeFailed{7} -- curl's vocabulary and nothing else's -- while
pgrep exits 1 to mean nothing matched and the unit query proves absence by
ANSWERING not-found. The positive control w_a_fully_retired_training_cell_is_ready
was therefore FAILING while this file was reported green: it had not been re-run
after the decoder was made discriminating. Each axis is now handed the evidence
its own tool produces.

Claims: 13/13 by execution via claim_batch --claim-run, including the three that
replace the exit-code claims. Every input in
w_a_connect_failure_never_establishes_endpoint_absence was ACCEPTED as absence
by the superseded evidence, 7 listed first because it was the code that used to
prove retirement.

Co-Authored-By: Claude Opus 5 (1M context) <noreply@anthropic.com>
Claude-Session: https://claude.ai/code/session_018dHefbXnLiW1XNs2vVxhwj

* Four rows on the artifact is not four operations on the host: the retirement path gets its apply readback

Three additions from the second review round.

The production-artifact claims established that the planner freezes four
retirement rows. They did not establish that the consumer which APPLIES frozen
rows handles the retirement arm at all. Apply executes the operations the plan
realized into each row, so a retirement effect that realized to no command
would arrive as four effects and zero steps -- reporting a retirement applied
while executing nothing, which is exactly the fail-open the freeze exists to
shut. The claim now continues through spark_serving_apply_reads_frozen_typed_rows
and counts what would actually run: four effects, four argv, stop and disable
each before rm, rm before daemon-reload, and no operation naming ollama or
loginctl -- the retained cache is not merely unplanned, it is unreachable from
what apply would execute.

The order is asserted over the applied argv rather than the frozen row order,
because the row claim cannot see whether apply preserves the schedule or
re-derives it. Only the two grounded precedences are asserted; the stop/disable
pair is not, because the partial order does not decide it.

spark_hosts_with_role filtered the raw assignment rows, so the two ways of
asking the same question disagreed: the point lookup refuses a duplicated host
as ambiguous while the roster still listed it, and spark_serving_desired_hosts
would carry a host the planner refuses individually. The roster now projects
through the lookup, so an ambiguous host is in no roster -- the fail-closed
reading of a cell whose purpose the roster does not decide.

Co-Authored-By: Claude Opus 5 (1M context) <noreply@anthropic.com>
Claude-Session: https://claude.ai/code/session_018dHefbXnLiW1XNs2vVxhwj

* Two rows authored ahead of their consumer, in the module that had just trimmed its neighbours

ss_query_exit_ok and ss_listening_tcp_port_argv_words had no caller. Only
ss_enumeration_is_empty is read, by training_ready's endpoint decoder. The
module's own authority note says only what this repository consumes is
modelled, so the file contradicted its own declared scope -- the same defect
that was trimmed from the curl and pgrep rows an hour earlier, reintroduced by
me in their replacement.

The invocation stays as cited prose and the success status is not minted at
all: the probe capture site that will issue this command is P0's wet half, and
a row waiting for a consumer that does not exist yet is the artifact-with-no-
final-consumer tell. The reason is now written into the module rather than
remembered.

Co-Authored-By: Claude Opus 5 (1M context) <noreply@anthropic.com>
Claude-Session: https://claude.ai/code/session_018dHefbXnLiW1XNs2vVxhwj

* sole_constructor protected the carrier while the ordering fact walked in as a bare Bool

Two review findings, both taken.

SparkUnifiedPoolObservation carried `observed_after_retirement: Bool`. Its own
comment argued that reading the pool before the retirement "would measure the
wrong machine state and would be indistinguishable in the type" -- and then let
the caller assert exactly that fact as a boolean, with the readiness gate
VALIDATING the assertion it had just been handed. That is validation standing
where construction was available, and it fails §5's own tell: the check is
satisfied by editing the argument while the realization lies.

The reading now carries the SparkServingRetirementObservation it followed, and
the producer refuses unless that retirement is established on all three
subjects. "Measured before the retirement" has no constructor any more: the
value proving the ordering did not exist until the retirement was observed.

Two things dissolved with it. The readiness gate took the retirement and the
pool as separate arguments and checked they agreed -- they cannot disagree now,
so it reads one carrier. And its retirement-not-established arm is gone,
because a pool cannot exist over an unestablished retirement. Per §4b the
production checks retire and their discriminating REDs do not: disabling the
producer's refusal reds three claims -- the unobserved-subject axis, the
still-serving endpoint, and the new pool-over-unestablished-retirement claim --
so the refusals those two arms used to carry are now enforced one stage earlier
rather than deleted.

spark_cell_role_eq matched the coproduct against itself in four arms, a second
enumeration of the variants beside spark_cell_role_wire: adding a role would
require both edited and only one would fail to compile. Equality now derives
from the wire projection, the idiom serving_membership's address equality and
serving_execution_schedule's fact equality already use for this question.

13/13 by execution, positive control included.

Co-Authored-By: Claude Opus 5 (1M context) <noreply@anthropic.com>
Claude-Session: https://claude.ai/code/session_018dHefbXnLiW1XNs2vVxhwj

* The correct model sat under a summary that still said a refused connection is evidence

Three module paragraphs survived the endpoint repair unchanged and now
contradicted the code beneath them: the absence sum's summary still said "a
refused connection is evidence", the per-subject list still said a dial exits 7
and only 7 to mean nothing is listening, and the observation producer still
framed the axes as disagreeing about what a refused connection means. Leaving a
superseded summary above a corrected model is worse than leaving neither --
DESIGN's §4c says an annotation must not restate what the declaration says, and
one that MISstates it will be read as the authority by anyone who reads the
prose first.

They are rewritten to the current model rather than annotated as historical: no
connection failure establishes endpoint absence; endpoint absence requires a
completed host-local listener enumeration whose exact target population is
empty; a failed or unreached enumeration establishes nothing.

Beside them, the boundary this repair does NOT close, stated so it cannot be
overgraded. SparkServingProbeCapture carries a completion status and stdout and
nothing about provenance, so nothing in the type says an enumeration ran on the
right host, filtered the right port, ran this module's argv, or ran after the
retirement it is read against. The decoder's semantics are correct; its
producer provenance is unmodelled, and replacing the generic capture with a
typed executed receipt is a P0-WET obligation for all three axes.

Co-Authored-By: Claude Opus 5 (1M context) <noreply@anthropic.com>
Claude-Session: https://claude.ai/code/session_018dHefbXnLiW1XNs2vVxhwj

* A refusal renderer no consumer called: the lookup sum is the shared authority, not one rendered sentence

spark_cell_role_lookup_refusal had no caller. Both authorization consumers
match the lookup arms directly and author their own causes, which is right
rather than a miss: the remedies differ, so the sentences should -- an ambiguous
role cannot authorize training CAPACITY in one and cannot select a desired
POPULATION in the other, and forcing both through one rendered string would
trade consumer-specific context for a shared phrasing nobody needed.

What is shared is the SUM, and that is already the single authority. This is
the same inert-declaration class trimmed from curl, pgrep and ss earlier in
this stack -- the fourth instance, which is the signal that the check belongs in
review habit and not in my memory of the last one.

Co-Authored-By: Claude Opus 5 (1M context) <noreply@anthropic.com>
Claude-Session: https://claude.ai/code/session_018dHefbXnLiW1XNs2vVxhwj

* Three enumerations of one coproduct and a predicate with one caller: its own claim

Two review findings on the teardown scheduler, and one nit from the base PR.

spark_serving_plan_direction enumerated MemberAction three times -- once to
filter removals, once installs, once refusals -- and walked the plan three times
to do it. Three enumerations of one coproduct is three places to edit when a
variant lands, and only one of them fails to compile if the others are
forgotten: the single-authority problem at the smallest possible scale. An
action is now classified once by SparkServingActionClass in one exhaustive
match, the census is one fold, and every population is derived from its answer,
so the three counts cannot describe different traversals of one plan.

spark_host_offerable_for_training matched the lookup sum into true/false and had
exactly one caller: its own claim. I declined this finding earlier on the
grounds that P3's supply module would consume it -- which is the
authored-ahead-of-its-consumer shape trimmed from the curl, pgrep and ss rows
earlier in this same stack, and no better for being a function than a constant.
It is deleted. spark_training_cell_hosts is the canonical surface and already
excludes serving cells, unassigned hosts, and hosts the roster cannot decide;
the claim now asserts over that roster, which is what P3 will actually consume.
A claim whose whole subject is a function nothing else calls measures its own
fixture.

spark_serving_desired_hosts_from_role was a function whose body was
spark_serving_cell_hosts() -- hollow_alias from the ledger, a second name for
one concept. Inlined.

Co-Authored-By: Claude Opus 5 (1M context) <noreply@anthropic.com>
Claude-Session: https://claude.ai/code/session_018dHefbXnLiW1XNs2vVxhwj

* The capture certified that a read happened while the number came from the caller

Two review findings, both the same residual one field apart, and both the shape
this module has now produced four times: an invariant argued in prose and then
admitted through the parameter list.

SparkUnifiedPoolObservation took free_pool: ByteSize and copied the capture's
stdout into a prose producer string. So an answered probe carrying any text at
all certified whatever capacity the caller passed -- a readiness proof mintable
from an unverified number. The paragraph directly above the producer already
said "a quantity with no read behind it is a number somebody typed"; the comment
was right and the signature did not implement it.

The parameter is gone. The count is parsed from the capture's own stdout with
parse_int, read as /proc/meminfo MemAvailable in KIBIBYTES per proc(5), and
converted through a modeled Kibibyte -> ByteSize rather than a bare multiply.
Text that does not parse is a typed refusal, not a zero and not a default:
w_a_capture_whose_body_is_not_a_count_is_refused hands it
"MemAvailable:   97656250 kB" and requires a refusal, which the previous shape
could not author at all because the quantity never came from the text.

Beside it, the host field. It sat next to `after`, and the readiness gate
compared the two -- two independently authored answers to one identity,
reconciled by a check, where the retirement observation already names the host.
Both the field and the parameter are deleted and every downstream host derives
from after.host, so the carrier cannot name two hosts and there is nothing left
to compare. The mismatch arm and its claim are deleted rather than replaced by a
predicate: the state has no constructor. This fixes the DRY defect only --
nothing in a generic capture establishes where it ran, which stays the P0-WET
provenance obligation.

std.measure gains kibibyte/kibibyte_count/kibibyte_to_byte_size beside the
gibibyte trio it mirrors.

The witness note is rewritten: it still advertised "a pool measured before the
retirement" as an authorable red. Three states it listed lost their constructor
across these changes, so the note now names them and says where the refusals
they used to prove moved.

Also repaired: deleting spark_host_offerable_for_training left its replacement
annotation trailing at end-of-file with no declaration to attach to, which is
the parse error CI caught. Every changed .dag was scanned for the same shape;
it was the only one.

13/13 by execution against a declared denominator of 13.

Co-Authored-By: Claude Opus 5 (1M context) <noreply@anthropic.com>
Claude-Session: https://claude.ai/code/session_018dHefbXnLiW1XNs2vVxhwj

* The .dag gained three functions and its seed projection did not: regenerate std_measure.rs

Extending std/measure.dag with kibibyte/kibibyte_count/kibibyte_to_byte_size
obliges the stage0 Rust realization generated from it, and I pushed the
authority without the projection. CI caught it as
`regen FAIL generated surface drift: std_measure.rs`.

The bytes here are the generator's own candidate output, taken from
target/stage0-regen-candidate/src/std_measure.rs rather than hand-written to
match. Hand-authoring a projection the model generates is the manual-application
tell, and a hand-written approximation that happened to compile would have
drifted again on the next regeneration.

Verified before pushing this time, which is the actual repair: the build lane's
regen phase now reports first_generation_equal=true, phases_run=1 failed=0. The
claim runs I had been treating as sufficient cover the MODEL; they say nothing
about the seed projection or the parse gate, which is why four heads went out
with problems a pre-push lane run would have caught.

Co-Authored-By: Claude Opus 5 (1M context) <noreply@anthropic.com>
Claude-Session: https://claude.ai/code/session_018dHefbXnLiW1XNs2vVxhwj

---------

Co-authored-by: gunbc-ci-auto-heal <gunbc-ci-auto-heal@users.noreply.github.com>
Co-authored-by: Claude Opus 5 (1M context) <noreply@anthropic.com>
briansrls pushed a commit that referenced this pull request Aug 30, 2026
…tion denominator made physical (#9731)

* The observer could not see the host it existed to retire: the observation denominator made physical

srv6 ran a live, enabled serving unit and the converge plan minted zero typed
actions and called the fleet clean. Measured, not reasoned: fleet-converge run
33296910990 reported observed_hosts=1, and run 33297525985 wrote
spark_serving_typed_actions.wire at 0 bytes.

spark_serving_observe_ci_wet enumerated spark_serving_desired_hosts. That was
correct while desired named every Spark, and became wrong the moment #9679 gave
desired a role scope. Retirement is planned from members observed and NOT
desired, so scoping observation by desired makes the retirement population
unobservable by construction -- observed minus desired is empty for every host,
always. The module already refused an EMPTY observation list on the ground that
emptiness is ignorance rather than a clean fleet; the roster held {srv5}, which
is not empty, so nothing fired.

That is empty_observation_narrow in its non-empty form.

The obvious repair -- union the two role rosters -- is rejected here and the
rejection is why this is a module rather than a one-line edit. Both role
projections answer through SparkCellRoleUnique and deliberately drop
SparkCellRoleAmbiguous and SparkCellRoleAbsent, so their concatenation silently
omits a host whose role assignment is broken. An absent or contradictory ROLE
says nothing about whether a serving PROCESS runs on the metal, and a broken
assignment is precisely when stale serving state must stay visible. That repair
would have fixed today's srv6 and preserved the class one input away.

So the scope reads the procurement authority, which derives the enrolled Spark
identities from the router bindings and knows nothing about roles. Roles decide
what may be DONE to a host; they do not decide whether it is LOOKED AT.

Coverage becomes an identity join and SUBSUMES the cardinality guard rather than
standing beside it -- two authorities for one question. Observed keys must equal
expected keys, each exactly once; missing, foreign and duplicated are carried
separately because they are different defects with different causes.

Evidence: 8 claims under v2.test.*, which the required floor executes -- the
same conclusion #9711 reached when it moved a RED witness under the required
prefix rather than leaving an inert wall. They are pure over host-identity lists
and reach no fleet planner. Discriminating control run: reinjecting the defect
reds 5 of 8; the 3 that stay green are the pure coverage-classifier claims,
which consult no scope producer and correctly test a different subject.

Co-Authored-By: Claude Opus 5 (1M context) <noreply@anthropic.com>
Claude-Session: https://claude.ai/code/session_018dHefbXnLiW1XNs2vVxhwj

* The comment claimed a replacement the diff had not performed

Review 57545 (non-blocking): SparkObservationCoverage is defined and exercised
by claims but not consumed by serving_observe, which still refuses on the count
the join was written to subsume -- while the module comment described the
subsumption as done.

That is the overstatement this module exists to correct, one layer up: prose
asserting a migration that execution has not performed. The PR body named the
gap; the source did not, and the source is what survives.

The comment now states that the replacement has NOT happened, that the two
authorities genuinely coexist until the observer's report is built from the
join, that the narrowed-roster case is currently caught by the claims rather
than by the production path, and that the cutover is the remaining P0-C
obligation. No behaviour change.

Co-Authored-By: Claude Opus 5 (1M context) <noreply@anthropic.com>
Claude-Session: https://claude.ai/code/session_018dHefbXnLiW1XNs2vVxhwj

* Both observer entries answer 'did the probe answer', and the causal chain is unwelded

Two corrections from tracing the live dataflow rather than reasoning about it.

TERMINAL POLICY (a defect this branch introduced). Widening the roster to the
physical Spark population put a RETIRED cell inside it, and a retired cell is
structurally SparkServingAbsent. The workstation entry terminated on
spark_serving_observation_report_succeeded, under which absent counts as
failure, so the exactly-correct final state -- srv5 serving, srv6 retired --
would have exited non-zero. Fail-closed, but the wrong answer for an instrument
whose job is to describe what is there. Both entries now terminate on
spark_serving_report_observations_were_made.

The row that is no longer the terminal is retained and documented rather than
deleted: its only consumers are claims in test.claim.spark.*, which is outside
the required gate closure, so editing that module would classify its witnesses
as changed-and-declined and red the required floor. It is blocked from deletion
by the same enrollment gap that owns the 59 nonterminal identities, and that is
recorded at the function rather than hidden.

CAUSAL CHAIN. The module comment cited fleet-converge run 33297525985's 0-byte
spark_serving_typed_actions.wire as evidence for the observer defect. False in
the live dataflow: fleet_converge_plan_cli passes observed_spark: [] and
fleet_converge_plan_artifact sets spark_serving_typed_actions to
scheduled_rows_empty() unconditionally, so the live planner reads NO Spark
observer and would have emitted an empty wire whatever this module answered.
Independent defects, repaired separately. The comment now says so, including why
the chain was tempting.

Evidence for the terminal swap is authored (spark_serving_observe_witness_test
asserts was_made && !succeeded) and does not execute. Restating it under
v2.test.* was attempted and abandoned on measurement: reaching the predicates
pulls this module's 46-import closure and kills the claim module during resolve.

8 scope claims re-verified 8/8 on the refreshed head.

Co-Authored-By: Claude Opus 5 (1M context) <noreply@anthropic.com>
Claude-Session: https://claude.ai/code/session_018dHefbXnLiW1XNs2vVxhwj

---------

Co-authored-by: gunbc-ci-auto-heal <gunbc-ci-auto-heal@users.noreply.github.com>
Co-authored-by: Claude Opus 5 (1M context) <noreply@anthropic.com>
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

None yet

Projects

None yet

Development

Successfully merging this pull request may close these issues.

1 participant