Repository navigation
Derive group B's serving route from the fabric assignment, not the retired GLM canary - #11617
Conversation
Co-Authored-By: Claude Opus 5 (1M context) <noreply@anthropic.com>
…sitive arm authorable Group B's assigned arm now claims all four member hosts, so admit_unplaced_host -- the one seam the source build, the image build and the published-image probe reach a host through -- refuses srv9-srv12. Two witnesses had certified the opposite: srv10 admitted for a build, and srv10 admissible for the v4.1 image build. srv10 was running a rank of the native GLM arm the whole time, with no placement claim, so a build would have competed with a serving rank for the unit's unified memory. admit_unplaced_host_among takes the unplaced set as a supplied value; admit_unplaced_host is the unchanged production wrapper over it (spark_host_is_unplaced was already `any` over spark_unplaced_hosts). Every refusal arm still runs the real path; the admitting arm is kept as a positive control over a supplied set, because no production Spark unit is unplaced any more and a seam that refused everything would otherwise read as tested. authority_held_and_ruling_derived_causes_are_separable_by_a_fold: srv10 was the ruling-only specimen only because the arm serving there had no claim. It now carries one cause of each kind like srv9, and the fold still discriminates (a non-separating fold counts two of each). Consequence to note: with every Spark unit placed, the v4.1 build lane has no admissible host until one is freed or it is given a non-serving build host. Co-Authored-By: Claude Opus 5 (1M context) <noreply@anthropic.com>
… canary citations Addresses review 67909 (REQUEST_CHANGES), both findings verified against the code. 1. spark_build_host_reachability_wet gated on a fold starting at true over map(spark_unplaced_hosts, ...). This PR makes that population empty in production (every Spark unit is placed), so the root would have written a receipt with no probe lines and exited zero -- the widening its own annotation records removing. The verdict is now spark_reachability_verdict, three arms: NoAdmissibleHost refuses by name, AllAnswered admits, SomeUnanswered refuses with the body. The earlier diff deleted the only assertion that would have caught this; the_reachability_verdict_refuses_an_empty_ population_rather_than_passing pins all three arms against supplied readings. 2. gunbc.harness.harness_seat still said group B's tolerant ceiling derives from gunbc.spark.glm_canary_converge's arm, and cited serving_canary_route_ceiling, which this PR deletes. Both now name gunbc.serving.serving_enrollment's declared rows and ServingRouteTurnCeiling's CeilingFromRouteLaunch arm; a historical correction about the canary's bound is put in the past tense rather than left asserting current state. Co-Authored-By: Claude Opus 5 (1M context) <noreply@anthropic.com>
|
Both findings in review 67909 were correct, and both are fixed in c5d7779. 1. The empty population exited zero. This was a real regression that this PR introduced. 2. CI's floor will verify the changed witnesses on this head. I couldn't finish the local run because the session container kept restarting partway through. — sent from bright-eagle-728 |
…per host, a production backfill entry with receipts, and a nonempty retained population Against the ninth side-chat hold and reviews 68027/68062: - Liveness: a host-effect claim fences until released; the stated term marks it overdue, never free. A second claim on a held host is refused at the fold, at the claim and in the standings (OccupiedByHostEffect). Hosts of no claimed group claim on their own host-effect/<host> partition with the same events and fold. - Backfill: fabric_event_log_event_refs_backfill_wet on the placed host over every refs/fabric/* partition through the envelope decoder, with a receipt ref per partition; the reader's refusal consults the receipt. - Population: an empty retained host list refuses at decode, at the transition and in the fold. - Merged origin/main (#11617's admit_unplaced_host_among: admit_unplaced_host_in is now that over a standings population); the stale spark_pair_apply_plan symbol reference is repointed (review 68062). Co-Authored-By: Claude Opus 5 (1M context) <noreply@anthropic.com>
What broke
A real dispatch on production (2026-09-18,
shell-typed-invocation) spawned a worker that exited withno serving offer was admissible. For group B, the harness probed192.168.1.236:30001, where nothing listens. That's the port of the retired exl3 GLM canary. The arm that actually serves group B answers on192.168.1.236:30000asglm-5.3-flash(checked live withGET /v1/models, snapshoteb9eb208).The cause: group B's facts named an arm, not the fabric
Group A's subject is
PairServingUnitOn { group }, which is anchored on the fabric. Group B's wasPlacedServingArm { arm: glm_canary_arm }, anchored on one specific engine's declaration. Everything built on it inherited that snapshot, and none of it moved when the arm changed:gunbc.serving.serving_enrollmentgunbc.serving.admitted_modelglm-5.3-flash-exl3-canarygunbc.spark.serving_arm_placementgunbc.spark.serving_subjectreservation…_group_b_is_glmreserved group B for the canary's subjectgunbc.spark.serving_deployment_selectiongunbc.spark.host_commitmentadmit_unplaced_hostThe change
gunbc.spark.fabric_serving_assignment: one row per group (glm_group_b_serving_assignment) holding the planner's inputs minus launch intent.fabric_serving_headtakes the rank-zero node from the samearm_node_rosterthe launch is planned from. Ranks other than zero run--headless, so rank zero is the only front door. A roster refusal becomes a typed refusal, never a guessed head. The head is not the plan'smaster, which is the RoCE rail address.serving_arm_placementglm_group_b_serving_armpoints atglm_group_b_serving_assignment, and the claims, subject, reservation, admissibility and selection snapshot all key on it. Changing which engine serves group B is now one edit inside the assignment row, and nothing else moves.serve_port, and its ceiling is an attributed allocation of 4. It is deliberately not the engine's--max-num-seqs 16: the engine's scheduler limit and how many turns this repository offers are separate questions.glm-5.3-flash.fabric_group_hosts, which is total). They aren't taken from the launch roster, because a roster refusal would yield no claims and let occupied hosts read as free.admit_unplaced_host_amongtakes the unplaced set as a supplied value, andadmit_unplaced_hostis its unchanged production wrapper. Refusal arms run the real path. The admitting arm stays authorable as a positive control even though no production unit is unplaced any more.Operational consequence to know about
With group B's arm claiming its hosts, every Spark unit is now placed: group A serves DeepSeek and group B serves GLM. So the v4.1 image-build lane has no admissible host until one is freed or a non-serving build host is provided. Before this change it would have silently built onto a host serving GLM.
Witnesses
Several witnesses were certifying the stale state: the
:30001endpoint, a ceiling of 1, the canary alias, "srv10–12 unplaced / reserved but idle", "srv10 admitted for a build", and srv10 as a ruling-only specimen. Each now asserts the true state and keeps its discriminating force. The annotations say what each used to assert and why that was wrong. The rung-drop population entry forharness_seat_ceiling_by_operator_policynames the new group-B row, anddocs/design-rung-drops.mdis regenerated.Not in this PR
cat-file 09ed6c96… refused). The fabric event log links events through a JSONparentfield that git never fetches. That's P0-B and gets its own PR.serving_arm_placement's own annotation names that trigger. This PR removes the stale snapshot but doesn't build the liveness check.glm_canary_arm,glm_canary_host_claim) stay because its evidence rows still cite them. Nothing live reads them.Verification
CI green at
aa21ca2: witnesses, build, heal and floor all SUCCESS. Locally: the routing, admission, host-commitment, v4.1 build, selection and chat-shape witnesses returnedtruebehind a control that must fail.🤖 Generated with Claude Code