Repository navigation
CONVERGE-0 acceptance: a merged main revision becomes observed fleet state, including offline catch-up — open until the receipt set exists - #8083
Conversation
…onverge circuit First subject of the fleet-convergence vertical: the desired per-slot TasksMax=512 (gunbc.runner_slot_allocation, operator ruling 2026-08-05) becomes an appliable, readback-proven converge knob instead of a row the fleet cannot act on. Live pre-state on srv1/srv2 is the 2026-06-21 tourniquet TasksMax=16384 (compute_envelope_model), which now reads Drifted, never Converged. - extdeps.systemd: TasksMax/TasksCurrent properties + SystemdCgroupTaskLimit parse (count-or-infinity), mirroring the memory limit parse with a declared consolidation dissolve-on. - gunbc.host_converge: per_slot_tasks_max knob on the property-generic PerSlotMemoryCap target (over-narrow name noted with dissolve-on to the P3 axis framework); desired threaded from runner_slot_allocation through RunnerHostDeployment. - gunbc.host_axis_caps: 50-fleet-tasks.conf joins the managed drop-in set (Owned, not foreign). - gunbc.host_converge_slice1: full tasks circuit — live TasksMax read -> verdict -> pre-apply TasksCurrent guard (starving clone() with EAGAIN is the guarded harm) -> set-property -> independent read-back -> reapply-noop; per-unit outcomes beside the memory knob on srv1/srv3; refusals typed and located throughout. - fleet_converge_cli: srv1 slice1 route handles the tasks arm; whole-loop refuses if the tasks receipt fails its validity gate. - Witnesses: knob shape on srv1+srv3, tourniquet-16384/infinity RED (Drifted), 512 converges + off-by-one drifts, empty/junk wire is Absent (fail-closed), legacy-arm routing, and a validity-gate RED where flags claim success but the read still shows 16384. Co-Authored-By: Claude Fable 5 <noreply@anthropic.com> Claude-Session: https://claude.ai/code/session_0168R31q15ZsPJyik18Roz3y
…conflict) Co-Authored-By: Claude Fable 5 <noreply@anthropic.com> Claude-Session: https://claude.ai/code/session_0168R31q15ZsPJyik18Roz3y
…84; 512 is the RED Operator ruling 2026-08-08: TasksMax=512 was the diagnosed EAGAIN outage value, not the repair — the earlier revision inverted the incident diagnosis. Corrections: - runner_slot_allocation: tasks_max 512 -> 16384 at the single authority, note updated with the ruling; any lower derived envelope value must be separately accepted and never below live task demand. - Witnesses flipped: 512 (and infinity, and off-by-one 16383) read Drifted, never Converged; the validity-gate RED reads 512. - Guard boundary tightened to >= (equality is zero headroom) via an extracted pure predicate, with an equality-boundary witness (manager review finding 2). - Reapply-noop guard bypass justified in-code by physics (finding 1). - No live apply executed at any point. Co-Authored-By: Claude Fable 5 <noreply@anthropic.com> Claude-Session: https://claude.ai/code/session_0168R31q15ZsPJyik18Roz3y
…able TasksConverged CLI arm Manager review on #8045: the ConvergeTarget comment described the tasks knob as TasksMax=512 — this PR's own prose citing the outage value as the knob's value; the desired is owned by gunbc.runner_slot_allocation and the literal is dropped. The fleet_converge_cli TasksConverged arm is annotated unreachable-by-construction so no new caller wires through it. Co-Authored-By: Claude Fable 5 <noreply@anthropic.com> Claude-Session: https://claude.ai/code/session_0168R31q15ZsPJyik18Roz3y
…o real producers, and the fleet-atomic-convergence fold Item 1 (B): host_phase_observation's hard-coded single-cell if-chain generalizes into ONE canonical observe_host_phase(subject, host, phase_label) seam — never a per-program observer beside it. Two real producers land as pure deductions over committed probe receipts (gunbc.host_reach_identity_probe), the same shape as the srv3 lease-table producer, so the matrix stays hermetic by construction: - ONE wet probe (authenticated short-hostname read over the management transport) feeds BOTH P0 cells: reach-secrets-network and host-identity-converge. - Four distinct typed outcomes by operator constraint: Reachable / TransportUnavailable / AuthenticationRefused / HostKeyMismatch — auth-refused and transport-unavailable stay distinct arms (different remedies) even where both project Inaccessible; host-key mismatch is a live class (srv5/srv6 shipped identical factory host keys) and maps to Conflict so a wrong machine's hostname never launders into an identity observation. - Wire classification is fail-closed: unclassifiable failure is TransportUnavailable with cause preserved, never fabricated Reachable. - Receipts registry is honestly EMPTY: every host reads UnknownRefused on both cells until a wet probe executes; coverage grows only by executed observation, never by wiring. Password auth on srv5/srv6 stays enabled until a Reachable receipt exists (ordering law recorded at the outcome type). Item 2: gunbc.fleet_convergence_verdict — the fleet-atomic fold joining knob-grain ConvergeCliReceipts with the lifecycle cells. Laws, each with an executing witness: a required unobserved cell PREVENTS fleet convergence (never defaults green); knob-converged receipts alone cannot converge the fleet (RED); an empty enrollment refuses; unobserved / unreachable / drifted / conflict stay distinct standings with distinct remedies; the not-converged arm carries full standings so the verdict IS the work list. The named current defect is now executable: the live fleet reads NOT converged with a positive unobserved-cell count (one real producer cell fleet-wide). Witnesses: 7 probe + 5 fold, green by execution; host_phase_status's 22 witnesses green under the new seam signature. Co-Authored-By: Claude Fable 5 <noreply@anthropic.com> Claude-Session: https://claude.ai/code/session_0168R31q15ZsPJyik18Roz3y
…srv1-at-512 control notes Item 3 (gunbc.fleet_main_revision): refs/fleet/desired as the single desired-state ref; FleetDesiredRevision (Admitted|Unobserved), admit_fleet_desired_advance is a CAS carrying the expected prior and REFUSES without floor-green; an unobserved ref yields NO converge target, never the invoking tree. 4 witnesses green incl. the floor-not-green refusal RED. Item 4 slice 1 (gunbc.fleet_converge_timer): single authority for the per-host ctrl-fleet-converge.service/.timer names, cadence (*:0/15, RandomizedDelaySec=90), and unit CONTENT — Option A (OnCalendar + Persistent=true, catch-up for offline hosts); the service converges SELF only via the pinned binary. 4 witnesses green. Stale-note recut (review finding, flagged twice): runner_lifecycle runner_lifecycle_memory_cap_apply_note and runner_capacity_operator_access srv1_already_converged_caps_control_note recorded srv1's hand-converged TasksMax=512 as part of the acceptance-row-6 no-op control; per the 2026-08-08 ruling 512 is the diagnosed outage value (desired 16384), so both notes now scope the zero-effects premise to the MEMORY axes and record srv1's tasks axis as live drift the per_slot_tasks_max knob corrects. Co-Authored-By: Claude Fable 5 <noreply@anthropic.com> Claude-Session: https://claude.ai/code/session_0168R31q15ZsPJyik18Roz3y
…ing-built unit INI scaffold) per_runner_tasks_max gains the 🟡 marker naming the std.measure TaskCount dissolution (bare-Int pids count beside ByteSize siblings, retiring the to_string re-serialization); fleet_converge_timer's concat-built unit text is marked as a tests-only scaffold dissolving on systemd-unit grammar-rows emit. Co-Authored-By: Claude Fable 5 <noreply@anthropic.com> Claude-Session: https://claude.ai/code/session_0168R31q15ZsPJyik18Roz3y
… row for string-built unit text fleet_converge_timer_randomized_delay rides std.measure Second (rendered via second_count at the INI seam); the prose-only string-built-medium marker becomes a Disposition = Scaffold row binding the live_deploy emit-shell dissolution trigger, matching the fleet_show_effective_read peer. Co-Authored-By: Claude Fable 5 <noreply@anthropic.com> Claude-Session: https://claude.ai/code/session_0168R31q15ZsPJyik18Roz3y
…-item grain The 4c annotation channel admits only standalone leading // blocks attached to module-scope declarations; the marker sat inside the RunnerHostDeployment body and refused. Moved above the type declaration; host_converge_delta witnesses green by execution. Co-Authored-By: Claude Fable 5 <noreply@anthropic.com> Claude-Session: https://claude.ai/code/session_0168R31q15ZsPJyik18Roz3y
…r ruling 2026-08-09) 16384 is annotated as an emergency parity rollback only — not a derived value, not the accepted model home. The named follow-up model distinguishes runner-slot task budget, shared compile-pool task budget (sccache children live in the server's cgroup), and host-parent reserve, each derived from one measured envelope with sum(slots)+pool+reserve <= parent; dissolve-on that model landing. Not built in this PR per the ruling. Co-Authored-By: Claude Fable 5 <noreply@anthropic.com> Claude-Session: https://claude.ai/code/session_0168R31q15ZsPJyik18Roz3y
…y, srv1-pinned) Approved by manager (msg_03ec1f78) to produce the operator-ruled blocking pre-apply receipt without consuming the supervised window. New module gunbc.srv1_tasks_preapply_observation: strictly read-only wet entry returning the typed Srv1TasksPreApplyReceipt — per slot unit: effective TasksMax + TasksCurrent (typed systemctl show seam), cgroupfs pids.max/pids.current at the ControlGroup path SYSTEMD reports (never a hand-escaped slice path — the decoy class), the ancestor slice pids.max, plus the observer's own /proc/self/cgroup membership so the receipt shows the read ran inside the observed universe. Refusal arms are typed per unit; empty enumeration refuses rather than reporting a vacuous receipt. extdeps.systemd gains the ControlGroup property arm. The job is authored in the ci_workflow typed builders, gated `github.event_name == 'workflow_dispatch'` at the job (never rides PR/push), pinned to the srv1 runner spec, needs only the release-bins build; ci.yml regenerated via the artifact gate. Hermetic witnesses green for the pure ancestor-path logic (parent-of-reported-ControlGroup + degenerate boundary); ci_workflow witness green. Co-Authored-By: Claude Fable 5 <noreply@anthropic.com> Claude-Session: https://claude.ai/code/session_0168R31q15ZsPJyik18Roz3y
First dispatch (run 31288091024) captured the full receipt but exited 2 through the NotProcessExit refusal arm. The wet entry now writes the typed receipt to target/srv1-tasks-preapply-receipt.txt (Filesystem.Write, workspace-only) and derives its exit from the receipt's own arm — Observed -> ExitSuccess, Refused -> exit_failure(cause) — never from transport exit codes; the job step cats the file so the receipt also lands in the log. Co-Authored-By: Claude Fable 5 <noreply@anthropic.com> Claude-Session: https://claude.ai/code/session_0168R31q15ZsPJyik18Roz3y
…ll; feature-tag the two task-count unit gates gunbc_ci_srv1_tasks_observation_invoke now carries the sibling pattern exactly (Scaffold disposition row binding the invoke decl + dissolution-trigger string emitted as the script's # 🟡 header); the per_runner_tasks_max marker and the SystemdCgroupTaskLimit consolidation note both gain the feature:task-count-measure-carrier tag naming the std.measure TaskCount dissolution. ci.yml regenerated; ci_workflow witness green. Co-Authored-By: Claude Fable 5 <noreply@anthropic.com> Claude-Session: https://claude.ai/code/session_0168R31q15ZsPJyik18Roz3y
…cepted 16384) The two acceptance-row-6 notes still framed srv1's tasks axis as live drift at 512 pending correction; the executed, operator-accepted pre-apply receipt (run 31288091024) reads all five slots at effective 16384 with matching cgroup pids.max, so both notes now record the observation-decided standing — the no-op control is exact on every converged axis, and no state-changing tasks apply is pending. Co-Authored-By: Claude Fable 5 <noreply@anthropic.com> Claude-Session: https://claude.ai/code/session_0168R31q15ZsPJyik18Roz3y
…in→desired-ref advancement CI red root cause: cell_standing_is_converged and host_knob_standing_is_converged matched closed coproducts with top-level wildcards — the exact class the non_fold_residue receipt refuses; both now enumerate their arms (construction over wildcard) and the receipt is green by execution. Loop leg: --host self — converge_cli_resolve_self resolves the literal "self" to the kernel-reported hostname at the cited procfs authority (ProcSysKernelHostname arm added), any other value routes unchanged, and an unreadable hostname falls through to the UnknownHost typed refusal, never a fabricated identity. Loop leg: main→desired-ref advancement — advance_fleet_desired job (push-to-main only, needs [ci] as the floor-green admission, contents:write) performs the CAS advance of refs/fleet/desired via --force-with-lease naming the exact observed prior (fleet_main_revision's from side); creation is the from=none arm. Script carries the scaffold row + emitted 🟡 header; ci.yml regenerated. Witnesses green: fleet_converge_cli routing, fleet_convergence_verdict (5), non_fold_residue receipt, ci_workflow. Co-Authored-By: Claude Fable 5 <noreply@anthropic.com> Claude-Session: https://claude.ai/code/session_0168R31q15ZsPJyik18Roz3y
…fixture correction)
…m a Class B refusal); flat_map for the per-unit outcome accumulator host_reach_identity_probe and fleet_converge_cli now import std.algebra trim explicitly. Review 50742: the slice1 per-unit outcome fold was a copied accumulator (concat(acc, [..]) per unit, O(n^2)); §6 bare-minimum-cost fixes a proven cost shape regardless of n — now units.flat_map. Slice1 witnesses green; generated-artifact regen green. Co-Authored-By: Claude Fable 5 <noreply@anthropic.com> Claude-Session: https://claude.ai/code/session_0168R31q15ZsPJyik18Roz3y
…ication on the self route Leg (c): FleetConvergeTimerUnit joins live_deploy membership on the belt-timer precedent exactly — per-instance names derived from the deployment stem (unconditional membership, twin-safe by path construction), one member owning .timer + .service, apply writes both units (service: oneshot root ExecStart '<deployed binary> converge --host self'; timer: OnCalendar/Persistent=true/ jitter derived from the gunbc.fleet_converge_timer authority rows) with daemon-reload/enable/restart, teardown removes both files, retirement stops the runtime. Persistent=true on the calendar timer is the offline-host catch-up property. Leg (d): on the SELF route the converge CLI publishes its receipt line to /opt/gunbc/receipts/<host>-converge-receipt.txt before printing; publication failure fails closed (converged=false, typed reason) because the loop's product is the published receipt. Non-self supervised routes unchanged. The deploy arm installs the receipts dir from the same authority row. Witnesses green by execution: the two new member witnesses (install content incl. catch-up + receipts dir; retract removes both files), the twin-safety controls, and the full 45-witness live_deploy emit corpus. Co-Authored-By: Claude Fable 5 <noreply@anthropic.com> Claude-Session: https://claude.ai/code/session_0168R31q15ZsPJyik18Roz3y
…and threads without fabrication Review 50695: the wire classifier stubbed expected_fingerprint as "" — conflating "enrolled expectation contradicted" (wrong-machine vs rotated-key, comparable) with "transport flagged verification, nothing enrolled to compare". The field is now String? (Absent = no enrollment), the wire classifier alone produces only Absent, and classify_reach_probe_wire_with_expectation carries an enrolled fingerprint into the same arm without re-parsing the wire. New discriminating witness; all 8 probe witnesses green. Co-Authored-By: Claude Fable 5 <noreply@anthropic.com> Claude-Session: https://claude.ai/code/session_0168R31q15ZsPJyik18Roz3y
… fleet reach probe job (loop leg e) CI red on 1178e7d was the mismatch-arm annotation sitting inside the coproduct body — hoisted above the type declaration (same 4c class as the earlier ci_runner_placement red). Leg (e): probe_fleet_reach_wet — ONE wet pass over srv1-srv6 through the typed SSH PROBE seam (typed_argv_probe_over_ssh, expect: OutcomeIsData), so a failed leg is DATA classified into its typed arm (Reachable / TransportUnavailable / AuthenticationRefused / HostKeyMismatch with Absent expectation — no enrollment registry exists yet), never a step failure and never an ExpectSuccess annotation standing where a check was owed. Receipt written as a typed value + step cat; only a receipt-write failure fails the run. Exposed as workflow_dispatch-only job probe_fleet_reach on the srv1 runner (the probe runs FROM inside the fleet), scaffold row + emitted 🟡 header per the transport discipline; ci.yml regenerated. Also: reapply-noop comment wording per review 50754 (non-atomic read/write stated; the class cannot be CAUSED by re-applying the effective value, regardless of concurrent spawns). Witnesses green: 8/8 probe (incl. expectation threading), ci_workflow golden. Co-Authored-By: Claude Fable 5 <noreply@anthropic.com> Claude-Session: https://claude.ai/code/session_0168R31q15ZsPJyik18Roz3y
…e sites host_reach_identity_probe's Filesystem.Write failed to resolve on the dispatched runner (undefined variable), while the srv1 observation module's identical call had resolved by pool-membership coincidence — the Class B accidental-coverage class exactly. All three modules using Filesystem.Write now import extdeps.filesystem.filesystem_io explicitly. Co-Authored-By: Claude Fable 5 <noreply@anthropic.com> Claude-Session: https://claude.ai/code/session_0168R31q15ZsPJyik18Roz3y
…le, never Conflict, never satisfied Operator emphasis: rightness over green. A ProbeHostKeyMismatch with Absent expectation cannot honestly claim a contradiction — the wire's 'verification failed' does not distinguish first contact from a changed key when nothing is enrolled — so both verdict projections now split on expectation presence: Present -> Conflict (a real contradiction to investigate), Absent -> Inaccessible (transport never completed; enroll the fingerprint registry). Both arms leave the cell NOT converged in the fleet fold; only the remedy differs. Witnessed in both directions on both projections; all 8 probe witnesses green. Co-Authored-By: Claude Fable 5 <noreply@anthropic.com> Claude-Session: https://claude.ai/code/session_0168R31q15ZsPJyik18Roz3y
…ate-receipt ambiguity wall; machine-id hazard checked Endpoint: new gunbc.fleet_reach_endpoint (downstream of procurement AND the probe, per bold-eagle's draft) projects SlotAssigned -> the cited CR1000A reservation address via render_ipv4_address and SlotUnassigned -> no endpoint, never a placeholder; the reach probe now transports to the resolved endpoint (Sparks probe at 192.168.1.222/223; name-resolvable hosts pass through) while receipts stay keyed by fleet identity — addressing is transport, not identity. Machine-id hazard: --host self derives from kernel hostname, never machine-id (note added stating both Sparks share one machine-id AND factory host keys, so hostname is sound only after host-identity-converge); and the fold's host_knob_standing no longer picks first-wins on duplicate receipts — 0 -> Unobserved, 1 -> judged, >1 -> NotConverged naming the ambiguity, so a two-machines-one-identity collision surfaces instead of one machine's receipt satisfying the other. New witnesses: duplicate-converged-receipts read ambiguous (RED-shaped) and Spark probe targets are modeled addresses. 9/9 probe + 6/6 fold witnesses green. No is-system-running usage anywhere in the loop's health reads. Co-Authored-By: Claude Fable 5 <noreply@anthropic.com> Claude-Session: https://claude.ai/code/session_0168R31q15ZsPJyik18Roz3y
…ismatch constructor split; receipt-dir re-homed to host_layout breaking the apply->emit->cli cycle Co-Authored-By: Claude Fable 5 <noreply@anthropic.com> Claude-Session: https://claude.ai/code/session_0168R31q15ZsPJyik18Roz3y
…remote/rev-parse ops, fleet_desired_observe standing, self-route revision gate + published revision cell Co-Authored-By: Claude Fable 5 <noreply@anthropic.com> Claude-Session: https://claude.ai/code/session_0168R31q15ZsPJyik18Roz3y
… srv2-4 via typed self-install plan (argv derived from the one timer authority, is-enabled readback), Sparks typed pre-enrollment Co-Authored-By: Claude Fable 5 <noreply@anthropic.com> Claude-Session: https://claude.ai/code/session_0168R31q15ZsPJyik18Roz3y
…er the typed argv seam, parsed by the backward read of the byte-locked grammar, and fed to the atomic fold; Absent/Unparseable stay typed evidence Co-Authored-By: Claude Fable 5 <noreply@anthropic.com> Claude-Session: https://claude.ai/code/session_0168R31q15ZsPJyik18Roz3y
…r — Spark host keys are DISTINCT; machine-id is the shared non-anchor Co-Authored-By: Claude Fable 5 <noreply@anthropic.com> Claude-Session: https://claude.ai/code/session_0168R31q15ZsPJyik18Roz3y
…is not TimerViaSelfInstall (srv1 landing would fork live_deploy ownership) Co-Authored-By: Claude Fable 5 <noreply@anthropic.com> Claude-Session: https://claude.ai/code/session_0168R31q15ZsPJyik18Roz3y
… main push (implicit success() cascade) — status guard + RED control (#8085) * Repair: push-to-main jobs silently skipped on every main push — condition now carries its own status guard, with a RED control Measured latent (>=30 consecutive main pushes): heal_generated_artifacts skips on ordinary pushes; ci survives via its own !cancelled(); but a needs:[ci] job whose if carries NO status function gets GitHub's implicit success() ANDed in, and success() is false downstream of any transitively skipped job — so deploy_dashboard_srv1, advance_fleet_desired, and collect_fleet_convergence skipped while every clause in their condition was true (an empty-observation narrow: 'dependency skipped' rendered as 'condition false'). Receipt: run 31320306318 (ci=success, all three skipped); no deploy ran in the last 30 main pushes. Fix at the one generating authority (ci_deploy_push_to_main_if): '!cancelled() && needs.ci.result == 'success' && <event clauses>'. ci.yml regenerated through the artifact gate. RED control push_to_main_family_carries_explicit_status_guard: reds if the guard leaves the authority row, if any push-to-main job's condition drops it, or if the family goes vacuous. Verified discriminating by execution (reverting the row -> FAIL). This is the focused repair arm under the CONVERGE-0 acceptance outcome (#8083 remains the acceptance carrier); items 1/2 of the receipt set become obtainable once this is on main. Co-Authored-By: Claude Fable 5 <noreply@anthropic.com> Claude-Session: https://claude.ai/code/session_0168R31q15ZsPJyik18Roz3y * Restore the authority row (RED-check revert had clobbered it before the first commit) Co-Authored-By: Claude Fable 5 <noreply@anthropic.com> Claude-Session: https://claude.ai/code/session_0168R31q15ZsPJyik18Roz3y --------- Co-authored-by: Brian Searls <briansrls@gmail.com> Co-authored-by: Claude Fable 5 <noreply@anthropic.com>
…e tests blanket sudo; the grant set is exact-argv — probe with sudo -n -l (run 31334610318 srv2 receipt) Co-Authored-By: Claude Fable 5 <noreply@anthropic.com> Claude-Session: https://claude.ai/code/session_0168R31q15ZsPJyik18Roz3y
…erged model Co-Authored-By: Claude Fable 5 <noreply@anthropic.com> Claude-Session: https://claude.ai/code/session_0168R31q15ZsPJyik18Roz3y
…o-converge; read-only multi-principal probe for srv3/srv4 foothold measurement - fleet_multi_principal_probe: read-only measurement (parent msg_e767a783) of whether the in-run fleet key authenticates as any pre-existing account (ubuntu/briansrls/ghrunner/root) on srv3/srv4 — hostname + sudo -n -l, no writes. Rides the existing WIF-authenticated probe job (one credential materialization). - fleet_sudoers_bootstrap_scaffold: the parent's hand-applied srv2 dropin + srv1/srv2 gunbc-fleet accounts reclassified from 'legacy bootstrap' to DRIFT-TO-CONVERGE; dissolution = the membership spine reproduces srv1/srv2 standing Unchanged (a hunk would be the receipt that the hand-work was drift in substance). No further hand-applied host state; the real defect is live_deploy targeting srv1 only. Co-Authored-By: Claude Fable 5 <noreply@anthropic.com> Claude-Session: https://claude.ai/code/session_0168R31q15ZsPJyik18Roz3y
…ncipal verification, failure-safe credential trap, key-identity check; + srv3/srv4 privileged foothold MEASURED #1 UNOBSERVED PRINCIPAL NEVER WIDENS TO AMBIENT SSH: FleetProbeTarget = ProbeTargetReady{host,endpoint,remote_principal} | ProbeTargetRefused{cause: PrincipalUnobserved|EndpointUnobserved}. ProbeUserPending (Sparks) now REFUSES — no bare endpoint, no ssh attempt. srv5/srv6 rows are ProbeTargetUnobserved. #2 REACH DOES NOT VERIFY PRINCIPAL: ProbeReachable carries observed_principal (second typed op 'id -un'); receipt carries requested_principal; identity Converged requires BOTH hostname AND principal to match (hostname-match-under-wrong-account is Drifted). #3 FAILURE-SAFE CLEANUP: trap cleanup EXIT armed before the key touches disk (AGENT_STARTED flag), so any set -e abort wipes the key file and kills the agent; outer always() kept as defense in depth. #4 KEY-IDENTITY VERIFICATION: before ssh-add, ssh-keygen -y | ssh-keygen -lf - compared to the modeled fingerprint authority; mismatch = ProbeCredentialIdentityMismatch, CONTACTS NO HOST. MEASURED (run 31336668532, read-only multi-principal probe): the fleet key ALREADY authenticates as a PRIVILEGED account on both srv3 (ubuntu, sudo -n -l LISTED) and srv4 (briansrls, sudo -n -l LISTED) — so srv3/srv4 bootstrap is dispatchable through CI with no operator grant. ghrunner is nologin on both; root refused on both. Collector consults the same target (refuses on unobserved principal). All probe/collector/host_phase/cli witnesses green. Co-Authored-By: Claude Fable 5 <noreply@anthropic.com> Claude-Session: https://claude.ai/code/session_0168R31q15ZsPJyik18Roz3y
…-job effects The ladder correctly caught that probe_fleet_reach and collect_fleet_convergence carry the identical key-agent + cleanup scripts. That is not cacheable cross-job duplication — an ssh-agent lives in one job's process env and cannot be memoized across isolated jobs (same category as per-job deploy stages). Declared both as fresh effects (ci_fresh_effect_scripts); cleanup extracted to gunbc_ci_fleet_key_agent_cleanup_script (one authority, referenced by workflow step + gate). Both ladder witnesses green. Co-Authored-By: Claude Fable 5 <noreply@anthropic.com> Claude-Session: https://claude.ai/code/session_0168R31q15ZsPJyik18Roz3y
# Conflicts: # dag/gunbc/fleet_converge_cli.dag
… CI URL + requirement row derive from it The secret pin lived in three places — SecretRef.version='latest', a hardcoded versions/1 in the CI curl URL, and pinned_version='versions/1' on the requirement row (§3 violation; and 'latest' contradicted the SECRET-VERSION-LIFECYCLE name-a-version rule). Now: - fleet_automation_ssh_privkey_secret.version = '1' (the blessed version; single authority). - fleet_automation_privkey_version_segment() and fleet_automation_privkey_sm_access_url() derive the path segment and the full SM access URL (project + secret + version) from that one SecretRef. - ci_spec's key-agent curl + mismatch echo consume the URL/segment fns; fleet_host_key_enrollment's pinned_version consumes the segment fn. Emitted ci.yml URL is byte-identical (proof the derivation reproduces the intended path). Rotating to v2 now edits one field. Co-Authored-By: Claude Fable 5 <noreply@anthropic.com> Claude-Session: https://claude.ai/code/session_0168R31q15ZsPJyik18Roz3y
|
Fixed in the latest commit (review 50872, RC — valid §3 finding). The secret version pin had three authorities; it now has one. |
…rator finding) EVIDENCE INFLATION (operator scan): srv1/srv2 observed_principal='gunbc-fleet' was INFERRED from the ssh target on a run (31333011288) that predated the id-un op — a value naming an observation nobody made, inverting repair #2. Corrected by EXECUTION: re-ran the probe (run 31346260819, probed_at 2026-08-10T01:11:35Z) which runs id-un; observed_principal is now a real remote-process readback (=gunbc-fleet on both), so identity Converged is genuinely two-dimensional. Provenance + comment updated; static transcription now reflects a run that actually observed the principal (condition 4 -> receipt; condition 5 advanced). §3 IMPORT FORKS (review 50880): - ci_fleet_wif_auth_step: workload_identity_provider + service_account now derive from probe_identity_secret_access_requirement (was hardcoded literals beside the row it names as authority); rotation is one edit site. - fleet_receipt_collector: fleet_reach_probe_targets imported from its definer gunbc.fleet_reach_endpoint (was the host_reach_identity_probe re-export — Class B). - fleet_multi_principal_probe: clock_now_probed_at imported from gunbc.clock_read definer (was the host_reach_identity_probe re-export). Emitted ci.yml WIF block byte-identical. All probe witnesses green. Co-Authored-By: Claude Fable 5 <noreply@anthropic.com> Claude-Session: https://claude.ai/code/session_0168R31q15ZsPJyik18Roz3y
|
Addressed the operator's evidence-inflation finding and review 50880's three §3 forks at HEAD c97d65f: Evidence inflation (the load-bearing one): srv1/srv2 review 50880 §3 forks:
All probe witnesses green; PR back out of draft. Conditions 1–5 of the operator's seven now met (1 body-actuals, 2 desired-ref CAS at 0f9bf74, 3 collector-compiles via #8090, 4 id-un executed, 5 transcription reflects a real id-un run); 6 (fold to one verdict) and 7 (offline-return control) remain as the acceptance set completes. — sent from wise-moth-430 |
Operator correction taken: stop modeling, get to a live canary. This is the first executable step and it settles the question no model can -- can the fleet controller authenticate to srv5, and as WHICH principal. Runs on a self-hosted fleet runner because the Sparks have no Actions runner, so nothing bootstraps them from inside; authenticated remote application from a LAN controller is the only mechanism. Uses the ONE permission measured as held (versions.access on fleet-automation-ssh-key, run 31336704811). READ-ONLY by construction: every command observes. Nothing is installed, no authorized_keys is written, BatchMode=yes cannot prompt so a missing credential fails instead of falling back to an interactive path, and the key is shredded in an always() step. Token goes to a 0600 curl config, never argv. The receipt states what a failure does NOT establish: it cannot distinguish missing authorized_keys from absent account from sshd policy, and those have different remedies. #8083's 'no verified managed principal' is a claim about our records; this observes the host. Co-Authored-By: Claude Opus 5 (1M context) <noreply@anthropic.com>
…witness imports Two findings on head c97d65f, both DESIGN-grounded and confirmed real: (1) §3 fork — probe_identity_secret_access_requirement re-minted secret_project/secret_name as literals beside the SecretRef authority in fleet_ssh_access. Both now derive from fleet_automation_ssh_privkey_secret (.project/.secret as String — the same casts fleet_automation_privkey_sm_access_url already proves), so one place carries the secret identity. Rotating edits one row. (2) Class B — fleet_host_key_enrollment_witness_test bound its cross-module symbols by pool-membership coincidence (no import lines). Added the explicit import block from the definers: LiveTreeDisposition/SubstrateInputsOnly from v2.std.live_tree, HostIdentity from product.placement_supply, the enrollment symbols from gunbc.fleet_host_key_enrollment, and the probe classifier + host-key arms from gunbc.host_reach_identity_probe — matching the sibling host_reach_identity_probe_witness_test import discipline. Neither change touches emitted ci.yml. Co-Authored-By: Claude Opus 4.8 <noreply@anthropic.com> Claude-Session: https://claude.ai/code/session_0168R31q15ZsPJyik18Roz3y
|
Bugbot is not enabled for your account, so this pull request was not reviewed. Enable Bugbot in the Cursor dashboard to get automatic reviews on future PRs. |
|
Addressed cursor review 50886 (both findings, DESIGN-grounded and real) at 497719b:
Neither change touches emitted — sent from wise-moth-430 |
… probe to enrolled-only known_hosts
Host-key audit (witty-raven-412 / bold-eagle-107 cross-check, 2026-08-10). Two
independent audits agree the blind-accept sites (StrictHostKeyChecking=no +
UserKnownHostsFile=/dev/null) are exactly the BMC/sshpass lane in
extdeps.exec.command, and the CONVERGE probe path (typed_argv_probe_over_ssh ->
ssh.Session.ExecArgv = `ssh {host} -- argv`) carries NONE of it. But the probe
did not pin UserKnownHostsFile to the enrolled authority either — it APPENDED the
fragment to $HOME/.ssh/known_hosts with no StrictHostKeyChecking, so host
verification rode the runner account's AMBIENT known_hosts of unaudited (OOBE /
prior-manual TOFU) provenance. Condition-4 fixed the principal half; the host half
was zero-dimensional.
Anchor search (fleet_physical_inventory, fleet_intent_network, enrollment
provenance) before concluding: srv1/srv2 have no cited reservation, no
procurement/serial, no external key authority — genuine TOFU, not the convenient
answer.
Two changes:
1. Mint HostKeyEnrollmentProvenance = ScannedFirstContact{scanned_from} |
IndependentlyAnchored{authority}. authority is a REQUIRED NonEmptyStr, so
"anchored but unnamed" has no constructor (construction over validation, §5 —
no witness restates it). All 6 enrollment rows carry ScannedFirstContact
(measured honest state); the srv5/srv6 DGX Sparks are the intended
IndependentlyAnchored inhabitant (cited CR1000A reservation) but stay
first-contact until their slot->box assignment resolves — asserting it now
would encode the 50/50 guess DESIGN §5 forbids. Arm ready for that lane to
flip without a widening edit (second-consumer discipline). Receipt carries the
provenance label so "found" never reads as verification the row doesn't claim.
2. Pin the probe non-destructively: enroll_apply writes a SCOPED ssh config
(`Host <fleet endpoints>` -> UserKnownHostsFile = enrolled-only fragment +
StrictHostKeyChecking yes), replacing the ambient-known_hosts append. For fleet
hosts ssh now consults ONLY this registry and refuses unknown/mismatched keys
(a key change trips ProbeHostKeyMismatch). Scoped to fleet endpoints so
non-fleet ssh is untouched even on durable HOME; marker-guarded so a repeat run
appends once. Readback verifies against the pinned fragment.
Net climb: ambient-unaudited -> enrolled-authority-pinned, first-contact ceiling
carried by the provenance type. Banked run 31346260819 pre-dates the pin (rode
ambient); enrolled-anchored re-observe lands post-merge (probe jobs run on
main/dispatch, skip on PR), same window as the host-receipt item.
Green by execution: both fleet_host_key_enrollment witnesses return true; the
module compiles clean through analyses with the new provenance field. Pin is a
runtime .dag effect inside enroll_apply, not emitted ci.yml — no regen.
Co-Authored-By: Claude Opus 4.8 <noreply@anthropic.com>
Claude-Session: https://claude.ai/code/session_0168R31q15ZsPJyik18Roz3y
…; the honest blocker is the unperformed anchor join Factual correction (witty-raven-412, read against gunbc.dgx_spark_procurement). My prior scanned_from prose said the srv5/srv6 slot->box assignment is "an open operator decision" — false and would propagate. dgx_spark_router_binding_operator_allocation (SPARK-ALLOC-LAND-0, 2026-08-07) records the operator DELEGATED the tie-break and it was decided on ascending address order: srv5->192.168.1.222, srv6->192.168.1.223, both carried as SlotAssigned rows. No 50/50 guess is outstanding. ScannedFirstContact is still the correct label, for the right reason: no anchored enrollment has been PERFORMED — nobody has enrolled a Spark host key, observed the MAC at that address, and matched it against the cited CR1000A reservation. IndependentlyAnchored would claim work that has not happened. The anchor is AVAILABLE but UNUSED. This matters practically: bold-eagle is NOT blocked on an operator decision (it was made three days ago), only on running the join — so the Spark host-identity condition does not acquire a phantom blocker. Prose-only; no type, value, or behavior change. Compiles clean through analyses. Co-Authored-By: Claude Opus 4.8 <noreply@anthropic.com> Claude-Session: https://claude.ai/code/session_0168R31q15ZsPJyik18Roz3y
…its SSH legs (same path as probe) Real defect (cursor review 50900, §3/§5). collect_fleet_convergence_wet SSHs to every fleet host but never called fleet_host_key_enroll_apply(), unlike probe_fleet_reach_wet. The collector runs in its OWN isolated main-push job with a fresh ephemeral HOME (CI overrides HOME per job) and WIF-materialized key but no durable known_hosts — so the enrolled-only pin (StrictHostKeyChecking yes + UserKnownHostsFile=fragment) that enroll_apply writes never landed there, leaving the collector on the ambient-trust path the pin was written to close, and on a different trust path than the probe. Fix: call fleet_host_key_enroll_apply() FIRST in collect_fleet_convergence_wet and carry its readback in the collect receipt, mirroring the probe entry exactly — one enrollment authority, one trust path. Green by execution: the collector closure compiles and runs to ExitSuccess with the enroll call (no effect/cycle error). Co-Authored-By: Claude Opus 4.8 <noreply@anthropic.com> Claude-Session: https://claude.ai/code/session_0168R31q15ZsPJyik18Roz3y
|
Addressed cursor review 50900 at 80aa480 — a real §3/§5 defect, confirmed.
Fix: the collector now calls — sent from wise-moth-430 |
…operator request) Operator: "can we simply have a single fleet convergence job — it's overwhelming to see so many ... it can include all deployment/convergence i.e. srv1 deploy etc." The ten former jobs (deploy_dashboard_srv1, collect_fleet_convergence, advance_fleet_desired, probe_fleet_reach, scan_fleet_host_keys, enroll_fleet_host_keys, observe_probe_identity, install_fleet_converge_timer, enroll_runner_authorized_key, observe_srv1_tasks_preapply) collapse into ONE `fleet` job, so a normal push/PR shows a single (mostly-skipped) row instead of ~nine. Each former job's work becomes a STEP carrying the event guard its old job's if_condition held, applied via with_step_if: - main-push convergence steps (deploy/collect/advance): if github.event_name == 'push' - workflow_dispatch toolbox steps: if github.event_name == 'workflow_dispatch' Job-level gate ci_fleet_job_if runs the whole job only on (main push + ci success) OR workflow_dispatch — so PRs skip it as one row. Shared prelude + release-bins + WIF/ssh-agent setup run ONCE (were duplicated across jobs — a materialization win), key-agent cleanup runs always(). runs-on srv1; needs [ci]; permissions are the union (contents:write for advance + id_token:write for WIF); deploy's blue/green concurrency group carries to the whole job. Tradeoff, stated: needs [ci] means manual workflow_dispatch fleet ops now wait for the ci floor before running (the cost of one consolidated job; the convergence steps must gate on ci success, which requires needs [ci]). Cleanup: deleted the 11 now-dead per-op job builders, the unused gunbc_ci_deploy_job_backstop_timeout_minutes and ci_fleet_credential_permissions, and the orphaned ci_deploy_push_to_main_if / ci_workflow_dispatch_only_if expressions (their skip-cascade rationale folded onto ci_fleet_job_if, which now carries the !cancelled() + needs.ci.result guard the witness checks). The skip-cascade wall is preserved: ci_fleet_job_if's push arm carries !cancelled() + needs.ci.result == 'success', so push_to_main_family_carries_explicit_status_guard still holds (count is now 1 job, not >=2). Updated ci_deploy / ci_deploy_target_host witnesses to assert deploy-as-step-in-fleet; consumer_jobs -> ["fleet"]. Green by execution: main_wet regen idempotent (ci.yml 10 jobs -> 5); the emitted fleet job carries the correct per-event step guards; ci_workflow_witness_holds, push_to_main_family_carries_explicit_status_guard, and both updated deploy witnesses return true. Runtime main-push behavior validates post-merge (fleet convergence steps run on main push only). Co-Authored-By: Claude Opus 4.8 <noreply@anthropic.com> Claude-Session: https://claude.ai/code/session_0168R31q15ZsPJyik18Roz3y
…both DGX Sparks (#8073) * Give the fleet an automation principal and a service principal fleet-automation@gunbc is an SSH KEY COMMENT, not a POSIX username. The tree had no automation account at all, so every modeled host access resolved FleetAccountOperator -- the human -- and automation reached hosts as a person. Adds FleetAccountAutomation (gunbc, uid/gid 1500) and FleetAccountInferenceService (gunbc-inference, uid/gid 1501) to the single POSIX-account authority. No Spark-local account enum beside it. Privilege is one closed root-owned helper, not blanket sudo and not docker group membership: the Docker control socket is root-equivalent, so joining that group to fix a refused socket grants far more than deployment needs. The serving principal's grant list is EMPTY by construction -- it runs a model and answers requests, and never needs to change its host. UID/GID pinned identically on both Sparks only because it was OBSERVED: 2026-08-09, both hosts carried exactly one uid>=1000 account, and both names plus uids 1500/1501 were free on BOTH. Recorded rather than smoothed over: these two rows are DESIRED and exist nowhere, while every other PosixUser here is OBSERVED with a probe that confirmed it. Their probes therefore record the availability check actually run, not a fabricated existence check. That is gunbc#8068's authored-versus- executed conflation one carrier over; the split is deliberately NOT invented here, and dissolves when the apply creates the accounts. 25/25 existing principal witnesses PASS; module compiles clean. * Revert the static-account commit; no host mutation may proceed against it Rolls dag/gunbc/fleet_posix_accounts.dag back to main. The reverted commit fused semantic identity with the operating-system label -- static usernames gunbc / gunbc-inference and cross-host-equal uid/gid 1500/1501 -- which is the same defect the hostname authority already corrected one carrier over, reintroduced in a commit that argued for single authority. It carried three further defects, all confirmed against the tree: - a duplicate declaration of the privileged-helper path, beside the existing gunbc_deploy_privileged_helper_path in the same module; - new accounts unreachable through fleet_posix_account_from_name; - availability probes placed in PosixAccountDeclaredProbe, whose invariant is exact equality with concat('id ', name) -- the rows would have FAILED it, and the 25/25 witness run reported as validation never enumerated the new arms, so nothing checked. The working tree was rolled back earlier but the commit was never committed or pushed, so the remote head still carried it while it was reported reverted. That gap is the reason this is its own commit rather than a silent amend. * Split principal intent from account allocation from observation The fourth route. Neither an optional fleet_posix_principal nor an overlapping allocated-account enum: one semantic principal population, with its fused result separated into intent -> occurrence -> allocation -> observation -> grounding. WHY NOT OPTIONAL. Absent cannot carry a remedy. Intent-absent, allocated-but-unapplied, applied-but-unobserved, observation-contradicts- allocation, drifted and grounded are six situations with six different next actions, and a two-valued answer makes them one. Proven by execution: flipping AccountUnallocated to report PrincipalUnobserved reds the unallocated control while the unobserved control stays green, so the states are distinguished rather than collapsed. NameRequirements is a shared vocabulary -- audience, readability, permitted semantic content, authority role, stability, uniformity, allocation method, retirement -- and NOT a universal name type: DNS labels and POSIX login names keep separate lexical authorities. Automation and service names are required to carry NoDomainMeaning and to be IndependentPerOccurrence; the human label is the stated exception, because a person must know and type it. Consistency is four requirements, not one. The reverted commit asserted UNIFORMITY (equal uid across hosts is one identity) when it needed per-occurrence STABILITY. UID/GID are host-local numeric allocations in a PosixIdentityDomain; equality across srv5 and srv6 is permitted, never required, and never evidence of one principal. An inode records the number, so a uid change is an ownership migration rather than a rename. Both Spark automation occurrences are UNALLOCATED. No name chosen, no uid reserved, nothing applied -- so nothing can render useradd, User=, a home path, sudoers or authorized_keys, which is the point. HumanAccessOnly is carried structurally on the human principal, because host_identity_access and ci_deploy_access both resolve it today and that fusion is live. Consumes the existing gunbc_deploy_privileged_helper_path rather than declaring a second one with the same literal. 7 new controls reach the new arms, 7/7 PASS, the wall mutation-tested. Legacy witnesses unchanged: 25/25 grounded_principal, 11/11 ci_deploy. * Delete the fabricated principal; the missing state was the defect BLOCKING REPAIR. The recut removed a fabricated GroundedPosixPrincipal (static gunbc, uid 1500) and replaced it with a DIFFERENT fabricated GroundedPosixPrincipal: name UNALLOCATED-PRINCIPAL, uid -1, defended in a comment calling it unreachable-by-contract while the type made it an ordinary value every existing caller could consume into an effect, a path, a comparison or a log without any match forcing acknowledgement. A conspicuous poison value is still fabricated success. WHY IT EXISTED, which is the actual finding: fleet_posix_principal takes no host, and an account is per-host. There was no honest answer available, so one was invented. The repair is the missing STATE -- PrincipalRequiresHostScope -- plus total per-account accessors for the four legacy principals whose accounts really were observed on srv1/srv3. The placeholder is deleted, not relocated. Also repaired: - allocation standings are occurrence-bound; an unallocated answer for srv5 can no longer be consumed as srv6's with nothing to detect it; - allocation conflicts are preserved as their own outcome instead of being converted to unallocated, which erased both the attempt and the causes needing remediation; - conflict populations are first-plus-rest, so 'conflicted with no causes' has no constructor; - PosixAccountObservation carries its OWN occurrence rather than a copy of the desired allocation, and grounding compares the two subjects -- a copied observation is now drift. Copying the allocation in would have made every observation agree with the plan by construction, which is the authored-versus-executed defect reproduced inside its own repair; - the inference service realization is Unselected rather than an unallocated persistent account, which had already chosen a /etc/passwd realization before the runtime exists to say whether a supervisor dynamic identity suffices. STILL OWED BEFORE ANY APPLY, named in-carrier rather than silently missing: process receipt, raw-output identity, decoder identity and timestamp on the observation; separation of account grounding from access grounding, since desired sudo grants currently ground an account whose sudoers may never have been installed; POSIX lexical authority instead of NonEmptyStr; and the semantic-content / readability axis split. 10/10 new controls (3 added: host-scope refusal, cross-host observation is drift, service realization unselected). Legacy unchanged: 25/25, 11/11. * Receipt-backed observation; account grounding split from access grounding Completes the carrier repairs owed before any apply. RECEIPT-BACKED OBSERVATION. PosixAccountObservation now carries a ProcessReceipt, a raw-output identity and a timestamp. Without them any caller could author a record asserting an account exists -- #8068's authored-versus-executed defect at account grain, where the carrier proves an account was DESCRIBED rather than that a command ran. The grounding fold also stops synthesizing a probe with the literal 'observed' and carries the observation's own timestamp. ACCOUNT GROUNDING IS NOT ACCESS GROUNDING. The previous fold copied DESIRED sudo grants into a grounded result, so an account whose sudoers was never installed still read as subsumed -- it proved policy, not observed access. ground_host_access separates four facts that can each hold while the next does not: account matches allocation, key population observed, password login refused, narrow privilege proven to execute. Only HostAccessGrounded may authorize a serving deployment, which is exactly the srv5 canary progression. NAMING AXES SPLIT. An earlier revision made NoDomainMeaning and MnemonicOnly peer variants, a false exclusion: a pronounceable arbitrary word is both a human mnemonic and free of domain meaning, and that pair is precisely what an automation label should be. NameSemanticContent and NameHumanFactors are now independent, and a control asserts the combination the fused coproduct made unrepresentable. NameIntent (what the name must be) is separated from NameAllocationPolicy (how it is chosen and retired). POSIX LEXICAL AUTHORITY. PosixLoginName lands in extdeps/access/posix.dag beside the Open Group anchor, cited to the portable filename character set and the leading-hyphen constraint -- deliberately NOT shared with DnsLabel, whose grammar, length limits and authority differ. Scheme and scope identities are branded; name_attempt is Nat, so a negative attempt has no representation. 12/12 controls (2 added: account-grounded-does-not-imply-access-grounded, unallocated-cannot-reach-access-grounding). Legacy 25/25 and 11/11. * Author the Spark bootstrap secret producer (containers + IAM + material write) The refs in gunbc.spark_bootstrap_credential were named with nothing creating them. This is the producer, modeled on the live-proven gunbc.assimilate.bmc_bootstrap_provision pattern against the same project. Container provisioning and material write are separate patterns because their idempotence differs: CreateSecret converges (409 on re-run), AddVersion does not (every call mints a version). Collapsing them behind one verb would have carried the BMC runbook's 'safe to re-run' promise onto a half where it is false. Payloads are pattern parameters, never data rows. Readback names the exact version written rather than the moving 'latest' alias. Token acquisition is a parameter rather than a direct shell.GCloud.AuthPrintAccessToken() call, so this does not mint a third authority beside gunbc.auth.access_token_source.ensure_access_token. Compiles green over a 225-source closure; bindings evaluate to the real least-privilege SA. NOT YET APPLIED: no admin access token is reachable from this host. Co-Authored-By: Claude Opus 5 (1M context) <noreply@anthropic.com> * Correct a convergence claim refuted by execution; split the convergent op out The authority note and the prior commit message both asserted that CreateSecret converges -- 'a second run answers 409 and the desired state is reached either way'. That was written from reading the API, not from running it. The reapply was then executed and returned HTTP 409 ALREADY_EXISTS as a hard pattern failure. The claim is corrected to what was measured. The failure also exposed a design error: SetSecretIamPolicy (convergent, sets a policy to a desired value) was bundled behind CreateSecret (not convergent). Because creation runs first and aborts, a re-run never reached the bindings -- so the only operation that could repair drifted least-privilege bindings was gated behind one guaranteed to fail on every host after the first apply. Split into spark_secret_containers_create (one-shot, typed loud failure on re-run) and spark_secret_iam_converge (re-runnable). The remaining non-convergence carries a dissolve-on naming its next rung: route CreateSecret through an UpsertClassification mapping 409 to Noop, at which point the two patterns rejoin. Measured 2026-08-09: first apply created both secrets and set both bindings; reapply failed 409 on the first secret. Co-Authored-By: Claude Opus 5 (1M context) <noreply@anthropic.com> * Record the executed re-runnability receipt for IAM convergence spark_secret_iam_converge re-run against already-existing secrets succeeds where the previously bundled pattern died on 409, which is the discriminating receipt for the split. Stated at measured strength: re-running leaves binding content byte-identical but is NOT a no-op -- the policy etag advances each run, because the pattern writes the desired policy rather than comparing and skipping. Re-runnable and idempotent-in-result, not idempotent-in-effect. Co-Authored-By: Claude Opus 5 (1M context) <noreply@anthropic.com> * Add executing controls over the Spark grant shape The producer module landed with applied receipts but no controls. A successful apply proves the API accepted the operations and proves nothing about whether the roles granted were the least ones sufficient -- an owner-level binding applies exactly as cleanly -- so least privilege has to be asserted structurally rather than inferred from a green run. Five controls, 5/5 PASS by execution: - least privilege: exactly two bindings, only the assimilator, allowed roles - both intended roles present (control 1 alone passes if secretAccessor were granted twice and versionAdder never) - escalated roles absent: the discriminating negative, red if someone resolves a permission error by reaching for admin - the two secret identities are distinct: a collision would silently write both credentials into one secret as successive versions, since AddVersion never overwrites - neither collides with the BMC secret sharing this project and SA, where the assimilator already holds versionAdder so the write would be permitted Co-Authored-By: Claude Opus 5 (1M context) <noreply@anthropic.com> * Recut Spark secrets as convergence subjects; delete the mutation patterns Operator verdict on the interactive apply. The three patterns that dispatched CreateSecret / SetSecretIamPolicy / AddVersion are DELETED rather than repaired, because each was unconditional -- create with no prior Get, SetIamPolicy with no GetIamPolicy and no etag (setIamPolicy REPLACES a policy, so unrelated bindings on any secret carrying them would have been dropped), AddVersion with no attempt identity -- and because leaving that shape callable is what made a manual apply possible at all. Actuation belongs to an OIDC/WIF workflow that consumes this model. The population is FOUR host-scoped occurrences, not two global names. The factory hotspot value differs per device, so one global hotspot secret would have to hold two credentials, and since AddVersion never overwrites the second write would silently become a prior version of the first. This is the same host-scope error the POSIX principal split fixed one carrier over. The two containers created interactively classify ObservedLegacyUnbound: real starting-state observations, not accepted output, not the authority for any occurrence, not edited or deleted from here. Neither carries a version. The BMC assimilator principal is replaced by an unallocated FleetCloudConvergencePrincipal. Binding a principal declared for the srv3 BMC secret to Spark credentials made BMC assimilation the authority for Spark provisioning -- a cross-domain expansion, and the nickname trap in its usual form. Six controls, 6/6 PASS. Note for the record: spark_bootstrap_credential's own dissolve-on already required a receipt naming the exact version so an interrupted retry reuses it rather than minting a second password. The deleted AddVersion pattern violated a law written in the module it was extending. Co-Authored-By: Claude Opus 5 (1M context) <noreply@anthropic.com> * Record the reported cloud census; split principal identity from population binding witty-raven-412 reports (msg_1e6d1b42) the fleet key secret, a pre-existing fleet-automation@ consumer, and a new fleet-cloud-convergence@ SA WIF-bound for gunb-ai/gunbc. Recorded as REPORTED observations carrying their source, not as measurements: this session has no gcloud, ADC, metadata server or credential env, so it cannot verify any of them. Two separations the report's phrasing would otherwise collapse: Identity vs binding. The convergence SA exists and holds accessor on ONE key secret, which does not reach the four Spark occurrences. Principal allocated and population reachable are different facts; SparkPopulationBindingStanding keeps them apart so 'the principal is set up' cannot read as 'the population is bound'. Identity join vs causal claim. The scratchpad key and SM version 1 share an ED25519 fingerprint, so they are the same keypair. That does NOT establish the local copy was materialized FROM that version: a fingerprint is symmetric, a common origin or a copy predating the version yields identical evidence, and no fetch was observed. Recording it as a materialization would assert an event with no receipt. Control 4 was also wrong in the other direction and is fixed: it asserted the principal stays unallocated, which would have gone red the moment the correct thing happened. The real invariant is that the Spark authority is dedicated rather than borrowed, so it now reds on the BMC assimilator or the fleet-automation consumer being substituted. 7/7 PASS. Co-Authored-By: Claude Opus 5 (1M context) <noreply@anthropic.com> * Container observe-before-create classifier and IAM membership diff The two operations the deleted patterns did blind, modeled as decidable classifications with no actuation. Container: four verdicts, because absent, already-correct, present-but-not-ours, and unreadable have four different remedies. Calling Create and reading a 409 was an error arm standing in for an observation nobody made. A container that ALREADY carries a version classifies Conflicted, never Converged -- AddVersion never overwrites, so writing there leaves two live credentials for one occurrence. An unallocated occurrence refuses rather than computing a probable name, which is the reverse inference the carried-never-derived law forbids. IAM: a delta against an observed policy, with an owned-universe parameter that makes 'unrelated' decidable. A member outside it is preserved even when undesired, because this model does not own it. A write with no observed etag refuses -- setIamPolicy replaces rather than amends, so a write that cannot name the policy it read silently overwrites a concurrent update. Seven controls, 7/7 PASS, including the two discriminating negatives: an existing version conflicts rather than converging, and an unrelated binding is preserved rather than dropped (the exact defect the deleted pattern shipped). Co-Authored-By: Claude Opus 5 (1M context) <noreply@anthropic.com> * Materialization attempt identity, receipt, and the hotspot source asymmetry At most one AddVersion per attempt identity. Three attempt states, because never tried, issued-but-unconfirmed, and confirmed have three different resolutions -- and the middle one is the point: it resolves by OBSERVING the store, never by issuing a second AddVersion. A lost process outcome is not evidence the write did not land, and treating it as such sprawls versions for one intended credential. A confirmed receipt is discharged by an INDEPENDENT read agreeing with it, not by having been written: a mismatched or unavailable readback refuses. A readback naming the moving alias latest is Unavailable rather than Matched, since it is compatible with another writer having landed in between. THE ASYMMETRY, raised with the operator and modeled as a wall rather than a note: the convergence shape says material is generated or fetched inside the admitted run, which does not cover a FACTORY hotspot credential. That value is printed on the device -- there is nothing to generate and nothing to fetch it from -- so its only honest source is an operator observation through the sensitive-input boundary. Generating one fabricates an observation of physical hardware that every downstream check would accept while the credential was simply wrong, so the pairing refuses. The administrator arm still admits generation, proving the refusal is kind-specific rather than a blanket ban. Five controls, 5/5 PASS. Nineteen on the module total. Co-Authored-By: Claude Opus 5 (1M context) <noreply@anthropic.com> * Extract the repository-level GitHub-to-GCP federation as one authority Operator directive: extract the federation and consume it from both domains rather than copying the BMC rows into the Spark lane. Nothing about a pool for gunb-ai/gunbc, an OIDC issuer URI, or the fleet secrets project is a BMC fact. The tell was in the names: bmc_secrets_project_id held gunbai-secrets, the FLEET secrets project, so a second domain had to either import a bmc_ symbol for a non-BMC fact or copy the literal. Copying is how the second authority gets minted, and the copy is invisible afterwards because both sides read correct. BMC now consumes the extracted rows; the values have one authority. Proven by execution against BMC's own witnesses: 25/25 PASS across all three files (bootstrap 4, token federation 7, delegation chain 14), including the ones that would notice a changed value -- provider resource well-formed, principal set pins the repo, least privilege, closed allowlist. Residual naming fork left marked rather than silently carried: seven bmc_-prefixed names still stand for repository-level concepts. Weaker than the value fork just removed since no row can now disagree, but still a nickname; the dissolve-on names the consumer rename that retires them. Two process notes worth recording. The first run reported 0 PASS with no failures -- that was ZERO ENROLLMENT from --roster-from-discovery with --entry, not success, and a FAIL-grep would have read it as green. The second run failed 17/25 on 'no such function', because the new module was not in the entry closure: bare cross-module references resolve only by pool-membership coincidence, so the fix is explicit imports in both directions. Co-Authored-By: Claude Opus 5 (1M context) <noreply@anthropic.com> * Independence is a grant property: project into std.access, drop the global standing Operator correction. Independence belongs to authorization policy, not to usernames, uids, service-account counts, or how many credential values exist. THE DISHONEST ARM, and it was mine: the carrier returned FleetCloudPrincipalAllocated while its own note said the allocation was not independently verified. Those cannot coexist -- the carrier reported a rung its evidence did not establish, in the same commit that recorded the evidence as second-hand. Split into Unallocated | Reported | Allocated; the live standing is Reported until the WIF workflow reads it back, and a Reported principal cannot authorize actuation. SparkPopulationBindingStanding deleted. A global present/absent could not say which operation on which resource, so 'the principal is bound' read as coverage while naming nothing. Replaced by exact principal x operation x resource cells over a scope carrying FrozenAtGrant vs TracksNamespaceMembership -- so adding srv7 cannot silently inherit today's authority. No second access kernel: std.access supplies AccessDecision, the typed refusal causes and decision_meet. This module supplies only the domain operation population, resource identity and scope rows. Action-not-granted and object-not-granted stay distinct refusal coordinates rather than one blurred no. Five new controls test independence as a GRANT property and mention no username or uid: srv5 does not reach srv6; one global principal with one exact grant reaches both admitted resources and not a third; AccessVersion does not imply AddVersion and AddVersion does not imply DestroyVersion; a Reported principal cannot actuate while an Allocated one can; an allocated principal is still bounded by scope. Full module roster 22/22 PASS. Co-Authored-By: Claude Opus 5 (1M context) <noreply@anthropic.com> * Model the login shell; record the ghrunner contradiction and the applied allocation Three defects surfaced by witty-raven-412's live readback on srv1 and srv2. SHELL WAS UNMODELED, and it is load-bearing: sshd refuses remote command execution under a nologin shell, so an account can exist, carry the right key, and still be unusable for the exact thing automation needs. PosixUser gains a PosixLoginShell, modeled as an OBSERVATION rather than a required literal -- forcing a value would have invented three shells nobody read, which is exactly how the uid rows beside them went wrong. An unobserved shell answers false to posix_shell_permits_command_execution, because 'we did not look' is not 'it works'. THE FALSE RECEIPT, which is the sharper defect: fleet_posix_ci_runner_user carries uid 1001/gid 1001 and a declared_probe stamping probe_command 'id ghrunner' on srv1. That exact command on that exact host returns 999/987, so the row claims a probe whose own output contradicts the values beside it -- either it never ran or its result was discarded for authored numbers. A wrong integer is visible; the provenance stamp is what made it look checked. Recorded as a typed contradiction with source, NOT overwritten: replacing one authored pair with another I cannot verify from here is a better-sourced guess, and minting a PosixAccountObservation to hold someone else's reading would defeat the ProcessReceipt wall that carrier exists to be. ALLOCATION APPLIED: gunbc-fleet on srv1 and srv2, same label on every host, because independence is a property of the grant and never of the string. srv3/srv4 stay AllocationPending -- an unreachable host has no allocation rather than a presumed one. Carried with the uid coincidence: gunbc-fleet took 1001 as the next free id, which is what the ghrunner row wrongly claims, so two rows now read 1001 for different accounts on one host. Harmless (uid is host-local and never principal identity) but a reading hazard that raises the priority of grounding ghrunner. Also recorded: srv1's duplicate sudoers dropin makes effective privilege the UNION of two files, so a grant readback reading one reports a strict subset and would classify an over-granted host as exact -- a measurement hazard, not untidiness. 46/46 across the three affected rosters (posix 12, managed_directory 9, grounded_principal 25). Co-Authored-By: Claude Opus 5 (1M context) <noreply@anthropic.com> * Remove two fabricated-success paths and join the receipt to itself Review findings, all three mine and all three the class this module exists to prevent. AN EXACT VERSION ADDRESS IS NOT A PAYLOAD MATCH. admit_spark_readback_version turned any non-latest string into SparkReadbackMatched while consuming no AccessVersion response, no decoded payload, no expected identity and no comparison -- fabricated success producing the very verdict that discharges a receipt. Split into an address ADMISSION and a separate compare_spark_readback taking both identities, so Matched is reachable only when two supplied identities actually agree. The old control asserted the defective behaviour, so the control and the bug agreed; it is rewritten to prove admission alone yields nothing. THE AT-MOST-ONCE WALL WAS ADVISORY. spark_materialize_once let any caller mint SparkMaterializeOnce without consulting the attempt record -- the wall in the prose, the bypass in the interface. Deleted. A first write is now reachable only through SparkAttemptJournaled, a durable record written BEFORE the effect issues, so a crash between journal and issue lands in IssuedUnconfirmed and resolves by observation rather than a second AddVersion. THE RECEIPT DID NOT JOIN ITS OWN FIELDS. A receipt naming exact_version 7 whose readback verified version 8 produced Noop, because nothing compared them. Two fields that must agree and are never checked are two facts, not one. 24/24 PASS. NOT addressed in this commit and still open from the review: the parallel grant system (no AccessRequest/AccessPolicy construction, no authorization_profile, no effect_grant reach, a second Secret Manager operation enum disagreeing with the extdeps surface), missing workflow/revision/purpose/duration/use/approval context, IAM diffing at flat-member rather than role x member x condition grain, the consumer-derived secret demand, the local spark_srv5/srv6 identity roster, the legacy container carrier holding no receipt, and every POSIX fail-open. No srv5 apply is safe from this head. Co-Authored-By: Claude Opus 5 (1M context) <noreply@anthropic.com> * Read-only WIF effective-access observation (temporary push trigger) The first thing on this branch that changes observed reality rather than its description. Authenticates via GitHub OIDC to WIF as fleet-cloud-convergence@, mutates nothing, and emits the effective-permission map. What it decides: fleet_cloud_principal_standing currently returns CloudPrincipalReported -- an allocation another session reported and this one has never verified -- and the model refuses to let a Reported principal actuate. This run either grounds it with a receipt or refutes it. Expected standing is EffectiveAccessIncomplete: the principal is reported to hold accessor on ONE secret, so the Spark containers should refuse. An unexpected permit is the more interesting outcome and classifies Overbroad. Both are real receipts; neither is a failure. Read-only by contract. The testIamPermissions calls are POST because that is the API's shape; they return the granted subset of the supplied permissions and change no state. No AccessVersion call is made, so no payload is read, and the token is never echoed. Declared scaffold on two axes, each with its dissolution named in the file: the YAML and shell are hand-authored where the project's rules say a foreign-executor workflow should be emitted from .dag, and the push trigger is temporary so the observation runs without a manual press. Co-Authored-By: Claude Opus 5 (1M context) <noreply@anthropic.com> * Ground the cloud principal by execution; record measured effective access FIRST OBSERVED RECEIPT ON THIS BRANCH. GitHub Actions run 31335059188 at head 6b79d061c4a: OIDC to WIF issued a token for fleet-cloud-convergence@ with no key, no pasted credential and no gcloud, then probed its own permissions read-only. CloudPrincipalReported -> CloudPrincipalAllocated. The allocation another session reported is now confirmed by an observation this session can cite, so a principal my own model refused to let actuate has a receipt behind it. EFFECTIVE ACCESS, MEASURED: secretmanager.versions.access on fleet-automation-ssh-key, and NOTHING on either Spark container. testIamPermissions returned an empty object for both, which is a positive statement of no permission rather than a 403's ambiguity between denied and absent -- the reason the probe asks from the resource side instead of inferring from status codes. versions.destroy and secrets.setIamPolicy were probed explicitly and denied everywhere, so the grant is narrow BY MEASUREMENT and there is no Overbroad finding. Standing is EffectiveAccessIncomplete with the seven missing cells enumerated rather than guessed. Two facts nobody had recorded: this principal cannot read secrets.get anywhere, so it cannot independently verify the legacy containers (what I know about them came from an operator token, not this path); and the Cloud Resource Manager API is disabled on the project. CONTROL REPAIRED, second instance of the same shape: the reported-cannot-actuate control asserted against the LIVE standing, so grounding the principal by observation would have RED it -- a control forbidding progress rather than forbidding the defect. It now takes a Reported fixture, since the invariant is about that arm. 25/25 PASS. Co-Authored-By: Claude Opus 5 (1M context) <noreply@anthropic.com> * Correct the testIamPermissions overclaim; take the token out of argv Two defects in code I landed hours ago, both mine. THE INSTRUMENT CANNOT SUPPORT AN ABSENCE CLAIM. The prior receipt asserted that an empty testIamPermissions response is 'a positive statement of no permission rather than a 403 ambiguity', and concluded there was no Overbroad finding and an exact missing-cell population. That is backwards: the API returns the empty set for a resource that does NOT EXIST, is documented as a permission-aware-UI helper rather than an authorization check, and may fail open. So an empty response carries exactly the denied-or-absent ambiguity a 403 carries, and I had claimed it RESOLVED that ambiguity. Standing is now EffectiveAccessPartiallyObserved: one positively observed cell, seven requested-but-not-returned (never 'missing'), and overbreadth explicitly UNOBSERVED -- no policy was enumerated, and a probe cannot find a binding it did not ask about. WIF issuance is split into its own standing, because the credential path working is not evidence about what the credential may reach. THE TOKEN WAS IN ARGV. The workflow passed the bearer header as a curl -H argument, placing the credential in curl's command line where any process on the runner can read it from /proc -- the same argv exposure this session already recorded for a CLI flag, repeated in the file whose own comment claimed the token was never exposed. It is now written once to a 0600 curl config by heredoc, passed with --config, and shredded in an always() step. Also recorded in the workflow rather than left implicit: it emits log text, not a typed artifact receipt, so the .dag rows are hand-transcribed from it -- a second authority that closes when the workflow is emitted from the model. 26/26 PASS, each witness named individually in the untruncated log. Co-Authored-By: Claude Opus 5 (1M context) <noreply@anthropic.com> * Repair the GCP IAM surface; reconcile at cell grain; split the two principals Commit 1 of the directed sequence. Four separable defects, one measurement. EXTDEPS DID NOT CARRY THE OPERATIONS THE MODEL NEEDS. Added GetSecret, a secret-scoped GetIamPolicy, and TestSecretIamPermissions. SetSecretIamPolicy was wrong three ways at once: it took a bare binding list with NO etag and returned only an etag, so it silently dropped every binding the caller did not author, had no optimistic concurrency to refuse a concurrent edit with 412, and never showed the caller the policy it actually installed. It now carries the full GcpPolicy including the etag it was read at, carries updateMask, and returns the RESULTING policy so a write is checked against what landed rather than what was sent. THE PROBE CANNOT PRODUCE ABSENCE, SO THE TYPE DOES NOT LET ANYONE WRITE IT. SmPermissionProbeCell has Held and Undetermined and NO third constructor. This is the same overclaim corrected in the previous commit, moved one rung up the ladder: there it was a prose correction to a receipt, here the invalid state has no representation, so an "exact missing-permission set" is unwritable rather than merely discouraged. GetSecretIamPolicy is the only instrument that settles it. NO ROLE-PERMISSION TABLE IS AUTHORED. Predefined role contents are Google-owned and change without us, so role_grants_permission reads a RolePermissionAuthority with an explicit Unobserved arm and returns a three-valued verdict rather than false. The citation behind the observed arm is a new gcp.IamRoles.GetRole -- its own service, because roles live on iam.googleapis.com while token minting lives on iamcredentials.googleapis.com, and hanging it off the credentials service would have silently requested the wrong host. THE IAM DIFF WAS AT THE WRONG GRAIN. It compared flat member strings, which cannot express the question a policy answers. One member under secretAccessor and under admin is ONE entry at member grain, so demoting a principal reads as "already present, nothing to do" and the admin binding survives. Reconcile now works on GcpIamBindingCell -- role x member x condition -- with condition equality part of cell identity, and ownership asked per cell so owning (M, accessor) does not imply authority to remove (M, admin). TWO PRINCIPALS, FORCED BY MEASUREMENT. Run 31336704811 at da9d440f8f4 probed fleet-cloud-convergence and got back exactly secretmanager.versions.access; secrets.getIamPolicy did not come back and secrets.get returned a live 403. That principal provably cannot read a policy, so overbreadth stays unobservable until a grant administrator exists. The data plane now reaches payloads and never policy; the control plane reaches policy and never payloads. The refusal subject is DERIVED from the grant -- the prior literal "FleetCloudConvergence" would have located an administrator's refusal against the wrong subject the moment the split landed. The grant is a thin profile over std.authorization_profile's EffectRequest and finally carries the context it needs to be adjudicated at all: workflow, revision, purpose, duration, max uses and approval. FrozenAtGrant names the roster identity it was frozen against, so "frozen" is checkable instead of a label. 34/34 on the spark suite, 12/12 on the secret-manager suite, rosters matched. Nothing actuates. Owed and NOT claimed here: a discrimination check on the cell-grain controls, consumer-derived secret demand, workflow generation from .dag, and the POSIX fail-opens. Co-Authored-By: Claude Opus 5 (1M context) <noreply@anthropic.com> * Give the two BMC callers the read-modify-write my extdeps change required CI fix. The break was mine and the class was one I had already been warned about: I reshaped SetSecretIamPolicy in extdeps and swept only the module I was looking at, leaving two real callers reading a field that no longer exists -- bmc_bootstrap_provision and bmc_wif_canary_bootstrap, both on bind.ok_etag. WHAT THE BREAK EXPOSED IS WORTH MORE THAN THE BREAK. Both callers were passing a bare binding list with no etag: a blind whole-policy overwrite, which is exactly the defect the extdeps repair exists to make unwritable. Reverting to green would have restored it. So the compile error was the wall working as intended, and both sites now observe the policy first, merge, and write with the observed etag -- the compare-and-swap the API actually requires. merge_policy_bindings lives in extdeps.cloud.gcp.iam beside GcpPolicy, not in a consumer: every caller needs the same preserve rule, and three copies of it would drift. Its bound is stated where it is defined rather than assumed -- it replaces at ROLE grain, so a caller authoring a role takes over that role's member list. Cell-grain preservation needs the caller's owned-cell universe, which that function does not have and must not guess. Both files also gained explicit imports for GcpPolicy/merge_policy_bindings. They previously carried no imports at all and reached everything as bare references, which resolves only when some unrelated module in the closure happens to drag the definer into the pool. VERIFIED BY CONTROLLED COMPARISON, not by the diff looking right. Same binary, same command, red head b7c0d19b41a vs this tree: ok_etag errors 2 -> 0 Empty errors 32 -> 32 (pre-existing, in complexity_accumulator_copy, not mine) diagnostic lines 79 -> 77 (exactly the two removed, nothing new) Both runs exit 1 on those pre-existing Empty errors, so this is a delta claim and NOT a whole-tree-green claim. Co-Authored-By: Claude Opus 5 (1M context) <noreply@anthropic.com> * Defork IAM reconciliation onto one cell-grain mechanism in the GCP authority I introduced this fork two commits ago and it is removed here rather than left to propagate into the demand work. THE FORK. The CI repair added merge_policy_bindings, which replaced the whole member population of every role the caller authored. That was a SECOND reconciliation semantics beside the role x member x condition mechanism the Spark module already carried -- the dual representation this lane exists to remove -- and its failure mode was concrete rather than theoretical: an unrelated member under a role the caller also uses is dropped with no diagnostic, because whole-role replacement cannot see members it does not author. Its own note admitted it could not preserve those members. A helper that has to disclaim the property callers need is not a shared mechanism; it is a second one wearing the first one's job. THE REPLACEMENT lives with GcpPolicy, not in a consumer, because every caller needs the same read-modify-write and three copies would drift. reconcile_gcp_policy takes an explicit GcpIamOwnedCellUniverse and: flattens observed bindings to cells, adds missing desired owned cells, removes stale cells ONLY inside the exact owned universe, preserves every unowned cell including another member under an authored role, regroups cells into wire bindings so an unauthored cell survives the round trip, and carries the observed etag so the write stays a compare-and-swap. GcpPolicyAlreadyConverged is its OWN arm rather than a delta that happens to be empty, so a caller cannot write an unnecessary setIamPolicy and read the resulting etag bump as progress. Both BMC callers and the Spark module now consume that one mechanism; Spark's local cell predicates delegate to it instead of duplicating it. merge_policy_bindings is DELETED, not parked beside the replacement -- a climb deletes the machinery it obsoletes rather than accumulating next to it. Also fixed, both caught by the compiler rather than by assumption: an annotation I placed inside a declaration body (annotations are modeled at module-item grain only), and a twelve-line comment block still describing merge_policy_bindings after I deleted it -- prose outliving what it names. VERIFIED. Spark suite 34/34, roster matched. Whole-tree dag compile against the pushed head, same binary and command: hard diagnostics 67 -> 67 (mid-state with my annotation bug was 69) diagnostic lines 77 -> 77 Empty errors 32 -> 32 (pre-existing, not mine) No new diagnostics. Both runs still exit 1 on the pre-existing Empty errors, so this is a no-regression claim and NOT a whole-tree-green claim. Co-Authored-By: Claude Opus 5 (1M context) <noreply@anthropic.com> * Derive access demand from selected effects; extdeps owns operation identity The desired access population is no longer authored. It is a projection of the effects this workflow will actually perform. WHAT WAS WRONG. SparkCloudGrant.operations x SparkCloudScope.resources was a hand-maintained roster crossed into cells: a second inventory of what the code needs, kept beside the code, so it drifts -- and the cross product granted every listed operation on every listed resource, so a plan that reads secret A and writes secret B acquired write on A and read on B for free. EXTDEPS OWNS THE UPSTREAM FACTS, following the github_effect_required_permissions precedent already in tree rather than inventing a shape. SecretManagerEffect carries the EXACT resource it acts on; secret_manager_effect_operation mints the OperationRef beside the operation declaration it names, so no consumer authors service/operation strings; secret_manager_effect_required_permissions states each operation's permission requirement, because which permissions an operation needs is Google's contract, not a workflow's guess. DEMAND IS ONE CELL PER (selected invocation, required permission), with the resource taken from the effect. There is no roster left to cross. Every cell carries source_effect, so "why is this permission requested" is answered by a plan step a reader can find. TWO FAIL-CLOSED ARMS, both deliberate. An operation whose permission projection is empty yields DemandIncomplete and BLOCKS actuation rather than contributing no demand -- an empty requirement would read as "needs nothing" and admit the effect with no grant at all. And one incomplete step refuses the WHOLE derivation, because a partial population read as total is how an ungranted effect slips through. THE CEILING STAYS NORMATIVE AND SEPARATE, now keyed on the real OperationRef. Selected code does not authorize itself: control 4 asserts BOTH halves -- a selected DestroyVersion produces a visible demand AND is refused. The enum-keyed ceiling is demoted to a shim delegating to the ref-keyed authority, so deleting SecretManagerOperation is mechanical rather than a risky half-migration. A SIDE FINDING, kept because it was load-bearing: this witness had NO imports and reached its entire subject by bare reference, resolving only by pool coincidence. Adding one explicit import scoped resolution and 39 of 40 controls died at once with "no such function" -- the transition is all-or-nothing, there is no partial state. The subject module is now named explicitly, which also means those 34 pre-existing controls no longer pass by coincidence of pool membership. Also hoisted ten annotations out of match arms and function bodies (annotations are modeled at module-item grain only) -- my debt from the two-principal split. VERIFIED. Suite 40/40, roster matched. Whole-tree dag compile vs pushed head: hard diagnostics 67 -> 57 (the ten hoisted annotations), Empty 32 -> 32, nothing new. Both runs still exit 1 on the pre-existing Empty errors, so this is a no-regression-plus-ten claim, NOT whole-tree green. NOT DONE, not claimed: SecretManagerOperation deletion, consumer-derived secret population, role realization from observed permissions, approval derivation, and controls 6-12. Control 6 is deliberately unwritten -- every current operation has a permission mapping, so testing it would require a planted fake, and a control that only passes against a fabricated input is not evidence about the real projection. Co-Authored-By: Claude Opus 5 (1M context) <noreply@anthropic.com> * Derive secret demand from named consumers, not from the credential roster THE FOUR-CREDENTIAL POPULATION WAS NEVER A DEMAND POPULATION. spark_credential_population is a roster of credentials that COULD exist for a host; it says nothing about whether any workflow still needs one. Reading it as demand made a post-OOBE factory hotspot credential an automatic blocker on serving, because "four occurrences, none allocated" reads as four outstanding obligations whether or not anything consumes them. SecretDemandStanding derives from a CONSUMER -- a named workflow plus the operation it will perform. Four arms, because these have different remedies and collapsing them is how a break-glass credential becomes live authority: Active (named consumers), RetainedForRecovery, RetiredAfterOobe, Unobserved. Only Active yields routine demand. TWO JUDGEMENTS, stated as judgements rather than derivations. RetainedForRecovery is INACTIVE: a break-glass credential should be reachable by an explicit recovery workflow, not by the standing grant every serving run carries. And the serving API credential is Unobserved rather than Active: the pinned artifact and its acquisition path are not selected yet, so an Active row would fabricate a consumer that exists in no plan -- it is the credential most likely to become active next, and it is still not asserted early. An Active arm with an empty consumer list claims consumers exist while naming none, so secret_demand_is_active refuses it rather than letting the arm's NAME stand in for the fact. SCOPE, stated so this is not overclaimed: this ADDS the demand authority beside the existing population rather than deleting it. Nothing yet consumes SecretDemandStanding in an acceptance condition, so the model now permits the correct answer rather than enforcing it, and the old four-occurrence roster is still present for the controls that reference it. On the guarantee ladder this is mitigatable moving toward mechanically preventable -- not structural, and NOT "the serving blocker is removed". VERIFIED. Suite 42/42, roster matched. Whole-tree dag compile: 57 hard diagnostics and 32 Empty, both exactly baseline -- no regression. Still exit 1 on the pre-existing Empty errors, so this is no-regression, not whole-tree green. Co-Authored-By: Claude Opus 5 (1M context) <noreply@anthropic.com> * Name the BMC no-op bypass instead of letting it read as protection Found in review, and it is mine from the defork commit. Both arms of the reconcile match in the BMC wrappers return the policy, so a CONVERGED policy is still written: GcpPolicyAlreadyConverged is computed and discarded at the only call site that mattered. An arm that exists and is consumed nowhere is worse than an absent one -- it reads as no-op protection while every run writes, and an unnecessary setIamPolicy advances the etag, so a convergent re-run looks like it changed something. Not repaired in place: the caller is a `node bind:` pattern invocation, which cannot branch on a verdict to skip its own effect. Making the write conditional requires the call site to become an effect-plan step that can be selected or not, which is the dissolution trigger recorded in both files. Named rather than silently left, because the alternative is a wrapper that looks correct. Control 11 (zero-delta reapply is a Noop) therefore does NOT hold at the BMC call sites today, and I am not claiming it does. Co-Authored-By: Claude Opus 5 (1M context) <noreply@anthropic.com> * srv5 canary step 1: observe managed-principal reach, read-only Operator correction taken: stop modeling, get to a live canary. This is the first executable step and it settles the question no model can -- can the fleet controller authenticate to srv5, and as WHICH principal. Runs on a self-hosted fleet runner because the Sparks have no Actions runner, so nothing bootstraps them from inside; authenticated remote application from a LAN controller is the only mechanism. Uses the ONE permission measured as held (versions.access on fleet-automation-ssh-key, run 31336704811). READ-ONLY by construction: every command observes. Nothing is installed, no authorized_keys is written, BatchMode=yes cannot prompt so a missing credential fails instead of falling back to an interactive path, and the key is shredded in an always() step. Token goes to a 0600 curl config, never argv. The receipt states what a failure does NOT establish: it cannot distinguish missing authorized_keys from absent account from sshd policy, and those have different remedies. #8083's 'no verified managed principal' is a claim about our records; this observes the host. Co-Authored-By: Claude Opus 5 (1M context) <noreply@anthropic.com> * Add branch-scoped push trigger so the canary probe can actually run workflow_dispatch resolves against the default branch, where this workflow does not exist, so the dispatch 404'd. Same constraint and same remedy as spark-cloud-observe. Scoped to this branch, not a required check, gates no PR, with deletion on canary completion as its dissolution trigger. Co-Authored-By: Claude Opus 5 (1M context) <noreply@anthropic.com> * Stop the canary's own harness from killing its primary observation srv5:22 OPEN and AccessVersion http=200 with a 419-byte key: reach and credential materialization through WIF on a fleet runner are now proven by execution. The principal question went unanswered because GitHub Actions runs `run:` under bash -e, so the first ssh returning non-zero aborted the step before any line printed. That is the probe's own harness converting its primary observation into a crash -- a failed login is exactly the data this step exists to collect, since the question is WHICH principal srv5 admits, not whether one succeeds. Each attempt now captures its exit code and continues, and the step's success no longer depends on any login working. Co-Authored-By: Claude Opus 5 (1M context) <noreply@anthropic.com> * The probe fabricated its own exit codes; make the principal verdict readable THIRD harness defect in this probe, and the worst kind: it manufactured the exact field a reader uses to judge success. I wrote OUT=$(...) || true followed by RC=$?, so $? captured the status of `true` -- every row reported rc=0, including ghrunner, ubuntu and nvidia which were plainly refused with Permission denied. What the run text does support: those three are genuinely REFUSED (publickey, password). gunbc-fleet is UNRESOLVED -- no denial, but no id output either, and a successful login would have printed a principal and uid. Absence of a denial is not a login, and I am not recording it as one. Two repairs. Exit status is captured with if/else so it is the ssh status. And the remote command prints an explicit PRINCIPAL= marker, because bare `id -un` output was indistinguishable from an empty capture -- 'no denial and no output' could not be told apart from a successful login whose text was lost. A login now counts as ADMITTED only when that marker is present; anything else is REFUSED-publickey or UNRESOLVED, which are different states with different remedies. Standing so far, by execution: srv5:22 reachable from the fleet LAN, and the fleet key materializes through WIF on a runner (http=200, 419 bytes). The admitted principal remains UNOBSERVED. Co-Authored-By: Claude Opus 5 (1M context) <noreply@anthropic.com> * Re-probe with briansrls: I omitted the principal my own evidence names Correct catch against me. My candidate list -- gunbc-fleet, ghrunner, ubuntu, nvidia -- came from naming CONVENTION, not from evidence. briansrls is the principal my own reports point at twice: the Sparks' factory OOBE account, and the principal under which wise-moth measured the fleet key ACCEPTED on srv4. Choosing candidates by convention while holding evidence that names a specific account is how a 'nothing admits the key' verdict gets manufactured. Now probing briansrls, root, dgxuser plus the original four, across BOTH srv5 and srv6. Same read-only construction, same corrected harness -- nothing installed, BatchMode cannot prompt, explicit PRINCIPAL= marker, real exit status. RECONCILIATION OWED regardless of outcome: an earlier report of mine claimed fleet-key login verified end-to-end on both Sparks, key-only. That and 'no principal admits the key' cannot both be true. If briansrls admits it, the earlier claim was probably right and my candidate list was the defect. If everything still refuses, the earlier claim was wrong and I need to establish which observation was mistaken -- that matters more than the unblock, because one of them is a false verification sitting in the record. Co-Authored-By: Claude Opus 5 (1M context) <noreply@anthropic.com> * Canary step 2: observe srv5 serving-relevant facts before pinning anything Step 1 answered its question: briansrls is ADMITTED on srv5 AND srv6 with the fleet key (PRINCIPAL=briansrls UID=1000, read back from the remote process), every other candidate REFUSED-publickey. My earlier "no managed principal admits the key" was a defect in the candidate list, not an observation of the hosts. This step observes what a pinned serving process would be pinned TO: accelerator, driver, container runtime, interpreter, privilege, free space, memory, existing listeners, and any serving process already running. Read-only; installs nothing. Every probe prints an explicit ABSENT marker on absence, because an empty capture is indistinguishable from a failed one -- the exact conflation that produced the false negative in step 1. * Fix empty-vs-absent in the two fields a serving canary relies on; interrogate :11000 `pgrep -af ... | head` left the || arm unreachable because head succeeds regardless, so a no-match and a failed probe printed the same empty string -- in existing_serving and hf_cache, the exact pair used to claim srv5 was not already serving. pgrep's own exit code separates them: 1 is "ran, matched nothing", >1 is "did not run". hf_cache now tests for the directory rather than inferring absence from empty du output. :11000 has an unidentified listener on the host about to run an inference server. Privilege will not name the process, so this asks it what it is over HTTP -- status, body, /v1/models, and the Server header. Silence records as a typed unknown; open-but-not-HTTP, closed, and probe-failed are three different states and none of them is "unrelated". * Probe serving-toolchain feasibility on GB10 aarch64 before the install step :11000 answered: an NVIDIA-branded web UI (HTML, nv-dark), not an OpenAI-compatible endpoint -- /v1/models redirects to /. It is occupied and localhost-bound, so the serving port must not be 11000. The two repaired fields now carry real observations: existing_serving=NONE-MATCHED and hf_cache=ABSENT-no-such-dir, both distinguishable from a failed probe. GB10 is aarch64 Blackwell, where "a serving runtime has an installable build" is a genuine question rather than a formality. Assuming a wheel exists is how an install discovers its blocker halfway through a mutation, so this observes CUDA toolkit, ollama, uv, pip, venv, torch and the resolvable vllm candidate list read-only first. A candidate list is not an install; it only separates "no build exists for this platform" from "a build exists and installing is next". * Canary step 2/3: install pinned vLLM user-unit on srv5, read back the LOADED identity First mutation in this lane. Scope is derived, not preferred: sudo -n NO and the refused docker socket are one cause with two closures (no system unit, no container), so user scope is what remains. venv + vllm==0.26.0, systemd --user unit with lingering, bound to 127.0.0.1:8000 -- 8000 measured free, since :11000 is NVIDIA's own localhost UI. Writes no system unit, no authorized_keys, nothing outside $HOME. THE READBACK IS NOT /v1/models. vLLM echoes the model id it was launched with and the revision is not in that response at all, so comparing it to the pin compares a string against itself -- it would answer identically if a different revision, a stale cache, or an unrelated checkpoint had loaded. That is a receipt supplying its own denominator. The independent fact is produced by the LOADING: the HF loader resolves a revision to hub/models--*/snapshots/<SHA>/ and the serving process mmaps the weight files from it, so the SHA in /proc/<pid>/maps is the path the process actually opened rather than an argument I typed. Open fds and the cache listing are recorded alongside it. /v1/models is still printed, labelled DEPENDENT, for contrast only. If no independent fact is observable the step says READBACK=UNESTABLISHED rather than falling back to the tautology. Three separate facts kept separate, because a resolve is not an install and an install is not a load: pip resolution (already observed), pip install rc, and a standalone import check, so an installed-but-broken build is a named outcome rather than a mysterious startup failure. A GB10 initialization failure is reported as a measured outcome; no substitute serving stack gets selected here. * Derive serving install scope from observed privilege; import HostIdentity from its definer The user-unit scope now exists in the model as a DERIVATION rather than an unexplained choice. srv5 was measured before anything was installed: sudo -n refuses and the container socket refuses the admitted principal. Those are not two independent limits but ONE cause with two consequences, since a system unit and a container both need an escalation this principal lacks. Modeled as derive_install_scope(HostPrivilegeObservation) -> ServiceInstallScope because an unexplained scope invites a later reader to "upgrade" it to a system unit, silently reintroducing the out-of-band human step the workflow path exists to remove. Stated as a consequence, that upgrade is visibly a change to the OBSERVATION: it requires granting privilege first. UserUnitScope carries its ground so the reason travels with the decision, and InstallScopeUnobserved is a separate arm so an unmeasured host cannot default into user scope and read as derived. Also fixes a pre-existing resolution defect this exposed: HostIdentity was used in five places here and imported nowhere, resolving only by pool-membership coincidence -- the Class B bare cross-module reference DESIGN.md records. A narrower single-entry compile fails on it. Now imported from its definer, product.placement_supply, and the module compiles with zero errors of its own. * Register the pip install as a declared scaffold; discriminate derive_install_scope both directions The serving dependency install is now a registered Scaffold with a named dissolution trigger binding to gunbc.host_effect_realize host_effect_apply, rather than debt described in a message. It dissolves when the venv construction, pinned package install, and systemd --user unit write are each a typed effect through host_effect_apply and the YAML carries no shell leaf. The Spark is a runtime-present site already reachable over an authenticated transport, so it has no foreign-executor or pre-runtime-bootstrap excuse for rendering bash. Debt that is only described is debt nobody schedules. derive_install_scope was asserted and compiled but discriminated by nothing, which is the inert mechanism class: wrong in exactly the way it exists to prevent, silently. Three controls, 3/3 PASS by execution against a roster of 3: - observed srv5 facts produce the user-scope arm - no privilege must REFUSE system-unit and container installability - privilege actually SELECTS the arm (sudo -> system, socket admits -> container) The third is load-bearing: the first two both pass on a function hardwired to return user scope, because srv5's observed facts ARE the no-privilege case. Without it the derivation is indistinguishable from a declaration and the function name does the body's work. The loopback bind is recorded as deliberate and explicitly NOT part of this scaffold, so a later reader dissolving the shell does not widen the endpoint with it. * Close endpoint auth, lock the resolved closure, verify loaded revision on EVERY start Three defects the audit surfaced, fixed in the order they constrain srv6. FLOATING TRANSITIVE DEPS. A pinned model and a pinned vllm still leave torch and everything under it resolving to whatever pip picks that day, so two hosts can differ while every pin reported is identical -- latent on one host, a real divergence the moment a second exists. The install now captures the resolved closure from the host that actually works (pip freeze -> constraints.txt) and emits it in the log for srv6 to install against. Replaying onto srv6 first and reconciling after is the one ordering that makes a difference indistinguishable between the day and the host. ENDPOINT AUTHENTICATION. Loopback bind plus the fleet key on the transport is real protection and is why the bind does not move, but "authenticated model" is an ENDPOINT property and the transport cannot stand in for it -- two different threats. vLLM now runs with --api-key. The key is generated on the host into a 0600 EnvironmentFile and never leaves it: not in the repo, not in CI logs, not in argv, not in the unit. Secret Manager is not its home because this principal holds versions.access only and cannot AddVersion. PER-START VERIFICATION. An initial-load readback describes a process that may since have restarted, and Restart=on-failure can return the server on whatever is in cache with nothing distinguishing that from the pinned revision. verify-loaded-revision.sh runs as ExecStartPost on every start, reads the same independent fact (the snapshot path the process actually mmapped, not /v1/models, which would echo the launch argument and pass regardless), and REFUSES -- failing the unit -- when the loaded revision disagrees, is unobservable, or no process exists. * Move verification out of the start path; stop the restart loop; verify cache after 20 SIGKILLs The hardening I added to close the restart gap is what broke the load. verify-loaded-revision.sh ran as ExecStartPost, and with Type=simple systemd bounds ExecStart plus ExecStartPost by TimeoutStartSec (90s default) -- so a verifier waiting for a multi-minute weight load was terminated and the LOADING ENGINE WAS SIGKILLED WITH IT, then restarted into the same race twenty times. A guard that blocks the thing it guards: the mirror of an absorbing fallback, and only visible by execution. I reasoned about what the check would PROVE and not about what it would DO to the lifecycle it was attached to. Verification now runs as its own oneshot unit ordered After= the service, which is the right fix for a principled reason rather than a practical one: inside the start deadline, "load failed" and "verifier timed out" produce the same journal outcome, so the verifier's failure is indistinguishable from its subject's -- a state-space conflation at the lifecycle layer. Ordered after, those two states separate permanently. It still refuses on mismatch, via ExecStopPost stopping the service on nonzero exit. TimeoutStartSec is deliberately NOT raised: removing the ExecStartPost removes the deadline problem, and inflating it would blunt a signal a genuinely hung start should still trip. Prior generations are stopped and reset-failed before the unit is rewritten, so the new definition does not race an old process for the port and the GPU. CACHE INTEGRITY GATE. Twenty starts each SIGKILLed partway through a 14.19 GiB checkpoint read i…
Immutable contract
Starting: implementation merged (#8045,
147ebbc283); acceptance open; live ending standing unobserved.Ending: main advances → every applicable host observes it → hosts reconcile → receipts collected → fleet verdict produced → one host absent during an advance returns and converges with no new commit.
Receipt set: desired revision · timer/service standing per host · host receipt per host · lifecycle observation per required cell · fleet fold · offline catch-up receipt.
Displacement, honestly: acceptance receipt 0/1. It does not advance until #8085's merge run actually executes deploy/advance/collect AND
refs/fleet/desired == #8085 merge revision, the collector's per-host reads, and the fold are observed.Current standing — the observation ladder (updated 2026-08-09, run 31333011288)
ScannedFirstContact: six host keys scanned + enrolled + materialized in-run. Honest name: this proves the key seen at scan equals the key later accepted, NOT that the first key belonged to the intended physical host (no BMC/console cross-check).gunbc-fleetprincipal,ProbeReachable, identity Converged, run 31333011288).gunbc-fleetnot created; typed auth refusals).cijob in progress); items 1/2 unobserved until it executes.Carried in this PR
push_to_main_family_carries_explicit_status_guard.gunbc-fleeton srv1/srv2;PrincipalAllocationPendingfor srv3/srv4.PrerequisiteAbsenttyped refusal + bootstrap receipts, reclassified as drift-to-converge scaffold (line-stop msg_af607b12).In-flight repair batch (operator scan of #8083, 7 items): principal-target refusal (no SSH on unobserved principal), principal verification (
id -un), failure-safe credential trap, private-key-vs-modeled-fingerprint check, host-targeted bootstrap roster, stronger scan/enroll readbacks + ephemeral known_hosts, and in-memory receipt fold (no static transcription as continuing authority).🤖 Generated with Claude Code
https://claude.ai/code/session_0168R31q15ZsPJyik18Roz3y