Skip to content

runner slot census: probe runs as typed argv on every host-effect transport; the rendered bridge retires - #12776

Merged
gunbai-bot[bot] merged 4 commits into
mainfrom
session/cool-bear-308
Sep 30, 2026
Merged

gunbai-bot[bot] merged 4 commits into
mainfrom
session/cool-bear-308

Conversation

@gunbai-bot

@gunbai-bot gunbai-bot Bot commented Sep 30, 2026 •

Copy link
Copy Markdown
Contributor

Sibling to #12579 (usb_gadget) and #12751 (nbd-proxy serve) in the shell→typed-argv lane. Files are disjoint from both except the shared argv_command admit list in extdeps/exec/command.dag.

What this does

observe_slot_population_over_host_effect_transport — the live read behind every runner-slot memory-cap decision, and the last census site still running its probe as a rendered script — now runs its probe as typed argv on every host-effect transport. The serialization was precisely what made the census unreachable over FleetSsh: run_shell_transport's fleet arm refuses a script BODY before any word is examined (its own refusal text says so), so the census the cap flow consumes could never reach a fleet-edge host.

  • One spelling. The probe is held as systemctl_list_all_service_units_command() in extdeps/systemd/systemctl.dag — an ArgvCommand on the PATH-resolved program identity (that row records why: the runner hosts are unobserved and the words begin with the bare name) — and systemctl_list_all_service_units_argv() is derived from it by argv_words. The rendered/typed fork this migration exists to remove cannot reappear; the words are byte-identical to what the census always ran, so this is a carrier swap, not a behavior change. Its other consumers (runner_host_file_converge, observe_slot_population_over_ssh) are untouched.
  • Per-arm dispatch. LocalShell → run_shell_command_capture (command_runner stays the only module that turns an ArgvCommand into a local process); SshShell/FleetSsh → run_typed_argv_transport — one catch-all arm for every non-local transport (EmitArtifactThenThinRun keeps refusing, now as a diagnosed CensusRefused rather than a fabricated success=false).
  • Fail-closed decode, one classifier. The typed-argv outcome → SlotCensus decode lives in the census module itself (slot_census_from_typed_argv_outcome, consolidated there on review — that module already imports the typed-exec records for its own fleet reader observe_slot_population_over_ssh, which now calls the same decode); the local seam's ProcessOutcome is flattened into the classifier's loose fields beside the only fn that holds that seam. Both feed the ONE classifier (slot_census_from_command_outcome); a typed refusal or a nonzero local exit decodes to CensusRefused — unobserved, never an empty population, so a shrink that cannot read a host stops. Cap-authorization logic is untouched.

The dissolution mechanics (two commits, deliberately)

  1. 2e8d0ebbf79 gives the rendered serialization its own declaration (observe_slot_population_census_script_body) and binds its disappearance with a std.dissolution DeclarationRetires trigger whose ref names a declaration that exists today — a row no declaration names cannot be retired by name, so the migration's retirement is a mechanism event, not a paragraph.
  2. 3c75e02dbd0 swaps the carrier and deletes fn+trigger together — a trigger citing an absent declaration is the dead vocabulary this repository removes; what remains is the counted removal, one row off the §5.E RetainedShellScript frontier, recorded as a receipt comment beside the census fn.

Witness

dag/test/claim/runner/runner_slot_census_typed_argv_witness_test.dag, 7 rows: words-derivation parity (reds if the command and the word list are ever edited apart), the two remote portability walls the census words cross as argv (the gates a script body never reached), and all four decode arms against constructed fixtures (converged fleet-shaped bytes → CensusDerived holding the fixture's slot row; typed refusal → CensusRefused with the transport cause; local refusal → the classifier's refused arm; local output → the same census the typed arms reach). Fixtures construct SshSessionExecResult directly, the srv3 unobservable-probe precedent; a FleetSsh execution context stays unforgeable/unfixturable by its KnownHostsAnchor seal, so the fleet arm's reachability is carried by the portability rows.

Review-driven revision (3295a6670ca)

review 73213 was right that the branch's first cut duplicated a decode: gunbc.runner_slot_population_census already imported the typed-exec records and already matched TypedArgvExecOutcome inside observe_slot_population_over_ssh — the "census module names no carrier" premise in the added comment was false, and two refusal prefixes is two places for one of them to render an unreadable host as an empty population. The fix moves the decode into the census module as the single slot_census_from_typed_argv_outcome (keeping that module's existing "fleet edge" refusal prefix verbatim, so the corpus fn's observable cause is byte-identical), has the fleet reader and the realizer's arms all call it, collapses the three identical remote arms to one catch-all arm (corpus precedent for _ arms: provider_standing.dag, build_cache_endpoint_observation.dag), deletes the false justification, and corrects the witness and this description (including the fleet-reach claim above — the fleet reader existed but was uncalled; what could not reach a fleet host was the cap flow's census path). The local ProcessOutcome flatten stays beside the fn that holds that seam: its record is not one the census module imports, and consolidating it would add an extdeps.shell.exec edge to that module — say the word if you want it consolidated too.

Verification — closed in CI

All seven witness rows executed and passed in the floor's changed-witness fold on this head (bc9a97be5fb, run 36731533741): words-derivation parity, both remote portability walls, and all four decode arms — outcome=passed each, zero non-pass outcomes in the fold — and parse OK 7126 file(s) parse-clean corpus-wide. The earlier session-long blocker (13 BuildBuddy dispatches OOM-killed at the ~5.9 GB pool load between 05:34Z and 14:55Z, builds themselves green throughout) is superseded: CI ran what the local runner could not.

gunbc-ci-auto-heal added 3 commits September 30, 2026 12:54
…ement (argv dissolution step 1)

The runner slot population census is the last census site still running its
probe as a rendered script: observe_slot_population_over_host_effect_transport
serializes typed systemctl argv to shell text and hands the BODY to
run_shell_transport, whose FleetSsh arm refuses a script body outright -- so
the census is unreachable over the fleet transport at all (docs/plans/
shell-to-dag-residual-census-and-arc-completion.md 0c: argv is never the
authority; 5.E: RetainedShellScript is the counted frontier).

A row that no declaration names cannot be retired by name. This commit gives
the rendering its own declaration, observe_slot_population_census_script_body,
and binds its disappearance with
observe_slot_population_census_script_retirement_trigger, a std.dissolution
DeclarationRetires row whose ref names a declaration that exists today -- so
the binding is verifiable the moment it is written. The next commit swaps the
carrier to typed argv and deletes fn and trigger together: a trigger citing an
absent declaration is the dead vocabulary this repository removes.

Witness: dag/test/claim/runner/runner_slot_census_typed_argv_witness_test.dag
asserts the obligation is bound and that the census words already cross both
remote portability walls (fleet portable remote words; typed-argv ssh
allowlist) -- the gates a script body never reached.
…transport; retire the rendered bridge

The migration the previous commit bound: observe_slot_population_over_host_effect_transport no
longer serializes the census probe to a shell script body. The probe's single spelling is now
extdeps.systemd.systemctl systemctl_list_all_service_units_command (an ArgvCommand on the
path-resolved program identity, for the reason that row records: the runner hosts are unobserved
and the words begin with the bare name), and systemctl_list_all_service_units_argv is DERIVED from
it by argv_words -- one spelling, one flattening, so the rendered/typed fork cannot reappear. The
census fn dispatches per transport: run_shell_command_capture on the local seam (command_runner
stays the only module that turns an ArgvCommand into a local process), run_typed_argv_transport on
SshShell and FleetSsh -- the arms a script BODY could never reach, since the fleet wall refuses a
script body before any word is examined. Two thin adapters in host_effect_realize flatten each
transport's observation record into the census module's ONE classifier
(slot_census_from_command_outcome); a typed refusal or a nonzero local exit decodes to
CensusRefused -- unobserved, never an empty population, so a shrink that cannot read a host stops.
EmitArtifactThenThinRun keeps refusing (typed mismatch), now as a diagnosed census refusal rather
than a fabricated success=false observation.

The rendered bridge and its std.dissolution retirement trigger deleted together, as the trigger's
own terms required -- a trigger citing an absent declaration is the dead vocabulary this repository
removes. The receipt stands beside the census fn: one RetainedShellScript row off the section 5.E
counted frontier, and a census that is reachable over FleetSsh for the first time.

The witness keeps the two portability-wall rows, adds the words-are-derived-from-the-command row,
and exercises every decode arm against constructed fixtures (converged fleet-shaped bytes, typed
refusal, local refusal, local output) -- the srv3 unobservable-probe precedent for constructing an
SshSessionExecResult a test can hold. Cap-authorization logic is untouched: this is a carrier swap,
not a policy change.
review 73213 on #12776: the typed-argv -> SlotCensus adapter this branch added
to host_effect_realize duplicated the match already living in
gunbc.runner_slot_population_census (observe_slot_population_over_ssh), with a
second refusal prefix -- two places for one of them to render an unreadable
host as an empty population, against DESIGN 2/3. The premise in the added
comment (census module names no carrier) was false: that module already
imports the typed-exec records for its own fleet reader.

- slot_census_from_typed_argv_outcome now lives in the census module and is
  the only decode of a typed-argv outcome as a census; its refused arm keeps
  the module's existing 'fleet edge' prefix verbatim, so the corpus fn's
  observable cause is unchanged and nothing forks.
- observe_slot_population_over_ssh calls the adapter instead of matching
  inline; it adds no judgment of its own.
- the realizer's three identical remote arms collapse to one catch-all arm
  through run_typed_argv_transport (corpus precedent for _ arms:
  provider_standing.dag, build_cache_endpoint_observation.dag); the local
  ProcessOutcome flatten stays beside the fn that holds that seam, its record
  being one the census module does not import.
- the false 'names no carrier' justification is deleted; the census fn comment
  now states the actual ownership (judgment + typed decode in the census
  module, transport binding here).
- witness: adapter imported from the census module, refusal-prefix assertion
  moved to the unified string, prose corrected (incl. the fleet-reach claim,
  now scoped to the census the cap flow consumes).
@gunbai-bot

gunbai-bot Bot commented Sep 30, 2026

Copy link
Copy Markdown
Contributor Author

Confirmed against the code and fixed in 3295a6670ca — every claim in review 73213 checked out:

  • The duplicate was real. gunbc.runner_slot_population_census already imports gunbc.typed_argv_exec { typed_argv_exec_over_fleet_ssh, TypedArgvExecConverged, TypedArgvExecRefused } (module line 15) and already matched TypedArgvExecOutcome inside observe_slot_population_over_ssh (line 263-278, refused arm at 269 with the "fleet edge" prefix). The premise in my justification comment — "the census module names no carrier, so it cannot hold these records" — was false; I had over-applied that module's header sentence (about the census verdict naming no carrier, to keep classifier imports one-way) to the record types. Both DESIGN.md §2 ("net concepts must not grow by re-invention") and §3 ("each fact lives in exactly one place") quotes verify verbatim.
  • The fix, as prescribed: slot_census_from_typed_argv_outcome now lives in the census module as the only decode of a typed-argv outcome as a census; observe_slot_population_over_ssh calls it instead of matching inline; the realizer's three identical remote arms are collapsed to one catch-all _ => arm through run_typed_argv_transport + the shared decode (catch-all precedent: provider_standing.dag:459, build_cache_endpoint_observation.dag:82); the false comment is deleted and replaced with the actual ownership (judgment + typed decode in the census module, transport binding in the realizer).
  • Behavior preservation: the unified decode keeps the census module's existing "runner slot population unreadable over the fleet edge: " prefix verbatim, so the corpus fn's observable cause string is byte-identical and no third string was minted; nothing in the corpus asserts either prefix (checked — the existing census witness discards cause). My divergent prefix is gone.
  • The local flatten stays. slot_census_from_process_outcome remains beside the fn that holds the local seam: ProcessOutcome is declared in extdeps.shell.exec (not this module), the census module does not import it, and consolidating would add a new extdeps.shell.exec edge to the census module. The review did not flag it — say the word if you want it consolidated too.
  • Also corrected: the witness (adapter now imported from the census module; refusal-prefix assertion moved to the unified string) and the PR description, which overclaimed in the other direction — "a fleet-edge host could never be censused" was wrong, since observe_slot_population_over_ssh predates this PR (uncalled); what could not reach a fleet host was the cap flow's census path. Both now say precisely that.

Net −16 lines. Witness re-verification on the new head is running now; the runner OOM situation from the description's caveat is unchanged so far (11 kills), results will be posted when a dispatch survives.

CI witnesses/floor on 3295a66 failed structural: 4x 'source annotation
sits inside a declaration body. Only module-item grain is modeled' at
host_effect_realize.dag:4194-4197 -- the collapsed-arm comment sat between
match arms. Its content moves into the fn's module-grain comment block; the
match itself is now comment-free. The floor's InterfaceConsumerPlanningUnavailable
refusal was downstream (no declaration index lent while parse failed), not a
separate defect.
@gunbai-bot

gunbai-bot Bot commented Sep 30, 2026

Copy link
Copy Markdown
Contributor Author

CI witnesses/floor on 3295a6670ca failed structural — my doing, found and fixed:

  • Cause (4 parse errors): the comment I placed between the collapsed match arms — dag models comments at module-item grain only: source annotation sits inside a declaration body at host_effect_realize.dag:4194-4197. The floor's InterfaceConsumerPlanningUnavailable refusal was downstream of that (no declaration index is lent while parse fails), not a separate defect.
  • Fix (bc9a97be5fb): the comment's content moved into the fn's module-grain comment block; the match itself is now comment-free. A sweep of every comment line this branch adds confirms all others sit between top-level declarations.
  • The floor re-runs on this push. Meanwhile the witness-execution situation from the caveat is unchanged: the BuildBuddy runner OOM-killed the pool load again (13th consecutive, rc=137, build itself green at 5m35s) — the witness's 7 rows remain unexecuted in this session, to be run by CI or a reviewer with ~6 GB available.

@gunbai-bot

gunbai-bot Bot commented Sep 30, 2026

Copy link
Copy Markdown
Contributor Author

Verification is now closed — all seven witness rows executed and passed in CI. The floor's changed-witness fold on this head (bc9a97be5fb, run 36731533741) planned and evaluated every row of runner_slot_census_typed_argv_witness_test.dag — outcome=passed for all seven (the_census_words_are_derived_from_the_census_command, both portability walls, and all four decode arms) — with zero non-pass outcomes in the fold, and the corpus-wide parse is clean (parse OK 7126 file(s) parse-clean). The 13 OOM-killed local dispatches described in the caveat below are superseded: the caveat's request to execute the witness before merge is satisfied by CI itself. (The fast witnesses check is a lane gate asserting floor+generated success, not the witness run — the floor is where the rows executed.)

On the wording remark ("unreadable over the fleet edge" now also serving the SshShell arm): accurate, and it is the trade-off documented in the consolidation reply — the census module's existing string was kept verbatim so the corpus fn's observable cause stayed byte-identical and no third string was minted. The follow-up shape, if wanted: give the adapter the edge noun as a parameter (the fleet reader passes "fleet edge" unchanged, the realizer's catch-all passes "the transport") and update the witness's prefix assertion — one small commit. I have deliberately not pushed it: the approval and the green floor are pinned to this head, and a wording-only push would re-open CI and stale the review for text that the refused reason itself disambiguates — the reason suffix that follows the noun names the concrete endpoint or the offending word for both ssh seams. Say the word and I'll do it post-merge, or now if you'd rather.

— sent from cool-bear-308

@gunbai-bot
gunbai-bot Bot added this pull request to the merge queue Sep 30, 2026
Merged via the queue into main with commit 4e14f03 Sep 30, 2026
4 checks passed
@gunbai-bot
gunbai-bot Bot deleted the session/cool-bear-308 branch September 30, 2026 18:02
briansrls pushed a commit that referenced this pull request Oct 1, 2026
…egenerate docs/design-rung-drops.md

Co-Authored-By: Claude Opus 5.5 (1M context) <noreply@anthropic.com>
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

None yet

Projects

None yet

Development

Successfully merging this pull request may close these issues.

0 participants