Skip to content

srv4 fleet subsumption + BMC virtual-media install path + modeling cleanup - #7027

Merged
briansrls merged 67 commits into
mainfrom
session/srv4-onboarding-credential-rotation-gap
Jul 23, 2026
Merged

briansrls merged 67 commits into
mainfrom
session/srv4-onboarding-credential-rotation-gap

Conversation

@briansrls

@briansrls briansrls commented Jul 21, 2026 •

Copy link
Copy Markdown
Contributor

Subsumes srv4 into the fleet and onto GitHub Actions runners, driving it through the modeled .dag workflow, plus the supporting install-path modeling and several dissolved shell/§3 forks found along the way.

Outcome

srv4 is enrolled and serving: 5 GitHub runners live (org gunb-ai), rootless Docker + sccache 0.15.0 — runner-layer parity with srv1/srv2. Subsumption is not claimed complete: host-convergence still carries interim hand-values and every step that was performed by hand is typed, with an acceptance bar and dissolution trigger, in the registered plan gunbc.plans.fleet_subsumption_manual_gaps (srv4's memory row likewise carries a declared observation deferral, the srv1/srv2 pre-observation shape). The plan retires when srv5 subsumes with zero hand-run shell.

Fleet membership (Layer 1)

  • srv4_host + samsung_970_evo_500gb_catalog (its actual drive, cited), LAN endpoint 192.168.1.196, srv4_offer, deployed_intent_v1_srv4. Placement solver now allocates srv4 slots (fleet_concurrent_runs 30→40).

Runner deploy (the width-INCREASE gap, now modeled)

  • runner_host_deploy.dag: RunnerHostDeploy intent citing the ctrl installer (install-actions-runner.sh) as the bound realization handler (§3 cite-upstream, not re-coined). Renders the CTRL_RUNNER_* invocation, the App-key SecretRef, and the actions-runner@srv4-NN enables. srv4: user briansrls, 5 slots (disk-cap note for the 500GB NVMe vs 2TB fleet).

Execution-surface honesty + toolchain provisioning (§5 model-reality gap)

  • fleet_intent_execution_surface no longer lies: container_runtime: Present{Docker} + toolchains: [sccache] instead of none/[].
  • extdeps/cache/sccache.dag: SccacheBinaryRelease (pinned v0.15.0 + per-arch sha256, cited to mozilla/sccache/releases) + install script.
  • extdeps/container/docker_ce.dag: DockerCeAptRepo + packages + rootless-docker apt prereq install, cited to docs.docker.com.

BMC virtual-media install path (used to install srv4 end-to-end)

  • extdeps/storage/nbd.dag, extdeps/linux/usb_gadget.dag (typed ConfigfsOp list, not scattered concats), gunbc/bmc_virtual_media.dag: nbd-server → nbd-client → configfs mass_storage USB gadget. Unified under extdeps/bmc/virtual_media.dag VirtualMediaIntent with the existing nbd-proxy-websocat path (§3 fork dissolved).
  • extdeps/firmware/uefi_shell.dag + gunbc/bmc_netboot_shell_boot.dag: shell-native tftp/execute boot modeling.

Install bugfixes (root-caused live, captured in the model + witnesses)

  • grub.dag: quote kernel cmdline args containing ; (grub command separator) — ds=nocloud;s=... was truncated at ;, dropping the seedfrom, so autoinstall fell to interactive on every attempt. grub_cmdline_arg_render + the seeded-media builders quote it.
  • ubuntu_seeded_install_media_remaster.dag: xorriso -boot_image any replay (was mkisofs, which destroyed the arm64 El Torito/ESP boot structure); instance-id derived from hostname (was hardcoded srv3).

All new modeling lands with green witnesses and dissolution markers on the shell-as-string leaves.

Operational (out of tree)

  • srv4 provisioned via the modeled scripts; App-key in Secret Manager reconciled to the live key (revoked version disabled); temp credentials shredded.

🤖 Generated with Claude Code

briansrls and others added 4 commits July 3, 2026 17:35
…ne (T3/T4)

Rebased onto main: StoragePolicyDirectLayout on seeded ISO autoinstall,
split reconcile modules (core/types/receipt/apply/dry_run/record_approval),
honest freeze scope and printf receipt echo, fail-closed observed_at parse,
ServeReady virtual-media session match, and review-driven witness coverage.

Co-authored-by: Cursor <cursoragent@cursor.com>
Serve receipt echo was concatenated into curl_tail as HTTP_CODE=if test…,
breaking live reconcile observe on srv1. Witness guards the regression.

Co-authored-by: Cursor <cursoragent@cursor.com>
$(curl … || echo 000) had a stray ) breaking bash on live srv1 observe.

Co-authored-by: Cursor <cursoragent@cursor.com>
…d not provisioned

srv4 racked, BMC at FactoryDefault (403 PasswordChangeRequired). The rotation
workflow (bmc_converge_credential_idempotent, wired into host_standup_spine) is
fully modeled and fail-closed, but bottoms out on the srv3-hardcoded
new_altra_onboarding_plan and assumes gcloud is present. Documents 4 gaps
(G1 per-host parameterization [dispositive], G2 gcloud self-provision, G3
operator-token handler, G4 srv4 identity rows) for review before modeling.

Co-Authored-By: Claude Opus 4.8 (1M context) <noreply@anthropic.com>
@cursor

cursor Bot commented Jul 21, 2026

Copy link
Copy Markdown

Bugbot is not enabled for your account, so this pull request was not reviewed.

Enable Bugbot in the Cursor dashboard to get automatic reviews on future PRs.

Brian Searls and others added 21 commits July 21, 2026 20:27
…ovision, operator-token handler, srv4 identity

Full close of the four gaps from the analysis doc (PR #7027), so BMC onboarding
is hands-off per-host instead of srv3-hardcoded:

G1 — per-host parameterization: altra_onboarding_plan(bmc_host, secret_name)
  constructor + srv3_onboarding_plan / srv4_onboarding_plan rows replace the
  srv3-literal new_altra_onboarding_plan. Threaded `plan` through every
  bmc_onboard func; de-nicknamed srv3_gcp_project -> bmc_secrets_gcp_project
  (fleet-wide). Zero-arg per-host entries srv3_converge_credential /
  srv4_converge_credential are what the standup decl_ref + executor invoke.

G2 — gcloud self-provision: modeled gcloud_cli_tool (extdeps/tools/gcloud.dag)
  + package_google_cloud_cli, and bmc_credential_actuator_toolchain_requirement
  (curl + gcloud) mirroring the OS-install toolchain-ensure.

G3 — token de-fork: gunbc.auth.access_token_source with
  AccessTokenSource = GcloudPrintToken | OperatorSuppliedToken{token} and
  resolve_access_token, so an operator-supplied token is a first-class handler
  (drives rotation with no gcloud on the actuator).

G4 — srv4 identity: operator_host_srv4 + srv4_bmc_endpoint (.195) in
  fleet_intent_network.

Verified: full-corpus typecheck clean (850 modules, 0 errors) + 11 witnesses
green by execution across every touched module, incl. a new discriminating
srv4_plan_targets_195_with_srv4_secret and the updated endpoint-count witness.

Co-Authored-By: Claude Opus 4.8 (1M context) <noreply@anthropic.com>
…3/srv4 rotation)

First live execution of the rotation flow (srv3's was never run green) surfaced
that mint_bmc_credential's base64 octets are rejected by OpenBMC password
validation (PropertyValueFormatError). A firmware-policy divergence also showed:
srv4 (newer OpenBMC) accepts alphanumeric, but srv3 (OpenBMC 2.07.00, pwquality,
MinPasswordLength 9 / MaxPasswordLength 20) requires a 4th character class.

Fix: Urandom.ReadPassword — composition-guaranteed generator (>=1 upper/lower/
digit/special from the shell/JSON/basic-auth-safe set _.@#%-); mint_bmc_credential
mints a 16-char such password (within 9-20, accepted by both firmwares). The
fail-closed read-back gate correctly aborted every base64/alnum attempt before
rotating, so no lockout. Both srv3 (secret bmc-srv3-admin v6) and srv4 (v2) are
now live-rotated off factory 0penBmc; orphaned pre-rotation versions destroyed.

Full-corpus typecheck clean (850 modules, 0 errors) + witnesses green.

Co-Authored-By: Claude Opus 4.8 (1M context) <noreply@anthropic.com>
…eys (durable in repo)

SSH-who, the third principal facet alongside POSIX-who (fleet_posix_accounts)
and GCP-who (fleet_operator_gcp_iam_member):

- extdeps/access/ssh.dag: SshPublicKey type + authorized_keys line renderer.
- gunbc/fleet_ssh_access.dag: operator MacBook key (global break-glass, logs in
  as briansrls) + fleet-automation key (machine access). Both PUBLIC keys grounded
  in the repo (public keys aren't secret). fleet_authorized_keys = both, applied
  to every host by breadth.
- fleet_automation_ssh_privkey_secret → SecretRef to Secret Manager
  'fleet-automation-ssh-key' (project gunbai-secrets, v1). The private key's only
  copy lives there; generated 2026-07-21, local copy shredded.
- keys/fleet-ssh-public-keys.txt: plain-text backup of both public keys.

FOLLOW-UP: fleet-automation-ssh-key has no secret-level IAM binding yet (project-
scoped access). Lock to a dedicated automation SA with secretAccessor, mirroring
bmc-assimilator on bmc-srv*-admin.

Co-Authored-By: Claude Opus 4.8 (1M context) <noreply@anthropic.com>
…-closed)

onboarding_secret_id matched plan.rotated_credential and, on the Chained arm,
fabricated a secret id (plan.bmc.host; pre-existing code fabricated a literal) —
a §5 "fabricated plausible output" fallback. Construction-first fix: narrow the
plan field from rotated_credential: CredentialFlow to secret_name: NonEmptyStr,
so the Chained state is unwritable and the function is total (plan.secret_name,
no match, no fallback). Dropped now-unused std.credentials imports.

Verified by execution: srv3/srv4 plan witnesses (now assert secret_name directly)
+ bmc_onboard load, all green. rotated_credential/Chained gone from the corpus.

Co-Authored-By: Claude Opus 4.8 (1M context) <noreply@anthropic.com>
… key

Created service account fleet-automation@gunbai-secrets (least-privilege:
secretAccessor on fleet-automation-ssh-key only, mirroring bmc-assimilator's
scoping). Grounded in the model: fleet_automation_sa_email + SecretOwner record
tying the SA to the private-key secret, so "who owns the private key" is answered
in the repo, not just in GCP. Binding applied out-of-band (provenance recorded);
full WIF SecretAccessGrant modeling is follow-up.

Co-Authored-By: Claude Opus 4.8 (1M context) <noreply@anthropic.com>
Wire the access model into the OS install: UbuntuAutoinstallPayload gains
ssh_authorized_keys (List<SshPublicKey>) + ssh_password_auth; os_install_emit
renders the subiquity ssh section with authorized-keys (operator + automation
public keys) and allow-pw. srv3 payloads set fleet_authorized_keys + allow-pw
false — installed hosts trust the fleet keys and refuse password SSH.

Verified by execution: srv3_os_install_emit witnesses green, incl. a new
discriminating one asserting both key lines present + "allow-pw: false".

Note: identity.password still carries the bootstrap hash; per-host strong
console break-glass credential is part of Step B (per-host install identity).

Co-Authored-By: Claude Opus 4.8 (1M context) <noreply@anthropic.com>
srv4 host identity baked at install time: srv4_autoinstall_identity (hostname
srv4, console break-glass hash; plaintext in Secret Manager host-srv4-console)
+ srv4_ubuntu_autoinstall_on_iso (NoCloudLocal, DirectLayout, fleet SSH keys,
allow-pw false). srv4 seeded-media artifact rows + srv4_seeded_install_media_remaster
mirror srv3, driven by the same install_media_remaster_script builder.

Dry-run verified (typechecks, script generates, ExitSuccess).

Co-Authored-By: Claude Opus 4.8 (1M context) <noreply@anthropic.com>
…FI GUI step)

The manual "enter UEFI setup, enable PXE / set boot" GUI step becomes a grounded,
solvable model:
- extdeps/firmware/uefi_shell.dag: UEFI Shell command surface (bcfg boot
  dump/add/mv/rm, map, reset) cited to the UEFI Shell 2.2 spec, with a renderer
  to the real command text + a script folder.
- gunbc/uefi_boot_config.dag: DesiredBootSource (install media | PXE entry) ->
  bcfg command sequence — the UEFI-shell realization of the same "what to boot"
  intent the Redfish BootSourceOverride path already models (§2, one intent / two
  realizations). srv4_uefi_install_boot targets fs0:\EFI\BOOT\BOOTAA64.EFI.

Verified by execution: witnesses assert the exact bcfg sequence for install-media
boot and for PXE-entry reorder.

Next: SOL send transport (extend serial_console, currently capture-only) to drive
these over obmc-console into srv4's UEFI shell.

Co-Authored-By: Claude Opus 4.8 (1M context) <noreply@anthropic.com>
…r IPMI SOL

Confirmed live first (operator's caution): the ASRock ALTRAD8UD OpenBMC 2.07.00
supports IPMI SOL (ipmitool -I lanplus sol info → Enabled, ADMINISTRATOR, port
623). That capability was unmodeled — grounded it now: BmcCapability gains
CapabilitySerialConsole, added to the 2.07.00 list with a live-probe provenance row.

Transport: extdeps.bmc.serial_console gains SolConsoleTransport::IpmiSol +
SolConsoleSendIntent + sol_console_send_script (ipmitool sol activate with piped
input via IPMI_PASSWORD -E; obmc-console send arm too; RedfishSerialInterface send
fail-closed as read-only).

Runner: gunbc.uefi_shell_over_sol turns the solved bcfg sequence into serial input
(\r-submitted) and a SOL send script; srv4_uefi_boot_config_sol_send_intent targets
.195. So the manual UEFI GUI step is now: solve DesiredBootSource → bcfg → drive
over SOL, fully executable.

Verified by execution: witnesses assert the srv4 send carries the bcfg sequence,
the script uses ipmitool sol activate, and the ASRock 2.07.00 row declares the
serial-console capability.

Co-Authored-By: Claude Opus 4.8 (1M context) <noreply@anthropic.com>
Co-Authored-By: Claude Opus 4.8 (1M context) <noreply@anthropic.com>
…ment

Back up and ground the real shell interaction (not ad-hoc poking):
- Command surface expanded: connect -r (ConnectRecursive), devices (ListDevices),
  ifconfig (list / set dhcp / set static) alongside bcfg/map/reset — the commands
  actually used driving srv4 over SOL, cited to the UEFI Shell 2.2 spec.
- Observable environment types: UefiNetworkInterface (+ UefiMediaState), UefiBootOption,
  UefiDeviceMapping, UefiShellEnvironment — so the interaction is observe->decide->act.
- gunbc.srv4_uefi_observed: srv4's ACTUAL environment captured live over SOL 2026-07-21
  (map -r, bcfg boot dump -v, ifconfig -l): NVMe with Windows Boot Manager + EFI Shell,
  eth0/eth2 media present (eth0 link-local 169.254.0.18), eth1/eth3 disconnected.

Finding that motivated this: UEFI network stack is ALREADY up in srv4's shell (eth0 has
media) — PXE-enable via GUI is unnecessary; ifconfig -s eth0 dhcp reaches the network
directly. Windows-on-disk confirmed (operator OK'd wipe).

Verified by execution: new commands render, srv4 observed env asserts eth0-has-link +
Windows-present.

Co-Authored-By: Claude Opus 4.8 (1M context) <noreply@anthropic.com>
…decide→act)

Generalize the "we're at a UEFI shell, now what?" case into a goal-directed
decision procedure over the observed environment:

- DesiredOutcome = InstalledFleetNode (the goal: wipe + unattended Ubuntu).
- diagnose_boot_install(env) -> BootInstallSituation: pure read of the observed
  UefiShellEnvironment → InstallMediaReady | NetworkReady | NetworkUpNeedsDhcp |
  NoBootSourceAvailable. Fail-closed: no media + no link says so, never pretends.
- decide_boot_install(situation, goal) -> BootInstallAction: goal-directed; refusal
  is a first-class outcome (RefuseNoSource), not a silent no-op.
- boot_install_commands(action) -> List<UefiShellCommand>?: Absent for a refusal —
  a caller cannot extract a "do nothing" sequence and mistake it for progress.

Grounded on srv4's real observed env: diagnoses NetworkUpNeedsDhcp{eth0} (media up,
link-local) → DhcpThenNetworkBoot → ifconfig -s eth0 dhcp. Discriminating fail-closed
control witness: no-media/no-link env → RefuseNoSource → Absent commands.

Verified by execution.

Co-Authored-By: Claude Opus 4.8 (1M context) <noreply@anthropic.com>
… server)

Answering "why srv1?" — it isn't needed at runtime. Model the BMC as the netboot
host: gunbc.bmc_netboot_serve.BmcNetbootServePlan + srv4 instance.
- bmc_netboot_serve_command: busybox httpd -f -p 8080 -h /tmp/netboot (the BMC
  hosts the ~88MB bootstrap: kernel/initrd/grub + a staged static busybox).
- bmc_netboot_grub_cfg: boots /vmlinuz + /initrd with url= at the Ubuntu MIRROR
  (host streams the ~1.5GB bulk directly, never on the BMC) and ds=nocloud-net;s=
  at the BMC's own seed dir.
- bmc_netboot_nocloud_user_data: the served seed = autoinstall_user_data(srv4
  payload) — carries fleet SSH keys + allow-pw:false, same emit as the on-ISO path.

So provisioning is BMC + internet: no central serve host at runtime; srv1's only
role is one-time (cacheable) extraction of the 88MB bootstrap from the ISO.
Scaffold-marked (medium-as-string ssh/httpd glue) like nbd_proxy_serve.

Verified by execution: grub.cfg targets mirror-bulk + BMC-seed, serve cmd is
busybox httpd, served seed carries the fleet keys.

Co-Authored-By: Claude Opus 4.8 (1M context) <noreply@anthropic.com>
…oncat blobs)

Per operator direction — model each piece appropriately in extdeps first, legibly,
instead of hand-built concat strings:

- extdeps/firmware/kernel_cmdline.dag: KernelCmdlineArg = KernelFlag | KernelKeyValue;
  kernel_cmdline_render via join/map (cited to kernel-parameters.rst).
- extdeps/bootloader/grub.dag: GrubConfig / GrubMenuEntry (structured), grub_config_render
  via join — replaces the string-blob grub cmdline pattern (cited to the GRUB manual).
- extdeps/tools/busybox.dag: busybox CliTool + service busybox.Httpd.Serve with
  structured argv transport (the idiomatic form, like curl.Http).
- extdeps/firmware/uefi_http_boot.dag: UefiHttpBootEntry + provisioning variants
  (cited to UEFI 2.10 HTTP Boot).

gunbc.bmc_netboot_serve recomposed to build a GrubConfig + UefiHttpBootEntry from the
plan (no bespoke concat); grub.cfg, http-boot target, and BMC-served nocloud seed all
derive from structured values. Bulk from the Ubuntu mirror, seed from the BMC.

Verified by execution: grub.cfg renders the full structured cmdline, http-boot entry
targets the BMC url, served seed carries the fleet keys.

Co-Authored-By: Claude Opus 4.8 (1M context) <noreply@anthropic.com>
- extdeps/bmc/ssh.dag: service bmc.Ssh (ExecScript + PutFile) over sshpass -e
  (password via SSHPASS env, never argv) — structured argv, mirrors curl.Http.
- gunbc.bmc_netboot_serve: staging modeled as a structured manifest
  (BmcStagedFile / StagedContentSource = InlineText | LocalArtifact | FetchFromUri):
  busybox+kernel+initrd+grub as LocalArtifact, the rendered grub.cfg + nocloud
  user-data/meta-data as InlineText. Separates WHAT must be on the BMC from HOW
  it gets there.

Verified: manifest stages the rendered grub.cfg (ds=nocloud-net) and the seed
(fleet keys) inline, 7 files.

Co-Authored-By: Claude Opus 4.8 (1M context) <noreply@anthropic.com>
… blobs)

extdeps/exec/command.dag: ShellCommand { argv } + CommandTransport = LocalShell |
SshExec { ssh_target }; command_over_transport wraps a command's argv with the
transport prefix; shell_command_render joins to a string. One operation, chosen
transport — not a per-site command blob.

- busybox: service busybox.Httpd removed in favor of busybox_httpd_command ->
  ShellCommand (single authority for the command shape; runs local OR over BMC-ssh
  via the seam, no dual representation).
- bmc.Ssh: ExecScript removed (superseded by SshExec transport); PutFile kept for
  file transfer.
- bmc_netboot_serve: the serve command is busybox_httpd_command over bmc_transport
  (SshExec root@bmc); local and BMC renderings both derive from it.

Discriminating witness serve_command_one_shape_two_transports: the same command
renders "busybox httpd -p 8080 -h /tmp/netboot" locally and the sshpass-wrapped
form over the BMC transport.

Co-Authored-By: Claude Opus 4.8 (1M context) <noreply@anthropic.com>
bmc_netboot_provision: sequences the staged manifest + serve — mkdir + serve go
through the extdeps.exec.command seam (bmc_netboot_run_bmc), artifacts via
bmc.Ssh.PutFile, rendered grub.cfg/seed via Filesystem.Write then PutFile.
process_exit_first_failure collects the first failure (no fabricated success).
srv4_bmc_netboot_provision is the zero-arg entry. bmc.Ssh.PutFile gains a
mock_response for hermetic dry-run. Marked realization scaffold.

Co-Authored-By: Claude Opus 4.8 (1M context) <noreply@anthropic.com>
… by live run)

Live provision revealed the serve invoked the BMC's system busybox (no httpd
applet) instead of our staged static busybox. busybox_httpd_command now takes
busybox_bin; bmc_netboot_serve_command_local passes the staged path
(/tmp/netboot/busybox). Witness updated to the staged-path rendering.

Co-Authored-By: Claude Opus 4.8 (1M context) <noreply@anthropic.com>
…anual)

- extdeps/exec/command: shell_command_render now shell-quotes each arg (handles
  args with spaces like EXTRA_CFLAGS="-march=... -mfloat-abi=..."), robustness fix.
- gunbc/command_runner: run_shell_command / run_shell_commands — generic sequential
  runner over the seam with short-circuit (fold-with-effects, verified).
- extdeps/tools/busybox: busybox_source_1_36_1_url; apt package_gcc_arm_linux_gnueabi.
- gunbc/busybox_bmc_build: BusyboxCrossBuildPlan + busybox_build_commands (fetch →
  extract → defconfig → enable static → disable TC → cross-compile armv5te soft-float
  → install artifact) + busybox_bmc_build_run.

Solves the BMC-httpd wall found live: the AST2500 (armv6, no VFP) SIGILLs on prebuilt
busybox httpd; our conservative-flags build runs clean (verified: httpd serves HTTP 200
on the BMC). busybox_bmc_build_run regenerated the artifact from source end-to-end.

Co-Authored-By: Claude Opus 4.8 (1M context) <noreply@anthropic.com>
…d live)

eth0 has media but its UEFI DHCP falls back to link-local; eth2 leases 192.168.1.196
on the LAN segment with the BMC. Plan boot_interface + witness updated.

Co-Authored-By: Claude Opus 4.8 (1M context) <noreply@anthropic.com>
…eanup

Subsume srv4 into the fleet and get it onto GitHub Actions runners, plus the
supporting install-path modeling and several dissolved shell/§3 forks found
along the way.

Fleet membership (srv4):
- srv4_host + samsung_970_evo_500gb_catalog (its actual drive, cited), LAN
  endpoint 192.168.1.196, srv4_offer, deployed_intent_v1_srv4. Placement solver
  now allocates srv4 runner slots (fleet_concurrent_runs 30->40).

Runner deploy (the width-INCREASE gap, now modeled):
- runner_host_deploy.dag: RunnerHostDeploy intent citing the ctrl installer
  (install-actions-runner.sh) as the bound realization handler (§3 cite-upstream,
  not re-coined). Renders the CTRL_RUNNER_* invocation, the App-key SecretRef,
  and the actions-runner@srv4-NN enables. srv4: user briansrls, 5 slots
  (disk-cap note for the 500GB NVMe vs 2TB fleet).

Execution-surface honesty + toolchain provisioning (§5 model-reality gap):
- fleet_intent_execution_surface no longer lies: container_runtime Present{Docker}
  + toolchains [sccache] instead of none/[].
- extdeps/cache/sccache.dag: SccacheBinaryRelease (pinned v0.15.0 + per-arch
  sha256, cited to mozilla/sccache/releases) + install script.
- extdeps/container/docker_ce.dag: DockerCeAptRepo + packages + rootless-docker
  apt prereq install, cited to docs.docker.com.

BMC virtual-media install path (used to install srv4 end-to-end):
- extdeps/storage/nbd.dag, extdeps/linux/usb_gadget.dag (typed ConfigfsOp list,
  not scattered concats), gunbc/bmc_virtual_media.dag: nbd-server -> nbd-client
  -> configfs mass_storage USB gadget. Unified under extdeps/bmc/virtual_media.dag
  VirtualMediaIntent with the existing nbd-proxy-websocat path (§3 fork dissolved).
- extdeps/firmware/uefi_shell.dag + gunbc/bmc_netboot_shell_boot.dag: shell-native
  tftp/execute boot modeling.

Install bugfixes (root-caused live, captured in the model + witnesses):
- grub.dag: quote kernel cmdline args containing ';' (grub command separator) —
  ds=nocloud;s=... was truncated at ';', dropping the seedfrom, so autoinstall
  fell to interactive. grub_cmdline_arg_render + the seeded-media builders quote it.
- ubuntu_seeded_install_media_remaster.dag: xorriso -boot_image any replay (was
  mkisofs, which destroyed the arm64 El Torito/ESP boot structure); instance-id
  derived from hostname (was hardcoded srv3).

All new modeling lands with witnesses (green) and dissolution markers on the
shell-as-string leaves.

Co-Authored-By: Claude Opus 4.8 (1M context) <noreply@anthropic.com>
@briansrls briansrls changed the title srv4 BMC onboarding: parameterize per-host + gcloud self-provision + operator-token handler srv4 fleet subsumption + BMC virtual-media install path + modeling cleanup Jul 22, 2026
Brian Searls and others added 3 commits July 22, 2026 22:39
…ling backlog)

Modeled plan doc capturing every step of srv4 install+subsumption that was done
by hand — each a modeling gap. Two families: (A) credential handling (every temp
credential file = a missing MaterializedSecret lifecycle; reach-secrets ADC;
SecretRef liveness after the stale App-key 401) and (B) provisioning (runner SLOT
provisioning = the width-INCREASE gap fleet-converge.sh already names, documented
in full: pinned ActionsRunnerRelease + per-slot dir seed + count reconcile, all
hand-unrolled this time; fleet runner-user; fresh-host prereqs; configfs
virtual-media install actuator + media-detach lifecycle). Each item carries
acceptance tier + RED control; the plan's dissolution trigger retires item-by-item,
fully gone when srv5 subsumes with zero hand-run shell.

Co-Authored-By: Claude Opus 4.8 (1M context) <noreply@anthropic.com>
@gunbai-bot

gunbai-bot Bot commented Jul 23, 2026

Copy link
Copy Markdown
Contributor

Addressed (review 41809) — the gunbai-secrets project-ID fork is consolidated onto the single existing authority gunbc.assimilate.bmc_token_federation.bmc_secrets_project_id, on head 75ef068747:

  • bmc_onboard.dag: deleted its bmc_secrets_gcp_project decl, imports bmc_secrets_project_id, all three consumers repointed.
  • fleet_ssh_access.dag: deleted its fleet_ssh_secrets_project decl (the one this PR introduced), consumes bmc_secrets_project_id.
  • runner_host_deploy.dag: it imported the fleet_ssh_access fork transitively — repointed to the authority too, so there's exactly one source for the literal.
  • Now-unused GcpProjectId imports dropped from both files.

git grep bmc_secrets_gcp_project / fleet_ssh_secrets_project is now empty; no import cycle (bmc_token_federation imports none of these). Zero hard diagnostics on the gate-equivalent compile, base advisory count unchanged, and all affected witnesses PASS (bmc_onboard_body, bmc_token_federation, runner_host_deploy, tailscale_acl_phase2).

— sent from valiant-crab-81

@gunbai-bot

gunbai-bot Bot commented Jul 23, 2026

Copy link
Copy Markdown
Contributor

Thanks (review 41822). On the || true observation in the generated fleet-converge.sh bootstrap: agreed it's the softer §5 bar — the swallowed nonzero exits aren't counted, though as you note they sit in the declared pre-runtime bootstrap window and are followed by a loud gunbc converge that reds on a bad pin, so nothing fabricates green. I'm treating the "render each bootstrap step as a typed refusal incrementing host_failed" change as a follow-up in a separate PR rather than landing it here: it means editing the fleet_converge_emit.dag template and regenerating the artifact (re-touching the drift gate), which is orthogonal to this PR's srv4-subsumption scope and better isolated. Per the repo's merge policy (substantive follow-ups land in a new PR), I'd rather not widen this one further.

For the CI failures this PR hit on the prior head (all stale assertions the srv4 enrollment invalidated, not logic bugs): endpoint count 9→10 (srv4 adds a LAN + BMC endpoint), offers 3→4 (srv4_offer), and the srv3 grub-cmdline witness updated to the now-quoted autoinstall "ds=nocloud — the quoting is this PR's own fix for grub truncating the seedfrom at the unquoted ;. All three affected suites pass; head is now 0438ae743d.

— sent from valiant-crab-81

@gunbai-bot

gunbai-bot Bot commented Jul 23, 2026

Copy link
Copy Markdown
Contributor

Follow-up on review 41809 (the gunbai-secrets project-id consolidation): my first pass pointed the three config modules at gunbc.assimilate.bmc_token_federation.bmc_secrets_project_id, which turned out to break the compile-clean gate — surfaced on the merged head 752a9441. Root cause, proven by execution:

bmc_token_federation imports v2.* compiler-orchestration modules (emit_orchestration, bash_orchestration_emit, std.orchestration/collection/diagnostic) for its GHA token-smoke workflow emission. The compile-clean gate compiles the affected .dag closures against a dag-only pool (--target dag), where those v2.* modules don't resolve. Importing bmc_token_federation dragged them into the config modules' dag-scoped closure: gunbc compile --entry dag/gunbc/tools/bmc_onboard.dag --source-root dag (dag-only) → v2.compiler.emit_orchestration not found; adding --source-root src/v2 resolves it. That's why the full-tree compile and the witnesses were green but the scoped gate went red.

Revised fix (head e329b68f): extracted the shared authority into a new lightweight, dependency-free module gunbc.fleet_secrets_config (fleet_secrets_gcp_project) and pointed bmc_onboard, fleet_ssh_access, and runner_host_deploy there. The three config modules now consume one shared authority — no re-minted literal — without coupling to bmc_token_federation's v2 imports. Verified: all three consumer closures compile clean under the dag-only pool, full gate compile has zero hard diagnostics, base advisory count unchanged, witnesses pass.

One honest residual: bmc_token_federation keeps its own bmc_secrets_project_id decl. It can't consume the shared authority in this PR, because editing it (or importing it from a dag-scoped module) puts its own un-resolvable v2.* imports into the gate's dag-only closure. Decoupling bmc_token_federation from the v2.* compiler internals is a pre-existing cross-boundary issue and a better fit for a focused follow-up. Net effect here: the three config-module forks the srv4 work introduced/carried collapse to one lightweight authority.

— sent from valiant-crab-81

briansrls and others added 4 commits July 23, 2026 13:33
…etired; FleetConvergeArtifact deregistered)

Co-Authored-By: Claude Opus 4.8 (1M context) <noreply@anthropic.com>
@gunbai-bot

gunbai-bot Bot commented Jul 23, 2026

Copy link
Copy Markdown
Contributor

Thanks — all three findings from review 41895 are addressed on the current head (8636a93):

1. .github/fleet-converge.sh re-add — correct catch, and now removed. The file was transiently re-added when the WIP auto-committer resolved the modify/delete merge conflict by keeping HEAD's side. Commit 12f87ca accepts main's #7121 deletion; git cat-file -e HEAD:.github/fleet-converge.sh is now empty and FleetConvergeArtifact is absent from the registry, so srv4 enrollment flows through the typed converge_apply path, not the paper transport.

2. uefi_interface_has_link predicate dissolution — agreed. It was a total 2→2 relabel of UefiMediaState. Deleted; iface_dhcp_rank now matches iface.media directly, with the no-link case as the MediaDisconnected => 0 arm, and the witness matches the coproduct inline.

3. boot_install_is_refused predicate dissolution — agreed; it was witness-only. Deleted; the two witnesses now branch on BootInstallAction's RefuseNoSource variant directly.

Both affected witness suites are green by execution (uefi_boot_install_decision, uefi_shell_environment).

— sent from valiant-crab-81

briansrls and others added 7 commits July 23, 2026 15:44
…tered in #7121)

The generated ci.yml auto-heal git-add list is derived from the artifact
registry; #7121 removed FleetConvergeArtifact, so the regenerated list no
longer names .github/fleet-converge.sh. Fixes generated_artifact_drift_gate.

Co-Authored-By: Claude Opus 4.8 (1M context) <noreply@anthropic.com>
…k line (serializer drift)

A bad merge had reverted main's belt-B roadmap row from roadmap_authority.dag;
restored from main. Regenerating with a fresh seed also drops a trailing blank
line the current markdown serializer no longer emits (main's committed copy is
stale, kept green there by the auto-heal job). Fixes generated_artifact_drift_gate.

Co-Authored-By: Claude Opus 4.8 (1M context) <noreply@anthropic.com>
…railing line)

Branch was behind main's v1_compiler_emit_rust.rs (Lane D #7098); merged main
and rebuilt the seed. Emitted-Rust artifacts now match; ROADMAP.md re-regenerated
without main's stale trailing blank line. Fixes generated_artifact_drift_gate.

Co-Authored-By: Claude Opus 4.8 (1M context) <noreply@anthropic.com>
@gunbai-bot

gunbai-bot Bot commented Jul 23, 2026

Copy link
Copy Markdown
Contributor

On review 41905's bmc_staged_file_basename fold-as-last note (dag/gunbc/bmc_netboot_serve.dag:153): agreed the accumulator is inert and it reads as a disguised last. Worth flagging that this exact idiom is already the established one in-tree — gunbc.gitignore_emit.path_basename (dag/gunbc/gitignore_emit.dag:239) is byte-for-byte the same fold(split(...), init: "", f: (_, seg) => seg). So the clean fix per DESIGN §2/§3 is not a local rewrite of this one site but consolidating both onto a single last-element / basename accessor (no such accessor exists in std today). I've deliberately scoped that consolidation out of this srv4 subsumption PR: it's a cross-module refactor touching gitignore_emit, and every extra push here re-runs the full floor against main's currently-active emitter/stage0-partition churn (this PR has already had to chase three emitter/partition merges to keep the generated-artifact drift gate green). Filing it as the honest follow-up rather than destabilizing a green head. No behavior change either way — both are correct last-of-split.

— sent from valiant-crab-81

briansrls and others added 4 commits July 23, 2026 17:05
My PR added ~12 witness test entries; main's witness_entry_eligibility_census
(#7111) fail-closes the floor when a witness entry has no census TSV row
(panic at cli_run.rs:8508). Bumped the declared count 864->876, regenerated
the census TSV + histogram, updated the count assertion. Sync + count witnesses
green by execution; the previously-panicking argv_command_render entry is now
present.

Co-Authored-By: Claude Opus 4.8 (1M context) <noreply@anthropic.com>
Kept branch current with main's witness/census churn; regenerated the eligibility
census TSV+histogram so committed == emit(merged roster) (876 entries, emit's
internal roster==count check green). Generated-artifact drift clean.

Co-Authored-By: Claude Opus 4.8 (1M context) <noreply@anthropic.com>
@briansrls
briansrls merged commit 608749d into main Jul 23, 2026
3 checks passed
@briansrls
briansrls deleted the session/srv4-onboarding-credential-rotation-gap branch July 23, 2026 18:24
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

None yet

Projects

None yet

Development

Successfully merging this pull request may close these issues.

1 participant