CENSUS-IMAGE 0A: one seeded-image derivation, a measured identity, and the census envelope grammar

[gunbai-bot]

CENSUS-IMAGE 0A: one seeded-image derivation, a measured identity, and the census envelope grammar

Stage 0A of the Mt. Collins census image (work item adhoc-6b13c807-a40; parent decisions from eager-owl-205 are summarized below).

What recovery found

• Output recovered, source never committed. The earlier census's output is committed (artifacts/bmc/mtcollins1-host-capture.txt, attempt=3, Preserve mtcollins1 hardware-census and boot-path evidence #10965). It ran in the live environment (tmpfs workload, sr0 mounted), so no Curtin recut is needed. Its seed and program were never committed, and the medium's stock volume label shows the ISO wasn't built by any modeled route. The capture program here is therefore a reconstruction from that file's ENVELOPE/SECTION/ARGV/EXIT lines, and it is labelled as one.
• The derivation already exists. A modeled seeded-image derivation was already in the tree: extdeps.provisioning.ubuntu_seeded_install_media{,_remaster}, the xorriso -boot_image any replay that keeps the arm64 El Torito/ESP boot intact (srv4 fleet subsumption + BMC virtual-media install path + modeling cleanup #7027), used by srv3. A standalone CensusImageBuildInput would have been a second authority (§3). Parent ruled (A): generalize it, and rewire srv3 in the same change (root cut, no two representations).

The change

• extdeps.provisioning.ubuntu_seeded_install_media: UbuntuSeededInstallMediaBuildInput. The payload is a parameter (rendered user-data plus NoCloudSeedFiles); the volume id is a parameter; dates are pinned; the stem and builder revision are fields. Every field reaches seeded_install_media_build_key. Identity is measured: UbuntuSeededInstallMediaBuilt { build_key, output_digest, image_name }, where image_name = <stem>-<first 16 hex of the output sha256>.iso.
• …_remaster: seeded_install_media_stage verifies the stock sha256 before building (the upstream digest). It then extracts, writes the seed, patches grub, runs xorriso.Iso.ReplayBuildPinnedDates (-volume_date uuid / all_file_dates, so identical inputs reproduce identical bytes), and reads the digest off the written file. The old sidecar, the input-hash identity and the rebuild-in-place-when-stale path are deleted.
• gunbc.seeded_install_media_publish: publish_seeded_image / resolve_seeded_image.
  • Create-only publish. A hard link fails if the name exists. An existing name with the same digest counts as already published; with a different digest it refuses and replaces nothing.
  • Read-back, then record. The published file is read back, and only then is the derivation record written (build key → digest).
  • Resolution re-measures. Consumers resolve through the record and re-measure the file before trusting it.
  • Two write boundaries. BmcExportBoundary (admitted against the export's ownership requirement) and HostArtifactsDirectory.
• gunbc.machine_intake_boot_image_fetch: the stock fetch's publish is now create-only too (BootImagePresentWithDifferentDigest). boot_image_published_path is the single composition of export dir + name.
• gunbc.machine_intake_host_capture_envelope: one grammar, read in both directions (§4). The stage roster and framing are declared once; the program renderer and the verdict parsers read the same rows.
• gunbc.machine_intake_mtcollins1_census_image:
  • Parameterized by stage roster. The census image is mtcollins1_seeded_image_input(stages, stem, volume_id) applied to the census roster.
  • Seed. Upstream early-commands runs /cdrom/gunbc-census/capture.sh with interactive-sections: ["*"]. If the power-off fails, the unit waits at the installer and no unattended storage step is reachable.
  • Boot configuration. autoinstall "ds=nocloud;s=/cdrom/gunbc-census/" console=ttyAMA0,115200, label GUNBC_MTC1_CENSUS_2404_3.
  • Publish target. mtcollins1_census_image_publish builds from the stock medium in /srv/bmc and publishes into the same export.
• srv3 rewired onto the same derivation. Its NBD serve step and apply entry resolve the image, and an unresolved image refuses the serve step (ActuatorStepImageUnresolved). The install diagnostic compares measured sha256 to measured sha256. Two predicates that had no consumer and existed only over the deleted static path are removed.

Witnesses

• test.claim.mtcollins1_census_image (hermetic, supplied captures):
  • a complete envelope parses on every verdict axis;
  • RED: a workload token that appears only in the ARGV line is not a result;
  • RED: without the seed, the boot reaches the installer and is never census-complete;
  • the envelope is matched against its own machine;
  • the roster follows the brief's order;
  • the program writes every section, then END, then sync, then the halt;
  • the seed runs early and leaves every section interactive;
  • every build input reaches the derivation key (program byte, cmdline, label, both dates, builder revision, an appended stage), and the positive control reproduces it;
  • the image name is the stem plus the measured digest.
• test.claim.mtcollins1_census_image_local_wet (the inhabitance half, local-wet lane, triple-enrolled): the attempt=3 capture parses under the reconstructed grammar; the renderer's program executed under sh parses back, including a nonzero stage exit and shell quoting.
• The affected srv3 / NBD / diagnostic / selection witnesses now take a supplied resolved image. There is a new RED that the serve step serves the resolved path and refuses each unresolved arm.

Declared frontiers (not in this PR)

• 0B/0C. mtcollins1_census_image_publish runs on srv2 through the fleet-converge workflow (host=srv2), building and publishing in the same job so the image never leaves the host that serves it. The wet byte-level controls run there: rebuild twice gives the same digest, and changing one seed byte gives a different digest. Hermetic CI has no stock ISO or xorriso; the derivation key's sensitivity is what is witnessed here.
• 0D / approval loop D: converge materialization, loopback bind, mtcollins1_boot mode #11484.
  • Parser swap. eager-owl-205 will swap approval loop D: converge materialization, loopback bind, mtcollins1_boot mode #11484's literal parser onto gunbc.machine_intake_host_capture_envelope, and the verdict REDs (begin without end, nproc ≠ 160, 160 elsewhere, CE/UE > 0, write failure, media) land in approval loop D: converge materialization, loopback bind, mtcollins1_boot mode #11484's witness.
  • Media check flips with 0D. approval loop D: converge materialization, loopback bind, mtcollins1_boot mode #11484's boot-media check currently expects the stock point release. It must flip to the derived image's label and digest in 0D.
  • Machine name, one source. mtcollins1_census_machine is the envelope's machine name; approval loop D: converge materialization, loopback bind, mtcollins1_boot mode #11484's mtcollins1_boot_machine should import it rather than restate it.
• srv4 (gunbc.bmc_virtual_media srv4_virtual_media_source) names a hand-built ISO by literal path. No srv4 build input exists, so it is outside this derivation rather than a second representation of it. Flagged, not changed.
• Stage roster as a parameter. calm-ibex-117 (HW1-QUAL) adds its runner stage after end by calling mtcollins1_seeded_image_input with an extended roster. That gives a different image with its own digest and name; the signed boot subject binds only the census image.

🤖 Generated with Claude Code

[gunbai-bot]

Addressed review 68602 in 0a3703a:

• The local host_capture_sh_quote is deleted. Stage framing now quotes through extdeps.posix.shell_command_language posix_single_quote, the one authority for the POSIX §2.2.2 encoding. The local-wet round trip (echo 'it'\''s quoted') still exercises it through a real sh.
• Added host_capture_program_shell_emit_dissolution_trigger on the carrier. It names the four string-built renderers and dissolves on bash-emit (General orchestration intent to Bash emit fold over grammar rows #5828 / ROADMAP 6-shell-slice0 / shell-to-intent Phase 2) rendering a stage roster through orchestration emit. This replaces, beside the new program, the install_media_remaster_bash_emit_dissolution_trigger row that was deleted along with the old remaster's shell.
  — sent from sleek-ferret-265

[gunbai-bot]

Addressed review 68615 in 6dd390c:

1. Absent vs unreadable image. Agreed; it was a behavioural defect. resolve_seeded_image now distinguishes three unresolved states with different remedies:

   • SeededImageRecordedImageAbsent: the record exists but its image was removed. publish_seeded_image rebuilds into the free name.
   • SeededImageRecordDiverged: the image holds other bytes. Publish refuses and replaces nothing.
   • SeededImageUnobservable: the image exists but can't be read. Publish refuses and accuses no bytes.

   New claim test.claim.mtcollins1_census_image_local_wet a_removed_image_is_absent_not_diverged_by_real_execution drives the real resolver against a temp dir through absent, then diverged, then resolved (the digest is read off the written file). It is triple-enrolled. The NBD serve step refuses the new arm too, and a RED is added for it.

2. Builder revision. There is no evaluation-time content hash of a declaration closure today, so the revision cannot be derived yet. It is now a declared below-ceiling class: gunbc.recurring_failure_mode.a_hand_bumped_builder_revision_keys_a_derivation (invalid state, harm, ceiling = structurally impossible, capability-grain next-rung trigger plus its discriminating RED), cited on the carrier.

3. Roster narrowing. host_capture_census_stages now carries attempt=3's sections with attempt=3's argv, including dmidecode 16/17, meminfo, lscpu, numa, boot-cmdline and edac-find. dmesg-full is the one omission, declared on the carrier with its reason (the kernel log is already on the ttyAMA0 stream the envelope is written to) and what re-adds it.

4. Runbook. Hard constraint 2 and step 0b's expected output now name srv3_seeded_install_media_input / publish_seeded_image / srv3_seeded_install_media_resolve and the refusal arms, not the deleted symbol.
   — sent from sleek-ferret-265

[gunbai-bot]

Addressed review 68635 in dd1385d. mtcollins1_census_image_publish and mtcollins1_census_image_resolve now carry a declared-frontier annotation on the carrier (DESIGN §3c's second arm), naming each consumer and the trigger:

• publish: consumed by stage 0B/0C, a fleet-converge step run on srv2 (host=srv2, SM/WIF prelude, same job shape as mtcollins1_fetch_boot_image) that builds and publishes inside /srv/bmc and emits the name and digest as its receipt.
• resolve: consumed by stage 0D, gunbc.machine_intake_mtcollins1_boot_run (approval loop D: converge materialization, loopback bind, mtcollins1_boot mode #11484), whose signed boot subject binds the resolved image_name and output_digest in place of the stock ones.

The frontier retires when both land and call these functions. They are not wired in this PR because stage 0B/0C is the job that runs on srv2 hardware, and stage 0D edits #11484, which eager-owl-205 owns.
— sent from sleek-ferret-265

[gunbai-bot]

Addressed review 68664 in 07bd82c:

1. One tty. mtcollins1_census_console_tty ("ttyAMA0") and mtcollins1_census_console_baud are the rows; the device the program exec >s and the kernel's console= argument are both derived from them, so the two spellings cannot be edited apart. The witness asserts the derivation rather than the literals.
2. Dangling fixture. Rather than delete srv3_remote_bmc_probe_observation, I restored its reading as an executing claim: test.claim.srv3_os_install_diagnostic a_remote_bmc_probe_alone_needs_the_serve_observation runs diagnose_srv3_install over that fixture with the new intent fixture and asserts InconclusiveNeedsServeObservation. The deleted production fn had no caller, so this reading never executed; now it does, and the fixture has a consumer.
   — sent from sleek-ferret-265

[gunbai-bot]

Addressed review 68712 in 56655cf. The census witness module now declares its imports explicitly (std.logic / std.types / std.string_type, extdeps.provisioning.ubuntu_seeded_install_media, gunbc.machine_intake_mtcollins1_boot_artifact, gunbc.machine_intake_host_capture_envelope, gunbc.machine_intake_mtcollins1_census_image, v2.std.live_tree), matching every sibling in dag/test/claim/machine_intake/.

On whether it was executing: several witness files elsewhere in the tree carry no import block and are resolved implicitly, which is why it was written that way, and the floor's own refusals on this branch have been AmbiguousBareNameRead (a bare read that reaches two declarers) rather than an unresolved name — so bare reads do resolve. But I cannot show you a green execution of these eight claims yet, and that is the honest state: no witnesses run has completed on this branch. Explicit imports remove the question either way, and they also pre-empt the ambiguity class that has already refused this branch twice in extdeps.tools.coreutils_stat. I will report the executed result when the run on this head finishes.
— sent from sleek-ferret-265

[gunbai-bot]

Addressed review 68762 in 7f9e9da. Both halves were right.

1. Command order. -volume_date all_file_dates applied to the files then in the image, and it was issued before the two -map commands, so the freshly mapped seed directory and grub.cfg would have carried the build instant — the pinning would not have covered the files the derivation is about. Both -volume_date commands now follow the maps, and the operation's exit annotation says the ordering is load-bearing and why.
2. The premise was asserted, not executed. SeededInstallMediaPinnedDates now types it as a bet per §4d: what IS established (without pinning, a rebuild measures a different digest), what is NOT (that pinning these two dates is sufficient; xorriso may carry other build-varying state, and the ordering above is read off the tool's documented execution rather than measured), and what would settle it (the srv2 control: build twice from one input and compare measured digests, then change one seed byte and compare again).

Worth adding explicitly, since it bears on how much weight the bet carries: nothing silently depends on it. If a rebuild is not byte-identical it measures a different digest, so it publishes under a different name; create-only publication replaces nothing and the record names whichever bytes were actually built. The cost of the bet being wrong is a second published image, not a wrong one. That is now in the annotation too.

Executed evidence on this branch, since CI no longer runs the floor (main's #11742 replaced the required witnesses check with cargo build): I built the interpreter locally and ran both census witness files against the merged tree. All 13 claims PASS — 10 hermetic (roster order, halt ordering, the ARGV false-green red, every-input-reaches-the-key with its positive control, digest-derived name) and 3 wet (attempt=3's committed capture parses under the reconstructed grammar; the rendered program executed under sh -c and parsed back including a nonzero-exit stage and an embedded quote; the resolver's absent / diverged / resolved arms against a real directory).
— sent from sleek-ferret-265

[gunbai-bot]

CI note: this PR's heal-generated-artifacts red is inherited from main, not from this change. main's tip #11761 enforces the §4c annotation-attachment rule, and src/v2/workflow/floor_naming_hygiene.dag carries a trailing comment block (authored in #11573) that names no subject, so resolve refuses twelve lines of it. The file here is byte-identical to main's.

Repair is cut as its own one-file PR against a clean main base: #11780. Once it lands I will merge main again here and CI should clear. clippy and compiler are already green on this head.
— sent from sleek-ferret-265

[gunbai-bot]

Both findings fixed at 8cc7dd0 (review 69300). Verified both against the code first; both were real.

HistoricalStageNewlyIntroduced could never be accepted — this was the serious one. Your trace is exact: a newly-introduced stage has no section in the historical capture, so host_capture_bound_section_argv yields Absent and the outer match returned false before that disposition was ever reached, while roster_matches_bindings counted the stage as carried and passed. The first stage anyone added would have gone green on census_roster_matches_bindings and permanently red on the_historical_binding_holds_against_the_committed_capture_by_real_execution — and the wet failure would have read as a capture problem rather than a checker problem, which is the expensive part.

The disposition is now decided before the historical section is demanded. It still has to earn itself: if the capture does carry the section, the row is wrong about history and it fails. So the arm is reachable and discriminating in both directions rather than being a permanent false.

census_historical_capture / HistoricalCaptureIdentity had no reader. Correct, and the sharper half of your finding is the consequence you name — the wet claims bound the capture by path literal, so the row’s stated job, constraining the producer to that committed digest and byte count, was performed by nothing.

I did not delete this one. Unlike historical_binding_names (deleted last round), the job here is real and the brief requires it: the bindings are statements about one artifact, read against a different file they are statements about nothing, and the file sits in a worktree where anything may rewrite it. So the row is now load-bearing — both existing claims read the path from it, and a new claim checks the recorded sha256 and byte count against the bytes on disk. That is also the brief’s digest disagreement → refuse RED, which had no executing claim until now.

The new claim is enrolled on both the wet schedule and the floor route-gap chunk — an unenrolled claim is the defect review 69157 caught here once already.

Evidence: mtcollins1_census_image_local_wet_test — 7 PASS / 0 FAIL. I also checked the new claim is not vacuously green: measured sha256 = a9b880c3…fd2a and 270880 bytes, both matching the committed rows, so it is comparing real values.

— sent from sleek-ferret-265

[briansrls]

SOURCE SIGN-OFF — exact head 9ab8c6f.

This supersedes my REQUEST_CHANGES review 5261076419. The remaining identity wall is closed:

• host_capture_run_identity_same is constructor-aware and total: HistoricalAttemptOrdinal compares only with HistoricalAttemptOrdinal, KernelBootId only with KernelBootId, and cross-constructor pairs are false regardless of equal payload text.
• host_capture_marker_matches consumes that equality rather than projecting both identities to String.
• the_same_text_under_two_identity_forms_is_not_one_run covers both directions (attempt=3 begin / boot-id=3 end, and the reverse).
• the_same_text_under_one_identity_form_is_one_run is the positive control for both identity forms.
• The recorded mutation is discriminating: restoring text-only comparison yields 21/22 with exactly the cross-form claim red while the positive control remains green; the repaired source yields 22/22.

The commit changes only the envelope comparison and the hermetic witness. The previously accepted bounded-envelope, workload pass-status/full-coverage, measure-derived footprint, one-observation-per-case, and 19-stage historical-binding walls remain intact. The 7/7 local-wet receipt is on the parent source; this exact delta does not touch the wet file, and its same-form behavior is directly covered by the new exact-head positive control.

All four exact-head checks are green. No srv2 publication is required here; 0B/0C/0D remain declared frontiers.

APPROVE for enqueue on this exact SHA.