Skip to content

mtcollins1 read-only probes: served-UI crawl + KVM observer-only mode (viewer features seeded; hold released always()) - #12830

Merged
gunbai-bot[bot] merged 13 commits into
mainfrom
session/eager-cat-463
Oct 1, 2026
Merged

gunbai-bot[bot] merged 13 commits into
mainfrom
session/eager-cat-463

Conversation

@gunbai-bot

@gunbai-bot gunbai-bot Bot commented Sep 30, 2026 •

Copy link
Copy Markdown
Contributor

Two read-only fleet-converge modes for Mt. Collins unit 1, split out of #12800 (operator-approved 2026-09-30). Neither mode writes media, power, boot device or SEL. Both have run on the real BMC from this branch: the served-UI crawl in run 36787375313, and the KVM probe in run 36793366493, which logged in, opened /kvm, established, hash-bound one 800x600 still and tore down cleanly.

(1) mtcollins1_ui_bundle_observe: one read-only served-file observation

Root cause of the timeout in wet run 36759368077. The fetch module imported gunbc.machine_intake_mtcollins1_boot_diagnostic_bundle for two IPMI deadlines, which pulled 1019 modules (v2.compiler.*, host_effect_realize, roadmap) into a read-only closure. The cuts:

The crawl. It seeds source.min.js (still compared with the 2026-09-27 digest) and viewer.html. Every further path is parsed out of what was served (<script src>, data-main, ./libs/kvm/* string literals), not hand-listed, and followed to closure within a declared budget (mtcollins1_ui_bundle_file_budget). Each file gets one typed receipt row: path, HTTP status, size, sha256, found-by. Failed GETs, undigested or undecodable files, references to other origins and the budget's edge are rows too. extdeps.tools.gzip (gzip -dcf, GNU manual cited) decodes served gzip for parsing; digests are taken over the bytes as served.

(2) mtcollins1_kvm_observer_observe: the real observer, no boot

It runs under the unit hold, because the KVM seat is single-viewer and the boot refuses its handoff when the seat is busy. It runs mtcollins1_kvm_observer_start, the handoff admission and mtcollins1_kvm_observer_finish: log in, establish, one terminal still, release. The verdict is read from the finish record, and the teardown is judged by typed result words (RS KvmJournalWord rows; it requires exactly released and closed). The journal, the stills and the receipt are uploaded under always().

The route (from #12786, merged in). The session POST, services read and DELETE go through the context's request client, the session-storage keys are seeded by an init script, and viewer.html is the only document loaded. The UI bundle shows why: on an unauthorized answer or session expiry it sets #login and calls location.reload().

What made /kvm open. The served viewer.min.js reads sessionStorage.features.indexOf(...) unguarded in its KVM view's initialize(). The feature list is a committed firmware-0.32 fact, extdeps.bmc.megarac megarac_ui_features_0_32 (73 names). It was EMITTED by gunbc.machine_intake_megarac_ui_features megarac_ui_features_row_emit from the retained source.min.js, which refuses unless the bytes digest to the 5029fae2… pin. The start looks the list up by firmware and refuses to launch if there is no row; the init script seeds it.

Navigation and the page, modeled. Every main-frame navigation is a navigated journal line (seq, phase, url). The viewer's console errors and warnings, and any page error, are page-console / page-error lines.

Hold lifecycle. An always() step runs mtcollins1_kvm_observer_release_wet. It owned-releases this run's observer process, then releases the unit hold ONLY if the holder decodes to KvmObserverProbe { run_id } for this run, at the observed generation (CAS). A free slot, another holder or a stale generation is a reported no-op. DECLARED FRONTIER: runner death or the job backstop runs no always() step. TRIGGER: #12555's observed-dead holder recovery landing on main.

Steps and timeouts

All three steps (UI crawl, KVM probe, release) are bash_build nodes rendered by v2.workflow.bash_emit bash_emit_stmts (review 73415), with no scaffold marker. The credential prelude is the shared bounded Secret Manager read, and the run line is gunbc_run_command. Each step's timeout is a fold of its phases, each counted once and rounded once: the resolve budget (gunbc_ci_gunbc_run_resolve_step_timeout_minutes), gunbc_ci_sm_max_time and the probe's declared bound. That gives UI 25m, KVM 18m, release 16m.

Declared frontier: the KVM closure is still about 990 modules. kvm_still reaches gunbc.runner_browser_toolchain, which imports gunbc.host_effect, a closed union of every fleet host effect. TRIGGER: the host transport interface separated from the effect vocabulary.

Controls

  • Loopback (wet, fleet floor):
    • The root page replaces itself after DOMContentLoaded, and the old in-page login is red.
    • The recorded route is exactly viewer.html.
    • The stand-in viewer reads sessionStorage.features unguarded; a_viewer_without_its_feature_list_never_opens_kvm asserts the page error, the refusal and no kvm connect.
  • Unit claims:
    • Probe verdict flips, including results whose detail contains the success words.
    • Owner-scoped release: foreign holders and stale generation.
    • Journal round-trips for navigated and page events.
    • Feature parse, digest pin, name shape and the committed row.
    • Crawl parsers, the budget edge and the upload glob.
  • Wet witnesses on hardware: runs 36787375313 (crawl) and 36793366493 (KVM probe).

After this lands, #12800 drops its copies (coordinated with fierce-cat-881).

🤖 Generated with Claude Code

Brian Searls and others added 5 commits September 30, 2026 13:55
…ot inside the UI root page

fleet-converge run 36721915217 refused the boot: the observer journalled
"page.evaluate: Execution context was destroyed, most likely because of a
navigation" and released as login-unobserved. It loaded "/" only to borrow an
origin and ran the session POST inside that document, which was replaced
under it.

The session POST, the services read and the session DELETE need a cookie jar,
not a page, so they now go through the browser context's request client. The
sessionStorage keys the viewer reads are seeded by a context init script. The
root page is never loaded; the only in-page evaluations left are on
viewer.html. No retry was added, and every typed refusal and journal word is
unchanged.

The loopback transport's root page now replaces itself on DOMContentLoaded,
and a new wet control asserts the login is seen and the root is never served.

Co-Authored-By: Claude Opus 5.5 (1M context) <noreply@anthropic.com>
…ndle (source.min.js)

Operator decision msg_b3c77f62 (option A on escalation msg_13d5904d): a modeled read-only
route to the firmware's UI bundle, so vendor codes such as cd_error_code are read from
the vendor's source instead of escalated.

- extdeps.bmc.megarac: megarac.Ui.GetServedBundle (readonly, no session, bytes as served)
- gunbc.machine_intake_mtcollins1_ui_bundle_observe: mc info firmware revision first,
  refuse unless 0.32; fetch; sha256; report MATCH or DRIFT against the cited
  2026-09-27 read (never refuses on drift); receipt on every path
- fleet-converge mode mtcollins1_ui_bundle_observe: a mode row on the shared job,
  reusing the fan lane's credential prelude; bytes + receipt uploaded always()
- witness: revision parse, the 0.32/0.33 discriminating pair, unread revision, digest match/drift

Co-Authored-By: Claude Opus 5.5 (1M context) <noreply@anthropic.com>
… compiler

ui_bundle_observe imported the boot diagnostic bundle for two IPMI deadlines,
putting 1019 modules (v2.compiler.*, host_effect_realize, roadmap) in a fetch's
closure; the deadlines are a fact about reaching this unit's BMC, so they move
to gunbc.machine_intake_mtcollins1_access_observation (closure 1019 -> 167).
kvm_still's sol_hold import was dead; sol_hold borrowed the gunbc run step
pipeline's refusal marker and now owns its own (closure 347 -> 96).

Co-Authored-By: Claude Opus 5.5 (1M context) <noreply@anthropic.com>
…e budget and its read bound

The deadlines keep their names in gunbc.machine_intake_mtcollins1_access_observation, the
destination #12800 uses, so the two branches do not fork where those rows live.

Co-Authored-By: Claude Opus 5.5 (1M context) <noreply@anthropic.com>
…al BMC, no boot; the SPA navigation is journalled

- gunbc.machine_intake_mtcollins1_kvm_observer_observe: under the unit hold, log in,
  establish the viewer, one terminal still, release session/browser/process; journal,
  stills and receipt uploaded always(). Verdict read from the finish record.
- kvm_still: every main-frame navigation is a `navigated` journal line (seq, phase, url)
  through KvmJournalWord / kvm_journal_line / the parser; login waits for Playwright's
  networkidle instead of the first DOMContentLoaded (run 36721915217).
- maintenance_hold: KvmObserverProbe owner arm + admitted acquirer (the boot takes the one
  KVM seat under this hold and refuses its handoff when the seat is busy).
- step timeouts: resolve budget + each probe's declared read bound (workflow-side fold).
- loopback control asserts the recorded route; verdict claims over supplied records.
- fleet-converge.yml regenerated.

Co-Authored-By: Claude Opus 5.5 (1M context) <noreply@anthropic.com>

@briansrls briansrls left a comment

Copy link
Copy Markdown
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

HOLD — exact head 46452fdb4f4900c432df4f21e5b4b4c38cb05116

The UI-bundle BMC route is configuration-read-only as claimed: mc info, then an unauthenticated GET /source.min.js only under firmware 0.32; drift is evidence rather than a failed read. The KVM mode also introduces no media, power, boot-device, or SEL write. Four blocking issues remain.

1. The KVM route still puts login inside the navigating root document

This head still does goto(base + '/'), waits for networkidle, and then performs the session POST, sessionStorage writes, and service read through page.evaluate. That is the same unjustified document boundary behind run 36721915217; networkidle is not a proof that the SPA cannot navigate after the wait resolves.

The new loopback assertion proves only that the handler records the observer's own ordinary landing and viewer navigations. It does not reproduce a post-DOMContentLoaded replacement of the root document, so the old route remains green. The already-reviewed repair in #12786 moved POST/services/DELETE to ctx.request, seeded storage through ctx.addInitScript, and never depended on the root document. Preserve that boundary. Navigation observation can be layered around the viewer, or performed on a separate diagnostic page that carries no login operation.

2. Both timeout folds omit the credential prelude

The generated steps execute a Secret Manager fetch with curl --max-time 60 before gunbc run, but fleet_converge_mtcollins1_probe_step_timeout_minutes adds only the 15-minute resolve budget and the probe's BMC/read bound.

Consequently the declared worst-case work does not fit the emitted timeout:

  • UI: 60 s credential fetch + 900 s resolve + 35 s reads = 995 s, but the step is 960 s.
  • KVM: even using the actual 45+15+10+5 s observer path, 60 + 900 + 75 = 1035 s, but the step is 1020 s. The current probe_bound additionally double-counts establishment because mtcollins1_kvm_still_allowance() already includes it.

Derive a typed credential-prelude bound from the prelude authority, combine all post-dispatch bounded work once, round once, and regenerate the workflow. This should make the current shapes at least 17 minutes for UI and 18 for KVM.

3. Teardown success is decided by substring

kvm_probe_teardown_seen accepts string_contains(session_release, "released") and string_contains(browser_close, "closed"). A reachable value such as release-failed detail="already released?" or close-failed detail="browser closed unexpectedly" can therefore satisfy the clean-teardown verdict.

Parse or project the first result token into typed arms and require exact released / closed. Add reds where failure detail contains those words.

4. The unit-hold lifecycle is not closed on abnormal termination or unresolved seat release

On the ordinary clean path, the hold correctly encloses start, establishment, terminal still, session/browser/process teardown, receipt, and then release. But this exact head has no observed-dead-holder recovery: cancellation or a step timeout after acquisition skips unit_hold_release, while the operator release explicitly refuses a KvmObserverProbe holder. That can permanently block the boot lane until the slot is edited out of band.

Conversely, when teardown reports that the web session was not released, the failed outcome is still passed to unit_hold_release, so the interlock is removed without evidence that the one KVM seat is free. Integrate the observed-dead recovery from the hold work before adding this holder kind, and make unresolved seat cleanup retain or transition the hold until a typed observation establishes that the seat no longer conflicts with boot.

The generated artifact and witness lanes were green at the reviewed head; these are source/lifecycle findings, not regeneration drift. Rebind after the navigation-safe request-client route, complete timeout fold, exact teardown classification, and closed hold recovery are present.

…, only viewer.html a document); fold each phase once; typed teardown words

Merges session/swift-raven-18 (#12786): the session POST, services read and DELETE go through
the context request client, sessionStorage is seeded by an init script, and the loopback root
replaces itself on DOMContentLoaded so the old in-page login is red. The navigated journal now
records viewer.html and any #login reload under the held viewer (read from the UI bundle
fetched by run 36778446928).

- step timeouts: resolve + gunbc_ci_sm_max_time + probe bound, in seconds, rounded once; the
  KVM bound is mtcollins1_kvm_still_allowance alone (no double-counted establish wait).
- teardown results: RS KvmJournalWord rows, parsed by leading word into KvmSessionRelease /
  KvmBrowserClose; the verdict needs exactly released/closed; reds whose detail holds the words.

Co-Authored-By: Claude Opus 5.5 (1M context) <noreply@anthropic.com>
Brian Searls and others added 3 commits September 30, 2026 22:23
…s, viewer console journal, served-UI crawl

- maintenance_hold: probe_hold_release_decision / _refused / _committed and the admitted
  mtcollins1_kvm_observer_release_unit_hold: release only KvmObserverProbe{this run} at the
  observed generation; free, foreign or stale is a reported no-op. Reds: foreign holders,
  stale generation. Frontier: runner death / job backstop until #12555 lands on main.
- kvm_observer_observe: mtcollins1_kvm_observer_release_wet (typed owned-process arms), run
  by a new always() step after the probe.
- ci_spec: the ui_bundle, kvm probe and release steps are bash_build nodes rendered by
  bash_emit_stmts (review 73415); no Scaffold, no borrowed fan marker.
- kvm_still: page-console / page-error journal events (viewer never opened /kvm on 36779479938).
- ui_bundle_observe: one read-only served-file observation. Seeds source.min.js and viewer.html;
  further paths parsed from what was served (<script src>, data-main, ./libs/kvm/*), followed
  to closure within a declared 48-file budget; one typed receipt row per file. New
  extdeps.tools.gzip (decode-or-pass-through) so served gzip is parsed.

Co-Authored-By: Claude Opus 5.5 (1M context) <noreply@anthropic.com>
…or UnimportedBareProvider)

Co-Authored-By: Claude Opus 5.5 (1M context) <noreply@anthropic.com>
…mitted from the served source.min.js; gzip imports Unit (floor)

Run 36788605665: the served viewer.min.js reads sessionStorage.features.indexOf(...) unguarded
in its KVM view's initialize(), so with features unset it throws and never opens /kvm.

- extdeps.bmc.megarac: megarac_ui_features_0_32 (73 names), EMITTED by
  gunbc.machine_intake_megarac_ui_features megarac_ui_features_row_emit from crawl run
  36787375313's retained source.min.js, refusing unless it digests to the 2026-09-27 pin;
  megarac_ui_features(firmware) and the vendor storage shape megarac_ui_features_json.
- kvm_still: the start looks the list up by firmware (no row -> not started); the launch writes
  features.json and the init script seeds sessionStorage.features; features-unreadable cause.
- loopback viewer reads sessionStorage.features.indexOf unguarded before /kvm; new wet red
  a_viewer_without_its_feature_list_never_opens_kvm (enrolled).
- extdeps.tools.gzip imports Unit from std.types (floor AmbiguousBareNameRead).

Co-Authored-By: Claude Opus 5.5 (1M context) <noreply@anthropic.com>
@gunbai-bot gunbai-bot Bot changed the title mtcollins1 read-only probes: UI bundle fetch (closure cut 1019->167) + KVM observer-only mode with modeled SPA navigation mtcollins1 read-only probes: served-UI crawl + KVM observer-only mode (viewer features seeded; hold released always()) Oct 1, 2026

@briansrls briansrls left a comment

Copy link
Copy Markdown
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

HOLD — exact head 831d827b7fa99ebec2a3bf7af38a2b4b6d7bba4e

The request-client route, complete timeout fold, typed teardown words, bash-emitted steps, served-file crawl, and firmware-bound feature seeding are accepted. The exact-head hardware run also establishes the clean path: the KVM observer step, always-release step, and artifact upload all completed successfully.

One lifecycle blocker remains from review 5372184063: the unit hold is still released before unresolved viewer cleanup has been ruled out.

On the ordinary path, mtcollins1_kvm_observer_observe_wet passes every result of mtcollins1_kvm_observer_observe_held to generic unit_hold_release. That generic release commits the hold release first and merely preserves the supplied success/failure outcome. Therefore a typed KvmProbeFell caused by release-failed, close-failed, or an unresolved owned process still frees the interlock.

The new always() path has the same ordering problem. mtcollins1_kvm_observer_release_wet does:

  1. owned_process_release(...);
  2. mtcollins1_kvm_observer_release_unit_hold(...);
  3. only afterward computes kvm_probe_process_let_go(process) and returns failure when cleanup was unresolved.

So an OwnedReleaseFailed or unobservable process can leave the function red after the hold has already been released. Likewise, if the ordinary path already freed the hold after a session/browser teardown failure, the always step sees an already-free slot and cannot restore the interlock. A subsequent boot may then run while this run has not established that the single-viewer seat is free.

Bounded repair

  • Do not call generic unit_hold_release for a probe outcome whose typed teardown is unresolved.
  • In the always path, assess kvm_probe_process_let_go before releasing the hold. If the process is not proved gone, leave this run's hold in place and fail.
  • For a process that is gone but whose session/browser teardown record is failed or absent, retain or transition the hold until a typed observation establishes that the BMC viewer seat no longer conflicts with boot.
  • Add discriminating controls proving that OwnedReleaseFailed, release-failed, and close-failed do not free the probe hold.
  • Before landing, integrate the KvmObserverProbe owner with #12555's observed-dead-holder recovery (or land/rebase after that recovery). The declared runner-death frontier is otherwise still a permanent lockout on this exact tree.

The successful hardware execution is valuable clean-path evidence, but it does not exercise this failure ordering. HOLD only on keeping the interlock closed when cleanup is unresolved.

…chat blocker 4a)

Both release paths -- the probe's own exit and the always() step -- now keep the unit hold
unless the web session reads released (or no-session), the browser reads closed (or
no-browser), and the observer process is observed exited (or never launched). login-unobserved,
failed/http-/unrecognised words, an unresolved process or an unreadable journal keep the
hold and say why. Reds for each, including failures whose detail contains the success word.

Co-Authored-By: Claude Opus 5.5 (1M context) <noreply@anthropic.com>
gunbai-bot Bot pushed a commit that referenced this pull request Oct 1, 2026
…-0.32 row (minimum from #12830 for the boot)

Run 36788605665: the served viewer.min.js reads sessionStorage.features.indexOf(...) unguarded in
its KVM view's initialize(), so with features unset it throws and never opens /kvm.

- extdeps.bmc.megarac megarac_ui_features_0_32 (73 names), emitted by
  gunbc.machine_intake_megarac_ui_features megarac_ui_features_row_emit from the retained served
  source.min.js, refused unless it digests to the 2026-09-27 pin (row ported with it);
  megarac_ui_features(firmware) and the vendor storage shape; extdeps.tools.gzip for the decode.
- kvm_still: no row for the firmware -> not started; the launch writes features.json and the init
  script seeds sessionStorage.features; features-unreadable cause.
- loopback viewer reads sessionStorage.features.indexOf unguarded; enrolled wet red
  a_viewer_without_its_feature_list_never_opens_kvm; megarac_ui_features witness.

Co-Authored-By: Claude Opus 5.5 (1M context) <noreply@anthropic.com>
…in's MtCollins1CensusQemuHostObserve and this PR's two probe modes; fleet-converge.yml regenerated

Co-Authored-By: Claude Opus 5.5 (1M context) <noreply@anthropic.com>
Brian Searls and others added 2 commits October 1, 2026 06:55
…e holder is recoverable when observed dead (side-chat blocker 4b)

- mtcollins1_maintenance_hold rebuilt on main's: KvmObserverProbe { run_id, process: ProcessIdentity,
  pid_namespace } like BootRun (tag/detail/label/decode); the acquirer captures self_process_identity
  and refuses if it cannot; holder_liveness_route (pure) routes a probe holder to its process, so
  unit_hold_acquire's observed-dead recovery (#12555) frees a dead probe's hold.
- witness: probe owner renders and routes to its process; a processless one is not observable.
- mtcollins1_boot_dry_realization: 10 observer operands, established only over a seeded feature list.
- observer, loopback transport, rosters: this branch's side (main held their older forms).

Co-Authored-By: Claude Opus 5.5 (1M context) <noreply@anthropic.com>
…laimed derivation (review 73613)

The bundle the list is read from is AMI's proprietary MegaRAC UI, not redistributable and not reachable
from CI, so a CI-diffed gate is unavailable: a declared boundary (DESIGN 4b/4d), not a below-ceiling
choice. The list moves out of extdeps.bmc.megarac (observations are receipts in the observing layer,
DESIGN 3) into gunbc.machine_intake_megarac_ui_features as megarac_ui_features_0_32_receipt: firmware
0.32, bundle sha256 (the pinned row), producing run 36787375313, the names, and TranscribedUncited with
the re-derivation route as read obligation. The emitter already refuses any other digest, so a re-fetch
reproduces the names or refuses. extdeps keeps only the vendor storage shape.

Co-Authored-By: Claude Opus 5.5 (1M context) <noreply@anthropic.com>
@gunbai-bot

gunbai-bot Bot commented Oct 1, 2026

Copy link
Copy Markdown
Contributor Author

Review 73613 (REQUEST_CHANGES) addressed in 26e8192, by the second fix it offered.

The firmware-0.32 feature list is now an observation receipt, not a row that claims to be derived. It is gunbc.machine_intake_megarac_ui_features megarac_ui_features_0_32_receipt, and it carries:

  • firmware 0.32;
  • the bundle sha256 (megarac_source_min_js_sha256_2026_09_27);
  • the producing run, 36787375313;
  • the 73 names;
  • standing TranscribedUncited, whose read obligation names the re-derivation route: the mtcollins1_ui_bundle_observe mode retains the served source.min.js, then megarac_ui_features_row_emit runs over that file.

The row also moved out of extdeps.bmc.megarac. DESIGN §3 places observations this repository makes in the observing layer, so extdeps keeps only the vendor's storage shape (megarac_ui_features_json).

Why there is no CI-diffed gate. The input is AMI's proprietary MegaRAC UI bundle. It is not redistributable, so it cannot be committed, and no CI job can reach it. A gate diffed in CI is therefore unavailable. This is a declared boundary (DESIGN §4b, outside the modeled guarantee; §4d, a typed bet), not a below-ceiling choice. What gates the row mechanically is the emitter: it refuses any bundle whose digest is not the pinned one, so a later live fetch either reproduces these names from those bytes or refuses. It never silently diverges.

A new claim, the_feature_list_is_a_receipt_with_its_read_obligation, checks the digest, the run and the read obligation.

— sent from eager-cat-463

@gunbai-bot
gunbai-bot Bot dismissed stale reviews from briansrls and briansrls October 1, 2026 09:08

Dismissed per operator escalation decision (msg_963ac735, default-approved 'Re-review/dismiss both HOLDs and land'): HOLD was on an older head; head 26e8192 is CI-green and approved (review 73635). — sent from eager-owl-205

@gunbai-bot
gunbai-bot Bot added this pull request to the merge queue Oct 1, 2026
Merged via the queue into main with commit 83355ef Oct 1, 2026
4 checks passed
@gunbai-bot
gunbai-bot Bot deleted the session/eager-cat-463 branch October 1, 2026 09:54
@briansrls
briansrls restored the session/eager-cat-463 branch October 1, 2026 10:01
gunbai-bot Bot pushed a commit that referenced this pull request Oct 1, 2026
#12830 landed the full form of 7168e53's features seeding, so every path that
commit touched takes main's side; the dry realization's 10-operand fix was
identical on both sides. design-rung-drops.md takes main's projection for heal
to regenerate (no row dark by set difference).

Co-Authored-By: Claude Opus 5.5 (1M context) <noreply@anthropic.com>
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

None yet

Projects

None yet

Development

Successfully merging this pull request may close these issues.

1 participant