Skip to content

Floor slot class at 41/40 GiB, three per host (B1 of 2): operator-approved stopgap - #12885

Closed
gunbai-bot[bot] wants to merge 13 commits into
mainfrom
session/keen-pike-73
Closed

gunbai-bot[bot] wants to merge 13 commits into
mainfrom
session/keen-pike-73

Conversation

@gunbai-bot

@gunbai-bot gunbai-bot Bot commented Oct 1, 2026

Copy link
Copy Markdown
Contributor

B1 of 2. Floor slot class, at 41/40 GiB, three per host. This is an operator-approved stopgap (2026-10-01, option B by default), carried as a scaffold row.

Why

#12799's floors pin at the fleet slot's 25 GiB memory.high from prepared-subject-warm on, at ~2.7k-module subjects. Runs 36787476687 and 36792656867 ended MemoryStallRefusedPageThrash; 36768748506 was green.

Under B only the floor gets a bigger leaf. Every other slot stays on the 26/25 fleet row.

Retirement trigger (gunbc.runner_slot_desired gunbc_runner_floor_slot_class_stopgap_2026_10_01)

This is a capability: a ~2.7k-module floor subject peaks (cgroup charge) under the original 25 GiB memory.high, measured by //gunbc/instruments:floor-memory-qualification at that subject.

Whatever delivers that retires the class: warm-phase demand, the type_env reduction, or both.

Headroom (per host, derived from the modeled envelope)

host runner envelope width before -> after GitHub slots
srv1 315.9 GiB (52 GiB shakedown charged first) 11 -> 9 8
srv3 465.3 GiB 17 -> 16 14
srv4 465.3 GiB 17 -> 16 14
srv2 unresolved (no runners live; read by neat-boar-16) - 0

The fleet goes from 40 to 36 GitHub slots, with 9 floor slots. Conservation holds by construction: the width charges the class block first (floor_class_then_fleet_width). The slot wall now sums each identity's own row (gunbc_runner_slot_identity_demand_sum), so its red (one slot past the width) is authorable again.

What lands

  • Class row: gunbc_runner_floor_slot_desired. Swap, tasks and CPU are read from the fleet row.
  • Class identities: slot_is_floor_class, fleet indices 1..3 on every host whose width resolves. gunbc_runner_slot_desired_for grants the class row to these slots.
  • Leaf: written as per-instance drop-ins (actions-runner@<slot>.service.d/80-…, 81-…). They go through the existing retirement-ordered cap decision, so the raise waits for surplus slots to retire. They are read back per host in the fleet-converge observer.
  • Floor readiness: the gunbc-floor label drop-in is written only by runner_browser_toolchain_converge, and only when its readback holds. It is revoked when the readback does not hold. A runner whose browser toolchain was never converged cannot become floor-selectable; that is the gap behind the 5 real-Chromium wet refusals (witty-cat-761).
  • Per-slot label check: a class slot conforms with or without the granted label. Anything else diverges.
  • Rung-drop populations updated: microvm_shakedown_slot_row_above_fleet_row (class as a second per-slot arm) and slot_row_pinned_below_demonstrated_demand_unrefused.
  • Regenerated: docs/design-rung-drops.md, srv1/3/4 sudoers (srv1-09, srv3-17, srv4-17 leave with the narrower widths). witnesses.yml is unchanged in B1.

Order (B2 is deliberately separate)

  1. Merge B1.
  2. Run fleet converge FullHost on srv1/3/4. This retires surplus slots, then raises the leaf.
  3. Run runner_browser_toolchain_converge once per host. This grants the labels.
  4. Land B2: the floor job's runs-on gains gunbc-floor, and the browser-toolchain pool follows the floor spec.
  5. Rerun EdgeLabel cut: Authored | StructuralLabel (CoreMarker | ProductionEdge) | Positional, root-first #12799's floor once.

In one PR, the floor would require a label no runner carries until after the converge, and every floor would queue.

Open

  • Privileges: the label write uses install, tee and rm under sudo as the administrator principal over the admin edge. I have not verified those grants live.
  • Witnesses: the core runner modules (slot allocation, microVM controller, floor class) are green locally. The wider witness sweep is still running, and this PR's floor is the authoritative run.

🤖 Generated with Claude Code

@gunbai-bot
gunbai-bot Bot marked this pull request as ready for review October 1, 2026 06:09
@gunbai-bot
gunbai-bot Bot marked this pull request as draft October 1, 2026 07:00
@gunbai-bot

gunbai-bot Bot commented Oct 1, 2026

Copy link
Copy Markdown
Contributor Author

HELD (deep-ferret-305, 2026-10-01): #12890 dropped #12799's floor peak to 25.29 GB, under the 26.84 GB line, which is this stopgap's retirement trigger. If #12890 lands and #12799's floor goes green on the normal fleet leaf, this PR (and B2) close unmerged. It resumes if #12890's differential fails or real floors still thrash. The branch is kept.

@gunbai-bot
gunbai-bot Bot marked this pull request as ready for review October 1, 2026 07:00
@gunbai-bot gunbai-bot Bot changed the title Floor slot class at 41/40 GiB, three per host (B1 of 2): operator-approved stopgap [HELD, do not land] Floor slot class at 41/40 GiB, three per host (B1 of 2): operator-approved stopgap Oct 1, 2026
@gunbai-bot gunbai-bot Bot changed the title [HELD, do not land] Floor slot class at 41/40 GiB, three per host (B1 of 2): operator-approved stopgap Floor slot class at 41/40 GiB, three per host (B1 of 2): operator-approved stopgap Oct 1, 2026
@gunbai-bot

gunbai-bot Bot commented Oct 1, 2026

Copy link
Copy Markdown
Contributor Author

UN-HELD (deep-ferret-305, 2026-10-01): #12890's headroom is about 270 subject modules, about one week of corpus growth, so B lands as the declared next step (neat-boar-16's rule). The retirement trigger stays the capability: a ~2.7k-module floor subject peaking under 25 GiB memory.high, measured by floor-memory-qualification. The type_env PR-2 is the durable fix that would retire it.

Sequencing note with the browser-readiness gate (separate PR, session/keen-pike-73-browser-ready): both write LABELS_JSON. That gate writes a template drop-in (host set + gunbc-browser-ready), while this PR's class drop-in sits in the instance directory and overrides it. Whichever lands second must make the class labels drop-in carry gunbc-browser-ready too. Both are granted by the same ready-gated converge, so the class set is host + gunbc-floor + gunbc-browser-ready.

…iminator through the production width fold; shakedown fixture re-sized)
gunbc-ci-auto-heal added 2 commits October 1, 2026 07:24
@gunbai-bot

gunbai-bot Bot commented Oct 1, 2026

Copy link
Copy Markdown
Contributor Author

B2 patch, preserved for whoever lands B2 after the fleet converge (keen-pike-73 wind-down). Apply on top of B1, then regenerate .github/workflows/witnesses.yml through tools.generated_artifact_gate main_wet_verified (never hand-edit). The floor-class witness that checks only the floor spec carries gunbc-floor belongs with it.

b2.patch
diff --git a/dag/gunbc/ci/ci_runner_target.dag b/dag/gunbc/ci/ci_runner_target.dag
index 55d5e1a9dae..d16b2b7615b 100644
--- a/dag/gunbc/ci/ci_runner_target.dag
+++ b/dag/gunbc/ci/ci_runner_target.dag
@@ -1,10 +1,10 @@
 module gunbc.ci_runner_target
 
 import extdeps.cloud.ubicloud { UbicloudRunnerCatalogRow, ubicloud_standard_16_arm_catalog }
-import extdeps.github.actions { RunnerSpec, SelfHosted, HostedRunner }
+import extdeps.github.actions { RunnerSpec, SelfHosted, HostedRunner, RunsOnExpression }
 import extdeps.github.hosted_runners { GithubHostedRunnerCatalogRow }
 import gunbc.ci_fleet { gunbc_ci_fleet_offer }
-import gunbc.runner_slot_desired { RunnerSlotDesired, gunbc_runner_slot_desired }
+import gunbc.runner_slot_desired { RunnerSlotDesired, gunbc_runner_slot_desired, gunbc_runner_floor_slot_class_label }
 import gunbc.ci_floor_measurement { gunbc_ci_runner_slot_ram_speed_ceiling }
 import gunbc.runner_spec_from_offer { runner_spec_from_offer }
 import std.measure { ByteSize }
@@ -69,3 +69,26 @@ fn selected_ci_runner_target() -> CiRunnerTarget {
 fn gunbc_ci_selected_runner_spec() -> RunnerSpec {
   ci_runner_target_spec(target: selected_ci_runner_target())
 }
+
+// THE FLOOR JOB SELECTS THE FLOOR SLOT CLASS BY ITS LABEL, and only the floor job does (operator-
+// approved stopgap 2026-10-01, gunbc.runner_slot_desired gunbc_runner_floor_slot_class_stopgap_2026_10_01).
+// The spec is the selected target's spec with the class label added, so a switch of target still
+// moves the floor with everything else: on the fleet the floor queues for a class slot; on a cloud
+// target there is no slot class and the label is not added, because a label no runner carries would
+// queue the floor forever rather than route it.
+fn ci_runner_target_floor_spec(target: CiRunnerTarget) -> RunnerSpec {
+  match target {
+    FleetSelfHosted =>
+      match ci_runner_target_spec(target: target) {
+        SelfHosted { labels: labels } => SelfHosted { labels: concat(labels, [gunbc_runner_floor_slot_class_label]) }
+        HostedRunner { label: l } => HostedRunner { label: l }
+        RunsOnExpression { expression: e } => RunsOnExpression { expression: e }
+      }
+    UbicloudRunner { row: _ } => ci_runner_target_spec(target: target)
+    GithubHostedRunner { row: _ } => ci_runner_target_spec(target: target)
+  }
+}
+
+fn gunbc_ci_floor_runner_spec() -> RunnerSpec {
+  ci_runner_target_floor_spec(target: selected_ci_runner_target())
+}
diff --git a/dag/gunbc/runner/runner_browser_toolchain.dag b/dag/gunbc/runner/runner_browser_toolchain.dag
index 2559c5b906b..3d967d635b5 100644
--- a/dag/gunbc/runner/runner_browser_toolchain.dag
+++ b/dag/gunbc/runner/runner_browser_toolchain.dag
@@ -57,8 +57,10 @@ import gunbc.host_effect_realize {
   srv3_ensure_apt, AptPresenceSubject, AptBinaryOnPath, AptPackageInstalled,
 }
 import product.placement_supply { HostIdentity }
-import gunbc.ci_runner_target { gunbc_ci_selected_runner_spec }
-import gunbc.runner_registration_labels { runner_registration_labels, RegistrationLabelsResolved, RegistrationLabelsRefused }
+import gunbc.ci_runner_target { gunbc_ci_floor_runner_spec }
+import gunbc.runner_slot_allocation { host_floor_class_slots }
+import gunbc.build_cache_instance { RunnerSlotIdentity }
+import gunbc.runner_registration_labels { runner_slot_registration_labels, RegistrationLabelsResolved, RegistrationLabelsRefused }
 import gunbc.runner_host_deploy { RunnerHostDeploy, RunnerHostSpec, fleet_runner_host_deploys }
 import gunbc.fleet_posix_accounts { fleet_posix_home_directory }
 import gunbc.fleet_reach_endpoint { fleet_probe_endpoint_for }
@@ -286,7 +288,7 @@ fn runner_browser_toolchain_node_command(
 // ---------------------------------------------------------------------------------------------
 
 // THE POPULATION IS DERIVED, NOT LISTED (operator condition, 2026-09-28). The floor job runs on
-// gunbc.ci_runner_target gunbc_ci_selected_runner_spec; a rostered runner host belongs to its pool
+// gunbc.ci_runner_target gunbc_ci_floor_runner_spec; a rostered runner host belongs to its pool
 // exactly when that host's registration labels (gunbc.runner_registration_labels, the same
 // composition every registration reads) carry every label the spec asks for. So a host that stops
 // registering a floor label leaves the pool, and a new host that registers them joins it, with no
@@ -300,15 +302,26 @@ fn labels_cover(have: List<String>, want: List<String>) -> Bool {
   fold(want, init: true, f: (acc, w) => acc && fold(have, init: false, f: (found, h) => found || h == w))
 }
 
-fn host_runs_floor_labels(deploy: RunnerHostDeploy, floor_labels: List<String>) -> Bool {
-  match runner_registration_labels(host: (deploy.host_label as String) as HostIdentity) {
+// THE FLOOR RUNS ON A FLOOR SLOT CLASS SLOT (gunbc.ci_runner_target gunbc_ci_floor_runner_spec), so a
+// host runs the floor exactly when one of its committed class slots registers every label the floor
+// spec asks for. Asking the HOST's set would never cover the class label, which only class slots
+// carry, and the pool would converge nothing.
+fn slot_runs_floor_labels(slot: RunnerSlotIdentity, floor_labels: List<String>) -> Bool {
+  match runner_slot_registration_labels(slot: slot) {
     RegistrationLabelsResolved { labels: ls } => labels_cover(have: ls, want: floor_labels)
     RegistrationLabelsRefused { host: _, cause: _ } => false
   }
 }
 
+fn host_runs_floor_labels(deploy: RunnerHostDeploy, floor_labels: List<String>) -> Bool {
+  any(
+    host_floor_class_slots(host: (deploy.host_label as String) as HostIdentity),
+    slot => slot_runs_floor_labels(slot: slot, floor_labels: floor_labels)
+  )
+}
+
 fn runner_browser_toolchain_pool_of(deploys: List<RunnerHostDeploy>) -> BrowserToolchainPool {
-  match gunbc_ci_selected_runner_spec() {
+  match gunbc_ci_floor_runner_spec() {
     SelfHosted { labels: floor_labels } =>
       BrowserToolchainPoolDerived {
         hosts: fold(deploys, init: [], f: (acc, d) =>
diff --git a/dag/gunbc/witness/compiler_gate_workflow.dag b/dag/gunbc/witness/compiler_gate_workflow.dag
index bfbd87523d7..2ec206cd06b 100644
--- a/dag/gunbc/witness/compiler_gate_workflow.dag
+++ b/dag/gunbc/witness/compiler_gate_workflow.dag
@@ -120,7 +120,7 @@ import gunbc.witness_floor_workflow {
   required_ci_measurement_scripts_are_renderable,
   WitnessFloorGenerationOutcome, WitnessFloorGenerated, WitnessFloorGenerationRefused
 }
-import gunbc.ci_runner_target { gunbc_ci_selected_runner_spec }
+import gunbc.ci_runner_target { gunbc_ci_selected_runner_spec, gunbc_ci_floor_runner_spec }
 import gunbc.ci_failure_class { wrap_command_with_floor_attempt_receipt, wrap_command_is_renderable }
 import gunbc.ci_spec {
   gunbc_ci_generated_artifact_verify_invoke,
@@ -722,11 +722,16 @@ fn compiler_gate_floor_scripts_are_renderable() -> Bool {
     && required_ci_measurement_scripts_are_renderable()
 }
 
+// THE FLOOR RUNS ON THE FLOOR SLOT CLASS (operator-approved stopgap 2026-10-01): its runs-on carries
+// the gunbc-floor label, so only a class slot at the 41/40 GiB leaf can pick it up, while the
+// generated, unit-test and aggregate jobs keep the plain fleet spec and the fleet row's leaf. The
+// class, its retirement trigger and its width cost are gunbc.runner_slot_desired
+// gunbc_runner_floor_slot_class_stopgap_2026_10_01; floors past the fleet's class slots queue.
 fn compiler_gate_floor_job() -> Job {
   Job {
     id: compiler_gate_floor_job_id,
     name: none,
-    runner: gunbc_ci_selected_runner_spec(),
+    runner: gunbc_ci_floor_runner_spec(),
     environment: none,
     steps: list_map(xs: compiler_gate_floor_bound_steps(), f: fn(b) { b.step }),
     needs: [],

— sent from keen-pike-73

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

None yet

Projects

None yet

Development

Successfully merging this pull request may close these issues.

1 participant