CI compile-jobs from the model (slice A): derive cargo build parallelism, fail-closed serial, live nextest-gate consumer

[briansrls]

What & why

The rust-test floor node was the CI tentpole. I measured the lever (the original deliverable): the win is cargo nextest run (process-per-test) — 1571s libtest → 341s nextest, 4.6× — NOT floor-node sharding (one 128-core runner; nextest already saturates ~46 cores). That swap landed in #5427.

nextest exposed the next tentpole: the single-threaded compile clamped by CARGO_BUILD_JOBS=1 (set when sccache is active; its trigger is a sccache spawn/pids EAGAIN, not memory). This PR derives the cargo job count from the model instead of the hand-set clamp — the operator's "ban the hand-tuned flag; derive from the dag; compilation fails if there isn't enough information."

This is slice A: the reusable mechanism + its first live consumer (the modeled nextest gate). It is additive and behavior-preserving (fail-closed serial) and mergeable on its own.

This PR (slice A — green by execution)

• std/realization_width.dag: process_bounded_job_count + process_memory_aware_spawn_width — the 3-term min(cores, ⌊0.8·mem÷per_rustc_peak⌋, ⌊pids÷procs_per_job⌋), a literal extension of the floor's memory_aware_spawn_width min-fold (same 0.8 safety + fail-closed-on-zero), adding the pids/process term. + 4 discriminating witnesses.
• gunbc/ci_compile_measurement.dag: CompileJobMeasurement = CompileJobMeasured{per_rustc_peak, procs_per_job} | CompileJobUnmeasured. Committed state is CompileJobUnmeasured → fail-closed by construction (an explicit variant, not a 0/None sentinel).
• gunbc/ci_compile_jobs.dag: the derive → DerivedCompileJobs = CompileJobs{count} | CompileJobsRefuse{reason}. Refuses on Unmeasured rather than guessing. compile_jobs_count_or_serial maps Refuse → serial 1. force_serial models the EAGAIN cold-retry precedence.
• extdeps/rust/cargo_build.dag: typed build_jobs_args slot on Build + Nextest (argv splices it; [] default = backward-compat); spelling helpers cargo_build_jobs_flag/cargo_nextest_build_jobs_flag — extdeps owns the -j/--build-jobs spelling (§3); the count is workflow policy.
• gunbc/operator_fleet.dag: operator_fleet_host_memory_bytes() reuses the single-authority compute_host_ram_bytes_total; operator_fleet_kernel_pid_max.
• tools/rust_gates_ci.dag: run_gates derives the nextest gate's build parallelism via ci_compile_jobs keyed on the committed fleet budget → currently --build-jobs 1 (fail-closed serial while CompileJobUnmeasured; Refuse short-circuits before the budget), passed to the live cargo.Build.Nextest gate. Auto-widens when the committed measurement flips.
• witnesses (new + extended): pids-binds, memory-still-binds, force-serial-overrides, unreadable-budget-fallback, Unmeasured-refuses, flag-spelling, count-or-serial map, live-gate-fail-closed-serial.

Verified: witness aggregates green-by-execution; whole-tree dsl_compile_clean_gate = ExitSuccess. No ci.yml drift (the rust gate runs at CI runtime via gunbc, not emitted).

Follow-on (slice B — separate PR)

The actual ~16m displacement is the pre-gunbc bootstrap build step (cargo build … --bins in load-bearing ci_spec.dag), which can't run the .dag derive in-process. Since the fleet is uncapped (host-measured: memory.max=max everywhere), B bakes an emit-time constant -j N into ci.yml, removes the CARGO_BUILD_JOBS env clamp, and re-derives -j 1 on EAGAIN retry. B carries a §5 budget fix: the emit-time N must divide host RAM by the co-residence count (host RAM is shared once across placement × within-build), not claim the full 128 GiB per build. B is sequenced after quick-ant's cgroup-measurement step + the co-residence-authority lock, and the committed measurement must not flip to Measured until B's divisor lands.

[gunbai-bot]

Addressed in a56ee73 — valid finding, this was the unit-modeling hard-block (flat-scalar Int where ByteSize is the carrier on both sides).

Threaded ByteSize end-to-end:

• ci_compile_jobs / ci_compile_jobs_for: memory_budget_bytes: Int → memory_budget: ByteSize, passed straight into process_memory_aware_spawn_width (no byte_size() re-wrap; dropped the byte_size import).
• rust_gates_ci: rust_gate_build_jobs_args now passes compute_host_ram_bytes_total(host: srv1_host) (a ByteSize) directly.
• operator_fleet: deleted operator_fleet_host_memory_bytes() (existed only to unwrap) and its now-unused imports.
• witnesses: memory_budget call sites pass byte_size(...).

pids_budget stays Int (a dimensionless process count, not a measure). Witness aggregate + whole-tree dsl_compile_clean_gate green by execution.

— sent from sleek-cat-446

[gunbai-bot]

Addressed in fcd3f566b7 — folded the pids budget into the measurement carrier so it's fail-closed-symmetric with the memory term.

You were right that operator_fleet_kernel_pid_max = 4194304 was a guessed host-kernel fact authored as a flat scalar with no measurement-vs-default disposition — while the memory term refuses to guess (CompileJobUnmeasured → refuse), the pids term silently used a default. That asymmetry is exactly the §5 tell.

Fix: CompileJobMeasured now carries pids_budget: Int alongside per_rustc_peak / procs_per_job, so CompileJobUnmeasured refuses on all three unknowns at once — no standalone pids guess survives. Deleted the constant (and the now-unused Int import in operator_fleet). The real fleet pid ceiling will be sourced from quick-ant's measured pids.max in slice B, stamped into the same carrier. Witnesses + direct compile green by execution.

— sent from sleek-cat-446