Host admission reads occupancy: a held Spark host running its pair worker refuses GPU/memory effects

[gunbai-bot]

Admission gap found 2026-09-26 on srv8: gunbc.spark.host_commitment admit_host_held_by_subject admitted srv8 for V4.1 work because the Group A pair subject holds it. That subject's own V4 pair worker (gunbc-spark-pair-worker.service) was holding ~99.7 GB of the unified pool, leaving 15 GiB; adding a probe on top made the host thrash, and it needed a power cycle. Ownership was read as vacancy. The design was approved by proud-deer-538 before building, with two additions: HostMemoryShort, and occupancy that is not limited to known units.

What changes

• gunbc.compute.host_occupancy (fabric/compute domain): an occupancy reading taken on the host, made of three modeled reads (systemctl is-active per serving unit, nvidia-smi compute processes, /proc/meminfo). A pure admission fold admit_host_occupancy returns one of:

  • HostOccupiedBy{occupant, others, gpu_used}
  • HostMemoryShort{available, need}
  • HostOccupancyNotRead{cause}
  • HostUnoccupied

  Every GPU compute process is an occupant, whether or not a known unit owns it, so a stray container also refuses. A failed read (Unread) refuses and never counts as vacant. The leg that runs the commands is supplied, the same pattern as host_effect_quiescence.

• extdeps.nvidia.system_management_interface: the nvidia.Smi.QueryComputeApps operation (cited upstream), plus a parse in which a withheld per-process figure ([N/A]) is its own arm, not zero. A row that does not parse is Unread, not skipped.

• gunbc.host_operation_exec: new ProcfsReadMeminfo, NvidiaSmiQueryComputeApps and SystemctlStart variants. Every argv is derived from its extdeps declaration and none is re-spelled.

• gunbc.spark.host_occupancy_admission:

  • admit_spark_host_unoccupied reads the realization's pair head and worker units and the two retired units.
  • spark_ctx_argv_run adapts the privileged leg the V4.1 effects already hold.
  • with_spark_pair_vacated is the modeled vacate: a bracket that stops exactly the serving units the reading found active (stop, not disable), re-reads, and runs the body only when the host reads vacant. It then starts exactly those units and reads each back. Restoration is structural, and nobody has to remember it.

• Callers: the V4.1 image build (v41_runtime_image_converge) and the row-store encode (v41_row_store_encode_run) now require both held-by-subject and unoccupied. Each declares its memory need (vllm_build_host_memory_need 48 GiB, v41_row_store_encode_memory_need 16 GiB). Both figures are declared bets with a read obligation, not measurements.

• recurring_failure_mode host_ownership_admitted_as_vacancy: new row. Found at rung 1, now at rung 2. The ceiling is stated, and the next trigger is that every memory/GPU door arrives through this seam.

Checkpoint materialize and row-store readback keep ownership-only admission. They stage or read bytes at rest and declare no memory need.

Declared frontiers (DESIGN §3c)

• with_spark_pair_vacated has no real-host consumer in this PR, by ruling: no real-host vacate or restore here, and srv8 stays as it was hand-left. Its first consumer is the V4.1 cutover of the V4 pair on srv5–srv8 and the V4.1 capacity-measurement serve. That door must also bind the vacate to the pair-serving authority, because stopping a worker degrades the peer head.
• The Engram probe and capacity-measurement serve doors do not exist yet. When they land, they must admit through admit_spark_host_unoccupied.

Evidence

claim_batch on test.claim.spark.host_occupancy_admission_witness: 11/11 PASS. The REDs:

• the pair worker holding 99700 MiB refuses and names the unit;
• an unattributed GPU process refuses;
• a withheld figure still occupies;
• memory short with nothing running refuses;
• an unreachable or unparseable read refuses;
• the vacate refuses on a failed stop, and believes the re-read over the stop's exit code.

The GREEN control is that a vacated host admits. The mutation control (dropping GPU processes from the reading) turns the pair-worker RED and the unattributed-process RED to FAIL while the vacated control stays PASS. The existing v41_row_store_encode_run (5/5) and v41_runtime_image_produce (7/7) witnesses pass over the rewired callers. Each fixture answers only the argv that host_operation_exec derives, so drift in an argv turns the admitting claims red.

🤖 Generated with Claude Code

[gunbai-bot]

Addressed review 71643 in 318639d. gunbc.compute.host_occupancy now reads unit state through extdeps.systemd: parse_systemd_unit_active_state parses the is-active word, and Absent stays unread. Occupancy is an exhaustive match on SystemdUnitActiveState. ServingUnitActive.state carries the enum, and the wire rendering goes through systemd_unit_active_state_wire_label. refreshing is dropped rather than added to the enum, because I have no citation for it. A word the model does not parse refuses as unread, and the new red an_unmodeled_unit_state_refuses_as_unread covers that. 12/12 claims pass on a remote claim_batch run.

— sent from still-seal-656