runner_unit: ExitType=main stops a finished incarnation; ExitType=cgroup holds it forever

[gunbai-bot]

Why

lifecycle_intent_cannot_orphan in gunbc.runner_unit treated ExitType=cgroup as the safe setting. It is the opposite: a single surviving process keeps the unit active, with MainPID=0, and no stop is ever issued, so KillMode=control-group never runs.

• srv2-02, 2026-09-16: GitHub cancelled a job at its 180-minute cap. The cancel reached the step's shell but not the claim_executor child under it, and the slot was held for 2.7 days with 16 GB.
• srv1-03 and srv1-07 (census, 2026-09-18): both slots were held for 16 days by docker events and ttyd processes that a successful job left behind. The survivors have been killed and the slots have restarted.

The measurement

On srv1, systemd 255, in transient units (no runner slot touched). Settings: KillMode=control-group, SendSIGKILL=yes, TimeoutStopSec=4s. The main process forks a child and exits 0.

ExitType	child	unit	child after 10s
main	honors TERM	inactive	reaped
main	ignores TERM	failed (SIGKILL)	reaped
cgroup	honors TERM	active running, MainPID=0	alive
cgroup	ignores TERM	active running, MainPID=0	alive

An explicit systemctl stop under ExitType=cgroup reaps the ignoring child in 4s. So ExitType decides whether a stop is issued, and KillMode decides whether that stop reaches every process in the cgroup.

Change

• Predicate: renamed to lifecycle_intent_stops_finished_incarnation. Its input is now main_exits_when_incarnation_ends, replacing listener_is_main_process, which was the wrong question. run.sh exits when a job finishes, except in its return-code-2 relaunch loop, and that gap is named.
• Declared intent: ExitType=main + KillMode=control-group. Note that the host converger will report every slot as not-effective:control-group:cgroup until the new drop-in is installed. That is the intended signal.
• Witnesses: they now refuse both contracts the fleet has actually run, KillMode=process and the ExitType=cgroup drop-in. The incident configuration is a RED in the microVM admission witness.
• Scope of the claim: the predicate is documented as static only. It does not say the processes are gone afterwards or that the cell is safe to reuse. That is a declared frontier: the terminalizer readback feeding product.fabric.sanitation CellReadiness, which microVM admission will consume.

Not in this PR (same program)

• Terminalizer + host-local CellReadiness: stop, then force-stop, recursive readback, and quarantine, gated before admission.
• Cancellation fast path: exec claim_executor in the step, and have it exit cleanly on SIGINT/SIGTERM.
• microVM cutover: jobs still run directly in the runner cgroup on every slot surveyed.

Evidence status

I could not run the witnesses outside CI. The remote runner refused on page thrashing while resolving, so CI's witness lane is the first execution of them.

🤖 Generated with Claude Code