srvN build-cache: the sccache unit still self-exits — idle-timeout fix + DaemonUnsupervised refusal (follow-up to #7206)

dag/extdeps/cache/sccache.dag

@@ -150,6 +150,30 @@ data sccache_misconfigured_log_fragments: List<String> = [
   sccache_failed_to_execute_compile_log_fragment,
 ]
 
+data sccache_server_lifetime_note: String = "sccache's server EXITS ON ITS OWN. Three environment knobs, all documented by the module's cited authority extdeps_external_authority_anchor (github.com/mozilla/sccache/blob/main/docs/Configuration.md), decide the process lifetime a supervisor sees, and they are modeled here because they are facts about SCCACHE — not about systemd, and not about gunbc. Before 2026-07-25 the P1b unit emitter spelled SCCACHE_START_SERVER=1 / SCCACHE_NO_DAEMON=1 as bare literals of its own, which made gunbc.host_build_cache_provision_script a SECOND authority for sccache's environment vocabulary (§3) — and the knob that was missing from that second authority is exactly the one that broke: SCCACHE_IDLE_TIMEOUT. Cited text: 'how long the local daemon process waits for more client requests before exiting, in seconds. Set to 0 to run sccache permanently.' The default is 600 seconds, confirmed by execution rather than read off the doc: on srv3 2026-07-25 the foreground-supervised unit started at 06:28:47 UTC, served no compile request, and EXITED 0 after `Duration: 10min 8.215s`, leaving `inactive (dead)` — Restart=on-failure could not fire because the exit was a success. Consequence for any supervisor: a Type=simple unit whose ExecStart is sccache is only durable when the idle timeout is disabled; otherwise it dies on the first quiet ten minutes and the host reverts to sccache's own unsupervised per-invocation auto-start (the very no-op state the foreground form was introduced to remove). Whether the srv1/srv2 production ctrl-sccache.service sets this knob is UNVERIFIED — neither host is reachable from a session container today, and the only in-tree description of that unit (#7206's note) lists SCCACHE_START_SERVER and SCCACHE_NO_DAEMON and no idle-timeout line, which is an absence in a summary, not evidence of absence in the file. What its month-plus uptime does NOT establish is that the unit is durable: those hosts run CI continuously, so traffic keeps the daemon below the idle threshold whether or not the timer is disabled — the uptime is consistent with both. This row therefore stands on the cited doc plus the srv3 receipt as a DEDUCTION; it is explicitly NOT claimed as a swap toward a production form nobody has read."
+
+data sccache_start_server_env_var: String = "SCCACHE_START_SERVER"
+
+data sccache_no_daemon_env_var: String = "SCCACHE_NO_DAEMON"
+
+data sccache_idle_timeout_env_var: String = "SCCACHE_IDLE_TIMEOUT"
+
+data sccache_idle_timeout_never_value_note: String = "The value whose MEANING is 'never exit', which is why it is a named row rather than a literal inside the assignment: '0' is not an ordinary duration on this axis (a zero-second idle timeout would mean the opposite), it is the cited doc's sentinel for running permanently. A future consumer that wants a bounded lifetime supplies its own seconds value; a consumer that wants a supervised daemon reads THIS row."
+
+data sccache_idle_timeout_never_value: String = "0"
+
+fn sccache_start_server_env_assignment() -> String {
+  join([sccache_start_server_env_var, "=1"], "")
+}
+
+fn sccache_no_daemon_env_assignment() -> String {
+  join([sccache_no_daemon_env_var, "=1"], "")
+}
+
+fn sccache_idle_timeout_never_env_assignment() -> String {
+  join([sccache_idle_timeout_env_var, "=", sccache_idle_timeout_never_value], "")
+}
+
 data sccache_local_facts: CacheInterfaceCatalogFacts = {
   identity: sccache_local_id
   backing_surface: "sccache-local-disk"

dag/gunbc/host_build_cache_provision.dag

@@ -39,12 +39,15 @@ data ci_sccache_opportunistic_detect_scaffold: Disposition = Scaffold {
 }
 data build_cache_path_authority_observation_note: String = "CI and ctrl-build sessions consume the cache as bare `sccache` on PATH (RUSTC_WRAPPER=sccache), so provisioning the modeled binary is NOT sufficient — the PATH-visible sccache must BE the modeled one. A second sccache earlier in PATH (a distro package, a stale copy) is precisely the two-authority state this lane exists to remove, and it is invisible to a read-back that probes the absolute path: that read-back greens on the binary we provisioned while the compiler calls a different one. DaemonPathShadowed carries BOTH paths so the refusal is located (which binary won, which one was expected) rather than merely typed. This judgement is modeled HERE, in the verdict fold (proven by design_witness_path_shadow_refuses_fail_closed against a synthetic DaemonPathShadowed observation), not decided by the provisioning shell. Its LIVE observation is STAGED, not emitted: per the shell→dag census this module's script side is an A5 `decompose` row that must not grow its emitted shell surface, so the `command -v sccache` resolve waits for the typed-argv dissolve-on (#5828) rather than landing as hand-shell now (review 42343). The model carries the state and its refusal today; only the live producer is deferred."
 
+data build_cache_supervision_observation_note: String = "A stats read-back CANNOT see whether the daemon it just talked to is supervised — and this is not a gap in the probe, it is the probe's semantics: any sccache invocation AUTO-STARTS a server if none is listening, so `--show-stats` manufactures the very evidence it reports. On srv3 2026-07-25 it printed a full stats table while the modeled unit had been `inactive (dead)` for six hours and no sccache process existed beforehand. Read through the verdict fold as it stood, that host observed DaemonStatsOk and therefore converged: the provisioning act would have reported success over a host with NO durable cache — a fabricated plausible output (§5), and the same shape as DaemonPathShadowed (a read-back that greens on evidence it produced rather than on the state the lane exists to establish). DaemonUnsupervised splits that conflation: 'a daemon answered' and 'the modeled supervisor is running it' are different states with different remedies, so they get different names rather than one Ok. The LIVE observation is STAGED exactly as the PATH-shadow one is — the supervisor read (`systemctl --user is-active`) is not emitted as hand-shell here, because this module's script side is an A5 `decompose` census row that may shrink but must not grow (review 42343); it lands with the typed-argv dissolve-on (#5828). The model carries the state and its refusal today; only the live producer is deferred. Construction covers the case this observation would catch on a host provisioned BY THIS LANE — build_cache_unit_daemon_runs_until_stopped makes the self-exiting unit unwritable — so the observation's standing job is the residue construction cannot reach: a host whose unit was stopped, masked, or replaced out of band."
+
 type BuildCacheDaemonObservation
   = DaemonStatsOk { stats_line: String }
   | DaemonAbsent
   | DaemonUnresponsive { detail: NonEmptyStr }
   | DaemonMisconfigured { matched_fragment: String, detail: NonEmptyStr }
   | DaemonPathShadowed { resolved: String, expected: String }
+  | DaemonUnsupervised { unit: String, unit_state: String }
 
 type BuildCacheProvisionRefusal
   = RefusalDaemonAbsent
@@ -53,6 +56,7 @@ type BuildCacheProvisionRefusal
   | RefusalPrivilegeDenied
   | RefusalCatalogUnknown
   | RefusalPathShadowed
+  | RefusalDaemonUnsupervised
 
 type BuildCacheProvisionVerdict
   = ProvisionConverged { catalog_id: CacheInterfaceId, stats_line: String }
@@ -82,6 +86,11 @@ fn provision_build_cache_absorbing_fallback_widen_sketch(
         catalog_id: catalog_id,
         stats_line: concat("path-shadow-widen-sketch: ", concat(r, concat(" vs ", e))),
       }
+    DaemonUnsupervised { unit: u, unit_state: s } =>
+      ProvisionConverged {
+        catalog_id: catalog_id,
+        stats_line: concat("unsupervised-widen-sketch: ", concat(u, concat(" is ", s))),
+      }
   }
 }
 
@@ -118,6 +127,12 @@ fn build_cache_provision_gate_accepts(
         ProvisionConverged { catalog_id: _, stats_line: _ } => false
         ProvisionRefused { cause: _, reason: _ } => false
       }
+    DaemonUnsupervised { unit: _, unit_state: _ } =>
+      match verdict {
+        ProvisionRefused { cause: RefusalDaemonUnsupervised, reason: _ } => true
+        ProvisionConverged { catalog_id: _, stats_line: _ } => false
+        ProvisionRefused { cause: _, reason: _ } => false
+      }
   }
 }
 
@@ -158,6 +173,17 @@ fn provision_build_cache_verdict(
               ),
             ) as NonEmptyStr,
           }
+        DaemonUnsupervised { unit: u, unit_state: s } =>
+          ProvisionRefused {
+            cause: RefusalDaemonUnsupervised,
+            reason: concat(
+              concat("host_build_cache_provision: sccache answers but its supervisor unit ", u),
+              concat(
+                concat(" is ", s),
+                "; the daemon is only alive because an invocation auto-started it, so it exits on its own idle timeout and nothing restarts it. Fail-closed, no local-build fallback.",
+              ),
+            ) as NonEmptyStr,
+          }
       }
   }
 }