fix(gateway): detect nonexistent user units in TimeoutStopSec alignment check

[DannyDulgheru]

Problem

systemctl --user show <unit> returns the compiled-in default TimeoutStopUSec (usually 1min 30s / 90s) with exit code 0 for nonexistent units. This makes it indistinguishable from a real unit that legitimately has a 90-second timeout.

When the gateway is installed as a system-level service (no user-scope unit), the check in check_systemd_timing_alignment queries --user first, gets the default 90s, and emits a false-positive warning:

Stale systemd unit detected: hermes-gateway.service has TimeoutStopSec=90s but drain_timeout=180s (expected >=210s). systemd may SIGKILL the gateway mid-drain.

The actual system-level unit has TimeoutStopSec=210s — the warning is spurious.

Fix

1. Add _unit_exists_in_scope() helper that uses systemctl --{scope} list-unit-files to verify the unit is actually registered before trusting its show output.

2. Query system scope first, then user scope as fallback. For --user results, skip the value when the unit doesn't exist in the user scope so the system-level fallback prevails.

3. Reset timeout_us to None inside the loop so stale values from a previous iteration cannot leak through.

Testing

>>> _unit_exists_in_scope('hermes-gateway.service', 'user')
False
>>> _unit_exists_in_scope('hermes-gateway.service', 'system')
True

Before the fix: gateway warns on every startup about 90s vs 210s mismatch.
After the fix: warning disappears because the user-scope query is correctly identified as a phantom default.

[alt-glitch]

Related: #36755 (the false-positive this fixes) and #34734 (competing open fix). Same goal — suppress the spurious "TimeoutStopSec=90s" stale-unit warning when the gateway runs as a system-level service and systemctl --user show returns the compiled-in 90s default for a phantom user unit. Different mechanism: this PR adds a _unit_exists_in_scope() list-unit-files probe and queries system scope first; #34734 adds --property=LoadState and skips values where LoadState != loaded. Maintainer to pick one approach.

[teknium1]

Thanks for catching the systemctl --user show default-timeout false-positive — that's a real correctness bug in the alignment check (it returns the compiled-in 90s default for nonexistent units, indistinguishable from a real 90s timeout).

We addressed the underlying crash loop in #54066 by setting the restart_drain_timeout default to 0: a restart now interrupts in-flight work immediately, so systemd's kill-timer always exceeds a near-instant drain and the mismatch can't cause a SIGKILL-mid-cleanup loop. That makes the alignment check moot for default installs — the warning only fires now if a user explicitly opts into a large drain.

Your detection-hardening is still genuinely useful for that opt-in case. I'm closing this as superseded by #54066 for the reported crash loop, but if you'd like to re-target the false-positive fix as a standalone hardening PR against current main, we'd take it — it stands on its own merit. Appreciate the careful root-cause work.