You signed in with another tab or window. Reload to refresh your session.You signed out in another tab or window. Reload to refresh your session.You switched accounts on another tab or window. Reload to refresh your session.Dismiss alert
query LoadState together with TimeoutStopUSec in check_systemd_timing_alignment()
skip user-scope systemd results whose unit is not-found, masked, or error before falling back to the system unit
add regression coverage for a system service where systemctl --user show hermes-gateway returns LoadState=not-found plus the user-manager default timeout
Why
On a system-level Hermes gateway deployment, the current check tries systemctl --user show hermes-gateway --property=TimeoutStopUSec first. On Ubuntu/systemd, that can return success for a missing user-scope unit with LoadState=not-found and the manager default TimeoutStopUSec=1min 30s. Hermes then logs a false stale-unit warning even when the real system service has an aligned timeout.
Observed on a live v0.18.0 system-service gateway:
Duplicate of #34734 — same mechanism (query LoadState alongside TimeoutStopUSec in check_systemd_timing_alignment() and skip user-scope units whose LoadState is not-found/masked/error). #34734 is the earliest open PR using this approach; #36766 and #37324 are the same fix. #54396 solves the same #36755 false-positive via a different mechanism (detect the systemd manager from /proc/self/cgroup) and is related rather than a duplicate. Flagging the cluster so a maintainer can pick one.
Closing as duplicate after triage. This fix belongs to the existing upstream cluster for #36755; #34734 is the earliest open PR using the same LoadState-based approach, and #37324 carries the same approach with regression coverage. Our production verification still confirms the bug is present in v0.18.0, but keeping another duplicate PR open would add noise for maintainers.
This file contains hidden or bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
comp/gatewayGateway runner, session dispatch, deliveryduplicateThis issue or pull request already existsP3Low — cosmetic, nice to havetype/bugSomething isn't working
2 participants
Add this suggestion to a batch that can be applied as a single commit.This suggestion is invalid because no changes were made to the code.Suggestions cannot be applied while the pull request is closed.Suggestions cannot be applied while viewing a subset of changes.Only one suggestion per line can be applied in a batch.Add this suggestion to a batch that can be applied as a single commit.Applying suggestions on deleted lines is not supported.You must change the existing code in this line in order to create a valid suggestion.Outdated suggestions cannot be applied.This suggestion has been applied or marked resolved.Suggestions cannot be applied from pending reviews.Suggestions cannot be applied on multi-line comments.Suggestions cannot be applied while the pull request is queued to merge.Suggestion cannot be applied right now. Please check back later.
Summary
LoadStatetogether withTimeoutStopUSecincheck_systemd_timing_alignment()not-found,masked, orerrorbefore falling back to the system unitsystemctl --user show hermes-gatewayreturnsLoadState=not-foundplus the user-manager default timeoutWhy
On a system-level Hermes gateway deployment, the current check tries
systemctl --user show hermes-gateway --property=TimeoutStopUSecfirst. On Ubuntu/systemd, that can return success for a missing user-scope unit withLoadState=not-foundand the manager defaultTimeoutStopUSec=1min 30s. Hermes then logs a false stale-unit warning even when the real system service has an aligned timeout.Observed on a live
v0.18.0system-service gateway:LoadState=not-found,TimeoutStopUSec=1min 30sStale systemd unit detected: hermes-gateway.service has TimeoutStopSec=90s...Fixes #36755.
Test
pytest tests/gateway/test_shutdown_forensics.py -q