test(docker): poll for settled dashboard 'down' slot instead of racing boot - #63523
Closed
jethac wants to merge 2 commits into
Closed
test(docker): poll for settled dashboard 'down' slot instead of racing boot#63523jethac wants to merge 2 commits into
jethac wants to merge 2 commits into
Conversation
This was referenced Jul 13, 2026
jethac
force-pushed
the
fix/dashboard-slot-down
branch
from
July 13, 2026 04:00
40c19e6 to
29a7f39
Compare
Contributor
Author
jethac
force-pushed
the
fix/dashboard-slot-down
branch
from
July 14, 2026 08:41
29a7f39 to
879c483
Compare
Contributor
|
Thanks for isolating this Docker-test race. Current The proposed Automated hermes-sweeper review. |
jethac
force-pushed
the
fix/dashboard-slot-down
branch
from
July 18, 2026 02:02
879c483 to
63b658b
Compare
My commits on this PR use my personal email (no longer at Google). Add it to AUTHOR_MAP alongside the existing jetha@google.com entry (which still attributes an earlier commit) — both resolve to the same GitHub user. Fixes check-attribution. Co-Authored-By: Claude Opus 4.8 <noreply@anthropic.com>
…g boot
test_dashboard_slot_reports_down_when_disabled did a single immediate
s6-svstat check with no retry, unlike its sibling
test_dashboard_slot_reports_up_when_enabled which polls. The dashboard is
an always-declared supervised s6 longrun; s6-rc exec's its run wrapper once
when it brings the user bundle up, so svstat can briefly report
`up (...) 0 seconds` before run exits 0 and finish exits 125 (permanent-down)
settle the slot to `down`. The immediate check races that one-time boot flap
and intermittently reads the transient `up`:
AssertionError: Dashboard slot should be 'down' without HERMES_DASHBOARD;
svstat reports: 'up (pid 155 pgid 155) 0 seconds'
The service is correctly gated (run exits 0, no daemon; finish exits 125,
no restart — verified, and the sibling pgrep test confirms no process runs),
so the slot's steady state is down. Poll for it, anchored on `^down` so a
transitioning `up (...) want down` line can't false-match. Also correct the
docstring, which still described the deleted cont-init.d/03-dashboard-toggle
down-marker mechanism.
Co-Authored-By: Claude Opus 4.8 <noreply@anthropic.com>
jethac
force-pushed
the
fix/dashboard-slot-down
branch
from
July 29, 2026 06:03
63b658b to
d4af8e7
Compare
Contributor
Author
This file contains hidden or bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
Sign up for free
to join this conversation on GitHub.
Already have an account?
Sign in to comment
Add this suggestion to a batch that can be applied as a single commit.This suggestion is invalid because no changes were made to the code.Suggestions cannot be applied while the pull request is closed.Suggestions cannot be applied while viewing a subset of changes.Only one suggestion per line can be applied in a batch.Add this suggestion to a batch that can be applied as a single commit.Applying suggestions on deleted lines is not supported.You must change the existing code in this line in order to create a valid suggestion.Outdated suggestions cannot be applied.This suggestion has been applied or marked resolved.Suggestions cannot be applied from pending reviews.Suggestions cannot be applied on multi-line comments.Suggestions cannot be applied while the pull request is queued to merge.Suggestion cannot be applied right now. Please check back later.
Problem
tests/docker/test_dashboard.py::test_dashboard_slot_reports_down_when_disabledis flaky in the Docker build/test job:Note the
0 secondsuptime — svstat is catching the service in its first moments.Root cause
The dashboard is an always-declared supervised s6 longrun (
docker/s6-rc.d/dashboard/). WhenHERMES_DASHBOARDis unset:runexits 0 immediately — no dashboard process is ever exec'd;finishexits 125, s6's "permanent failure, do not restart" marker (≈s6-svc -O), so s6-supervise leaves the slot down and never restarts it.So the slot's steady state is
down, with no process running — corroborated by the siblingtest_dashboard_not_running_by_default(apgrepcheck) which passes.But s6-rc exec's the
runwrapper once when it brings the user-services bundle up, sos6-svstatcan briefly reportup (...) 0 secondsbeforerunexits andfinishsettles the slot todown. The failing test did a single immediates6-svstatcheck with no retry — unlike its siblingtest_dashboard_slot_reports_up_when_enabled, which polls — so it intermittently reads that one-time boot flap.Fix
Poll for the settled
downstate (mirroring the "up" test), anchored on^downso a transitioningup (...) want downline can't false-match a baredown. Also corrects the docstring, which still described the deletedcont-init.d/03-dashboard-toggledown-marker mechanism (the real mechanism is the run-exit-0 / finish-exit-125 pair).Test-only change — no s6 service or product code touched; it only relaxes an immediate check into a bounded poll for the same condition the service already guarantees in steady state.