Repository navigation
fix(embed): harden daemon startup lifecycle - #4700
Merged
nicoloboschi merged 6 commits intoSep 25, 2026
Merged
Conversation
diatche
changed the title
nicoloboschi
force-pushed
the
fix/embed-cleanup-startup-timeout
branch
from
September 25, 2026 12:57
7bee3de to
466afa2
Compare
Collaborator
|
Thanks! I rebased onto main and pushed a few review fixes: the timeout cleanup now signals the child's whole process group on POSIX (under the |
Review fixes on top of the startup-lifecycle hardening: - signal the child's whole process group on POSIX: under the uvx fallback the spawned child is the uv launcher, so terminating only its pid could leave the real hindsight-api running as an orphan (Windows keeps Popen.kill) - detect a child that exited during startup via poll() and fail immediately; with the stability probe now non-fatal, a real crash would otherwise wait out the full startup deadline - correct the --daemon comment: _HINDSIGHT_DAEMON_CHILD already puts the API in daemon mode, so the flag was redundant (older APIs forked on it) - tests: shared startup fixtures, a constant clock instead of a fixed time.time() sequence, coverage for fail-fast exit, group kill and Windows; give test_daemon_client's Popen stub a live process
- patch os.killpg with raising=False: it does not exist on Windows runners - hindsight-all's Popen stub now reports a live process, so the new fail-fast exit check does not read it as a crashed daemon
nicoloboschi
force-pushed
the
fix/embed-cleanup-startup-timeout
branch
from
September 25, 2026 13:15
466afa2 to
8b6672e
Compare
signal.SIGKILL and os.killpg do not exist there, so faking platform.system() is not enough to exercise that path on a Windows runner.
This file contains hidden or bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
Sign up for free
to join this conversation on GitHub.
Already have an account?
Sign in to comment
2 participants
Add this suggestion to a batch that can be applied as a single commit.This suggestion is invalid because no changes were made to the code.Suggestions cannot be applied while the pull request is closed.Suggestions cannot be applied while viewing a subset of changes.Only one suggestion per line can be applied in a batch.Add this suggestion to a batch that can be applied as a single commit.Applying suggestions on deleted lines is not supported.You must change the existing code in this line in order to create a valid suggestion.Outdated suggestions cannot be applied.This suggestion has been applied or marked resolved.Suggestions cannot be applied from pending reviews.Suggestions cannot be applied on multi-line comments.Suggestions cannot be applied while the pull request is queued to merge.Suggestion cannot be applied right now. Please check back later.
Summary
Popenhandle and terminate that exact child when startup reaches its deadlineWhy
The API can become temporarily unresponsive immediately after binding its port while startup backlog work occupies its event loop. The embed manager currently treats one missed stability probe as a crash. Separately, when startup reaches its 180-second deadline,
daemon startreturns without stopping the detached child. A supervisor retry can then create overlapping API processes.Together these changes keep a live daemon inside the existing bounded readiness loop and guarantee cleanup if it never becomes stably healthy.
Verification
uv run pytest tests/test_embed_manager.py -q— 33 passeduv run ruff check hindsight_embed/daemon_embed_manager.pyuv run ruff check --ignore E721 tests/test_embed_manager.py(the file has an unrelated pre-existing E721)uv run ruff format --check hindsight_embed/daemon_embed_manager.py tests/test_embed_manager.pyRelated: #4580 (separate detached-process lifecycle issue; not closed by this PR).