fix(web_server): reap finished action subprocesses to prevent zombie accumulation - #38045
Closed
liuhao1024 wants to merge 1 commit into
Closed
fix(web_server): reap finished action subprocesses to prevent zombie accumulation#38045liuhao1024 wants to merge 1 commit into
liuhao1024 wants to merge 1 commit into
Conversation
…accumulation get_action_status() calls proc.poll() to check if a dashboard action has finished, but never calls proc.wait() afterward. On POSIX systems the kernel retains the process table entry until a blocking waitpid() is issued, so every completed action remains as a zombie for the lifetime of the web server. After poll() returns a non-None exit code, call proc.wait(timeout=1) to reap the child and remove the handle from _ACTION_PROCS. Fixes NousResearch#38032
Collaborator
|
Competing PR: #38040 also fixes #38032 and is broader in scope — it reaps the dashboard action zombies (same |
Contributor
|
Superseded by #38049 (commit 78e2101), now merged to main — closes #38032. Your fix was the same approach (reap finished children via proc.wait() on poll); thanks for the clean, well-scoped patch. The merged version additionally migrates the exit code into _ACTION_RESULTS so repeat status polls stay consistent. |
This file contains hidden or bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
Sign up for free
to join this conversation on GitHub.
Already have an account?
Sign in to comment
Add this suggestion to a batch that can be applied as a single commit.This suggestion is invalid because no changes were made to the code.Suggestions cannot be applied while the pull request is closed.Suggestions cannot be applied while viewing a subset of changes.Only one suggestion per line can be applied in a batch.Add this suggestion to a batch that can be applied as a single commit.Applying suggestions on deleted lines is not supported.You must change the existing code in this line in order to create a valid suggestion.Outdated suggestions cannot be applied.This suggestion has been applied or marked resolved.Suggestions cannot be applied from pending reviews.Suggestions cannot be applied on multi-line comments.Suggestions cannot be applied while the pull request is queued to merge.Suggestion cannot be applied right now. Please check back later.
What does this PR do?
Reaps finished dashboard action subprocesses in
get_action_status()to prevent zombie process accumulation. Previously,proc.poll()was called to check liveness butproc.wait()was never issued, leaving every completed action as a<defunct>zombie for the lifetime of the web server.Related Issue
Fixes #38032
Type of Change
Changes Made
hermes_cli/web_server.py: Afterproc.poll()returns a non-None exit code, callproc.wait(timeout=1)to reap the child process and.pop()the handle from_ACTION_PROCS. This prevents zombie accumulation for all dashboard actions (gateway-restart, hermes-update, etc.).tests/hermes_cli/test_web_server.py: Addedtest_finished_action_proc_is_reaped_and_removed— injects a mockFinishedProcinto_ACTION_PROCS, calls the status endpoint, and asserts thatproc.wait()was called and the entry was removed from the dict.How to Test
pytest tests/hermes_cli/test_web_server.py::TestWebServerEndpoints::test_finished_action_proc_is_reaped_and_removed -xvspytest tests/hermes_cli/test_web_server.py -k "action" -xvspytest tests/hermes_cli/test_web_server.py -xChecklist
Code
fix(scope):,feat(scope):, etc.)pytest tests/hermes_cli/test_web_server.py -qand all tests passDocumentation & Housekeeping
docs/, docstrings) — or N/Acli-config.yaml.exampleif I added/changed config keys — or N/ACONTRIBUTING.mdorAGENTS.mdif I changed architecture or workflows — or N/ACode Intelligence
hermes_cli/web_server.py:get_action_status()(caller: FastAPI route handler, flows: dashboard action status polling)_ACTION_RESULTSdict already handles completed non-spawned actions; this fix extends the same cleanup to spawned subprocesses