fix(cron): clean up output dir when auto-removing repeat-limited job - #22066
Open
nftpoetrist wants to merge 1 commit into
Open
fix(cron): clean up output dir when auto-removing repeat-limited job#22066nftpoetrist wants to merge 1 commit into
nftpoetrist wants to merge 1 commit into
Conversation
mark_job_run() pops a job from jobs.json when its repeat limit is reached (lines 737-741) but did not delete OUTPUT_DIR/<job_id>. save_job_output() creates that directory on every run, so completed one-shot jobs leave one orphaned output directory per job. remove_job() gained the identical cleanup in PR NousResearch#21882 — this extends the fix to the auto-delete path inside mark_job_run(), which is the only other code path that removes a job. Fix: add the same OUTPUT_DIR/<job_id> rmtree guard used by remove_job(), scoped to the repeat-limit branch. No behavior change for recurring jobs or jobs without a repeat limit. Fixes NousResearch#22065
13 tasks
Contributor
|
Thanks for identifying the orphaned-output path. The premise is still valid on current main: Problems
Suggested changes
Automated hermes-sweeper review. |
This file contains hidden or bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
Sign up for free
to join this conversation on GitHub.
Already have an account?
Sign in to comment
Add this suggestion to a batch that can be applied as a single commit.This suggestion is invalid because no changes were made to the code.Suggestions cannot be applied while the pull request is closed.Suggestions cannot be applied while viewing a subset of changes.Only one suggestion per line can be applied in a batch.Add this suggestion to a batch that can be applied as a single commit.Applying suggestions on deleted lines is not supported.You must change the existing code in this line in order to create a valid suggestion.Outdated suggestions cannot be applied.This suggestion has been applied or marked resolved.Suggestions cannot be applied from pending reviews.Suggestions cannot be applied on multi-line comments.Suggestions cannot be applied while the pull request is queued to merge.Suggestion cannot be applied right now. Please check back later.
What does this PR do?
mark_job_run()removes a job fromjobs.jsonwhen itsrepeatlimit is reached, but the correspondingOUTPUT_DIR/<job_id>directory (written bysave_job_output()on every run) was never deleted. Each completed one-shot job leaves behind an orphaned output directory that accumulates indefinitely.remove_job()gained the identical cleanup in PR #21882 (merged today). This PR extends that fix tomark_job_run(), the only other code path that deletes a job — parity withremove_job()lines 700–703.One-line fix in
mark_job_run(), scoped to the repeat-limit branch. No behavior change for recurring jobs or jobs without a repeat limit.Related Issue
Fixes #22065
Type of Change
Changes Made
cron/jobs.py: addshutil.rmtree(OUTPUT_DIR / job_id)guard in the repeat-limit auto-delete branch ofmark_job_run()(+4 lines)tests/cron/test_jobs.py: two new tests inTestMarkJobRun(+33 lines)test_repeat_limit_cleans_up_output_dir— verifies the dir is removed after auto-deletetest_repeat_limit_no_output_dir_is_safe— verifies no crash when no output was savedHow to Test
python3.11 -m pytest tests/cron/test_jobs.py::TestMarkJobRun -v --override-ini="addopts="All 12 tests pass (1 skip for missing
croniter— pre-existing on macOS).Checklist
Code
Documentation & Housekeeping
shutil.rmtreeis cross-platform)