fix(cron): keep active manual runs heartbeating - #92018
Open
qdivan wants to merge 1 commit into
Open
Conversation
Contributor
|
This file contains hidden or bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
Sign up for free
to join this conversation on GitHub.
Already have an account?
Sign in to comment
Add this suggestion to a batch that can be applied as a single commit.This suggestion is invalid because no changes were made to the code.Suggestions cannot be applied while the pull request is closed.Suggestions cannot be applied while viewing a subset of changes.Only one suggestion per line can be applied in a batch.Add this suggestion to a batch that can be applied as a single commit.Applying suggestions on deleted lines is not supported.You must change the existing code in this line in order to create a valid suggestion.Outdated suggestions cannot be applied.This suggestion has been applied or marked resolved.Suggestions cannot be applied from pending reviews.Suggestions cannot be applied on multi-line comments.Suggestions cannot be applied while the pull request is queued to merge.Suggestion cannot be applied right now. Please check back later.
What does this PR do?
Keeps the calling turn's activity heartbeat alive for the full duration of an inline manual cron run.
The heartbeat currently stops after a hard-coded six-hour ceiling even when
run_one_jobis still active. Once it stops, the parent inactivity watchdog can terminate a legitimate long-running job at a deterministic wall-clock threshold. Elapsed time alone is not evidence that the child is wedged: persistent gateway sessions already dispatch manual runs through the async-delegation rail, while synchronous fallbacks remain governed by the cron run's own timeout/cancellation policy (including an explicit unlimited setting).This removes the independent heartbeat ceiling and leaves the heartbeat tied to the actual
run_one_joblifetime. The existingfinallyblock still stops and joins the heartbeat thread as soon as the job exits.Related Issue
No linked issue; this is a self-found regression in the manual-run heartbeat introduced by #76675. It is distinct from the opt-in per-job caps proposed in #45809.
Type of Change
Changes Made
tools/cronjob_tools.py: remove the elapsed-time gate that drops activity while an inline cron run is still executing.tests/tools/test_cronjob_run_immediate.py: replace the old ceiling contract with a deterministic monotonic-clock regression proving heartbeats continue beyond six hours until completion.How to Test
scripts/run_tests.sh tests/tools/test_cronjob_run_immediate.py tests/tools/test_cronjob_run_background.py tests/tools/test_cronjob_tools.py -q(94 passed).scripts/run_tests.sh tests/cron/ -q(873 passed, 1 Windows-only test skipped on Linux).uv run --with ruff ruff check tools/cronjob_tools.py tests/tools/test_cronjob_run_immediate.py.The focused regression was verified RED before the production change: the old code logged that the ceiling was reached, delivered no heartbeat, and failed while the fake job remained active. It passes after this change.
Checklist
Code
tests/suite (focused tool and cron suites above are green)Documentation & Housekeeping
cli-config.yaml.exampleupdate — N/A; no config keys changedCONTRIBUTING.md/AGENTS.mdupdate — N/A; no workflow changed