fix(cron): parallelize script jobs with workdirs - #61774
Conversation
Related: this lands in a saturated competing cluster on |
teknium1
left a comment
There was a problem hiding this comment.
Thanks for isolating the no-agent cwd to the subprocess; the premise is present on current main: cron/scheduler.py:2535-2549 mutates the process cwd, and tick() serializes every workdir job at cron/scheduler.py:3640-3647.
Problems
cron/scheduler.py:3349in this PR derivesscheduled_atfromnext_run_at. That is not valid for recurring external-scheduler fires:cron/scheduler_provider.py:100-105claims before reloading the job, whilecron/jobs.py:1708-1710and1743-1747advancenext_run_atduring that claim. The proposed artifact can therefore record the next future occurrence and a zero lag instead of the occurrence that fired.
Suggested changes
- Keep the subprocess-cwd/pool change focused, and split the dispatch metadata; alternatively carry the pre-claim scheduled time explicitly into
run_one_job(). - Cover the
CronScheduler.fire_due()recurring-job path if retaining the metadata.
Automated hermes-sweeper review.
| @@ -3346,6 +3347,37 @@ def _teardown_cron_agent(agent, job_id: str) -> None: | |||
| logger.debug("Job '%s': failed to reap stale auxiliary clients: %s", job_id, e) | |||
|
|
|||
|
|
|||
There was a problem hiding this comment.
next_run_at is not reliably the occurrence being dispatched. CronScheduler.fire_due() claims before it reloads the job, and claim_job_for_fire() advances recurring next_run_at; this will save a future scheduled time and zero lag for external recurring fires. Preserve the pre-claim fire time explicitly, or split this telemetry from the cwd fix.
Summary
Proof
HERMES_HOME=$(mktemp -d) EMAIL_HOME_ADDRESS= uv run pytest -q tests/cron— 673 passeduv run ruff checkon touched files — passedgit diff --check— passed