fix: dispatch cronjob(action='run') to the background like delegate_task - #80807
Merged
Conversation
A manual cronjob run executed the job synchronously on the calling agent's tool thread. A cron job is a full agent run that routinely takes minutes to hours, so the parent turn sat inside ONE tool call the whole time: uninterruptible (the interrupt flag is only checked between loop iterations) and serial (a batch of manual runs executed one by one). A Telegram session that kicked off dozens of new jobs 'right now' was wedged for hours ignoring every interrupt. action='run' now rides the async-delegation rail delegate_task background mode uses: the at-most-once claim is taken synchronously (so paused/missing/already-firing jobs still report immediately), the run executes on the shared daemon executor, the tool returns at once with a delegation handle, and the job's outcome re-enters the conversation as a type='async_delegation' completion event through the existing completion-queue drains (CLI + gateway) — preserving message-role alternation and the prompt cache. Sync fallbacks preserved: - no routable session (direct Python callers, hermes cron run) - async delivery unsupported (hermes -z, cron child sessions, Kanban workers, stateless HTTP) - dispatch pool at capacity (claim already taken — runs inline rather than stranding it) The completion block reports ok/failure, delivery target, next scheduled run, and an excerpt of the job's saved output.
Contributor
૮ >ﻌ< ა ci reviewran on acec730
|
alt-glitch
added
type/bug
Salvaged from PR #53395 by @izumi0uu: the fire claim's 300s TTL is routinely outlived by real cron jobs, so claim_job_for_fire alone cannot stop a manual cronjob(action='run') from double-firing a job the ticker (or another manual run) is still executing. Extract the ticker's _submit_with_guard running-set check into shared module-level helpers (try_register_running_job / release_running_job) and register manual runs through the same set — one dedupe owner, no drift. Manual runs also become visible to get_running_job_ids (the gateway shutdown drain, #60432) and mark_running_jobs_interrupted, which previously could not see them. The background dispatch path pre-checks the running set so a mid-run job reports 'already running' in the tool response immediately instead of as a delayed error completion event; the authoritative atomic check remains in _run_claimed_job on the worker. Co-authored-by: izumi0uu <izumi0uu@gmail.com>
This was referenced Aug 7, 2026
teknium1
added a commit
that referenced
this pull request
Aug 7, 2026
Covers the behavior shipped in #80807 (background dispatch for cronjob action='run') and #80838 (per-run '## Run Context' prompt, gateway-loop delivery): immediate return with handle, completion re-entering the conversation, in-flight dedupe, transient context injection with prompt scanning, and the sync fallbacks.
POWERFULMOVES
pushed a commit
to POWERFULMOVES/PMOVES-hermes-agent
that referenced
this pull request
Aug 10, 2026
Covers the behavior shipped in NousResearch#80807 (background dispatch for cronjob action='run') and NousResearch#80838 (per-run '## Run Context' prompt, gateway-loop delivery): immediate return with handle, completion re-entering the conversation, in-flight dedupe, transient context injection with prompt scanning, and the sync fallbacks.
randlee
pushed a commit
to randlee/hermes-agent
that referenced
this pull request
Aug 11, 2026
Covers the behavior shipped in NousResearch#80807 (background dispatch for cronjob action='run') and NousResearch#80838 (per-run '## Run Context' prompt, gateway-loop delivery): immediate return with handle, completion re-entering the conversation, in-flight dedupe, transient context injection with prompt scanning, and the sync fallbacks.
This file contains hidden or bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
Sign up for free
to join this conversation on GitHub.
Already have an account?
Sign in to comment
2 participants
Add this suggestion to a batch that can be applied as a single commit.This suggestion is invalid because no changes were made to the code.Suggestions cannot be applied while the pull request is closed.Suggestions cannot be applied while viewing a subset of changes.Only one suggestion per line can be applied in a batch.Add this suggestion to a batch that can be applied as a single commit.Applying suggestions on deleted lines is not supported.You must change the existing code in this line in order to create a valid suggestion.Outdated suggestions cannot be applied.This suggestion has been applied or marked resolved.Suggestions cannot be applied from pending reviews.Suggestions cannot be applied on multi-line comments.Suggestions cannot be applied while the pull request is queued to merge.Suggestion cannot be applied right now. Please check back later.
Summary
cronjob(action='run')now dispatches the job to the background likedelegate_task: the tool call returns immediately with a handle, and the job's outcome re-enters the conversation as a completion event when it finishes. Manual runs also gain in-flight dedupe shared with the scheduler ticker (salvaged from #53395 by @izumi0uu).Root cause of the incident this fixes: a manual run executed the full cron job (a complete agent run, minutes to hours) synchronously inside ONE tool call on the calling agent's thread. The parent turn was uninterruptible for the whole window (the interrupt flag is only checked between loop iterations) and a batch of manual runs executed strictly serially — a Telegram session that kicked off dozens of new jobs "right now" was wedged for hours ignoring every interrupt. Reported earlier as #52705 (first fix attempt by @Tranquil-Flow in #52720).
Changes
tools/cronjob_tools.py:_execute_job_now()split into claim +_run_claimed_job()halves (behavior unchanged)._try_dispatch_background_run(): takes the at-most-once claim synchronously (paused / missing / already-firing / already-running still report immediately in the tool response), then fires the run throughtools.async_delegation.dispatch_async_delegationon the shared daemon executor. Completion rides the existingprocess_registry.completion_queuerail (CLI drain + gateway_async_delegation_watcher) — role alternation and prompt cache preserved, zero new drain loops.hermes cron run),async_delivery_supported()false (one-shot runners, cron child sessions, Kanban workers, stateless HTTP), and dispatch pool at capacity (claim already taken — runs inline rather than stranding it).cron/scheduler.py(salvaged from fix(cron): run manual jobs without blocking the caller #53395, credited via co-author):try_register_running_job()/release_running_job(); the ticker's_submit_with_guardand manual runs now use ONE owner. The fire claim's 300s TTL is routinely outlived by real jobs, so the claim alone couldn't stop a manual run from double-firing a mid-run job.get_running_job_ids()(gateway shutdown drain, /update bypasses drain and interrupts in-flight cron tool work #60432) andmark_running_jobs_interrupted(), which previously couldn't see them.tests/tools/test_cronjob_run_background.py: 13 tests — immediate return with handle, completion event on the shared queue (ok + failure), claim-lost immediate report, all three sync fallbacks, in-flight dedupe (both directions + release), tool-level integration for both modes.Validation
hermes cron run/ one-shot runtimesE2E: real job store in temp
HERMES_HOME, realrun_one_jobbody (agent-run boundary stubbed with the real 4-tuple signature) — tool returns in 0.04s, completion event lands with correctsession_key/status/output excerpt,last_statuspersisted; second manual run while the first is mid-flight is refused immediately and the running set is empty after completion. Targeted suites: 501/501 passing (tests/cron/+ all cronjob tool tests + shutdown-interrupt/execution-ledger).Infographic