fix(cron): make cronjob(action='run') non-blocking (#52705)

[Tranquil-Flow]

Problem

cronjob(action="run") executes the job synchronously and inline inside the tool handler via _execute_job_now() → run_one_job(). The calling agent (or CLI) is blocked until the entire cron session — LLM loop, output save, delivery — completes before the tool returns.

This was introduced by #50025 (commit 65d7c7faf) which fixed #41037 (jobs never executing when no ticker was alive) by replacing the fire-and-forget trigger_job with synchronous _execute_job_now.

Fix

Two-path non-blocking design (as suggested in the issue):

1. Gateway alive → trigger_job(job_id) sets next_run_at = now; the scheduler ticker fires it asynchronously via ThreadPoolExecutor.submit(run_one_job) on its next cycle (~60s). At-most-once handled by the ticker's own claim_job_for_fire.

2. Gateway not alive (CLI/oneshot) → claim_job_for_fire(job_id) + daemon thread that calls run_one_job() directly. The thread runs the job to completion while the tool returns immediately.

Either way, run_one_job is never called synchronously inside the tool handler. The response reports triggered: true, trigger_mode: "scheduled"|"background", and executed (whether a claim/trigger was taken), directing the caller to cronjob(action="list") for post-run status.

Reliability & at-most-once

• No Cron jobs never execute: last_run_at always null after manual trigger #41037 regression: Path 1 arms the job (ticker fires it); Path 2 claims + daemon thread (guaranteed execution). No silent drop.
• At-most-once: Path 2 claims via claim_job_for_fire (atomic CAS that advances next_run_at), preventing double-fire if a ticker starts concurrently. Path 1 relies on the ticker's own claim.
• Error handling: daemon thread worker catches exceptions and calls mark_job_run(job_id, False, error) so the claim does not wedge.

Tests

10 new tests in tests/tools/test_cronjob_run_immediate.py:

• Gateway-alive path: uses trigger_job, never calls run_one_job synchronously
• Gateway-not-alive path: spawns daemon thread with daemon=True
• Claim-lost: no thread spawned, no trigger called
• Response contract: triggered + trigger_mode present, execution_success absent
• Core invariant: run_one_job never called synchronously regardless of gateway state
• Background worker: success + error paths

All 10 pass. Broader cron+tools suite: 84/84 pass, no regressions.

Files changed

• tools/cronjob_tools.py: replaced _execute_job_now with _trigger_job_nonblocking + _is_gateway_active + _run_job_in_background
• hermes_cli/cron.py: updated CLI display for trigger mode
• tests/tools/test_cronjob_run_immediate.py: 10 non-blocking behavior tests

Auto-published by Moonsong via Path B automated pipeline.

[teknium1]

Thanks for addressing the real blocking path: current main still calls run_one_job() inline from tools/cronjob_tools.py:641.

Problems

• tools/cronjob_tools.py:580 uses trigger_job() whenever a gateway PID is present. cron/jobs.py:1433-1441 makes trigger_job() re-enable paused jobs, while current manual-run logic intentionally rejects paused/disabled jobs at tools/cronjob_tools.py:630-637.
• PID liveness is not scheduler liveness: gateway/status.py:1535-1541 only checks for a running PID. Chronos intentionally has no in-process tick loop (plugins/cron_providers/chronos/__init__.py:103-113), and this trigger path does not call provider reconciliation.
• The daemon at tools/cronjob_tools.py:588-590 is not durable for a standalone CLI invocation: the process may exit immediately after returning from hermes cron run.

Suggested changes

• Keep the existing claim/precondition behavior and use a scheduler-owned, lifecycle-managed dispatch path rather than trigger_job() plus a per-call daemon thread.
• Cover paused jobs, external providers, and CLI/no-gateway process lifetime.

This is an automated hermes-sweeper review.

[teknium1]

is_gateway_running() only proves that a gateway PID exists, not that an in-process ticker is available. This also calls trigger_job(), which re-enables paused jobs; current manual-run behavior intentionally rejects paused/disabled jobs. Please route through a provider-aware dispatch path that preserves the existing claim/precondition semantics.

[teknium1]

A daemon thread is abandoned when a standalone hermes cron run process exits, so the no-gateway path does not guarantee execution. Use work owned by a process/service whose lifetime is explicitly managed, and add a process-lifecycle regression test.

[teknium1]

Thanks @Tranquil-Flow — you were the first to take a run at #52705, and your diagnosis was exactly right: cronjob(action='run') executing the full job synchronously inside the tool handler blocked the calling agent for the job's entire runtime.

This is now fixed on main via #80807, which takes a different mechanism than the trigger-next-tick / daemon-thread split here: the run dispatches through the async-delegation rail (same as delegate_task background mode), so the tool returns a handle immediately AND the job's outcome re-enters the conversation as a completion event when it finishes — no polling action='list' needed, and it works uniformly whether or not the gateway ticker is alive. Closing this one in favor of that, with credit for the first fix attempt.