fix(cron): execute job immediately on action=run instead of waiting for next tick (#41037)

[rodboev]

Summary

cronjob(action="run") used to stamp next_run_at and rely on the next scheduler tick. That left CLI-only and Windows setups with a success response but no actual execution, and it made manual runs depend on whether a live gateway loop happened to be polling.

This change dispatches manual runs through the scheduler execution path immediately. It preserves paused or future schedule state for one-off manual runs, queues delivery targets that need live gateway adapters for the next gateway tick, and shares the per-job run claim between manual dispatch and normal ticks so the same job is not double-fired across processes.

Changes

• tools/cronjob_tools.py: route run and its aliases through immediate dispatch, preserve schedule snapshots, and fall back to queueing when live delivery context is required
• cron/scheduler.py: add immediate-run helpers, manual-run schedule restore helpers, live-delivery checks, and shared per-job claim handling for both immediate runs and tick submissions
• hermes_cli/cron.py: update run output to report whether the job was dispatched now or queued for the next tick
• tests/tools/test_cronjob_run_immediate.py: cover immediate dispatch, live-context queueing, schedule restore, and already-running rejection
• tests/cron/test_scheduler.py: cover the cross-process overlap where tick must skip a job already claimed by an immediate run

Validation

Scenario	Before	After
action=run, no gateway ticker	returns success but never executes	dispatches immediately in a background worker
action=run, gateway running	waits for the next tick	dispatches immediately
action=run, live-adapter-only delivery with no live context	may stamp metadata with no viable delivery path	preserves the schedule snapshot and queues the manual run for the next gateway tick
Immediate run overlaps a scheduler tick in another process	both paths can dispatch the same job	tick skips the job when the immediate-run claim is held
Non-run cron actions	unchanged	unchanged

Test plan

• python -m pytest tests/tools/test_cronjob_run_immediate.py -v
• python -m pytest tests/cron/test_scheduler.py -k TestParallelTick -v

Fixes #41037

[liuhao1024]

Review: advance_next_run race condition

The run_job_immediate() implementation has a sequencing issue that can cause a scheduled run to be silently skipped when an immediate execution fails.

The problem

In run_job_immediate(), advance_next_run(job["id"]) is called before the job is dispatched to the thread pool:

advance_next_run(job["id"])  # ← bumps next_run_at NOW

pool = _get_sequential_pool() if ...
def _run_and_release(j=job, ctx=_ctx):
    try:
        success, output, final_response, error = ctx.run(run_job, j)
        ...
    except Exception as e:
        mark_job_run(j["id"], False, str(e))  # ← failure path

If the job fails (exception, agent timeout, empty response), mark_job_run records it as failed — but next_run_at was already advanced. For a job scheduled every 1h, a failed immediate run means the next scheduled execution is pushed out by the full interval, even though this run didn't produce useful output.

Suggested fix

Move advance_next_run() into the success path of _run_and_release, after run_job returns successfully:

def _run_and_release(j=job, ctx=_ctx):
    try:
        success, output, final_response, error = ctx.run(run_job, j)
        if success:
            advance_next_run(j["id"])  # ← only advance on success
        ...

This preserves the double-execution guard (the job is already in _running_job_ids) while keeping the schedule accurate on failure.

Note

The _running_job_ids guard correctly prevents concurrent execution, so removing the early advance_next_run doesn't introduce a double-fire risk — the scheduler tick will see the job in _running_job_ids and skip it.

[rodboev]

Addressed the follow-up issues in the manual-run / gateway-only path.

1. Live-delivery gating now checks the actual resolved delivery targets, so explicit plugin targets only dispatch immediately when the required live adapter is really present.
2. Queued gateway-only manual runs stay pending if a later tick() cannot claim the job, instead of being consumed by an advance_next_run() before execution starts.
3. tick() now uses one resolved live adapter / loop pair consistently for both gateway-only admission and delivery.
4. Live-context checks can reuse the in-memory gateway runtime config when a config reload is transiently unavailable.

Validation:

• python -m pytest -o addopts='' -p no:timeout tests/cron/test_trigger_immediate.py

[rodboev]

Closing this now. The linked issue was completed by merged PR #50025, and review also found an advance_next_run race in this older branch.