fix(cron): deliver manual runs on profile-owned gateway loop

[ciabata-git]

What does this PR do?

Manual cronjob(action="run") executions inside a live gateway currently call run_one_job(job) without the gateway adapter map or the event loop that owns those adapters. Matrix then may await its live mautrix/aiohttp client from a fresh asyncio.run() loop and fail with:

Timeout context manager should be used inside a task

This PR passes the live, profile-owned adapter map and its owning gateway loop into the existing scheduler delivery path. Unsafe or unavailable gateway context preserves the exact standalone run_one_job(job) behavior.

This carries forward Fly's original commit from #63586, preserving authorship, and adds the multiplex-profile isolation requested in that PR's maintainer review. Unlike the original revision, a secondary profile can never inherit the primary profile's adapter credentials when its own registry entry is missing or partial.

Related Issue

Fixes #61495

• Supersedes fix(cron): deliver manual runs on gateway loop #63586 with its requested multiplex-profile hardening while preserving the original commit and author.
• Complements fix(cron): isolate standalone fallback from live adapters #68359, which protects the separate second-stage fallback after a live delivery attempt has already failed.

Type of Change

• 🐛 Bug fix (non-breaking change that fixes an issue)
• ✨ New feature (non-breaking change that adds functionality)
• 🔒 Security fix
• 📝 Documentation update
• ✅ Tests (adding or improving test coverage)
• ♻️ Refactor (no behavior change)
• 🎯 New skill (bundled or hub)

Changes Made

• tools/cronjob_tools.py
  • Resolve the existing gateway.run._gateway_runner_ref without adding another registry or a Matrix-specific branch.
  • Reuse context only when the runner is active and its loop is present, open, and running.
  • Compare context-scoped get_hermes_home() with process-owned get_process_hermes_home() unconditionally.
  • For a scoped secondary profile, require the exact _profile_adapters[get_active_profile_name()] map; missing provenance fails closed instead of borrowing runner.adapters.
  • Pass adapters and their owning loop together to run_one_job; otherwise retain the exact one-argument standalone call.
• tests/tools/test_cronjob_run_immediate.py
  • Cover live gateway context, exact standalone compatibility, stale runner/loop states, real HERMES_HOME profile resolution, metadata enabled/disabled/missing, absent registries, and partial secondary maps.
  • Feed selected maps into the real delivery resolver to assert adapter identity rather than only inspecting mocks.

How to Test

1. Run the focused and adjacent cron suites:

   HERMES_PYTHON=/usr/local/lib/hermes-agent/venv/bin/python \
     HERMES_TEST_FILE_RETRIES=0 \
     scripts/run_tests.sh \
       tests/tools/test_cronjob_run_immediate.py \
       tests/tools/test_cronjob_tools.py \
       tests/cron/test_run_one_job.py \
       tests/cron/test_ticker_stall_60703.py \
       tests/cron/test_scheduler_provider.py -q

   Result: 155 passed, 0 failed.

2. Run static checks:

   python -m ruff check tools/cronjob_tools.py tests/tools/test_cronjob_run_immediate.py
   python -m py_compile tools/cronjob_tools.py tests/tools/test_cronjob_run_immediate.py
   git diff --check origin/main...HEAD

   Result: all passed.

3. Causal regressions were also verified during development:

   • the original live-context assertion failed on base because manual execution used the one-argument path;
   • the multiplex assertion failed when the primary map was selected;
   • the absent-registry resolver assertion failed when primary Matrix credentials were selected;
   • all pass on this branch.

Full-suite note: scripts/run_tests.sh tests/ -q was attempted locally but exceeded the 600-second execution cap at 29% of roughly 43,000 collected tests. Thirteen path-completion failures observed before timeout reproduce on untouched origin/main in this agent worktree environment. One unrelated routing assertion sees the configured Matrix home because this worktree sits beneath the live HERMES_HOME; it is outside the changed paths. I therefore leave the full-suite checklist item unchecked and rely on CI for the clean-environment full run.

Checklist

Code

• I've read the Contributing Guide
• My commit messages follow Conventional Commits (fix(scope):, feat(scope):, etc.)
• I searched for existing PRs; this preserves and extends fix(cron): deliver manual runs on gateway loop #63586 rather than discarding its contributor credit
• My PR contains only changes related to this fix
• I've run pytest tests/ -q and all tests pass — see the bounded full-suite note above
• I've added tests for my changes
• I've tested on Linux (7.0.14-4-pve, Python 3.11)

Documentation & Housekeeping

• Relevant documentation update — N/A; no user-facing behavior or configuration changes
• cli-config.yaml.example update — N/A; no config keys changed
• CONTRIBUTING.md or AGENTS.md update — N/A; no workflow or architectural contract changed
• Cross-platform impact considered; implementation uses pathlib, existing runtime APIs, and no platform-specific branches
• Tool descriptions/schemas update — N/A; tool schema is unchanged

Screenshots / Logs

Not applicable. The failure and fix are covered by causal unit/integration regressions and real delivery-resolver identity checks.

[teknium1]

Thanks for preserving the existing scheduler delivery path while adding the missing manual-run context.

Current main still calls run_one_job(job) without gateway context at tools/cronjob_tools.py:605. The scheduler already uses safe_schedule_threadsafe(..., loop) for live adapter delivery at cron/scheduler.py:1801-1808, so the PR targets the verified missing handoff rather than adding a second delivery mechanism. Its secondary-profile selection also matches the runtime scope contract in gateway/run.py:1777-1809 and the separate _profile_adapters ownership described at gateway/run.py:5528-5536.

Automated hermes-sweeper review.

[GottZ]

This was generated by AI during triage.

Summary

29 PRs address or reference this cron issue complex across stalled inference, manual execution, schema validation, prompt propagation, status reporting, standalone scheduling, Matrix delivery, and Desktop Git discovery. The diffs show distinct root causes rather than one universal fix; #71132 is the strongest current Matrix-delivery continuation, while #61503 is the only PR addressing the separate UGit resolver failure.

Related pull requests

• fix(cron): skip context files for scheduled jobs #7696 [closed] related — (+1/-0) — unrelated: adds only a scheduler test assertion for skip_context_files; the reported inference-loop stall remains unaddressed. Closed after the implementation was reported in fix(cron): pass skip_context_files=True to AIAgent in run_job #7958, so it remains relevant only as a superseded reference.
• fix(cron): action='run' executes inline when no gateway ticker is running #17516 [closed] best fix — (+643/-58) — partial fix for Bug: cronjob run 无法实际触发执行 (job stuck in state: scheduled) #16612: executes manual runs inline when no gateway ticker exists, but the contributor closed it as overbuilt and the later merged fix(cron): execute job immediately on action=run #50025 became the reference implementation.
• fix(cron): clarify schedule is required for create in tool schema #32448 [closed] partial — (+14/-3) — duplicate of fix(cron): clarify schedule is required for create in tool schema #32804: adds description/schema guidance for action=create; the effective description change was cherry-picked into fix(cron): clarify schedule is required for create in tool schema #32804 as commit 5101326, while the conditional schema layer was dropped.
• fix(cron): clarify schedule is required for create in tool schema #32804 [merged] best fix — (+43/-2) — best_fix for cronjob tool fails to include required schedule parameter when using Grok models #32427: ships the minimal description-driven schedule/prompt guidance with regression tests. Merged reference implementation; it does not affect manual execution or delivery.
• fix(cron): add schedule to required array in CRONJOB_SCHEMA #33980 [closed] partial — (+1/-1) — rejected schema approach: making schedule globally required would break list/remove/pause/resume/run actions. Closed against the documented action-specific design and remains relevant as a rejected alternative.
• fix(tools): improve cronjob tool descriptions to prevent missing required params #34173 [closed] related — (+122/-7) — duplicate of fix(cron): clarify schedule is required for create in tool schema #32804: rewrites the same description-driven cron schema guidance and adds a repro, but the maintainer found the remaining wording redundant and guard-regressing.
• fix(cronjob): accept schedule param aliases (cron, when, ...) (#34120) #34255 [closed] partial — (+145/-8) — rejected alias approach: normalizes model-emitted cron/when aliases, but the contributor decision explicitly rejects open-ended synonym handling as masking schema noncompliance.
• fix(tools): clarify cronjob required-params to prevent missing-schedule errors #34483 [closed] related — (+25/-6) — duplicate of fix(cron): clarify schedule is required for create in tool schema #32804: rewords already-shipped required-parameter descriptions and enriches the error, but its wording regressed the existing schema guard.
• fix(cron): enforce schedule in required schema properties to prevent validation failure #37431 [closed] partial — (+0/-0) — no effective diff: its global schedule requirement would reject non-create actions; the contributor withdrew it in favor of the existing description-driven design.
• fix(cron): clarify schedule and prompt requirements in parameter descriptions #37859 [closed] partial — (+2/-2) — duplicate of fix(cron): clarify schedule is required for create in tool schema #32804: proposes the same targeted description clarification, already implemented on main by commit 5101326 and covered by tests/cron/test_cronjob_schema.py:12.
• fix(cron): execute job immediately on manual trigger (#41037) #41130 [closed] partial — (+72/-1) — partial fix for Cron jobs never execute: last_run_at always null after manual trigger #41037: runs a triggered job immediately, but duplicates the scheduler pipeline and lacks the later atomic shared execution path. Closed as stale/superseded by fix(cron): execute job immediately on action=run #50025.
• fix(cron): add standalone cron daemon mode #41167 [closed] related — (+255/-7) — partial standalone-daemon implementation, bundled with unrelated provider UX changes; the contributor closed it in favor of focused fix(cron): add standalone cron daemon mode #41363, which is itself superseded by the later daemon continuation.
• fix(cron): execute job immediately on action=run instead of waiting for next tick (#41037) #41269 [closed] fixes — (+604/-202) — partial fix for Cron jobs never execute: last_run_at always null after manual trigger #41037: adds asynchronous immediate dispatch and delivery handling, but review found an advance_next_run race; closed after fix(cron): execute job immediately on action=run #50025 completed the accepted direction.
• fix(cron): don't skip first-run cron jobs past grace window #41307 [closed] partial — (+8/-0) — complementary first-run grace-window fix for Cron jobs never execute: last_run_at always null after manual trigger #41037, not the manual-trigger fix itself. Closed because the behavior was implemented on main through fix(cron): run missed-grace jobs once instead of deferring forever #50062, commit 6777a6b.
• fix(cron): add standalone cron daemon mode #41363 [closed] related — (+58/-5) — duplicate of fix(cron): add standalone cron daemon mode #41167: extracts the standalone cron daemon but leaves the same architecture and was later recreated by fix(cron): add standalone cron daemon mode #43864; no direct fix for the manual-trigger path.
• fix(cron): add standalone cron daemon mode #43864 related — (+59/-2) — partial standalone-daemon feature for Cron jobs never execute: last_run_at always null after manual trigger #41037. Keep open with a salvage path despite the automated keep_open review: rebase onto resolve_cron_scheduler().start(), register daemon startup hooks, and add provider/lifecycle tests and documentation.
• fix(cronjob): add schedule and job_id to JSON schema required array #49792 [closed] partial — (+1/-1) — rejected global-schema fix for Bug: cronjob tool create action fails with 'schedule is required for create' (v0.17.0) #49695: requiring schedule and job_id together over-constrains mutually exclusive actions. Closed in favor of the handler-boundary discussion, while the maintainer ultimately retained canonical schema enforcement.
• fix(cron): execute job immediately on action=run #50025 [merged] best fix — (+154/-4) — best_fix for Cron jobs never execute: last_run_at always null after manual trigger #41037: merged reference implementation using the atomic fire claim and shared run_one_job pipeline, so manual runs execute without a ticker without duplicating scheduler behavior.
• fix(cron): make cronjob(action='run') non-blocking (#52705) #52720 partial — (+232/-100) — partial fix for cronjob(action='run') blocks the calling agent until the cron session finishes #52705: removes synchronous blocking, but the contributor keep_open review identifies PID-based liveness, paused-job reactivation, external-provider handling, and non-durable daemon-thread gaps; keep open only with a scheduler-owned lifecycle salvage.
• fix(cronjob): accept schedule param aliases (cron, when, ...) (#34120) #53246 [closed] partial — (+145/-8) — rejected alias-normalization approach for [Bug]: cronjob tool: create action always fails with "schedule is required for create" even when parameters are provided #34120, functionally related to fix(cronjob): accept schedule param aliases (cron, when, ...) (#34120) #34255; the maintainer explicitly declined synonym coverage because it masks model schema failures.
• fix(cron): run manual jobs without blocking the caller #53395 best fix — (+357/-164) — best_fix for cronjob(action='run') blocks the calling agent until the cron session finishes #52705: queues manual runs through scheduler-owned pools and preserves shutdown/claim guards after the contributor follow-up. Keep open with a salvage path until the reviewed branch is rebased and those guards are verified in the current tree.
• fix(cron): thread per-run prompt through cronjob(action="run") execution chain #57342 best fix — (+90/-20) — best_fix for [Bug]: cronjob action='run' silently drops the per-run prompt — no context reaches the triggered job #57331: threads transient manual-run context through _build_job_prompt → run_job → run_one_job and tests assembled prompt content without persistence. It is the canonical earlier implementation, but remains open for the reviewed lifecycle integration.
• fix(cron): pass manual run prompt context #57360 fixes — (+101/-10) — duplicate of fix(cron): thread per-run prompt through cronjob(action="run") execution chain #57342: uses the same per-run context chain under different names. Despite the keep_open review, close as duplicate because fix(cron): thread per-run prompt through cronjob(action="run") execution chain #57342 is the recorded best_fix; the review-required defer_agent_teardown gap must be preserved if salvaged.
• fix: report immediate cron run status for depleted repeat jobs #57689 related — (+120/-9) — partial status-reporting fix: preserves success reporting when repeat-limited jobs are deleted after execution. Keep open with a salvage path requiring the contributor-reviewed real repeat=1 integration regression, not only mocked get_job/run_one_job tests.
• fix(plugins): use asyncio.wait_for instead of ClientTimeout in Matrix standalone send #61502 partial — (+15/-7) — competing partial fix for [Bug]: Matrix manual cron delivery fails with "Timeout context manager should be used inside a task" #61495: changes Matrix standalone timeout handling, but the keep-open review and subsequent live test show that the active manual path bypasses _standalone_send and that direct runtime_adapter.send() can still fail intermittently when not run as a task on the adapter's owning loop. Keep open only as a callee-side fallback after reproducing that route.
• fix(cron,desktop): pass gateway loop to run_one_job for Matrix cron delivery; discover UGit git binary (#61495, #61494) #61503 [closed] best fix — (+60/-4) — best_fix for [Bug]: #61494 and partial competing fix for [Bug]: Matrix manual cron delivery fails with "Timeout context manager should be used inside a task" #61495: dynamically discovers %LOCALAPPDATA%/UGit/app-*/resources/app/git/cmd/git.exe and forwards gateway delivery context, but is closed and bundled two issues; retain as the UGit reference and Matrix salvage source.
• fix(cron): deliver manual runs on gateway loop #63586 best fix — (+49/-1) — partial fix for [Bug]: Matrix manual cron delivery fails with "Timeout context manager should be used inside a task" #61495: forwards the default gateway adapters and loop, but the contributor keep_open review found secondary profiles still use the wrong adapter map. Close as superseded by fix(cron): deliver manual runs on profile-owned gateway loop #71132, which carries the requested profile hardening.
• fix(cron): detach manual gateway runs #63774 partial — (+230/-83) — alternative non-blocking manual-run design for cronjob(action='run') blocks the calling agent until the cron session finishes #52705, but the keep_open review identifies claim replacement/TTL correctness gaps; keep open only with a salvage path that distinguishes already-running from submission failure and renews claims.
• fix(cron): deliver manual runs on profile-owned gateway loop #71132 best fix — (+236/-2) — best_fix for [Bug]: Matrix manual cron delivery fails with "Timeout context manager should be used inside a task" #61495: forwards the profile-owned gateway adapters and loop, fails closed for stale or incomplete secondary-profile context, and adds focused coverage. Keep open with a salvage path as the canonical Matrix caller-side fix, incorporating the automated keep_open review and the live evidence that adapter selection alone is insufficient unless send() runs as a task on its owning loop.

Duplicates

#32448/#32804/#34173/#34483/#37859 are the same cron schema-description change, with #32804 as the merged reference. #34255/#53246 are the same rejected alias-normalization approach; #41130/#50025/#41269 are the manual-run execution cluster, with #50025 as the merged reference; #41167/#41363/#43864 are the standalone-daemon cluster; #57342/#57360 are duplicates, with #57342 canonical; #61503/#63586/#71132 are competing Matrix caller-side fixes, with #71132 superseding #63586; #61502 is a distinct callee-side Matrix approach.

Suggested consolidation

Keep open #71132 with a salvage path as the canonical current fix for #61495: retain its profile-owned adapter/loop selection and verify under repeated manual runs that Matrix send() executes as a task on the adapter's owning loop. Close #63586 as superseded by #71132, close #57360 as duplicate of #57342, and retain #43864, #52720, #53395, #57689, and #61502 only with their explicit salvage paths; close the remaining closed/redundant PRs against their recorded references. For the separate Desktop issue, retain #61503 as the closed UGit reference implementation, while its bundled Matrix portion is superseded by #71132.

Complex graph

flowchart LR
    classDef open fill:#dbeafe,stroke:#1d4ed8,color:#1e3a8a
    classDef merged fill:#dcfce7,stroke:#15803d,color:#14532d
    classDef closed fill:#e5e7eb,stroke:#6b7280,color:#1f2937
    classDef unverified fill:#f3f4f6,stroke:#9ca3af,color:#374151
    classDef best stroke-width:3px,stroke:#b45309
    classDef target stroke-width:3px,stroke:#4338ca
    I61495(["issue #61495 (open)"])
    subgraph Dup61503 ["PRs duplicating each other"]
        P61503["PR #61503 (closed)"]
        P63586["PR #63586 (open)"]
        P71132["PR #71132 (open)"]
    end
    P71132 -->|best fix| I61495
    class I61495 open
    class P61503 closed
    class P63586 open
    class P71132 open
    class P61503 best
    class P63586 best
    class P71132 best
    class P71132 target
    click I61495 "https://github.com/NousResearch/hermes-agent/issues/61495"
    click P61503 "https://github.com/NousResearch/hermes-agent/pull/61503"
    click P63586 "https://github.com/NousResearch/hermes-agent/pull/63586"
    click P71132 "https://github.com/NousResearch/hermes-agent/pull/71132"

Loading

Graph: solid arrow = fixes / best fix, dashed arrow = partial or unverified (see edge label); boxed group = PRs duplicating each other; amber border = best fix; indigo border = target; gray node = closed (state tag in the node label).

Cross-PR triage: Reviewed 29 pull requests and 11 issues in this complex. Each diff was read against this issue; Assessment working set: 287 kB of PR diffs, 82 kB of issue/PR text, 63 kB of discussion (79 comments), 51 verify verdicts. verdicts reflect diff content, not PR titles. Part of an automated triage batch.