Skip to content

fix(delegate): declare stateless channel in one-shot and cron so background delegations return results - #66617

Merged
teknium1 merged 1 commit into
mainfrom
hermes/delegate-stateless-63866
Jul 18, 2026
Merged

fix(delegate): declare stateless channel in one-shot and cron so background delegations return results#66617
teknium1 merged 1 commit into
mainfrom
hermes/delegate-stateless-63866

Conversation

@teknium1

Copy link
Copy Markdown
Contributor

Summary

delegate_task background=true and terminal notify_on_complete now fall back to synchronous execution in hermes -z one-shot runs and cron jobs, instead of handing out a completion handle that is silently dropped when the single-turn process/turn ends.

Salvages @cgarwood82's PR #63866 (earliest and only complete fix in the cluster). Fixes #53027.

Root cause

Top-level delegation is forced async-by-default. That's correct for the interactive CLI and gateway (their processes persist and drain process_registry.completion_queue), but a one-shot prints once and exits, and a cron job delivers its result then ends — neither has a live consumer left. The daemon thread dies mid-flight and the async_delegations row is orphaned at state: running forever. async_delivery_supported() defaulted True when unbound, and neither runner bound it.

Changes

  • hermes_cli/oneshot.py: run_oneshot() calls declare_stateless_channel() before the turn starts.
  • cron/scheduler.py: run_job() passes async_delivery=False to set_session_vars().
  • gateway/session_context.py: adds declare_stateless_channel() — binds only the delivery capability var, deliberately NOT the _session_context_engaged latch (which set_session_vars trips), so a one-shot declaring a capability doesn't flip its subprocess env bridge into ContextVar-authoritative mode.
  • tools/delegate_tool.py, tools/terminal_tool.py: user-facing fallback notes now name one-shot / cron alongside the stateless HTTP API.

Validation

Check Result
PR's own suite (test_async_delivery_capability.py) 15/15 pass
E2E: unbound (CLI/gateway) async_delivery_supported() → True
E2E: one-shot via declare_stateless_channel() → False, engaged latch stays False
E2E: cron via set_session_vars(async_delivery=False) → False
E2E: background=true batch in stateless channel falls back to SYNC, returns both results inline

Related

Supersedes the partial fixes #64609 (one-shot only, no tests), #65491 (one-shot only), #53062 (cron only) — closing those with credit pointing here.

Infographic

async-delegation-fix

…gate_task returns results

run_agent._dispatch_delegate_task forces background=True for every top-level
delegation, and async_delivery_supported() returns True for any session that
never binds the capability. On runners that cannot receive a completion after
their turn ends, that combination silently discards every subagent result: the
model gets a dispatch handle, ends its turn, and reports 'waiting for results'.

Two such runners never bind the capability:

* hermes -z (one-shot) prints one final response and exits. It bypasses cli.py,
  so nothing drains process_registry.completion_queue (only the interactive
  process_loop and the gateway watchers do).

* cron run_job clears the HERMES_SESSION_* routing keys, so a completion event
  carries session_key="" — _enrich_async_delegation_routing cannot resolve it
  and _inject_watch_notification drops it ("no routing metadata"). By then
  run_job has already shipped the job's final response via _deliver_result;
  there is no turn left to re-enter. Worse, get_current_session_key() can fall
  back to the ambient os.environ HERMES_SESSION_KEY, so a cron subagent's output
  can be routed into an unrelated user chat rather than merely dropped.

Add declare_stateless_channel() and bind it in both runners, routing
delegate_task to its existing inline/synchronous path — the same fallback the
stateless HTTP adapter already relies on, and the fix suggested in #63142. The
helper binds only the capability: set_session_vars() would also latch
_session_context_engaged, which a pure single-process one-shot must not trigger.

Also correct two agent-facing strings that hardcoded 'stateless HTTP API' as the
only channel without async delivery (delegate_tool, terminal_tool); they now name
the actual condition.

Repro (before): hermes -z 'Use delegate_task to spawn a subagent that replies
BANANA. Report its reply.' -> "Waiting for the subagent's response...", exit 0,
no BANANA. After: BANANA is returned in-turn.

Fixes #53027
Fixes #63142
@alt-glitch alt-glitch added type/bug Something isn't working P2 Medium — degraded but workaround exists comp/cli CLI entry point, hermes_cli/, setup wizard comp/cron Cron scheduler and job management comp/gateway Gateway runner, session dispatch, delivery tool/delegate Subagent delegation tool/terminal Terminal execution and process management sweeper:risk-session-state Sweeper risk: may lose/corrupt/mis-associate session or context state sweeper:risk-message-delivery Sweeper risk: may drop, duplicate, misroute, or suppress messages labels Jul 18, 2026
@alt-glitch

Copy link
Copy Markdown
Collaborator

This was generated by AI during triage.

Related to #63866 and the #53027 family: this maintainer salvage carries the stateless-channel approach across both one-shot and cron. Keep it as an active review alternative rather than marking it duplicate.

@tonydwb tonydwb left a comment

Copy link
Copy Markdown

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Code Review Summary

Verdict: Approved with Comment

PR #66617fix(delegate): declare stateless channel in one-shot and cron so background delegations return results

  • Fixes stateless channel declaration in one-shot and cron modes for background delegations.
  • 168 additions, 11 deletions — targeted fix with clear root cause.
  • No security concerns.

Suggestions

  • Ensure the fix is covered by integration tests for the delegation path.

Reviewed by Hermes Agent

@tonydwb tonydwb left a comment

Copy link
Copy Markdown

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

{
"event": "COMMENT",

Code Review Summary

Verdict: Comment (prior COMMENT activity noted)

Fixes cron/one-shot background delegation so that delegate_task uses its inline/synchronous path (results return within the job's turn) rather than async delivery. Adds declare_stateless_channel() function in gateway/session_context.py that sets only the delivery capability without latching _session_context_engaged (which would switch the env bridge mode). Passes async_delivery=False to set_session_vars in run_job.


Reviewed by Hermes Agent",
"comments": []
}

@teknium1
teknium1 merged commit 3d9be27 into main Jul 18, 2026
33 checks passed
@teknium1
teknium1 deleted the hermes/delegate-stateless-63866 branch July 18, 2026 07:05
Bartok9 added a commit to Bartok9/hermes-agent that referenced this pull request Jul 30, 2026
Closes NousResearch#69145

Root cause: NousResearch#66617 bound declare_stateless_channel for hermes -z and
cron so delegate_task runs inline when the channel cannot wake after
the turn. Webhook (and msgraph_webhook) keep supports_async_delivery=
True by default; gateway marks the parent session ended after one
response, then NousResearch#55578 drops ASYNC DELEGATION BATCH COMPLETE injections.

Fix: set supports_async_delivery=False on WebhookAdapter and
MSGraphWebhookAdapter so run.py _set_session_env propagates the
capability like APIServerAdapter.

Verification: pytest tests/gateway/test_async_delivery_capability.py -q
teknium1 added a commit that referenced this pull request Aug 15, 2026
…n after child completes

End-to-end #86632 reproduction: a real AIAgent child (mocked LLM) with the
post-turn skill-review trigger armed, dispatched through
delegate_task(background=True) on a session runtime where async delivery is
unsupported and no origin session id is bound (cron, post-#66617) — forcing
the synchronous fallback. Asserts (1) delegate_task returns the child's
result, and (2) the automatic background-review fork never spawns inside the
delegated child (the wedge site: the fork replayed the conversation on the
child's finalize path, and _child_future.result(timeout=None) never returned;
heartbeat went stale after 15 idle cycles and the cron watchdog killed the
job).

Verified RED on pre-fix main (fork spawns and wedges), GREEN with the
_delegate_depth guard in AIAgent._spawn_background_review.

Fixes #86632
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

comp/cli CLI entry point, hermes_cli/, setup wizard comp/cron Cron scheduler and job management comp/gateway Gateway runner, session dispatch, delivery P2 Medium — degraded but workaround exists sweeper:risk-message-delivery Sweeper risk: may drop, duplicate, misroute, or suppress messages sweeper:risk-session-state Sweeper risk: may lose/corrupt/mis-associate session or context state tool/delegate Subagent delegation tool/terminal Terminal execution and process management type/bug Something isn't working

Projects

None yet

Development

Successfully merging this pull request may close these issues.

[Bug]: Async delegation completion events lost in cron jobs

4 participants