Skip to content

fix(a2a): do not fail tasks on observation timeout - #96983

Open
RuniThomsen wants to merge 1 commit into
NousResearch:mainfrom
runi-services:wolf/a2a-timeout-not-failed
Open

RuniThomsen wants to merge 1 commit into
NousResearch:mainfrom
runi-services:wolf/a2a-timeout-not-failed

Conversation

@RuniThomsen

Copy link
Copy Markdown

Bug Description

Inbound A2A message/send waits for the live Hermes session, then _await_reply writes TASK_STATE_FAILED with [agent did not reply in time] when the caller window (default 300s) elapses. The task is still running. Abel's board then archives the row. Timeout is not a terminal state (A2A §3.1.3).

Same-family as Seed #1412 (session-inbound never-ack → honest terminal). This cut is the caller observation seam on native a2a-platform, not Josh/Ib pickup-ack.

Root Cause

_await_reply used fut.result(timeout=max(0, remaining)). Past the deadline that is a 0s wait, then STATE_FAILED. The orphan watchdog could also fail a working task that still had a live pending waiter.

Fix

  • After the observation window, keep waiting on the same future. Late complete() still lands.
  • fail_orphans(..., skip_ids=live pending) so a live waiter is not "orphaned".
  • Profile-CLI subprocess timeout is unchanged (that process was killed).

How to Verify

python -m pytest tests/plugins/test_a2a_plugin.py::TestObservationTimeoutNotFailed tests/plugins/test_a2a_phase23.py::TestTaskStore::test_fail_orphans -v

Sabotage: restore timeout=max(0.0, remaining) plus the STATE_FAILED / [agent did not reply in time] return → the late-complete test is RED; restore GREEN.

Test Plan

  • Late reply after deadline → TASK_STATE_COMPLETED, not failed
  • Watchdog skip live pending ids
  • Existing fail_orphans still fails true orphans
  • test_a2a_plugin.py + test_a2a_phase23.py 153 passed

Risk Assessment

Low–medium. Holding message/send threads now wait until the agent actually replies (or a real failure / client disconnect). Callers that already treat HTTP timeout as local observation keep GetTask as working and can receive a late push. PR ≠ merge ≠ live gateway (Wolf gateway not restarted).

Exclusions

  • No Seed bump (that is PR 800 / 7.6.20 pickup-ack).
  • No Kenneth-box / :3979 bounce, no board file write.

A2A §3.1.3: GetTask returns current server-owned state. A caller wait
window is not a terminal failure. Keep waiting for the live session
instead of TASK_STATE_FAILED "[agent did not reply in time]". Skip
orphan-fail for ids that still have a pending waiter.
@Enough1122

Copy link
Copy Markdown

AI code review — automated review for reference; please use your judgment.

fix(a2a): do not fail tasks on observation timeout (A2A 3.1.3) — correctly separates caller observation window from task terminal state.

  • plugins/platforms/a2a/adapter.py:38 _await_reply — replaces immediate STATE_FAILED/[agent did not reply in time] on deadline with one-time logger.info observation window elapsed then loops on fut.result(timeout=_SSE_KEEPALIVE) (or remaining window). Only keepalive() raise or exception maps to STATE_FAILED; late answer still returns STATE_COMPLETED. Preserves keepalive semantics for SSE.
  • plugins/platforms/a2a/adapter.py:25 watchdog path — snapshot _pending under lock then fail_orphans(skip_ids=live); protocol.py:100 adds skip_ids filter so live waiter ids never fail even if created_at is stale. Orphan watchdog still fails detached tasks after 5 min.
  • plugins/platforms/a2a/DESIGN.md:12 documents non-terminal observation vs orphan distinction.
  • Tests test_await_reply_keeps_waiting_after_deadline_then_completes and test_watchdog_skips_tasks_with_live_waiters cover late answer and live-skip.

Approve.

RuniThomsen pushed a commit to runi-services/hermes-agent that referenced this pull request Sep 9, 2026
Inbound message/send returned TASK_STATE_FAILED with
"[agent did not reply in time]" when A2A_REPLY_TIMEOUT elapsed.
The live session was still composing. After the window, keep waiting
with a positive interval. fail_orphans skips ids still in _pending.
Subscribe/watch still uses on_timeout.

Supersedes NousResearch#96983 (stale vs _await_future).

This branch has not been deployed

No deployments
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

comp/plugins Plugin system and bundled plugins P3 Low — cosmetic, nice to have type/bug Something isn't working

Projects

None yet

Development

Successfully merging this pull request may close these issues.

3 participants