Skip to content

feat(delegate): extend timeout diagnostics for N-API-call subagents - #17312

Open
YeahloYip wants to merge 2 commits into
NousResearch:mainfrom
YeahloYip:fix/delegate-acp-inherit
Open

feat(delegate): extend timeout diagnostics for N-API-call subagents#17312
YeahloYip wants to merge 2 commits into
NousResearch:mainfrom
YeahloYip:fix/delegate-acp-inherit

Conversation

@YeahloYip

Copy link
Copy Markdown

Summary

When a subagent running via delegate_task times out after making N>0 API calls, the lead agent previously received no visibility into what tool was last running or whether its result was partial. This made it impossible to distinguish:

  1. 'Tool completed, next LLM request stuck' — the tool finished but the provider response handling froze
  2. 'Tool itself hung' — the tool call never returned (network partition, blocked I/O)

Background

Two prior commits addressed adjacent cases:

This PR fills the remaining gap: N-API-call timeouts had no structured diagnostics.

Changes

tools/delegate_tool.py (+74 lines)

1. Enriched error message (lines ~1508-1525):

_last_tool = _summary.get("current_tool") if _summary else None
if _last_tool:
    _err = (
        f"Subagent timed out after {child_timeout}s with "
        f"{child_api_calls} API call(s) completed — "
        f"last tool was '{_last_tool}' (likely slow response). "
        f"The tool may have completed; check tool_trace for result_bytes."
    )
else:
    _err = ( ... generic message ... )

Replaces the generic "stuck on a slow API call" with a specific tool name.

2. tool_trace reconstruction on timeout (lines ~1535-1575):

tool_trace: list[Dict[str, Any]] = []
last_tool: Optional[str] = None
last_tool_status: Optional[str] = None
# ... build from result[messages] (assistant tool_calls + tool role responses) ...
return {
    ...
    "tool_trace": tool_trace,
    "last_tool": last_tool,
    "last_tool_status": last_tool_status,
}

Mirrors the tool_trace logic already present in the normal-completion path (~line 1556).

skills/software-development/subagent-timeout-diagnostics/SKILL.md (+183 lines)

Documents the full diagnosis procedure for lead agents: diagnosis matrix, step-by-step workflow, common pitfalls, and verification checklist.

Diagnosis Matrix

Scenario api_calls last_tool last_tool_status Interpretation
Normal completion N tool_N ok Tool succeeded, loop continued
0-API timeout 0 Child never made first request
N-API timeout N tool_K ok tool_K succeeded, provider request stalled
N-API timeout N tool_K error tool_K itself failed/hung
N-API timeout N No summary captured yet

Verification

grep -n "last_tool.*tool_trace\|tool_trace.*last_tool" tools/delegate_tool.py
# Expected: lines ~1587-1589

grep -n "last_tool = " tools/delegate_tool.py
# Expected: line ~1567

ls skills/software-development/subagent-timeout-diagnostics/
# Expected: SKILL.md

## Related Issues

- Fixes the gap described in #17308
- Related to #1175 (tool_trace in normal completion)
- Related to #15105 (0-API-call timeout diagnostics)

yeahlo added 2 commits April 28, 2026 15:41
When override_provider is set (from delegation.provider config),
clear effective_acp_command and effective_acp_args to prevent
subagents from inheriting the parent's ACP transport. This ensures
subagents use direct API calls with the configured provider instead
of copilot-acp transport.

Fixes: subagent using copilot-acp (qwen3.5-397b-a17b) instead of
the configured delegation.provider/model (e.g. minimax-cn).
When a subagent times out after making N>0 API calls, the lead agent previously
received no visibility into what tool was last running or whether its result was
partial. This made it impossible to distinguish "tool ran to completion, next LLM
request stalled" from "tool itself is hanging".

Two prior commits addressed adjacent cases:
- NousResearch#1175 (commit 7997569): tool_trace + tokens added to normal-completion results
- NousResearch#15105 (commit 7634c13): diagnostic_path log written for 0-API-call timeouts

This patch fills the gap for N-API-call timeouts by:

1. Building tool_trace from result["messages"] in the timeout path (mirrors the
   logic already present in the normal-completion path at ~line 1556). Returns
   tool_trace, last_tool, and last_tool_status in the timeout result dict so
   the lead can inspect the final tool outcome without a second round-trip.

2. Enriching the error message for api_calls>0 timeouts to include the value of
   current_tool from get_activity_summary(), replacing the generic "stuck on a
   slow API call" message with e.g.:

   "Subagent timed out after 300s with 3 API call(s) completed — last tool was
   'web_fetch' (likely slow response). The tool may have completed; check
   tool_trace for result_bytes."

3. Adding a SKILL.md that documents the full diagnosis procedure for lead agents:
   diagnosis matrix (api_calls × last_tool × last_tool_status), step-by-step
   workflow, common pitfalls, and verification checklist.

Also adds: skill/skills/software-development/subagent-timeout-diagnostics/
@alt-glitch alt-glitch added type/feature New feature or request P3 Low — cosmetic, nice to have comp/agent Core agent runtime: loop, agent_init, prompt builder, context-compression, responses endpoint tool/delegate Subagent delegation labels Apr 29, 2026

@teknium1 teknium1 left a comment

Copy link
Copy Markdown
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Thanks for targeting an observability gap that still exists on current main: tools/delegate_tool.py:2003-2022 returns only a generic N-API-call timeout result.

Problems

  • The new timeout trace cannot populate. The child result is assigned only after _child_future.result() returns (tools/delegate_tool.py:1927-1929); on timeout, the exception path is entered first. The changed tools/delegate_tool.py:1539 then reads undefined result, and its broad except clears tool_trace.
  • No regression test covers the proposed fields. Existing tests/tools/test_delegate_subagent_timeout_diagnostic.py:269-284 asserts the old N>0 behavior, while this PR changes no test file.
  • The skill documents duration_ms (SKILL.md:104), but the existing trace schema has no duration field (tools/delegate_tool.py:2071-2085).

Suggested changes

  • Build the timeout trace from live child state such as _session_messages, and distinguish an unmatched final tool call from a completed tool. Linked #17329 contains a tested implementation direction.
  • Add timeout-path tests for completed, in-progress, error, and parallel tool calls.
  • Drop the ACP inheritance commit already implemented on main at tools/delegate_tool.py:1238-1244 (commit 6b6fc28e).

Automated hermes-sweeper review.

Comment thread tools/delegate_tool.py
last_tool: Optional[str] = None
last_tool_status: Optional[str] = None
try:
_msgs = result.get("messages") or []

Copy link
Copy Markdown
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

On a real FuturesTimeoutError, result was never assigned because _child_future.result(...) raised before returning. This broad handler therefore always clears the trace on the path this code is meant to diagnose. Read a live source such as child._session_messages instead, then add a timeout regression test.

@@ -0,0 +1,183 @@
---
name: subagent-timeout-diagnostics
description: Use when a subagent times out and you need to diagnose what happened — identify whether it froze before any API call, stalled mid-request, or encountered an error in a long-running tool.

Copy link
Copy Markdown
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

This description is well over the repository's 60-character skill-description limit. Shorten it to one sentence ending with a period before adding the skill.

if tool_trace:
last = tool_trace[-1]
print(f"Last tool: {last.get('tool')}")
print(f"Duration: {last.get('duration_ms')}ms")

Copy link
Copy Markdown
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

tool_trace currently contains args_bytes, result_bytes, and status, but no duration_ms field (tools/delegate_tool.py:2071-2085). Remove this claim or implement and test that field.

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

comp/agent Core agent runtime: loop, agent_init, prompt builder, context-compression, responses endpoint P3 Low — cosmetic, nice to have sweeper:blast-moderate Sweeper blast radius: moderate — a subsystem or single platform tool/delegate Subagent delegation type/feature New feature or request

Projects

None yet

Development

Successfully merging this pull request may close these issues.

3 participants