Conversation
Fixes NousResearch#9814 Root cause: _get_tool_usage() used max() to merge tool_name and tool_calls sources, assuming they recorded the same calls. But across mixed session sources (gateway/CLI/ACP), they capture disjoint sets of calls. Fix: Sum counts from both sources to properly count all tool invocations.
teknium1
left a comment
There was a problem hiding this comment.
Thanks for identifying the real mixed-session undercount: current main still applies a global per-tool max() in agent/insights.py:281.
Problems
- The proposed addition at
agent/insights.py:280double-counts normal completed calls.agent/tool_executor.py:938appends a tool-result message, andrun_agent.py:1882-1895persists both assistanttool_callsand tool-resulttool_name. The existing fixture records both representations and expects two terminal calls attests/agent/test_insights.py:348-350; this change would report four. - No regression test accompanies the behavior change. The linked related PR #9896 uses
(session_id, tool_name)aggregation before the per-sourcemax, which preserves paired calls while adding disjoint sessions.
Suggested changes
- Merge per
(session_id, tool_name), then total by tool name. - Add paired-source and disjoint-session regression cases in
tests/agent/test_insights.py.
Automated hermes-sweeper review.
| # disjoint tool calls across different session sources. | ||
| all_tools = set(tool_counts) | set(tool_calls_counts) | ||
| merged = Counter() | ||
| for tool in all_tools: |
There was a problem hiding this comment.
This sums globally aggregated counters, so it doubles ordinary completed calls that have both assistant tool_calls and a tool-result tool_name row. Merge by (session_id, tool_name) first, then aggregate by tool name.
|
Salvaged onto current |
Summary
The Insights tool was undercounting tool usage when
tool_nameandtool_callssources coexisted across different sessions.Root Cause
_get_tool_usage()inagent/insights.pyusedmax()to merge the two tool-count sources, assuming they were duplicate views of the same calls. However, across mixed session sources (gateway / CLI / ACP), one session may record viatool_namewhile another records only viatool_calls, making them disjoint. When the same tool appeared once in each source, the count wasmax(1, 1) = 1instead of the correct2.Fix
Changed the merge logic from
max()to sum (+), sincetool_name(from tool execution results) andtool_calls(from assistant-side call records) can capture disjoint sets of calls across different session sources.Test Plan
Closes #9814