Skip to content

fix: tool_output_limits re-reads config file on every tool call - #22155

Closed
amathxbt wants to merge 1 commit into
NousResearch:mainfrom
amathxbt:fix/tool-output-limits-no-caching
Closed

fix: tool_output_limits re-reads config file on every tool call#22155
amathxbt wants to merge 1 commit into
NousResearch:mainfrom
amathxbt:fix/tool-output-limits-no-caching

Conversation

@amathxbt

@amathxbt amathxbt commented May 9, 2026

Copy link
Copy Markdown
Contributor

Bug

tools/tool_output_limits.py calls _load_config() on every invocation of get_output_limit(). _load_config opens and parses the YAML config file from disk each time.

Impact

In workloads that invoke many tool calls per second (e.g. batch file editing, large agentic loops), this creates continuous disk I/O and YAML parsing overhead. On network-mounted or slow storage the latency compounds.

Fix

Cache the parsed config in a module-level dict with a TTL (default 60 s). Adds _reset_config_cache() for tests and hot-reload.

@amathxbt

amathxbt commented May 9, 2026

Copy link
Copy Markdown
Contributor Author

Reproduction (profiling)

import time
from tools.tool_output_limits import get_output_limit

N = 1000
start = time.perf_counter()
for _ in range(N):
    get_output_limit('read_file')
elapsed = time.perf_counter() - start
print(f'{N} calls: {elapsed:.3f}s')
# Typically 0.5–2 s depending on disk speed — every call opens + parses YAML

Root cause: _load_config() has no cache — it opens the YAML file on every call.

Fix: Module-level dict cache with 60-second TTL. In a 100-tool-call workload (typical agentic loop) this reduces config I/O from 100 file reads to 1–2.

@alt-glitch alt-glitch added type/perf Performance improvement or optimization comp/tools Tool registry, model_tools, toolsets P3 Low — cosmetic, nice to have labels May 11, 2026
kshitijk4poor added a commit to kshitijk4poor/hermes-agent that referenced this pull request May 31, 2026
…align test fixtures + AUTHOR_MAP

Self-review follow-up on top of the salvaged perf fixes:

- gateway/run.py (both watcher-drain sites): the salvaged O(n^2) fix
  (NousResearch#32708) replaced `while pending_watchers: pop(0)` with iterate-then-
  `watchers.clear()`, but `watchers` aliased the registry's live list.
  A watcher appended by a concurrent session during the `await
  asyncio.sleep(0)` yield would be cleared without ever being scheduled.
  Detach the batch atomically (`pending_watchers = []`) before iterating.

- gateway/platforms/bluebubbles.py: normalize the salvaged _guid_cache
  LRU (NousResearch#30523) to match feishu/codebase precedent — module-level
  `_GUID_CACHE_SIZE` constant, `while len > cap`, and drop the redundant
  post-insert `move_to_end` (a fresh insert is already most-recent).

- gateway/platforms/feishu.py: drop the same redundant post-insert
  `move_to_end` from the salvaged _message_text_cache LRU (NousResearch#23706).

- scripts/release.py: add AUTHOR_MAP entries for the salvaged commits'
  authors (amathxbt NousResearch#22155, ErnestHysa NousResearch#32636/NousResearch#32708) so the contributor
  audit passes when these commits land on main.

- tests/tools/test_tool_output_limits.py: autouse fixture resets the new
  module-level limits cache between tests.

- tests/gateway/test_feishu.py: hand-built adapter fixture seeded
  _message_text_cache as a plain dict; it's now an OrderedDict, so the
  fixture type had to match.
kshitijk4poor added a commit that referenced this pull request May 31, 2026
…align test fixtures + AUTHOR_MAP

Self-review follow-up on top of the salvaged perf fixes:

- gateway/run.py (both watcher-drain sites): the salvaged O(n^2) fix
  (#32708) replaced `while pending_watchers: pop(0)` with iterate-then-
  `watchers.clear()`, but `watchers` aliased the registry's live list.
  A watcher appended by a concurrent session during the `await
  asyncio.sleep(0)` yield would be cleared without ever being scheduled.
  Detach the batch atomically (`pending_watchers = []`) before iterating.

- gateway/platforms/bluebubbles.py: normalize the salvaged _guid_cache
  LRU (#30523) to match feishu/codebase precedent — module-level
  `_GUID_CACHE_SIZE` constant, `while len > cap`, and drop the redundant
  post-insert `move_to_end` (a fresh insert is already most-recent).

- gateway/platforms/feishu.py: drop the same redundant post-insert
  `move_to_end` from the salvaged _message_text_cache LRU (#23706).

- scripts/release.py: add AUTHOR_MAP entries for the salvaged commits'
  authors (amathxbt #22155, ErnestHysa #32636/#32708) so the contributor
  audit passes when these commits land on main.

- tests/tools/test_tool_output_limits.py: autouse fixture resets the new
  module-level limits cache between tests.

- tests/gateway/test_feishu.py: hand-built adapter fixture seeded
  _message_text_cache as a plain dict; it's now an OrderedDict, so the
  fixture type had to match.
KKT-OPT pushed a commit to KKT-OPT/hermes-agent that referenced this pull request May 31, 2026
…align test fixtures + AUTHOR_MAP

Self-review follow-up on top of the salvaged perf fixes:

- gateway/run.py (both watcher-drain sites): the salvaged O(n^2) fix
  (NousResearch#32708) replaced `while pending_watchers: pop(0)` with iterate-then-
  `watchers.clear()`, but `watchers` aliased the registry's live list.
  A watcher appended by a concurrent session during the `await
  asyncio.sleep(0)` yield would be cleared without ever being scheduled.
  Detach the batch atomically (`pending_watchers = []`) before iterating.

- gateway/platforms/bluebubbles.py: normalize the salvaged _guid_cache
  LRU (NousResearch#30523) to match feishu/codebase precedent — module-level
  `_GUID_CACHE_SIZE` constant, `while len > cap`, and drop the redundant
  post-insert `move_to_end` (a fresh insert is already most-recent).

- gateway/platforms/feishu.py: drop the same redundant post-insert
  `move_to_end` from the salvaged _message_text_cache LRU (NousResearch#23706).

- scripts/release.py: add AUTHOR_MAP entries for the salvaged commits'
  authors (amathxbt NousResearch#22155, ErnestHysa NousResearch#32636/NousResearch#32708) so the contributor
  audit passes when these commits land on main.

- tests/tools/test_tool_output_limits.py: autouse fixture resets the new
  module-level limits cache between tests.

- tests/gateway/test_feishu.py: hand-built adapter fixture seeded
  _message_text_cache as a plain dict; it's now an OrderedDict, so the
  fixture type had to match.
@teknium1

Copy link
Copy Markdown
Contributor

This has already landed on current main via the perf-fix salvage batch. This is an automated hermes-sweeper review.

Evidence:

@teknium1 teknium1 closed this Jun 11, 2026
@teknium1 teknium1 added the sweeper:implemented-on-main Sweeper: behavior already present on current main label Jun 11, 2026
alt-glitch pushed a commit that referenced this pull request Jun 14, 2026
…align test fixtures + AUTHOR_MAP

Self-review follow-up on top of the salvaged perf fixes:

- gateway/run.py (both watcher-drain sites): the salvaged O(n^2) fix
  (#32708) replaced `while pending_watchers: pop(0)` with iterate-then-
  `watchers.clear()`, but `watchers` aliased the registry's live list.
  A watcher appended by a concurrent session during the `await
  asyncio.sleep(0)` yield would be cleared without ever being scheduled.
  Detach the batch atomically (`pending_watchers = []`) before iterating.

- gateway/platforms/bluebubbles.py: normalize the salvaged _guid_cache
  LRU (#30523) to match feishu/codebase precedent — module-level
  `_GUID_CACHE_SIZE` constant, `while len > cap`, and drop the redundant
  post-insert `move_to_end` (a fresh insert is already most-recent).

- gateway/platforms/feishu.py: drop the same redundant post-insert
  `move_to_end` from the salvaged _message_text_cache LRU (#23706).

- scripts/release.py: add AUTHOR_MAP entries for the salvaged commits'
  authors (amathxbt #22155, ErnestHysa #32636/#32708) so the contributor
  audit passes when these commits land on main.

- tests/tools/test_tool_output_limits.py: autouse fixture resets the new
  module-level limits cache between tests.

- tests/gateway/test_feishu.py: hand-built adapter fixture seeded
  _message_text_cache as a plain dict; it's now an OrderedDict, so the
  fixture type had to match.
T02200059 pushed a commit to T02200059/hermes-agent that referenced this pull request Jun 18, 2026
…align test fixtures + AUTHOR_MAP

Self-review follow-up on top of the salvaged perf fixes:

- gateway/run.py (both watcher-drain sites): the salvaged O(n^2) fix
  (NousResearch#32708) replaced `while pending_watchers: pop(0)` with iterate-then-
  `watchers.clear()`, but `watchers` aliased the registry's live list.
  A watcher appended by a concurrent session during the `await
  asyncio.sleep(0)` yield would be cleared without ever being scheduled.
  Detach the batch atomically (`pending_watchers = []`) before iterating.

- gateway/platforms/bluebubbles.py: normalize the salvaged _guid_cache
  LRU (NousResearch#30523) to match feishu/codebase precedent — module-level
  `_GUID_CACHE_SIZE` constant, `while len > cap`, and drop the redundant
  post-insert `move_to_end` (a fresh insert is already most-recent).

- gateway/platforms/feishu.py: drop the same redundant post-insert
  `move_to_end` from the salvaged _message_text_cache LRU (NousResearch#23706).

- scripts/release.py: add AUTHOR_MAP entries for the salvaged commits'
  authors (amathxbt NousResearch#22155, ErnestHysa NousResearch#32636/NousResearch#32708) so the contributor
  audit passes when these commits land on main.

- tests/tools/test_tool_output_limits.py: autouse fixture resets the new
  module-level limits cache between tests.

- tests/gateway/test_feishu.py: hand-built adapter fixture seeded
  _message_text_cache as a plain dict; it's now an OrderedDict, so the
  fixture type had to match.
xyshanren pushed a commit to xyshanren/hermes-agent-cn that referenced this pull request Jun 25, 2026
…align test fixtures + AUTHOR_MAP

Self-review follow-up on top of the salvaged perf fixes:

- gateway/run.py (both watcher-drain sites): the salvaged O(n^2) fix
  (NousResearch#32708) replaced `while pending_watchers: pop(0)` with iterate-then-
  `watchers.clear()`, but `watchers` aliased the registry's live list.
  A watcher appended by a concurrent session during the `await
  asyncio.sleep(0)` yield would be cleared without ever being scheduled.
  Detach the batch atomically (`pending_watchers = []`) before iterating.

- gateway/platforms/bluebubbles.py: normalize the salvaged _guid_cache
  LRU (NousResearch#30523) to match feishu/codebase precedent — module-level
  `_GUID_CACHE_SIZE` constant, `while len > cap`, and drop the redundant
  post-insert `move_to_end` (a fresh insert is already most-recent).

- gateway/platforms/feishu.py: drop the same redundant post-insert
  `move_to_end` from the salvaged _message_text_cache LRU (NousResearch#23706).

- scripts/release.py: add AUTHOR_MAP entries for the salvaged commits'
  authors (amathxbt NousResearch#22155, ErnestHysa NousResearch#32636/NousResearch#32708) so the contributor
  audit passes when these commits land on main.

- tests/tools/test_tool_output_limits.py: autouse fixture resets the new
  module-level limits cache between tests.

- tests/gateway/test_feishu.py: hand-built adapter fixture seeded
  _message_text_cache as a plain dict; it's now an OrderedDict, so the
  fixture type had to match.
waefrebeorn pushed a commit to waefrebeorn/slermes that referenced this pull request Jul 2, 2026
…align test fixtures + AUTHOR_MAP

Self-review follow-up on top of the salvaged perf fixes:

- gateway/run.py (both watcher-drain sites): the salvaged O(n^2) fix
  (NousResearch#32708) replaced `while pending_watchers: pop(0)` with iterate-then-
  `watchers.clear()`, but `watchers` aliased the registry's live list.
  A watcher appended by a concurrent session during the `await
  asyncio.sleep(0)` yield would be cleared without ever being scheduled.
  Detach the batch atomically (`pending_watchers = []`) before iterating.

- gateway/platforms/bluebubbles.py: normalize the salvaged _guid_cache
  LRU (NousResearch#30523) to match feishu/codebase precedent — module-level
  `_GUID_CACHE_SIZE` constant, `while len > cap`, and drop the redundant
  post-insert `move_to_end` (a fresh insert is already most-recent).

- gateway/platforms/feishu.py: drop the same redundant post-insert
  `move_to_end` from the salvaged _message_text_cache LRU (NousResearch#23706).

- scripts/release.py: add AUTHOR_MAP entries for the salvaged commits'
  authors (amathxbt NousResearch#22155, ErnestHysa NousResearch#32636/NousResearch#32708) so the contributor
  audit passes when these commits land on main.

- tests/tools/test_tool_output_limits.py: autouse fixture resets the new
  module-level limits cache between tests.

- tests/gateway/test_feishu.py: hand-built adapter fixture seeded
  _message_text_cache as a plain dict; it's now an OrderedDict, so the
  fixture type had to match.
santhreal pushed a commit to santhreal/hermes-agent that referenced this pull request Jul 13, 2026
…align test fixtures + AUTHOR_MAP

Self-review follow-up on top of the salvaged perf fixes:

- gateway/run.py (both watcher-drain sites): the salvaged O(n^2) fix
  (NousResearch#32708) replaced `while pending_watchers: pop(0)` with iterate-then-
  `watchers.clear()`, but `watchers` aliased the registry's live list.
  A watcher appended by a concurrent session during the `await
  asyncio.sleep(0)` yield would be cleared without ever being scheduled.
  Detach the batch atomically (`pending_watchers = []`) before iterating.

- gateway/platforms/bluebubbles.py: normalize the salvaged _guid_cache
  LRU (NousResearch#30523) to match feishu/codebase precedent — module-level
  `_GUID_CACHE_SIZE` constant, `while len > cap`, and drop the redundant
  post-insert `move_to_end` (a fresh insert is already most-recent).

- gateway/platforms/feishu.py: drop the same redundant post-insert
  `move_to_end` from the salvaged _message_text_cache LRU (NousResearch#23706).

- scripts/release.py: add AUTHOR_MAP entries for the salvaged commits'
  authors (amathxbt NousResearch#22155, ErnestHysa NousResearch#32636/NousResearch#32708) so the contributor
  audit passes when these commits land on main.

- tests/tools/test_tool_output_limits.py: autouse fixture resets the new
  module-level limits cache between tests.

- tests/gateway/test_feishu.py: hand-built adapter fixture seeded
  _message_text_cache as a plain dict; it's now an OrderedDict, so the
  fixture type had to match.
donbowman pushed a commit to donbowman/hermes-agent that referenced this pull request Jul 13, 2026
…align test fixtures + AUTHOR_MAP

Self-review follow-up on top of the salvaged perf fixes:

- gateway/run.py (both watcher-drain sites): the salvaged O(n^2) fix
  (NousResearch#32708) replaced `while pending_watchers: pop(0)` with iterate-then-
  `watchers.clear()`, but `watchers` aliased the registry's live list.
  A watcher appended by a concurrent session during the `await
  asyncio.sleep(0)` yield would be cleared without ever being scheduled.
  Detach the batch atomically (`pending_watchers = []`) before iterating.

- gateway/platforms/bluebubbles.py: normalize the salvaged _guid_cache
  LRU (NousResearch#30523) to match feishu/codebase precedent — module-level
  `_GUID_CACHE_SIZE` constant, `while len > cap`, and drop the redundant
  post-insert `move_to_end` (a fresh insert is already most-recent).

- gateway/platforms/feishu.py: drop the same redundant post-insert
  `move_to_end` from the salvaged _message_text_cache LRU (NousResearch#23706).

- scripts/release.py: add AUTHOR_MAP entries for the salvaged commits'
  authors (amathxbt NousResearch#22155, ErnestHysa NousResearch#32636/NousResearch#32708) so the contributor
  audit passes when these commits land on main.

- tests/tools/test_tool_output_limits.py: autouse fixture resets the new
  module-level limits cache between tests.

- tests/gateway/test_feishu.py: hand-built adapter fixture seeded
  _message_text_cache as a plain dict; it's now an OrderedDict, so the
  fixture type had to match.
Gravezzz pushed a commit to Gravezzz/hermes-agent that referenced this pull request Jul 21, 2026
…align test fixtures + AUTHOR_MAP

Self-review follow-up on top of the salvaged perf fixes:

- gateway/run.py (both watcher-drain sites): the salvaged O(n^2) fix
  (NousResearch#32708) replaced `while pending_watchers: pop(0)` with iterate-then-
  `watchers.clear()`, but `watchers` aliased the registry's live list.
  A watcher appended by a concurrent session during the `await
  asyncio.sleep(0)` yield would be cleared without ever being scheduled.
  Detach the batch atomically (`pending_watchers = []`) before iterating.

- gateway/platforms/bluebubbles.py: normalize the salvaged _guid_cache
  LRU (NousResearch#30523) to match feishu/codebase precedent — module-level
  `_GUID_CACHE_SIZE` constant, `while len > cap`, and drop the redundant
  post-insert `move_to_end` (a fresh insert is already most-recent).

- gateway/platforms/feishu.py: drop the same redundant post-insert
  `move_to_end` from the salvaged _message_text_cache LRU (NousResearch#23706).

- scripts/release.py: add AUTHOR_MAP entries for the salvaged commits'
  authors (amathxbt NousResearch#22155, ErnestHysa NousResearch#32636/NousResearch#32708) so the contributor
  audit passes when these commits land on main.

- tests/tools/test_tool_output_limits.py: autouse fixture resets the new
  module-level limits cache between tests.

- tests/gateway/test_feishu.py: hand-built adapter fixture seeded
  _message_text_cache as a plain dict; it's now an OrderedDict, so the
  fixture type had to match.
leewenjie pushed a commit to leewenjie/hermes-agent that referenced this pull request Aug 7, 2026
…align test fixtures + AUTHOR_MAP

Self-review follow-up on top of the salvaged perf fixes:

- gateway/run.py (both watcher-drain sites): the salvaged O(n^2) fix
  (NousResearch#32708) replaced `while pending_watchers: pop(0)` with iterate-then-
  `watchers.clear()`, but `watchers` aliased the registry's live list.
  A watcher appended by a concurrent session during the `await
  asyncio.sleep(0)` yield would be cleared without ever being scheduled.
  Detach the batch atomically (`pending_watchers = []`) before iterating.

- gateway/platforms/bluebubbles.py: normalize the salvaged _guid_cache
  LRU (NousResearch#30523) to match feishu/codebase precedent — module-level
  `_GUID_CACHE_SIZE` constant, `while len > cap`, and drop the redundant
  post-insert `move_to_end` (a fresh insert is already most-recent).

- gateway/platforms/feishu.py: drop the same redundant post-insert
  `move_to_end` from the salvaged _message_text_cache LRU (NousResearch#23706).

- scripts/release.py: add AUTHOR_MAP entries for the salvaged commits'
  authors (amathxbt NousResearch#22155, ErnestHysa NousResearch#32636/NousResearch#32708) so the contributor
  audit passes when these commits land on main.

- tests/tools/test_tool_output_limits.py: autouse fixture resets the new
  module-level limits cache between tests.

- tests/gateway/test_feishu.py: hand-built adapter fixture seeded
  _message_text_cache as a plain dict; it's now an OrderedDict, so the
  fixture type had to match.
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

comp/tools Tool registry, model_tools, toolsets P3 Low — cosmetic, nice to have sweeper:implemented-on-main Sweeper: behavior already present on current main type/perf Performance improvement or optimization

Projects

None yet

Development

Successfully merging this pull request may close these issues.

3 participants