fix(T5): replace time.sleep(0.25) with asyncio.sleep in MCP auth reconnect poll - #32636
Closed
ErnestHysa wants to merge 1 commit into
Closed
fix(T5): replace time.sleep(0.25) with asyncio.sleep in MCP auth reconnect poll#32636ErnestHysa wants to merge 1 commit into
ErnestHysa wants to merge 1 commit into
Conversation
…nnect poll
PAIN BEFORE:
Inside _handle_auth_error_and_retry() (a sync function that runs on the MCP
event loop thread), there was a blocking polling loop:
while time.monotonic() < deadline:
if srv.session is not None and srv._ready.is_set():
break
time.sleep(0.25) # BLOCKS THE ENTIRE EVENT LOOP
Since _handle_auth_error_and_retry is invoked from tool handlers that run ON
the MCP event loop, time.sleep(0.25) blocked ALL concurrent MCP operations
(including other tools, keepalive heartbeats, OAuth refreshes) for 250ms per
iteration. With a 15-second deadline, worst case = 60 * 250ms = 15 seconds
of fully blocked concurrency.
WHAT WAS FIXED:
Extracted the blocking poll into an async helper _await_ready() that uses
asyncio.sleep(0.25) (non-blocking), and runs it via _run_on_mcp_loop().
_run_on_mcp_loop() properly awaits the coroutine on the event loop without
blocking the caller's thread. Added exception handling around the poll so
stuck reconnects still fall through to the error path.
The sync _handle_auth_error_and_retry now:
1. Fires reconnect signal (threadsafe)
2. Calls _run_on_mcp_loop(_await_ready(), timeout=15) — non-blocking
3. Returns; the event loop handles the polling
File: tools/mcp_tool.py
Lines: _handle_auth_error_and_retry() (~1886-1920)
Found by: exhaustive multi-pass audit (10 strategies, 1901 files, 913K lines)
alt-glitch
added
type/perf
kshitijk4poor
added a commit
to kshitijk4poor/hermes-agent
that referenced
this pull request
May 31, 2026
…align test fixtures + AUTHOR_MAP Self-review follow-up on top of the salvaged perf fixes: - gateway/run.py (both watcher-drain sites): the salvaged O(n^2) fix (NousResearch#32708) replaced `while pending_watchers: pop(0)` with iterate-then- `watchers.clear()`, but `watchers` aliased the registry's live list. A watcher appended by a concurrent session during the `await asyncio.sleep(0)` yield would be cleared without ever being scheduled. Detach the batch atomically (`pending_watchers = []`) before iterating. - gateway/platforms/bluebubbles.py: normalize the salvaged _guid_cache LRU (NousResearch#30523) to match feishu/codebase precedent — module-level `_GUID_CACHE_SIZE` constant, `while len > cap`, and drop the redundant post-insert `move_to_end` (a fresh insert is already most-recent). - gateway/platforms/feishu.py: drop the same redundant post-insert `move_to_end` from the salvaged _message_text_cache LRU (NousResearch#23706). - scripts/release.py: add AUTHOR_MAP entries for the salvaged commits' authors (amathxbt NousResearch#22155, ErnestHysa NousResearch#32636/NousResearch#32708) so the contributor audit passes when these commits land on main. - tests/tools/test_tool_output_limits.py: autouse fixture resets the new module-level limits cache between tests. - tests/gateway/test_feishu.py: hand-built adapter fixture seeded _message_text_cache as a plain dict; it's now an OrderedDict, so the fixture type had to match.
kshitijk4poor
added a commit
that referenced
this pull request
May 31, 2026
…align test fixtures + AUTHOR_MAP Self-review follow-up on top of the salvaged perf fixes: - gateway/run.py (both watcher-drain sites): the salvaged O(n^2) fix (#32708) replaced `while pending_watchers: pop(0)` with iterate-then- `watchers.clear()`, but `watchers` aliased the registry's live list. A watcher appended by a concurrent session during the `await asyncio.sleep(0)` yield would be cleared without ever being scheduled. Detach the batch atomically (`pending_watchers = []`) before iterating. - gateway/platforms/bluebubbles.py: normalize the salvaged _guid_cache LRU (#30523) to match feishu/codebase precedent — module-level `_GUID_CACHE_SIZE` constant, `while len > cap`, and drop the redundant post-insert `move_to_end` (a fresh insert is already most-recent). - gateway/platforms/feishu.py: drop the same redundant post-insert `move_to_end` from the salvaged _message_text_cache LRU (#23706). - scripts/release.py: add AUTHOR_MAP entries for the salvaged commits' authors (amathxbt #22155, ErnestHysa #32636/#32708) so the contributor audit passes when these commits land on main. - tests/tools/test_tool_output_limits.py: autouse fixture resets the new module-level limits cache between tests. - tests/gateway/test_feishu.py: hand-built adapter fixture seeded _message_text_cache as a plain dict; it's now an OrderedDict, so the fixture type had to match.
KKT-OPT
pushed a commit
to KKT-OPT/hermes-agent
that referenced
this pull request
May 31, 2026
…align test fixtures + AUTHOR_MAP Self-review follow-up on top of the salvaged perf fixes: - gateway/run.py (both watcher-drain sites): the salvaged O(n^2) fix (NousResearch#32708) replaced `while pending_watchers: pop(0)` with iterate-then- `watchers.clear()`, but `watchers` aliased the registry's live list. A watcher appended by a concurrent session during the `await asyncio.sleep(0)` yield would be cleared without ever being scheduled. Detach the batch atomically (`pending_watchers = []`) before iterating. - gateway/platforms/bluebubbles.py: normalize the salvaged _guid_cache LRU (NousResearch#30523) to match feishu/codebase precedent — module-level `_GUID_CACHE_SIZE` constant, `while len > cap`, and drop the redundant post-insert `move_to_end` (a fresh insert is already most-recent). - gateway/platforms/feishu.py: drop the same redundant post-insert `move_to_end` from the salvaged _message_text_cache LRU (NousResearch#23706). - scripts/release.py: add AUTHOR_MAP entries for the salvaged commits' authors (amathxbt NousResearch#22155, ErnestHysa NousResearch#32636/NousResearch#32708) so the contributor audit passes when these commits land on main. - tests/tools/test_tool_output_limits.py: autouse fixture resets the new module-level limits cache between tests. - tests/gateway/test_feishu.py: hand-built adapter fixture seeded _message_text_cache as a plain dict; it's now an OrderedDict, so the fixture type had to match.
alt-glitch
pushed a commit
that referenced
this pull request
Jun 14, 2026
…align test fixtures + AUTHOR_MAP Self-review follow-up on top of the salvaged perf fixes: - gateway/run.py (both watcher-drain sites): the salvaged O(n^2) fix (#32708) replaced `while pending_watchers: pop(0)` with iterate-then- `watchers.clear()`, but `watchers` aliased the registry's live list. A watcher appended by a concurrent session during the `await asyncio.sleep(0)` yield would be cleared without ever being scheduled. Detach the batch atomically (`pending_watchers = []`) before iterating. - gateway/platforms/bluebubbles.py: normalize the salvaged _guid_cache LRU (#30523) to match feishu/codebase precedent — module-level `_GUID_CACHE_SIZE` constant, `while len > cap`, and drop the redundant post-insert `move_to_end` (a fresh insert is already most-recent). - gateway/platforms/feishu.py: drop the same redundant post-insert `move_to_end` from the salvaged _message_text_cache LRU (#23706). - scripts/release.py: add AUTHOR_MAP entries for the salvaged commits' authors (amathxbt #22155, ErnestHysa #32636/#32708) so the contributor audit passes when these commits land on main. - tests/tools/test_tool_output_limits.py: autouse fixture resets the new module-level limits cache between tests. - tests/gateway/test_feishu.py: hand-built adapter fixture seeded _message_text_cache as a plain dict; it's now an OrderedDict, so the fixture type had to match.
T02200059
pushed a commit
to T02200059/hermes-agent
that referenced
this pull request
Jun 18, 2026
…align test fixtures + AUTHOR_MAP Self-review follow-up on top of the salvaged perf fixes: - gateway/run.py (both watcher-drain sites): the salvaged O(n^2) fix (NousResearch#32708) replaced `while pending_watchers: pop(0)` with iterate-then- `watchers.clear()`, but `watchers` aliased the registry's live list. A watcher appended by a concurrent session during the `await asyncio.sleep(0)` yield would be cleared without ever being scheduled. Detach the batch atomically (`pending_watchers = []`) before iterating. - gateway/platforms/bluebubbles.py: normalize the salvaged _guid_cache LRU (NousResearch#30523) to match feishu/codebase precedent — module-level `_GUID_CACHE_SIZE` constant, `while len > cap`, and drop the redundant post-insert `move_to_end` (a fresh insert is already most-recent). - gateway/platforms/feishu.py: drop the same redundant post-insert `move_to_end` from the salvaged _message_text_cache LRU (NousResearch#23706). - scripts/release.py: add AUTHOR_MAP entries for the salvaged commits' authors (amathxbt NousResearch#22155, ErnestHysa NousResearch#32636/NousResearch#32708) so the contributor audit passes when these commits land on main. - tests/tools/test_tool_output_limits.py: autouse fixture resets the new module-level limits cache between tests. - tests/gateway/test_feishu.py: hand-built adapter fixture seeded _message_text_cache as a plain dict; it's now an OrderedDict, so the fixture type had to match.
xyshanren
pushed a commit
to xyshanren/hermes-agent-cn
that referenced
this pull request
Jun 25, 2026
…align test fixtures + AUTHOR_MAP Self-review follow-up on top of the salvaged perf fixes: - gateway/run.py (both watcher-drain sites): the salvaged O(n^2) fix (NousResearch#32708) replaced `while pending_watchers: pop(0)` with iterate-then- `watchers.clear()`, but `watchers` aliased the registry's live list. A watcher appended by a concurrent session during the `await asyncio.sleep(0)` yield would be cleared without ever being scheduled. Detach the batch atomically (`pending_watchers = []`) before iterating. - gateway/platforms/bluebubbles.py: normalize the salvaged _guid_cache LRU (NousResearch#30523) to match feishu/codebase precedent — module-level `_GUID_CACHE_SIZE` constant, `while len > cap`, and drop the redundant post-insert `move_to_end` (a fresh insert is already most-recent). - gateway/platforms/feishu.py: drop the same redundant post-insert `move_to_end` from the salvaged _message_text_cache LRU (NousResearch#23706). - scripts/release.py: add AUTHOR_MAP entries for the salvaged commits' authors (amathxbt NousResearch#22155, ErnestHysa NousResearch#32636/NousResearch#32708) so the contributor audit passes when these commits land on main. - tests/tools/test_tool_output_limits.py: autouse fixture resets the new module-level limits cache between tests. - tests/gateway/test_feishu.py: hand-built adapter fixture seeded _message_text_cache as a plain dict; it's now an OrderedDict, so the fixture type had to match.
waefrebeorn
pushed a commit
to waefrebeorn/slermes
that referenced
this pull request
Jul 2, 2026
…align test fixtures + AUTHOR_MAP Self-review follow-up on top of the salvaged perf fixes: - gateway/run.py (both watcher-drain sites): the salvaged O(n^2) fix (NousResearch#32708) replaced `while pending_watchers: pop(0)` with iterate-then- `watchers.clear()`, but `watchers` aliased the registry's live list. A watcher appended by a concurrent session during the `await asyncio.sleep(0)` yield would be cleared without ever being scheduled. Detach the batch atomically (`pending_watchers = []`) before iterating. - gateway/platforms/bluebubbles.py: normalize the salvaged _guid_cache LRU (NousResearch#30523) to match feishu/codebase precedent — module-level `_GUID_CACHE_SIZE` constant, `while len > cap`, and drop the redundant post-insert `move_to_end` (a fresh insert is already most-recent). - gateway/platforms/feishu.py: drop the same redundant post-insert `move_to_end` from the salvaged _message_text_cache LRU (NousResearch#23706). - scripts/release.py: add AUTHOR_MAP entries for the salvaged commits' authors (amathxbt NousResearch#22155, ErnestHysa NousResearch#32636/NousResearch#32708) so the contributor audit passes when these commits land on main. - tests/tools/test_tool_output_limits.py: autouse fixture resets the new module-level limits cache between tests. - tests/gateway/test_feishu.py: hand-built adapter fixture seeded _message_text_cache as a plain dict; it's now an OrderedDict, so the fixture type had to match.
Collaborator
|
Thanks for this fix! This landed on main as your commit eb9bfd3 — |
santhreal
pushed a commit
to santhreal/hermes-agent
that referenced
this pull request
Jul 13, 2026
…align test fixtures + AUTHOR_MAP Self-review follow-up on top of the salvaged perf fixes: - gateway/run.py (both watcher-drain sites): the salvaged O(n^2) fix (NousResearch#32708) replaced `while pending_watchers: pop(0)` with iterate-then- `watchers.clear()`, but `watchers` aliased the registry's live list. A watcher appended by a concurrent session during the `await asyncio.sleep(0)` yield would be cleared without ever being scheduled. Detach the batch atomically (`pending_watchers = []`) before iterating. - gateway/platforms/bluebubbles.py: normalize the salvaged _guid_cache LRU (NousResearch#30523) to match feishu/codebase precedent — module-level `_GUID_CACHE_SIZE` constant, `while len > cap`, and drop the redundant post-insert `move_to_end` (a fresh insert is already most-recent). - gateway/platforms/feishu.py: drop the same redundant post-insert `move_to_end` from the salvaged _message_text_cache LRU (NousResearch#23706). - scripts/release.py: add AUTHOR_MAP entries for the salvaged commits' authors (amathxbt NousResearch#22155, ErnestHysa NousResearch#32636/NousResearch#32708) so the contributor audit passes when these commits land on main. - tests/tools/test_tool_output_limits.py: autouse fixture resets the new module-level limits cache between tests. - tests/gateway/test_feishu.py: hand-built adapter fixture seeded _message_text_cache as a plain dict; it's now an OrderedDict, so the fixture type had to match.
donbowman
pushed a commit
to donbowman/hermes-agent
that referenced
this pull request
Jul 13, 2026
…align test fixtures + AUTHOR_MAP Self-review follow-up on top of the salvaged perf fixes: - gateway/run.py (both watcher-drain sites): the salvaged O(n^2) fix (NousResearch#32708) replaced `while pending_watchers: pop(0)` with iterate-then- `watchers.clear()`, but `watchers` aliased the registry's live list. A watcher appended by a concurrent session during the `await asyncio.sleep(0)` yield would be cleared without ever being scheduled. Detach the batch atomically (`pending_watchers = []`) before iterating. - gateway/platforms/bluebubbles.py: normalize the salvaged _guid_cache LRU (NousResearch#30523) to match feishu/codebase precedent — module-level `_GUID_CACHE_SIZE` constant, `while len > cap`, and drop the redundant post-insert `move_to_end` (a fresh insert is already most-recent). - gateway/platforms/feishu.py: drop the same redundant post-insert `move_to_end` from the salvaged _message_text_cache LRU (NousResearch#23706). - scripts/release.py: add AUTHOR_MAP entries for the salvaged commits' authors (amathxbt NousResearch#22155, ErnestHysa NousResearch#32636/NousResearch#32708) so the contributor audit passes when these commits land on main. - tests/tools/test_tool_output_limits.py: autouse fixture resets the new module-level limits cache between tests. - tests/gateway/test_feishu.py: hand-built adapter fixture seeded _message_text_cache as a plain dict; it's now an OrderedDict, so the fixture type had to match.
Gravezzz
pushed a commit
to Gravezzz/hermes-agent
that referenced
this pull request
Jul 21, 2026
…align test fixtures + AUTHOR_MAP Self-review follow-up on top of the salvaged perf fixes: - gateway/run.py (both watcher-drain sites): the salvaged O(n^2) fix (NousResearch#32708) replaced `while pending_watchers: pop(0)` with iterate-then- `watchers.clear()`, but `watchers` aliased the registry's live list. A watcher appended by a concurrent session during the `await asyncio.sleep(0)` yield would be cleared without ever being scheduled. Detach the batch atomically (`pending_watchers = []`) before iterating. - gateway/platforms/bluebubbles.py: normalize the salvaged _guid_cache LRU (NousResearch#30523) to match feishu/codebase precedent — module-level `_GUID_CACHE_SIZE` constant, `while len > cap`, and drop the redundant post-insert `move_to_end` (a fresh insert is already most-recent). - gateway/platforms/feishu.py: drop the same redundant post-insert `move_to_end` from the salvaged _message_text_cache LRU (NousResearch#23706). - scripts/release.py: add AUTHOR_MAP entries for the salvaged commits' authors (amathxbt NousResearch#22155, ErnestHysa NousResearch#32636/NousResearch#32708) so the contributor audit passes when these commits land on main. - tests/tools/test_tool_output_limits.py: autouse fixture resets the new module-level limits cache between tests. - tests/gateway/test_feishu.py: hand-built adapter fixture seeded _message_text_cache as a plain dict; it's now an OrderedDict, so the fixture type had to match.
leewenjie
pushed a commit
to leewenjie/hermes-agent
that referenced
this pull request
Aug 7, 2026
…align test fixtures + AUTHOR_MAP Self-review follow-up on top of the salvaged perf fixes: - gateway/run.py (both watcher-drain sites): the salvaged O(n^2) fix (NousResearch#32708) replaced `while pending_watchers: pop(0)` with iterate-then- `watchers.clear()`, but `watchers` aliased the registry's live list. A watcher appended by a concurrent session during the `await asyncio.sleep(0)` yield would be cleared without ever being scheduled. Detach the batch atomically (`pending_watchers = []`) before iterating. - gateway/platforms/bluebubbles.py: normalize the salvaged _guid_cache LRU (NousResearch#30523) to match feishu/codebase precedent — module-level `_GUID_CACHE_SIZE` constant, `while len > cap`, and drop the redundant post-insert `move_to_end` (a fresh insert is already most-recent). - gateway/platforms/feishu.py: drop the same redundant post-insert `move_to_end` from the salvaged _message_text_cache LRU (NousResearch#23706). - scripts/release.py: add AUTHOR_MAP entries for the salvaged commits' authors (amathxbt NousResearch#22155, ErnestHysa NousResearch#32636/NousResearch#32708) so the contributor audit passes when these commits land on main. - tests/tools/test_tool_output_limits.py: autouse fixture resets the new module-level limits cache between tests. - tests/gateway/test_feishu.py: hand-built adapter fixture seeded _message_text_cache as a plain dict; it's now an OrderedDict, so the fixture type had to match.
This file contains hidden or bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
Sign up for free
to join this conversation on GitHub.
Already have an account?
Sign in to comment
2 participants
Add this suggestion to a batch that can be applied as a single commit.This suggestion is invalid because no changes were made to the code.Suggestions cannot be applied while the pull request is closed.Suggestions cannot be applied while viewing a subset of changes.Only one suggestion per line can be applied in a batch.Add this suggestion to a batch that can be applied as a single commit.Applying suggestions on deleted lines is not supported.You must change the existing code in this line in order to create a valid suggestion.Outdated suggestions cannot be applied.This suggestion has been applied or marked resolved.Suggestions cannot be applied from pending reviews.Suggestions cannot be applied on multi-line comments.Suggestions cannot be applied while the pull request is queued to merge.Suggestion cannot be applied right now. Please check back later.
Summary
time.sleep(0.25) was blocking the asyncio event loop during MCP auth reconnect polling, causing event loop stalls.
What Was Fixed
Replaced time.sleep(0.25) with asyncio.sleep(0.25) in MCP auth reconnect polling to avoid blocking the event loop.