Skip to content

fix: eliminate 'Event loop is closed' / 'Press ENTER to continue' during idle - #3398

Merged
teknium1 merged 1 commit into
mainfrom
hermes/hermes-4f6a1f8e
Mar 27, 2026
Merged

fix: eliminate 'Event loop is closed' / 'Press ENTER to continue' during idle#3398
teknium1 merged 1 commit into
mainfrom
hermes/hermes-4f6a1f8e

Conversation

@teknium1

Copy link
Copy Markdown
Contributor

Summary

Fixes the "Unhandled exception in event loop: ... RuntimeError: Event loop is closed ... Press ENTER to continue..." error that occurs ~1 minute after an agent turn completes (during CLI idle time).

Root cause

The OpenAI SDK's AsyncHttpxClientWrapper.__del__ schedules aclose() via asyncio.get_running_loop().create_task(). When an AsyncOpenAI client is garbage-collected while prompt_toolkit's event loop is running (the common CLI idle state), the aclose() task runs on prompt_toolkit's loop but the underlying TCP transport is bound to a different (dead) worker loop. The transport's self._loop.call_soon() then raises RuntimeError('Event loop is closed').

The existing _force_close_async_httpx fix only runs on cache access or CLI exit — there's a window between the agent thread finishing and the next cache access where GC can hit an uncleaned client.

Three-layer fix

  1. neuter_async_httpx_del() — Monkey-patches AsyncHttpxClientWrapper.__del__ to a no-op at CLI startup. Safe because cached clients are explicitly cleaned via _force_close_async_httpx, and uncached clients' TCP connections are cleaned by the OS on exit.

  2. Custom asyncio exception handler — Installed on prompt_toolkit's event loop to silently suppress RuntimeError('Event loop is closed'). Defense-in-depth for SDK upgrades that might change the class name.

  3. cleanup_stale_async_clients() — Called after each agent turn to proactively evict cache entries whose event loop is closed.

Test plan

  • 6 new tests in tests/test_async_httpx_del_neuter.py
  • Full test suite passes (6485/6485, 29 pre-existing failures from missing optional deps)
  • test_model_tools_async_bridge.py — 10/10 ✔
  • test_crossloop_client_cache.py — 5/5 ✔
  • test_cli_init.py — 20/20 ✔

…ing idle

The OpenAI SDK's AsyncHttpxClientWrapper.__del__ schedules aclose() via
asyncio.get_running_loop().create_task().  When an AsyncOpenAI client is
garbage-collected while prompt_toolkit's event loop is running (the common
CLI idle state), the aclose() task runs on prompt_toolkit's loop but the
underlying TCP transport is bound to a different (dead) worker loop.
The transport's self._loop.call_soon() then raises RuntimeError('Event
loop is closed'), which prompt_toolkit surfaces as the disruptive
'Unhandled exception in event loop ... Press ENTER to continue...' error.

Three-layer fix:

1. neuter_async_httpx_del(): Monkey-patches __del__ to a no-op at CLI
   startup before any AsyncOpenAI clients are created.  Safe because
   cached clients are explicitly cleaned via _force_close_async_httpx,
   and uncached clients' TCP connections are cleaned by the OS on exit.

2. Custom asyncio exception handler: Installed on prompt_toolkit's event
   loop to silently suppress 'Event loop is closed' RuntimeError.
   Defense-in-depth for SDK upgrades that might change the class name.

3. cleanup_stale_async_clients(): Called after each agent turn (when the
   agent thread joins) to proactively evict cache entries whose event
   loop is closed, preventing stale clients from accumulating.
@teknium1
teknium1 merged commit e0dbbdb into main Mar 27, 2026
2 checks passed
binhnt92 added a commit to binhnt92/hermes-agent that referenced this pull request Mar 29, 2026
…ed event loop

The AsyncOpenAI client was created once at __init__ and stored as an
instance attribute. process_directory() calls asyncio.run() which creates
and closes a fresh event loop. On a second call, the client's httpx
transport is still bound to the closed loop, raising RuntimeError:
"Event loop is closed" — the same pattern fixed by PR NousResearch#3398 for the
main agent loop.

Create the client lazily in _get_async_client() so each asyncio.run()
gets a client bound to the current loop.
teknium1 pushed a commit that referenced this pull request Mar 30, 2026
…ed event loop

The AsyncOpenAI client was created once at __init__ and stored as an
instance attribute. process_directory() calls asyncio.run() which creates
and closes a fresh event loop. On a second call, the client's httpx
transport is still bound to the closed loop, raising RuntimeError:
"Event loop is closed" — the same pattern fixed by PR #3398 for the
main agent loop.

Create the client lazily in _get_async_client() so each asyncio.run()
gets a client bound to the current loop.
teknium1 added a commit that referenced this pull request Mar 30, 2026
…ed event loop (#4013)

The AsyncOpenAI client was created once at __init__ and stored as an
instance attribute. process_directory() calls asyncio.run() which creates
and closes a fresh event loop. On a second call, the client's httpx
transport is still bound to the closed loop, raising RuntimeError:
"Event loop is closed" — the same pattern fixed by PR #3398 for the
main agent loop.

Create the client lazily in _get_async_client() so each asyncio.run()
gets a client bound to the current loop.

Co-authored-by: binhnt92 <binhnt.ht.92@gmail.com>
angelburgosrosado pushed a commit to angelburgosrosado/hermes-agent that referenced this pull request Apr 27, 2026
…ing idle (NousResearch#3398)

The OpenAI SDK's AsyncHttpxClientWrapper.__del__ schedules aclose() via
asyncio.get_running_loop().create_task().  When an AsyncOpenAI client is
garbage-collected while prompt_toolkit's event loop is running (the common
CLI idle state), the aclose() task runs on prompt_toolkit's loop but the
underlying TCP transport is bound to a different (dead) worker loop.
The transport's self._loop.call_soon() then raises RuntimeError('Event
loop is closed'), which prompt_toolkit surfaces as the disruptive
'Unhandled exception in event loop ... Press ENTER to continue...' error.

Three-layer fix:

1. neuter_async_httpx_del(): Monkey-patches __del__ to a no-op at CLI
   startup before any AsyncOpenAI clients are created.  Safe because
   cached clients are explicitly cleaned via _force_close_async_httpx,
   and uncached clients' TCP connections are cleaned by the OS on exit.

2. Custom asyncio exception handler: Installed on prompt_toolkit's event
   loop to silently suppress 'Event loop is closed' RuntimeError.
   Defense-in-depth for SDK upgrades that might change the class name.

3. cleanup_stale_async_clients(): Called after each agent turn (when the
   agent thread joins) to proactively evict cache entries whose event
   loop is closed, preventing stale clients from accumulating.
angelburgosrosado pushed a commit to angelburgosrosado/hermes-agent that referenced this pull request Apr 27, 2026
…ed event loop (NousResearch#4013)

The AsyncOpenAI client was created once at __init__ and stored as an
instance attribute. process_directory() calls asyncio.run() which creates
and closes a fresh event loop. On a second call, the client's httpx
transport is still bound to the closed loop, raising RuntimeError:
"Event loop is closed" — the same pattern fixed by PR NousResearch#3398 for the
main agent loop.

Create the client lazily in _get_async_client() so each asyncio.run()
gets a client bound to the current loop.

Co-authored-by: binhnt92 <binhnt.ht.92@gmail.com>
02356abc pushed a commit to 02356abc/hermes-agent that referenced this pull request May 14, 2026
…ing idle (NousResearch#3398)

The OpenAI SDK's AsyncHttpxClientWrapper.__del__ schedules aclose() via
asyncio.get_running_loop().create_task().  When an AsyncOpenAI client is
garbage-collected while prompt_toolkit's event loop is running (the common
CLI idle state), the aclose() task runs on prompt_toolkit's loop but the
underlying TCP transport is bound to a different (dead) worker loop.
The transport's self._loop.call_soon() then raises RuntimeError('Event
loop is closed'), which prompt_toolkit surfaces as the disruptive
'Unhandled exception in event loop ... Press ENTER to continue...' error.

Three-layer fix:

1. neuter_async_httpx_del(): Monkey-patches __del__ to a no-op at CLI
   startup before any AsyncOpenAI clients are created.  Safe because
   cached clients are explicitly cleaned via _force_close_async_httpx,
   and uncached clients' TCP connections are cleaned by the OS on exit.

2. Custom asyncio exception handler: Installed on prompt_toolkit's event
   loop to silently suppress 'Event loop is closed' RuntimeError.
   Defense-in-depth for SDK upgrades that might change the class name.

3. cleanup_stale_async_clients(): Called after each agent turn (when the
   agent thread joins) to proactively evict cache entries whose event
   loop is closed, preventing stale clients from accumulating.
02356abc pushed a commit to 02356abc/hermes-agent that referenced this pull request May 14, 2026
…ed event loop (NousResearch#4013)

The AsyncOpenAI client was created once at __init__ and stored as an
instance attribute. process_directory() calls asyncio.run() which creates
and closes a fresh event loop. On a second call, the client's httpx
transport is still bound to the closed loop, raising RuntimeError:
"Event loop is closed" — the same pattern fixed by PR NousResearch#3398 for the
main agent loop.

Create the client lazily in _get_async_client() so each asyncio.run()
gets a client bound to the current loop.

Co-authored-by: binhnt92 <binhnt.ht.92@gmail.com>
gweeteve pushed a commit to gweeteve/hermes-agent that referenced this pull request Jun 2, 2026
…ing idle (NousResearch#3398)

The OpenAI SDK's AsyncHttpxClientWrapper.__del__ schedules aclose() via
asyncio.get_running_loop().create_task().  When an AsyncOpenAI client is
garbage-collected while prompt_toolkit's event loop is running (the common
CLI idle state), the aclose() task runs on prompt_toolkit's loop but the
underlying TCP transport is bound to a different (dead) worker loop.
The transport's self._loop.call_soon() then raises RuntimeError('Event
loop is closed'), which prompt_toolkit surfaces as the disruptive
'Unhandled exception in event loop ... Press ENTER to continue...' error.

Three-layer fix:

1. neuter_async_httpx_del(): Monkey-patches __del__ to a no-op at CLI
   startup before any AsyncOpenAI clients are created.  Safe because
   cached clients are explicitly cleaned via _force_close_async_httpx,
   and uncached clients' TCP connections are cleaned by the OS on exit.

2. Custom asyncio exception handler: Installed on prompt_toolkit's event
   loop to silently suppress 'Event loop is closed' RuntimeError.
   Defense-in-depth for SDK upgrades that might change the class name.

3. cleanup_stale_async_clients(): Called after each agent turn (when the
   agent thread joins) to proactively evict cache entries whose event
   loop is closed, preventing stale clients from accumulating.
gweeteve pushed a commit to gweeteve/hermes-agent that referenced this pull request Jun 2, 2026
…ed event loop (NousResearch#4013)

The AsyncOpenAI client was created once at __init__ and stored as an
instance attribute. process_directory() calls asyncio.run() which creates
and closes a fresh event loop. On a second call, the client's httpx
transport is still bound to the closed loop, raising RuntimeError:
"Event loop is closed" — the same pattern fixed by PR NousResearch#3398 for the
main agent loop.

Create the client lazily in _get_async_client() so each asyncio.run()
gets a client bound to the current loop.

Co-authored-by: binhnt92 <binhnt.ht.92@gmail.com>
waefrebeorn pushed a commit to waefrebeorn/slermes that referenced this pull request Jul 2, 2026
…ing idle (NousResearch#3398)

The OpenAI SDK's AsyncHttpxClientWrapper.__del__ schedules aclose() via
asyncio.get_running_loop().create_task().  When an AsyncOpenAI client is
garbage-collected while prompt_toolkit's event loop is running (the common
CLI idle state), the aclose() task runs on prompt_toolkit's loop but the
underlying TCP transport is bound to a different (dead) worker loop.
The transport's self._loop.call_soon() then raises RuntimeError('Event
loop is closed'), which prompt_toolkit surfaces as the disruptive
'Unhandled exception in event loop ... Press ENTER to continue...' error.

Three-layer fix:

1. neuter_async_httpx_del(): Monkey-patches __del__ to a no-op at CLI
   startup before any AsyncOpenAI clients are created.  Safe because
   cached clients are explicitly cleaned via _force_close_async_httpx,
   and uncached clients' TCP connections are cleaned by the OS on exit.

2. Custom asyncio exception handler: Installed on prompt_toolkit's event
   loop to silently suppress 'Event loop is closed' RuntimeError.
   Defense-in-depth for SDK upgrades that might change the class name.

3. cleanup_stale_async_clients(): Called after each agent turn (when the
   agent thread joins) to proactively evict cache entries whose event
   loop is closed, preventing stale clients from accumulating.
waefrebeorn pushed a commit to waefrebeorn/slermes that referenced this pull request Jul 2, 2026
…ed event loop (NousResearch#4013)

The AsyncOpenAI client was created once at __init__ and stored as an
instance attribute. process_directory() calls asyncio.run() which creates
and closes a fresh event loop. On a second call, the client's httpx
transport is still bound to the closed loop, raising RuntimeError:
"Event loop is closed" — the same pattern fixed by PR NousResearch#3398 for the
main agent loop.

Create the client lazily in _get_async_client() so each asyncio.run()
gets a client bound to the current loop.

Co-authored-by: binhnt92 <binhnt.ht.92@gmail.com>
@fostrumm fostrumm mentioned this pull request Jul 15, 2026
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

None yet

Projects

None yet

Development

Successfully merging this pull request may close these issues.

1 participant