AsyncOpenAI occasionally throws "Runtime Error: Event loop is closed" on Windows

[ethan-tonic]

Confirm this is an issue with the Python library and not an underlying OpenAI API

• This is an issue with the Python library

Describe the bug

On Windows, if you try getting chat completions the openai library will occasionally throw an error saying RuntimeError: Event loop is closed. This seems to originate from httpx trying to close the connection. I noticed it primarily happens on short lived async functions that are run in rapid succession. It also only seems to happen if the AsyncOpenAI client is created multiple times. If it's created once (like outside the loop in the example code below) then the issue doesn't happen. I believe this may or may not be related to httpx's connection pooling trying to reuse a connection that no longer exists. Here is a related issue from httpx's repo.

Here are the logs from the exception in the code snippet provided in this issue.

Task exception was never retrieved
future: <Task finished name='Task-6' coro=<AsyncClient.aclose() done, defined at c:\Users\Ethan\tonic_validate_tester\venv\Lib\site-packages\httpx\_client.py:2011> exception=RuntimeError('Event loop is closed')>
Traceback (most recent call last):
  File "c:\Users\Ethan\tonic_validate_tester\venv\Lib\site-packages\httpx\_client.py", line 2018, in aclose
    await self._transport.aclose()
  File "c:\Users\Ethan\tonic_validate_tester\venv\Lib\site-packages\httpx\_transports\default.py", line 385, in aclose
    await self._pool.aclose()
  File "c:\Users\Ethan\tonic_validate_tester\venv\Lib\site-packages\httpcore\_async\connection_pool.py", line 313, in aclose
    await self._close_connections(closing_connections)
  File "c:\Users\Ethan\tonic_validate_tester\venv\Lib\site-packages\httpcore\_async\connection_pool.py", line 305, in _close_connections
    await connection.aclose()
  File "c:\Users\Ethan\tonic_validate_tester\venv\Lib\site-packages\httpcore\_async\connection.py", line 171, in aclose
    await self._connection.aclose()
  File "c:\Users\Ethan\tonic_validate_tester\venv\Lib\site-packages\httpcore\_async\http11.py", line 265, in aclose
    await self._network_stream.aclose()
  File "c:\Users\Ethan\tonic_validate_tester\venv\Lib\site-packages\httpcore\_backends\anyio.py", line 54, in aclose
    await self._stream.aclose()
  File "c:\Users\Ethan\tonic_validate_tester\venv\Lib\site-packages\anyio\streams\tls.py", line 202, in aclose
    await self.transport_stream.aclose()
  File "c:\Users\Ethan\tonic_validate_tester\venv\Lib\site-packages\anyio\_backends\_asyncio.py", line 1191, in aclose
    self._transport.close()
  File "C:\Users\Ethan\AppData\Local\Programs\Python\Python311\Lib\asyncio\proactor_events.py", line 109, in close
    self._loop.call_soon(self._call_connection_lost, None)
  File "C:\Users\Ethan\AppData\Local\Programs\Python\Python311\Lib\asyncio\base_events.py", line 761, in call_soon
    self._check_closed()
  File "C:\Users\Ethan\AppData\Local\Programs\Python\Python311\Lib\asyncio\base_events.py", line 519, in _check_closed
    raise RuntimeError('Event loop is closed')
RuntimeError: Event loop is closed

To Reproduce

1. Run the code snippet provided on Windows 11; Python v3.11
2. Sometime during while the for loop is running you should see in the logs that the event loop is closed

Code snippets

async def test() -> None:
    chat_completion = await client.chat.completions.create(
        messages=[
            {
                "role": "user",
                "content": "Say this is a test",
            }
        ],
        model="gpt-3.5-turbo",
    )

for i in range(50):
    # Note we create the client inside this loop as it triggers the bug
    client = AsyncOpenAI()
    asyncio.run(test())

OS

Windows 11

Python version

Python v3.11.7

Library version

openai v1.12.0

[rattrayalex]

Thanks for the report; we'll look into it.

[rattrayalex]

Can you share a complete repro script (eg including imports) and verify that the script works produces the error case?

[RobertCraigie]

I can't reproduce this on MacOS with 3.9 but I can reproduce on 3.11.

It also only seems to happen if the AsyncOpenAI client is created multiple times. If it's created once (like outside the loop in the example code below) then the issue doesn't happen

Can you share why you want to create the client multiple times in a loop?

[rattrayalex]

@ethan-tonic would you mind checking if using a context manager fixes the bug for you? would it be acceptable?

async with AsyncOpenAI() as client:

[RobertCraigie]

@ethan-tonic you're getting this error because, at least in your reproduction script, you're calling asyncio.run() multiple times which is heavily discouraged in the documentation: https://docs.python.org/3/library/asyncio-runner.html#asyncio.run

You can fix this by either:

• calling asyncio.run() once at the entrypoint of your program
• sharing the client instance between requests
• using the context manager
• calling client.close() after making the requests

[ethan-tonic]

@RobertCraigie Sorry I forgot about this issue, but we hit it again recently. So I wanted to respond.

So, running asyncio.run multiple times is useful because it emulates the behavior of some libraries like flask. So, in flask's case, it can create new event loops for different requests. The problem comes when you try to use the openai sdk inside of something like flask.

I think either this just needs to be documented better in the docs or cases where there's multiple event loops need to be handled more gracefully by the sdk. It definitely has caused a lot of trouble for us since OpenAI is one of the only libraries we've encountered bad handling of multiple event loops in.

[ethan-tonic]

Specifically, it is this bit of code inside the _base_client.py that is causing the issue

class AsyncHttpxClientWrapper(DefaultAsyncHttpxClient):
    def __del__(self) -> None:
        if self.is_closed:
            return

        try:
            # TODO(someday): support non asyncio runtimes here
            asyncio.get_running_loop().create_task(self.aclose())
        except Exception:
            pass

Even though the exception is caught, it still gets logged like crazy. Anyhow, I found two ways to solve this. Either passing extra_headers={"Connection": "close"} or by passing a custom http_client that is reused between requests.