Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Regression in aiohttp 3.10.3 when running multiple loops in multiple threads #9202

Closed
1 task done
hauntsaninja opened this issue Sep 19, 2024 · 2 comments
Closed
1 task done
Labels

Comments

@hauntsaninja
Copy link
Contributor

hauntsaninja commented Sep 19, 2024

Describe the bug

λ uv pip show aiohttp
Name: aiohttp
Version: 3.10.2
Location: .../lib/python3.11/site-packages
Requires: aiohappyeyeballs, aiosignal, attrs, frozenlist, multidict, yarl
Required-by: adlfs, aiogoogle, aiohttp-cors, aiohttp-retry, aiomonitor, aioresponses, aiosseclient, apache-airflow-providers-http, boostedblob, datasets, geoip2, kubernetes-asyncio, langchain, langchain-community, livekit-api, microsoft-kiota-authentication-azure, pytest-aiohttp, safety, twilio

λ cat a.py
import asyncio
import aiohttp
import concurrent.futures


async def aiohttp_get():
    async with aiohttp.ClientSession() as session:
        async with session.get("https://httpbin.org/get") as response:
            return await response.text()


def sync():
    asyncio.run(aiohttp_get())


futs = []
with concurrent.futures.ThreadPoolExecutor() as executor:
    for _ in range(10):
        futs.append(executor.submit(sync))
for fut in futs:
    fut.result()

# works fine with aiohttp==3.10.2
λ python a.py

λ uv pip install 'aiohttp==3.10.3'
Resolved 8 packages in 51ms
Uninstalled 1 package in 11ms
Installed 1 package in 5ms
 - aiohttp==3.10.2
 + aiohttp==3.10.3

# crashes with aiohttp==3.10.3 and newer
λ python a.py
Traceback (most recent call last):
  File "/Users/shantanu/devoai/openai/a.py", line 21, in <module>
    fut.result()
  File "/Users/shantanu/.pyenv/versions/3.11.8/lib/python3.11/concurrent/futures/_base.py", line 449, in result
    return self.__get_result()
           ^^^^^^^^^^^^^^^^^^^
  File "/Users/shantanu/.pyenv/versions/3.11.8/lib/python3.11/concurrent/futures/_base.py", line 401, in __get_result
    raise self._exception
  File "/Users/shantanu/.pyenv/versions/3.11.8/lib/python3.11/concurrent/futures/thread.py", line 58, in run
    result = self.fn(*self.args, **self.kwargs)
             ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
  File "/Users/shantanu/devoai/openai/a.py", line 13, in sync
    asyncio.run(aiohttp_get())
  File "/Users/shantanu/.pyenv/versions/3.11.8/lib/python3.11/asyncio/runners.py", line 190, in run
    return runner.run(main)
           ^^^^^^^^^^^^^^^^
  File "/Users/shantanu/.pyenv/versions/3.11.8/lib/python3.11/asyncio/runners.py", line 118, in run
    return self._loop.run_until_complete(task)
           ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
  File "/Users/shantanu/.pyenv/versions/3.11.8/lib/python3.11/asyncio/base_events.py", line 654, in run_until_complete
    return future.result()
           ^^^^^^^^^^^^^^^
  File "/Users/shantanu/devoai/openai/a.py", line 8, in aiohttp_get
    async with session.get("https://httpbin.org/get") as response:
  File ".../lib/python3.11/site-packages/aiohttp/client.py", line 1353, in __aenter__
    self._resp = await self._coro
                 ^^^^^^^^^^^^^^^^
  File ".../lib/python3.11/site-packages/aiohttp/client.py", line 657, in _request
    conn = await self._connector.connect(
           ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
  File ".../lib/python3.11/site-packages/aiohttp/connector.py", line 564, in connect
    proto = await self._create_connection(req, traces, timeout)
            ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
  File ".../lib/python3.11/site-packages/aiohttp/connector.py", line 975, in _create_connection
    _, proto = await self._create_direct_connection(req, traces, timeout)
               ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
  File ".../lib/python3.11/site-packages/aiohttp/connector.py", line 1285, in _create_direct_connection
    sslcontext = await self._get_ssl_context(req)
                 ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
  File ".../lib/python3.11/site-packages/aiohttp/connector.py", line 1028, in _get_ssl_context
    return await self._make_or_get_ssl_context(True)
           ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
  File ".../lib/python3.11/site-packages/aiohttp/connector.py", line 1033, in _make_or_get_ssl_context
    return await self._made_ssl_context[verified]
           ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
RuntimeError: Task <Task pending name='Task-2' coro=<aiohttp_get() running at /Users/shantanu/devoai/openai/a.py:8> cb=[_run_until_complete_cb() at /Users/shantanu/.pyenv/versions/3.11.8/lib/python3.11/asyncio/base_events.py:181]> got Future <Future pending> attached to a different loop

To Reproduce

See a.py in log above

Expected behavior

No crash

Logs/tracebacks

See above

Python Version

3.11.8

aiohttp Version

3.10.3 and newer

multidict Version

6.1

yarl Version

1.11.1

OS

macOS

Related component

Client

Code of Conduct

  • I agree to follow the aio-libs Code of Conduct
@Dreamsorcerer
Copy link
Member

#9020?

@hauntsaninja
Copy link
Contributor Author

Yup, can confirm that that fixes! Thank you (and thank you bdraco for the fix). I'll wait for 3.10.6 then :-)

(I missed this when searching for issues because I searched phrases from stacktrace, not future attached to a different loop because that has a lot of other hits)

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Labels
Projects
None yet
Development

No branches or pull requests

2 participants