Skip to content

Commit

Permalink
Prevent edge case that creates extra event loop
Browse files Browse the repository at this point in the history
`asyncio.get_event_loop_policy().get_event_loop() is asyncio.get_running_loop()` is not always true, as mentioned in python/cpython#96377 (comment). The AsyncioSelectorReactor runs in the second thread and uses the already initialised event loop in the main thread to run the crawler, so a single event loop will be running code from two threads.
  • Loading branch information
auxsvr authored Feb 15, 2023
1 parent c8547b0 commit c4ea9f1
Showing 1 changed file with 2 additions and 5 deletions.
7 changes: 2 additions & 5 deletions scrapy/utils/reactor.py
Original file line number Diff line number Diff line change
Expand Up @@ -90,7 +90,6 @@ def _get_asyncio_event_loop():

def set_asyncio_event_loop(event_loop_path):
"""Sets and returns the event loop with specified import path."""
policy = get_asyncio_event_loop_policy()
if event_loop_path is not None:
event_loop_class = load_object(event_loop_path)
event_loop = event_loop_class()
Expand All @@ -109,15 +108,13 @@ def set_asyncio_event_loop(event_loop_path):
message="There is no current event loop",
category=DeprecationWarning,
)
event_loop = policy.get_event_loop()
event_loop = asyncio.get_event_loop()
except RuntimeError:
# `get_event_loop` raises RuntimeError when called with no asyncio
# event loop yet installed in the following scenarios:
# - From a thread other than the main thread. For example, when
# using ``scrapy shell``.
# - Previsibly on Python 3.14 and later.
# https://github.com/python/cpython/issues/100160#issuecomment-1345581902
event_loop = policy.new_event_loop()
event_loop = asyncio.new_event_loop()
asyncio.set_event_loop(event_loop)
return event_loop

Expand Down

0 comments on commit c4ea9f1

Please sign in to comment.