replace run_in_thread decorator with run_in_process

[wochinge]

Current Situation We use the[ run_in_thread decorator|

rasa/rasa/server.py

Line 575 in cbb6e7e

def run_in_thread(f: Callable[..., Coroutine]) -> Callable:

] to process blocking, computanional intense operations asynchronously and to avoid blocking the server.

Issue We recently had an issue where we found out that the pykwalify library was not used in a thread safe way. This was fixed but as @joejuzl pointed out:

Maybe it would be safer going forward to just stop using multithreading? We could easily run into more issues like this in the future unless we ensure the threadsafety of all the code we write, and all the libraries we use! We could use multiprocessing instead (should be a very easy change).

Implementation notes: While this might seem trivial, it isn't as the decorated functions are local functions and hence can't be pickled and also the Sanic request object can't be picked. You can reproduce the errors using:

def _run_in_process(f, request: Request, *args: Any, **kwargs: Any) -> None:

1\. This is a new thread, so we need to create and set a new event loop
 thread_loop = asyncio.new_event_loop()
 asyncio.set_event_loop(thread_loop)

 try:
 return thread_loop.run_until_complete(f(request, *args, **kwargs))
 finally:
 thread_loop.close()

def run_in_thread(f: Callable[..., Coroutine]) -> Callable:
 """Decorator which runs request on a separate thread.

 Some requests (e.g. training or cross-validation) are computional intense requests.
 This means that they will block the event loop and hence the processing of other
 requests. This decorator can be used to process these requests on a separate thread
 to avoid blocking the processing of incoming requests.

 Args:
 f: The request handler function which should be decorated.

 Returns:
 The decorated function.
 """

 @wraps(f)
 async def decorated_function(
 request: Request, *args: Any, **kwargs: Any
 ) -> HTTPResponse:
 with concurrent.futures.ProcessPoolExecutor() as pool:
 return await request.app.loop.run_in_executor(
 pool, functools.partial(_run_in_process, f, request, *args, **kwargs)
 )

 return decorated_function

Definition of Done

• run_in_thread is replaced by run_in_process

[stale]

Exalate commented:

stale[bot] commented:

This issue has been automatically marked as stale because it has not had recent activity. It will be closed if no further activity occurs. Thank you for your contributions.

[stale]

Exalate commented:

stale[bot] commented:

This issue has been automatically marked as stale because it has not had recent activity. It will be closed if no further activity occurs. Thank you for your contributions.

[sync-by-unito]

➤ Maxime Verger commented:

💡 Heads up! We're moving issues to Jira: https://rasa-open-source.atlassian.net/browse/OSS.

From now on, this Jira board is the place where you can browse (without an account) and create issues (you'll need a free Jira account for that). This GitHub issue has already been migrated to Jira and will be closed on January 9th, 2023. Do not forget to subscribe to the corresponding Jira issue!

➡️ More information in the forum: https://forum.rasa.com/t/migration-of-rasa-oss-issues-to-jira/56569.