Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

In to_thread_run_sync(), add abandon_on_cancel= as an alias for the cancellable= flag #2841

Merged
merged 9 commits into from
Nov 2, 2023
2 changes: 1 addition & 1 deletion docs/source/reference-core.rst
Original file line number Diff line number Diff line change
Expand Up @@ -1836,7 +1836,7 @@ to spawn a child thread, and then use a :ref:`memory channel

You can also use :func:`trio.from_thread.check_cancelled` to check for cancellation from
a thread that was spawned by :func:`trio.to_thread.run_sync`. If the call to
:func:`~trio.to_thread.run_sync` was cancelled (even if ``abandon_on_cancel=False``!), then
:func:`~trio.to_thread.run_sync` was cancelled, then
:func:`~trio.from_thread.check_cancelled` will raise :func:`trio.Cancelled`.
It's like ``trio.from_thread.run(trio.sleep, 0)``, but much faster.

Expand Down
3 changes: 0 additions & 3 deletions newsfragments/2841.feature.rst

This file was deleted.

8 changes: 8 additions & 0 deletions newsfragments/2841.removal.rst
Original file line number Diff line number Diff line change
@@ -0,0 +1,8 @@
To better reflect the underlying thread handling semantics,
the keyword argument for `trio.to_thread.run_sync` that was
previously called ``cancellable`` is now named ``abandon_on_cancel``.
It still does the same thing -- allow the thread to be abandoned
if the call to `trio.to_thread.run_sync` is cancelled -- but since we now
have other ways to propagate a cancellation without abandoning
the thread, "cancellable" has become somewhat of a misnomer.
The old ``cancellable`` name is now deprecated.
28 changes: 22 additions & 6 deletions trio/_tests/test_threads.py
Original file line number Diff line number Diff line change
Expand Up @@ -337,7 +337,7 @@
q.get()
register[0] = "finished"

async def child(q, abandon_on_cancel):
async def child(q: stdlib_queue.Queue[None], abandon_on_cancel: bool) -> None:
record.append("start")
try:
return await to_thread_run_sync(f, q, abandon_on_cancel=abandon_on_cancel)
Expand Down Expand Up @@ -400,8 +400,8 @@
q1.get()
q2.put(threading.current_thread())

async def main():
async def child():
async def main() -> None:
async def child() -> None:
await to_thread_run_sync(thread_fn, abandon_on_cancel=True)

async with _core.open_nursery() as nursery:
Expand Down Expand Up @@ -556,7 +556,7 @@

# TODO: should CapacityLimiter have an abc or protocol so users can modify it?
# because currently it's `final` so writing code like this is not allowed.
await to_thread_run_sync(lambda: None, limiter=CustomLimiter()) # type: ignore[arg-type]

Check failure on line 559 in trio/_tests/test_threads.py

View workflow job for this annotation

GitHub Actions / Ubuntu (3.8, check formatting)

Mypy-Linux+Mac+Windows

trio/_tests/test_threads.py:559: Unused "type: ignore" comment [unused-ignore]

Check failure on line 559 in trio/_tests/test_threads.py

View workflow job for this annotation

GitHub Actions / Ubuntu (3.8, check formatting)

Mypy-Linux+Mac+Windows

trio/_tests/test_threads.py:(559:11 - 559:67): No overload variant of "to_thread_run_sync" matches argument types "Callable[[], None]", "CustomLimiter" [call-overload]

Check notice on line 559 in trio/_tests/test_threads.py

View workflow job for this annotation

GitHub Actions / Ubuntu (3.8, check formatting)

Mypy-Linux+Mac+Windows

trio/_tests/test_threads.py:(559:11 - 559:67): Error code "call-overload" not covered by "type: ignore" comment

Check notice on line 559 in trio/_tests/test_threads.py

View workflow job for this annotation

GitHub Actions / Ubuntu (3.8, check formatting)

Mypy-Linux+Mac+Windows

trio/_tests/test_threads.py:(559:11 - 559:67): Possible overload variants:

Check notice on line 559 in trio/_tests/test_threads.py

View workflow job for this annotation

GitHub Actions / Ubuntu (3.8, check formatting)

Mypy-Linux+Mac+Windows

trio/_tests/test_threads.py:(559:11 - 559:67): def [RetT] to_thread_run_sync(sync_fn: Callable[..., RetT], *args: object, thread_name: Optional[str] = ..., abandon_on_cancel: bool = ..., limiter: Optional[CapacityLimiter] = ...) -> Coroutine[Any, Any, RetT]

Check notice on line 559 in trio/_tests/test_threads.py

View workflow job for this annotation

GitHub Actions / Ubuntu (3.8, check formatting)

Mypy-Linux+Mac+Windows

trio/_tests/test_threads.py:(559:11 - 559:67): def [RetT] to_thread_run_sync(sync_fn: Callable[..., RetT], *args: object, thread_name: Optional[str] = ..., cancellable: bool = ..., limiter: Optional[CapacityLimiter] = ...) -> Coroutine[Any, Any, RetT]
assert record == ["acquire", "release"]


Expand All @@ -574,7 +574,7 @@
bs = BadCapacityLimiter()

with pytest.raises(ValueError) as excinfo:
await to_thread_run_sync(lambda: None, limiter=bs) # type: ignore[arg-type]

Check failure on line 577 in trio/_tests/test_threads.py

View workflow job for this annotation

GitHub Actions / Ubuntu (3.8, check formatting)

Mypy-Linux+Mac+Windows

trio/_tests/test_threads.py:577: Unused "type: ignore" comment [unused-ignore]

Check failure on line 577 in trio/_tests/test_threads.py

View workflow job for this annotation

GitHub Actions / Ubuntu (3.8, check formatting)

Mypy-Linux+Mac+Windows

trio/_tests/test_threads.py:(577:15 - 577:58): No overload variant of "to_thread_run_sync" matches argument types "Callable[[], None]", "BadCapacityLimiter" [call-overload]

Check notice on line 577 in trio/_tests/test_threads.py

View workflow job for this annotation

GitHub Actions / Ubuntu (3.8, check formatting)

Mypy-Linux+Mac+Windows

trio/_tests/test_threads.py:(577:15 - 577:58): Error code "call-overload" not covered by "type: ignore" comment

Check notice on line 577 in trio/_tests/test_threads.py

View workflow job for this annotation

GitHub Actions / Ubuntu (3.8, check formatting)

Mypy-Linux+Mac+Windows

trio/_tests/test_threads.py:(577:15 - 577:58): Possible overload variants:

Check notice on line 577 in trio/_tests/test_threads.py

View workflow job for this annotation

GitHub Actions / Ubuntu (3.8, check formatting)

Mypy-Linux+Mac+Windows

trio/_tests/test_threads.py:(577:15 - 577:58): def [RetT] to_thread_run_sync(sync_fn: Callable[..., RetT], *args: object, thread_name: Optional[str] = ..., abandon_on_cancel: bool = ..., limiter: Optional[CapacityLimiter] = ...) -> Coroutine[Any, Any, RetT]

Check notice on line 577 in trio/_tests/test_threads.py

View workflow job for this annotation

GitHub Actions / Ubuntu (3.8, check formatting)

Mypy-Linux+Mac+Windows

trio/_tests/test_threads.py:(577:15 - 577:58): def [RetT] to_thread_run_sync(sync_fn: Callable[..., RetT], *args: object, thread_name: Optional[str] = ..., cancellable: bool = ..., limiter: Optional[CapacityLimiter] = ...) -> Coroutine[Any, Any, RetT]
assert excinfo.value.__context__ is None
assert record == ["acquire", "release"]
record = []
Expand All @@ -583,7 +583,7 @@
# chains with it
d: dict[str, object] = {}
with pytest.raises(ValueError) as excinfo:
await to_thread_run_sync(lambda: d["x"], limiter=bs) # type: ignore[arg-type]

Check failure on line 586 in trio/_tests/test_threads.py

View workflow job for this annotation

GitHub Actions / Ubuntu (3.8, check formatting)

Mypy-Linux+Mac+Windows

trio/_tests/test_threads.py:586: Unused "type: ignore" comment [unused-ignore]

Check failure on line 586 in trio/_tests/test_threads.py

View workflow job for this annotation

GitHub Actions / Ubuntu (3.8, check formatting)

Mypy-Linux+Mac+Windows

trio/_tests/test_threads.py:(586:15 - 586:60): No overload variant of "to_thread_run_sync" matches argument types "Callable[[], object]", "BadCapacityLimiter" [call-overload]

Check notice on line 586 in trio/_tests/test_threads.py

View workflow job for this annotation

GitHub Actions / Ubuntu (3.8, check formatting)

Mypy-Linux+Mac+Windows

trio/_tests/test_threads.py:(586:15 - 586:60): Error code "call-overload" not covered by "type: ignore" comment

Check notice on line 586 in trio/_tests/test_threads.py

View workflow job for this annotation

GitHub Actions / Ubuntu (3.8, check formatting)

Mypy-Linux+Mac+Windows

trio/_tests/test_threads.py:(586:15 - 586:60): Possible overload variants:
assert isinstance(excinfo.value.__context__, KeyError)
assert record == ["acquire", "release"]

Expand Down Expand Up @@ -884,15 +884,15 @@
assert token is weak_reference()


async def test_unsafe_abandon_on_cancel_kwarg():
async def test_unsafe_abandon_on_cancel_kwarg() -> None:
# This is a stand in for a numpy ndarray or other objects
# that (maybe surprisingly) lack a notion of truthiness
class BadBool:
def __bool__(self) -> bool:
raise NotImplementedError

with pytest.raises(NotImplementedError):
await to_thread_run_sync(int, abandon_on_cancel=BadBool())
await to_thread_run_sync(int, abandon_on_cancel=BadBool()) # type: ignore[arg-type]

Check failure on line 895 in trio/_tests/test_threads.py

View workflow job for this annotation

GitHub Actions / Ubuntu (3.8, check formatting)

Mypy-Linux+Mac+Windows

trio/_tests/test_threads.py:895: Unused "type: ignore" comment [unused-ignore]
richardsheridan marked this conversation as resolved.
Show resolved Hide resolved


async def test_from_thread_reuses_task() -> None:
Expand Down Expand Up @@ -979,7 +979,7 @@
async def test_from_thread_check_cancelled() -> None:
q: stdlib_queue.Queue[str] = stdlib_queue.Queue()

async def child(abandon_on_cancel, scope):
async def child(abandon_on_cancel: bool, scope: CancelScope) -> None:
with scope:
record.append("start")
try:
Expand Down Expand Up @@ -1077,3 +1077,19 @@
async with _core.open_nursery() as nursery:
for _ in range(4):
nursery.start_soon(child)


async def test_cancellable_and_abandon_raises() -> None:
with pytest.raises(ValueError):
await to_thread_run_sync(bool, cancellable=True, abandon_on_cancel=False)

with pytest.raises(ValueError):
await to_thread_run_sync(bool, cancellable=True, abandon_on_cancel=True)
richardsheridan marked this conversation as resolved.
Show resolved Hide resolved


async def test_cancellable_warns() -> None:
with pytest.warns(DeprecationWarning):
await to_thread_run_sync(bool, cancellable=False)

with pytest.warns(DeprecationWarning):
await to_thread_run_sync(bool, cancellable=True)
46 changes: 39 additions & 7 deletions trio/_threads.py
Original file line number Diff line number Diff line change
Expand Up @@ -6,9 +6,10 @@
import inspect
import queue as stdlib_queue
import threading
import warnings
from collections.abc import Awaitable, Callable
from itertools import count
from typing import Generic, TypeVar
from typing import Generic, TypeVar, overload

import attr
import outcome
Expand Down Expand Up @@ -171,13 +172,35 @@
token.run_sync_soon(self.run_sync)


@overload
async def to_thread_run_sync(
sync_fn: Callable[..., RetT],
*args: object,
thread_name: str | None = None,
abandon_on_cancel: bool = False,
limiter: CapacityLimiter | None = None,
) -> RetT:
...

Check failure on line 183 in trio/_threads.py

View workflow job for this annotation

GitHub Actions / Ubuntu (3.8, check formatting)

Mypy-Linux+Mac+Windows

trio/_threads.py:(176:1 - 183:7): Type of decorated function contains type "Any" ("Callable[[Callable[..., RetT], VarArg(object), DefaultNamedArg(Optional[str], 'thread_name'), DefaultNamedArg(bool, 'abandon_on_cancel'), DefaultNamedArg(Optional[CapacityLimiter], 'limiter')], Coroutine[Any, Any, RetT]]") [misc]


@overload
async def to_thread_run_sync(
sync_fn: Callable[..., RetT],
*args: object,
thread_name: str | None = None,
cancellable: bool = False,
limiter: CapacityLimiter | None = None,
) -> RetT:
...

Check failure on line 194 in trio/_threads.py

View workflow job for this annotation

GitHub Actions / Ubuntu (3.8, check formatting)

Mypy-Linux+Mac+Windows

trio/_threads.py:(187:1 - 194:7): Type of decorated function contains type "Any" ("Callable[[Callable[..., RetT], VarArg(object), DefaultNamedArg(Optional[str], 'thread_name'), DefaultNamedArg(bool, 'cancellable'), DefaultNamedArg(Optional[CapacityLimiter], 'limiter')], Coroutine[Any, Any, RetT]]") [misc]


@enable_ki_protection # Decorator used on function with Coroutine[Any, Any, RetT]
async def to_thread_run_sync( # type: ignore[misc]
richardsheridan marked this conversation as resolved.
Show resolved Hide resolved
sync_fn: Callable[..., RetT],
*args: object,
thread_name: str | None = None,
abandon_on_cancel: bool | None = None,
cancellable: bool = False,
cancellable: bool | None = None,
limiter: CapacityLimiter | None = None,
) -> RetT:
"""Convert a blocking operation into an async operation using a thread.
Expand All @@ -201,8 +224,6 @@
arguments, use :func:`functools.partial`.
abandon_on_cancel (bool): Whether to abandon this thread upon
cancellation of this operation. See discussion below.
cancellable (bool): *Deprecated* synonym for ``abandon_on_cancel``.
Providing a value to ``abandon_on_cancel`` overrides this argument.
thread_name (str): Optional string to set the name of the thread.
Will always set `threading.Thread.name`, but only set the os name
if pthread.h is available (i.e. most POSIX installations).
Expand Down Expand Up @@ -266,9 +287,20 @@

"""
await trio.lowlevel.checkpoint_if_cancelled()
if abandon_on_cancel is not None:
cancellable = abandon_on_cancel
abandon_on_cancel = bool(cancellable) # raise early if cancellable.__bool__ raises
if cancellable is not None:
if abandon_on_cancel is not None:
raise ValueError(
"Cannot set `cancellable` and `abandon_on_cancel` simultaneously."
)
warnings.warn(
richardsheridan marked this conversation as resolved.
Show resolved Hide resolved
DeprecationWarning(
"`cancellable` keyword argument is deprecated, "
"use `abandon on cancel` instead."
)
)
abandon_on_cancel = cancellable
# raise early if abandon_on_cancel.__bool__ raises
abandon_on_cancel = bool(abandon_on_cancel)
if limiter is None:
limiter = current_default_thread_limiter()

Expand Down
Loading