Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

process = await nursery.start(run_process, ...) #1568

Merged
merged 16 commits into from
Oct 13, 2021
Merged
6 changes: 3 additions & 3 deletions docs/source/history.rst
Original file line number Diff line number Diff line change
Expand Up @@ -213,7 +213,7 @@ Features
worry: you can now pass a custom ``deliver_cancel=`` argument to
define your own process killing policy. (`#1104 <https://github.com/python-trio/trio/issues/1104>`__)
- It turns out that creating a subprocess can block the parent process
for a surprisingly long time. So `trio.open_process` now uses a worker
for a surprisingly long time. So ``trio.open_process`` now uses a worker
thread to avoid blocking the event loop. (`#1109 <https://github.com/python-trio/trio/issues/1109>`__)
- We've added FreeBSD to the list of platforms we support and test on. (`#1118 <https://github.com/python-trio/trio/issues/1118>`__)
- On Linux kernels v5.3 or newer, `trio.Process.wait` now uses `the
Expand Down Expand Up @@ -267,7 +267,7 @@ Deprecations and Removals
alternatives or make a case for why some particular class should be
designed to support subclassing. (`#1044 <https://github.com/python-trio/trio/issues/1044>`__)
- If you want to create a `trio.Process` object, you now have to call
`trio.open_process`; calling ``trio.Process()`` directly was
``trio.open_process``; calling ``trio.Process()`` directly was
deprecated in v0.12.0 and has now been removed. (`#1109 <https://github.com/python-trio/trio/issues/1109>`__)
- Remove ``clear`` method on `trio.Event`: it was deprecated in 0.12.0. (`#1498 <https://github.com/python-trio/trio/issues/1498>`__)

Expand Down Expand Up @@ -495,7 +495,7 @@ Deprecations and Removals
deprecated. (`#878 <https://github.com/python-trio/trio/issues/878>`__)
- It turns out that it's better to treat subprocess spawning as an async
operation. Therefore, direct construction of `Process` objects has
been deprecated. Use `trio.open_process` instead. (`#1109 <https://github.com/python-trio/trio/issues/1109>`__)
been deprecated. Use ``trio.open_process`` instead. (`#1109 <https://github.com/python-trio/trio/issues/1109>`__)


Miscellaneous internal changes
Expand Down
101 changes: 35 additions & 66 deletions docs/source/reference-io.rst
Original file line number Diff line number Diff line change
Expand Up @@ -664,22 +664,43 @@ Spawning subprocesses

Trio provides support for spawning other programs as subprocesses,
communicating with them via pipes, sending them signals, and waiting
for them to exit. The interface for doing so consists of two layers:
for them to exit.

* :func:`trio.run_process` runs a process from start to
finish and returns a :class:`~subprocess.CompletedProcess` object describing
its outputs and return value. This is what you should reach for if you
want to run a process to completion before continuing, while possibly
sending it some input or capturing its output. It is modelled after
the standard :func:`subprocess.run` with some additional features
and safer defaults.
Most of the time, this is done through our high-level interface,
`trio.run_process`. It lets you either run a process to completion
while optionally capturing the output, or else run it in a background
task and interact with it while it's running:

* `trio.open_process` starts a process in the background and returns a
`Process` object to let you interact with it. Using it requires a
bit more code than `run_process`, but exposes additional
capabilities: back-and-forth communication, processing output as
soon as it is generated, and so forth. It is modelled after the
standard library :class:`subprocess.Popen`.
.. autofunction:: trio.run_process

.. autoclass:: trio.Process

.. autoattribute:: returncode

.. automethod:: wait

.. automethod:: poll

.. automethod:: kill

.. automethod:: terminate

.. automethod:: send_signal

.. note:: :meth:`~subprocess.Popen.communicate` is not provided as a
method on :class:`~trio.Process` objects; call :func:`~trio.run_process`
normally for simple capturing, or write the loop yourself if you
have unusual needs. :meth:`~subprocess.Popen.communicate` has
quite unusual cancellation behavior in the standard library (on
some platforms it spawns a background thread which continues to
read from the child process even after the timeout has expired)
and we wanted to provide an interface with fewer surprises.

If `trio.run_process` is too limiting, we also offer a low-level API,
`trio.lowlevel.open_process`. For example, if you want to spawn a
child process that will and outlive the parent process and be
richardsheridan marked this conversation as resolved.
Show resolved Hide resolved
orphaned, then `~trio.run_process` can't do that, but
`~trio.lowlevel.open_process` can.


.. _subprocess-options:
Expand All @@ -705,58 +726,6 @@ with a process, so it does not support the ``encoding``, ``errors``,
options.


Running a process and waiting for it to finish
~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~

The basic interface for running a subprocess start-to-finish is
:func:`trio.run_process`. It always waits for the subprocess to exit
before returning, so there's no need to worry about leaving a process
running by mistake after you've gone on to do other things.
:func:`~trio.run_process` is similar to the standard library
:func:`subprocess.run` function, but tries to have safer defaults:
with no options, the subprocess's input is empty rather than coming
from the user's terminal, and a failure in the subprocess will be
propagated as a :exc:`subprocess.CalledProcessError` exception. Of
course, these defaults can be changed where necessary.

.. autofunction:: trio.run_process


Interacting with a process as it runs
~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~

If you want more control than :func:`~trio.run_process` affords, you
can use `trio.open_process` to spawn a subprocess, and then interact
with it using the `Process` interface.

.. autofunction:: trio.open_process

.. autoclass:: trio.Process

.. autoattribute:: returncode

.. automethod:: aclose

.. automethod:: wait

.. automethod:: poll

.. automethod:: kill

.. automethod:: terminate

.. automethod:: send_signal

.. note:: :meth:`~subprocess.Popen.communicate` is not provided as a
method on :class:`~trio.Process` objects; use :func:`~trio.run_process`
instead, or write the loop yourself if you have unusual
needs. :meth:`~subprocess.Popen.communicate` has quite unusual
cancellation behavior in the standard library (on some platforms it
spawns a background thread which continues to read from the child
process even after the timeout has expired) and we wanted to
provide an interface with fewer surprises.


.. _subprocess-quoting:

Quoting: more than you wanted to know
Expand Down
6 changes: 6 additions & 0 deletions docs/source/reference-lowlevel.rst
Original file line number Diff line number Diff line change
Expand Up @@ -106,6 +106,12 @@ The tutorial has a :ref:`fully-worked example
Trio's internal scheduling decisions.


Low-level process spawning
==========================

.. autofunction:: trio.lowlevel.open_process


Low-level I/O primitives
========================

Expand Down
3 changes: 3 additions & 0 deletions newsfragments/1104.feature.rst
Original file line number Diff line number Diff line change
@@ -0,0 +1,3 @@
You can now conveniently spawn a child process in a background task
and interact it with on the fly using ``process = await
nursery.start(run_process, ...)``. See `run_process` for more details.
5 changes: 5 additions & 0 deletions newsfragments/1104.removal.rst
Original file line number Diff line number Diff line change
@@ -0,0 +1,5 @@
``trio.open_process`` has been renamed to
`trio.lowlevel.open_process`, and the ``aclose`` method on `Process`
has been deprecated, along with ``async with process_obj``. We
recommend most users switch to the new
``nursery.start(trio.run_process, ...)`` API instead.
11 changes: 10 additions & 1 deletion trio/__init__.py
Original file line number Diff line number Diff line change
Expand Up @@ -70,7 +70,7 @@

from ._path import Path

from ._subprocess import Process, open_process, run_process
from ._subprocess import Process, run_process

from ._ssl import SSLStream, SSLListener, NeedHandshakeError

Expand Down Expand Up @@ -106,6 +106,15 @@

_deprecate.enable_attribute_deprecations(__name__)

__deprecated_attributes__ = {
"open_process": _deprecate.DeprecatedAttribute(
value=lowlevel.open_process,
version="0.20.0",
issue=1104,
instead="trio.lowlevel.open_process",
),
}

# Having the public path in .__module__ attributes is important for:
# - exception names in printed tracebacks
# - sphinx :show-inheritance:
Expand Down
Loading