Merge branch 'master' of github.com:python-trio/trio

Author: epellis

.coveragerc

@@ -6,6 +6,8 @@ source=trio
 omit=
   setup.py
   */ipython_custom_exc.py
+# Omit the generated files in trio/_core starting with _public_
+  */trio/_core/_generated_*
 # The test suite spawns subprocesses to test some stuff, so make sure
 # this doesn't corrupt the coverage files
 parallel=True

.yapfignore

@@ -0,0 +1 @@
+**/_generated*

check.sh

@@ -4,6 +4,10 @@ set -ex
 
 EXIT_STATUS=0
 
+# Test if the generated code is still up to date
+python ./trio/_tools/gen_exports.py --test \
+    || EXIT_STATUS=$?
+
 # Autoformatter *first*, to avoid double-reporting errors
 # (we'd like to run further autoformatters but *after* merging;
 # see https://forum.bors.tech/t/pre-test-and-pre-merge-hooks/322)

ci.sh

@@ -120,7 +120,7 @@ else
 
     INSTALLDIR=$(python -c "import os, trio; print(os.path.dirname(trio.__file__))")
     cp ../setup.cfg $INSTALLDIR
-    pytest -W error -ra --junitxml=../test-results.xml --run-slow --faulthandler-timeout=60 ${INSTALLDIR} --cov="$INSTALLDIR" --cov-config=../.coveragerc --verbose
+    pytest -W error -r a --junitxml=../test-results.xml --run-slow ${INSTALLDIR} --cov="$INSTALLDIR" --cov-config=../.coveragerc --verbose
 
     # Disable coverage on 3.8 until we run 3.8 on Windows CI too
     #   https://github.com/python-trio/trio/pull/784#issuecomment-446438407

docs-requirements.txt

@@ -26,10 +26,10 @@ pytz==2019.1              # via babel
 requests==2.22.0          # via sphinx
 six==1.12.0               # via packaging
 sniffio==1.1.0
-snowballstemmer==1.2.1    # via sphinx
+snowballstemmer==1.9.0    # via sphinx
 sortedcontainers==2.1.0
 sphinx-rtd-theme==0.4.3
-sphinx==2.1.1
+sphinx==2.1.2
 sphinxcontrib-applehelp==1.0.1  # via sphinx
 sphinxcontrib-devhelp==1.0.1  # via sphinx
 sphinxcontrib-htmlhelp==1.0.2  # via sphinx

docs/source/contributing.rst

@@ -167,6 +167,14 @@ to get feedback on, feel free to submit it as a PR. (In this case it's
 traditional to start the PR title with ``[WIP]``, for "work in
 progress".)
 
+When you are submitting your PR, you can include ``Closes #123``,
+``Fixes: #123`` or
+`some variation <https://help.github.com/en/articles/closing-issues-using-keywords>`__
+in either your commit message or the PR description, in order to
+automatically close the referenced issue when the PR is merged.
+This keeps us closer to the desired state where each open issue reflects some
+work that still needs to be done.
+
 
 .. _pull-request-tests:
 

docs/source/history.rst

@@ -220,9 +220,9 @@ Features
   (`#56 <https://github.com/python-trio/trio/issues/56>`__)
 - New and improved signal catching API: :func:`open_signal_receiver`. (`#354
   <https://github.com/python-trio/trio/issues/354>`__)
-- The low level :func:`trio.hazmat.wait_socket_readable`,
-  :func:`~trio.hazmat.wait_socket_writable`, and
-  :func:`~trio.hazmat.notify_socket_close` now work on bare socket descriptors,
+- The low level ``trio.hazmat.wait_socket_readable``,
+  ``wait_socket_writable``, and
+  ``notify_socket_close`` now work on bare socket descriptors,
   instead of requiring a :func:`socket.socket` object. (`#400
   <https://github.com/python-trio/trio/issues/400>`__)
 - If you're using :func:`trio.hazmat.wait_task_rescheduled` and other low-level
@@ -303,8 +303,8 @@ Features
   :exc:`ClosedResourceError`. ``ClosedStreamError`` and ``ClosedListenerError``
   are now aliases for :exc:`ClosedResourceError`, and deprecated. For this to
   work, Trio needs to know when a resource has been closed. To facilitate this,
-  new functions have been added: :func:`trio.hazmat.notify_fd_close` and
-  :func:`trio.hazmat.notify_socket_close`. If you're using Trio's built-in
+  new functions have been added: ``trio.hazmat.notify_fd_close`` and
+  ``trio.hazmat.notify_socket_close``. If you're using Trio's built-in
   wrappers like :class:`~trio.SocketStream` or :mod:`trio.socket`, then you don't
   need to worry about this, but if you're using the low-level functions like
   :func:`trio.hazmat.wait_readable`, you should make sure to call these

docs/source/reference-core.rst

@@ -1109,7 +1109,7 @@ Using channels to pass values between tasks
 different tasks. They're particularly useful for implementing
 producer/consumer patterns.
 
-The channel API is defined by the abstract base classes
+The core channel API is defined by the abstract base classes
 :class:`trio.abc.SendChannel` and :class:`trio.abc.ReceiveChannel`.
 You can use these to implement your own custom channels, that do
 things like pass objects between processes or over the network. But in
@@ -1125,14 +1125,23 @@ inside a single process, and for that you can use
    what you use when you're looking for a queue. The main difference
    is that Trio splits the classic queue interface up into two
    objects. The advantage of this is that it makes it possible to put
-   the two ends in different processes, and that we can close the two
-   sides separately.
+   the two ends in different processes without rewriting your code,
+   and that we can close the two sides separately.
+
+`MemorySendChannel` and `MemoryReceiveChannel` also expose several
+more features beyond the core channel interface:
+
+.. autoclass:: MemorySendChannel
+   :members:
+
+.. autoclass:: MemoryReceiveChannel
+   :members:
 
 
 A simple channel example
 ++++++++++++++++++++++++
 
-Here's a simple example of how to use channels:
+Here's a simple example of how to use memory channels:
 
 .. literalinclude:: reference-core/channels-simple.py
 
@@ -1244,14 +1253,13 @@ program above:
 .. literalinclude:: reference-core/channels-mpmc-fixed.py
    :emphasize-lines: 7, 9, 10, 12, 13
 
-This example demonstrates using the :meth:`SendChannel.clone
-<trio.abc.SendChannel.clone>` and :meth:`ReceiveChannel.clone
-<trio.abc.ReceiveChannel.clone>` methods. What these do is create
-copies of our endpoints, that act just like the original – except that
-they can be closed independently. And the underlying channel is only
-closed after *all* the clones have been closed. So this completely
-solves our problem with shutdown, and if you run this program, you'll
-see it print its six lines of output and then exits cleanly.
+This example demonstrates using the `MemorySendChannel.clone` and
+`MemoryReceiveChannel.clone` methods. What these do is create copies
+of our endpoints, that act just like the original – except that they
+can be closed independently. And the underlying channel is only closed
+after *all* the clones have been closed. So this completely solves our
+problem with shutdown, and if you run this program, you'll see it
+print its six lines of output and then exits cleanly.
 
 Notice a small trick we use: the code in ``main`` creates clone
 objects to pass into all the child tasks, and then closes the original

docs/source/reference-hazmat.rst

@@ -124,113 +124,54 @@ Universally available API
 
 All environments provide the following functions:
 
-.. function:: wait_socket_readable(sock)
+.. function:: wait_readable(obj)
    :async:
 
-   Block until the given :func:`socket.socket` object is readable.
+   Block until the kernel reports that the given object is readable.
 
-   On Unix systems, sockets are fds, and this is identical to
-   :func:`wait_readable`. On Windows, ``SOCKET`` handles and fds are
-   different, and this works on ``SOCKET`` handles or Python socket
-   objects.
+   On Unix systems, ``obj`` must either be an integer file descriptor,
+   or else an object with a ``.fileno()`` method which returns an
+   integer file descriptor. Any kind of file descriptor can be passed,
+   though the exact semantics will depend on your kernel. For example,
+   this probably won't do anything useful for on-disk files.
 
-   :raises trio.BusyResourceError:
-       if another task is already waiting for the given socket to
-       become readable.
-
-.. function:: wait_socket_writable(sock)
-   :async:
-
-   Block until the given :func:`socket.socket` object is writable.
-
-   On Unix systems, sockets are fds, and this is identical to
-   :func:`wait_writable`. On Windows, ``SOCKET`` handles and fds are
-   different, and this works on ``SOCKET`` handles or Python socket
-   objects.
+   On Windows systems, ``obj`` must either be an integer ``SOCKET``
+   handle, or else an object with a ``.fileno()`` method which returns
+   an integer ``SOCKET`` handle. File descriptors aren't supported,
+   and neither are handles that refer to anything besides a
+   ``SOCKET``.
 
    :raises trio.BusyResourceError:
        if another task is already waiting for the given socket to
-       become writable.
-   :raises trio.ClosedResourceError:
-       if another task calls :func:`notify_socket_close` while this
-       function is still working.
-
-
-.. function:: notify_socket_close(sock)
-
-   Notifies Trio's internal I/O machinery that you are about to close
-   a socket.
-
-   This causes any operations currently waiting for this socket to
-   immediately raise :exc:`~trio.ClosedResourceError`.
-
-   This does *not* actually close the socket. Generally when closing a
-   socket, you should first call this function, and then close the
-   socket.
-
-   On Unix systems, sockets are fds, and this is identical to
-   :func:`notify_fd_close`. On Windows, ``SOCKET`` handles and fds are
-   different, and this works on ``SOCKET`` handles or Python socket
-   objects.
-
-
-Unix-specific API
------------------
-
-Unix-like systems provide the following functions:
-
-.. function:: wait_readable(fd)
-   :async:
-
-   Block until the given file descriptor is readable.
-
-   .. warning::
-
-      This is "readable" according to the operating system's
-      definition of readable. In particular, it probably won't tell
-      you anything useful for on-disk files.
-
-   :arg fd:
-       integer file descriptor, or else an object with a ``fileno()`` method
-   :raises trio.BusyResourceError:
-       if another task is already waiting for the given fd to
        become readable.
    :raises trio.ClosedResourceError:
-       if another task calls :func:`notify_fd_close` while this
+       if another task calls :func:`notify_closing` while this
        function is still working.
 
-
-.. function:: wait_writable(fd)
+.. function:: wait_writable(obj)
    :async:
 
-   Block until the given file descriptor is writable.
-
-   .. warning::
+   Block until the kernel reports that the given object is writable.
 
-      This is "writable" according to the operating system's
-      definition of writable. In particular, it probably won't tell
-      you anything useful for on-disk files.
+   See `wait_readable` for the definition of ``obj``.
 
-   :arg fd:
-       integer file descriptor, or else an object with a ``fileno()`` method
    :raises trio.BusyResourceError:
-       if another task is already waiting for the given fd to
+       if another task is already waiting for the given socket to
        become writable.
    :raises trio.ClosedResourceError:
-       if another task calls :func:`notify_fd_close` while this
+       if another task calls :func:`notify_closing` while this
        function is still working.
 
-.. function:: notify_fd_close(fd)
 
-   Notifies Trio's internal I/O machinery that you are about to close
-   a file descriptor.
+.. function:: notify_closing(obj)
 
-   This causes any operations currently waiting for this file
-   descriptor to immediately raise :exc:`~trio.ClosedResourceError`.
+   Call this before closing a file descriptor or ``SOCKET`` handle
+   that another task might be waiting on. This will cause any
+   `wait_readable` or `wait_writable` calls to immediately raise
+   `~trio.ClosedResourceError`.
 
-   This does *not* actually close the file descriptor. Generally when
-   closing a file descriptor, you should first call this function, and
-   then actually close it.
+   This doesn't actually close the object – you still have to do that
+   yourself afterwards.
 
 
 Kqueue-specific API

docs/source/reference-io.rst

@@ -117,14 +117,19 @@ Abstract base classes
      - :class:`~trio.SocketListener`, :class:`~trio.SSLListener`
    * - :class:`SendChannel`
      - :class:`AsyncResource`
-     - :meth:`~SendChannel.send`, :meth:`~SendChannel.send_nowait`
+     - :meth:`~SendChannel.send`
      -
-     - :func:`~trio.open_memory_channel`
+     - `~trio.MemorySendChannel`
    * - :class:`ReceiveChannel`
      - :class:`AsyncResource`
-     - :meth:`~ReceiveChannel.receive`, :meth:`~ReceiveChannel.receive_nowait`
+     - :meth:`~ReceiveChannel.receive`
      - ``__aiter__``, ``__anext__``
-     - :func:`~trio.open_memory_channel`
+     - `~trio.MemoryReceiveChannel`
+   * - `Channel`
+     - `SendChannel`, `ReceiveChannel`
+     -
+     -
+     -
 
 .. autoclass:: trio.abc.AsyncResource
    :members:
@@ -165,6 +170,10 @@ Abstract base classes
    :members:
    :show-inheritance:
 
+.. autoclass:: trio.abc.Channel
+   :members:
+   :show-inheritance:
+
 .. currentmodule:: trio
 
 
@@ -662,12 +671,12 @@ for them to exit. The interface for doing so consists of two layers:
   the standard :func:`subprocess.run` with some additional features
   and safer defaults.
 
-* :class:`trio.Process` starts a process in the background and optionally
-  provides Trio streams for interacting with it (sending input,
-  receiving output and errors). Using it requires a bit more code
-  than :func:`~trio.run_process`, but exposes additional capabilities:
-  back-and-forth communication, processing output as soon as it is generated,
-  and so forth. It is modelled after the standard :class:`subprocess.Popen`.
+* `trio.open_process` starts a process in the background and returns a
+  `Process` object to let you interact with it. Using it requires a
+  bit more code than `run_process`, but exposes additional
+  capabilities: back-and-forth communication, processing output as
+  soon as it is generated, and so forth. It is modelled after the
+  standard library :class:`subprocess.Popen`.
 
 
 .. _subprocess-options:
@@ -713,12 +722,11 @@ course, these defaults can be changed where necessary.
 Interacting with a process as it runs
 ~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~
 
-If you want more control than :func:`~trio.run_process` affords,
-you can spawn a subprocess by creating an instance of
-:class:`trio.Process` and then interact with it using its
-:attr:`~trio.Process.stdin`,
-:attr:`~trio.Process.stdout`, and/or
-:attr:`~trio.Process.stderr` streams.
+If you want more control than :func:`~trio.run_process` affords, you
+can use `trio.open_process` to spawn a subprocess, and then interact
+with it using the `Process` interface.
+
+.. autofunction:: trio.open_process
 
 .. autoclass:: trio.Process