Merge pull request #69 from oremanj/deprecations

Clean up public API surface

Author: pquentin

docs/source/conf.py

@@ -33,6 +33,7 @@
 ]
 
 autodoc_inherit_docstrings = False
+default_role = "obj"
 
 # -- General configuration ------------------------------------------------
 

docs/source/history.rst

@@ -5,19 +5,27 @@ Release history
 
 .. towncrier release notes start
 
-Trio_Asyncio 0.8.1 (2018-08-25)
+trio-asyncio 0.8.1 (2018-08-25)
 -------------------------------
 
 Features
 ~~~~~~~~
 
-- `trio_ayncio` now contains an `allow_asyncio` wrapper which allows you to
-  seamlessly mix asyncio and trio semantics:: from trio_asyncio import run
-  allow_asyncio async def main(): print("Sleeping 1") await asyncio.sleep(1)
-  print("Sleeping 2") await trio.sleep(1) run(allow_asyncio, main) The problem
-  with this solution is that Trio's cancellations will no longer be converted
-  to asyncio's `CancelledError` within asyncio code. This may or may not be an
-  issue for your code. (`#30
+- `trio_asyncio` now contains an `allow_asyncio` wrapper which allows you to
+  seamlessly mix asyncio and trio semantics::
+
+      from trio_asyncio import run
+      @allow_asyncio
+      async def main():
+          print("Sleeping 1")
+	  await asyncio.sleep(1)
+          print("Sleeping 2")
+	  await trio.sleep(1)
+      run(allow_asyncio, main)
+
+  The problem with this solution is that Trio's cancellations will no
+  longer be converted to asyncio's `~asyncio.CancelledError` within asyncio
+  code. This may or may not be an issue for your code. (`#30
   <https://github.com/python-trio/trio-asyncio/issues/30>`__)
 
 

docs/source/index.rst

@@ -8,34 +8,50 @@
 trio-asyncio: A re-implementation of the asyncio mainloop on top of Trio
 ========================================================================
 
-Trio-Asyncio is *the* library of choice for a Python program that 
-contains both `trio`_ and `asyncio`_ code.
+trio-asyncio is *the* library of choice for a Python program that
+contains both `Trio <https://trio.readthedocs.io/en/stable/>`__ and
+`asyncio` code.
 
-.. _asyncio: https://docs.python.org/3/library/asyncio.html
+Trio has native concepts of tasks and task cancellation. Asyncio is based
+on callbacks and chaining Futures, albeit with nicer syntax, which make
+handling of failures and timeouts fundamentally less reliable, especially in
+larger programs. Thus, you really want to use Trio in your project.
+
+On the other hand, there are quite a few robust libraries that have been
+implemented using asyncio, while Trio's ecosystem is relatively younger. You
+really don't want to re-invent any wheels in your project.
+
+Thus, being able to use asyncio libraries from Trio is useful.
+trio-asyncio enables you to do that, and more.
 
 With trio-asyncio, you can:
 
-* incrementally convert your code to Trio. Start with a Trio mainloop, call
-  your existing asyncio code, then successively convert procedures to Trio
-  calling conventions.
+* Incrementally convert an asyncio application to Trio. Start with a
+  Trio mainloop, call your existing asyncio code, then successively
+  convert procedures to Trio calling conventions.
 
-* use any asyncio-capable library.
+* Use any asyncio-capable library in a Trio application.
 
-* use trio-asyncio as a building block for convincing other async-ish
+* Use trio-asyncio as a building block for convincing other async-ish
   libraries (Twisted, Promise, …) to be compatible with Trio.
 
-Trio-Asyncio passes the test suite of some complex programs like
-``home-assistant``.
+trio-asyncio is tested against the complete asyncio test suite as
+shipped with each supported version of Python, although a small number of tests
+fail due to our limited support for starting and stopping the same event
+loop multiple times. It has also passed the test suite of some complex
+asyncio libraries such as ``home-assistant``.
 
-In the past, the complete Python 3.6 test suite for asyncio and the tests
-for some well-known libraries like aiohttp also Just Work(ed). This is no
-longer the case because these tests rely heavily on stopping and restarting
-the ``asyncio`` event loop. ``trio_asyncio`` no longer supports this.
+.. note:: trio-asyncio is most useful for *applications*: it works best
+	  when you control the code that starts the event loop (such as
+	  the call to :func:`asyncio.run`). If you're writing a *library* and
+	  want to adopt a Trio-ish worldview without sacrificing asyncio
+	  compatibility, you might find `anyio <https://anyio.readthedocs.io/en/latest/>`__
+          helpful.
 
 Helpful facts:
 
 * Supported environments: Linux, MacOS, or Windows running some kind of Python
-  3.5.3-or-better (either CPython or PyPy3 is fine). \*BSD and illumus likely
+  3.5.3-or-better (either CPython or PyPy3 is fine). \*BSD and illumOS likely
   work too, but are untested.
 
 * Install: ``python3 -m pip install -U trio-asyncio`` (or on Windows, maybe
@@ -45,7 +61,7 @@ Helpful facts:
 
 * Bug tracker and source code: https://github.com/python-trio/trio-asyncio
 
-Inherited from `Trio <https://github.com/python-trio/trio>`_:
+Inherited from `Trio <https://github.com/python-trio/trio>`__:
 
 * Real-time chat: https://gitter.im/python-trio/general
 
@@ -57,10 +73,14 @@ Inherited from `Trio <https://github.com/python-trio/trio>`_:
   conduct <https://trio.readthedocs.io/en/latest/code-of-conduct.html>`_
   in all project spaces.
 
+=====================
+ trio-asyncio manual
+=====================
+
 .. toctree::
    :maxdepth: 2
 
-   rationale.rst
+   principles.rst
    usage.rst
    history.rst
 

docs/source/principles.rst

@@ -0,0 +1,177 @@
+.. currentmodule:: trio_asyncio
+
+++++++++++++
+ Principles
+++++++++++++
+
+--------------------------------------
+ Async function "flavors"
+--------------------------------------
+
+As you might recall from the discussion of async "sandwiches" in the
+`Trio tutorial
+<https://trio.readthedocs.io/en/stable/tutorial.html#async-sandwich>`__,
+every async function ultimately must do its useful work by directly or
+indirectly calling back into the same async library (such as asyncio
+or Trio) that's managing the currently running event loop. If a
+function invoked within a :func:`trio.run` calls
+:func:`asyncio.sleep`, or a function invoked within an
+:func:`asyncio.run` calls :func:`trio.sleep`, the sleep function will
+send a message to the event loop that the event loop doesn't know how
+to handle, and some sort of error will result.
+
+In a program that uses trio-asyncio, you probably have some async
+functions implemented in terms of Trio calls and some implemented in
+terms of asyncio calls. In order to keep track of which is which,
+we'll call these "Trio-flavored" and "asyncio-flavored" functions,
+respectively.  It is critical that you understand which functions in
+your program are Trio-flavored and which are asyncio-flavored, just
+like it's critical that you understand which functions are synchronous
+and which ones are async. Unfortunately, there's no syntactic marker
+for flavor: both Trio-flavored and asyncio-flavored functions are
+defined with ``async def fn()`` and call other async functions with
+``await other_fn()``.  You'll have to keep track of it some other
+way. To help you out, every function in trio-asyncio documents its
+flavor, and we recommend that you follow this convention in your own
+programs too.
+
+The general rules that determine flavor are as follows:
+
+* Every async function in the ``trio`` module is Trio-flavored.
+  Every async function in the ``asyncio`` module is asyncio-flavored.
+
+* Flavor is transitive: if async function ``foo()`` calls ``await
+  bar()``, then ``foo()`` has ``bar()``'s flavor. (If ``foo()``
+  calls ``await baz()`` too, then ``bar()`` and ``baz()`` had better
+  have the same flavor.)
+
+* trio-asyncio gives you the ability to call functions whose flavor is
+  different than your own, but you must be explicit about it.
+  :func:`trio_asyncio.aio_as_trio` takes an asyncio-flavored function
+  and returns a Trio-flavored wrapper for it;
+  :func:`trio_as_aio` takes a Trio-flavored function and
+  returns an asyncio-flavored wrapper for it.
+
+If you don't keep track of your function flavors correctly, you might
+get exceptions like the following:
+
+* If you call a Trio function where an asyncio function is expected: ``RuntimeError:
+  Task got bad yield:`` followed by either ``WaitTaskRescheduled(abort_func=...)``
+  or ``<class 'trio._core._traps.CancelShieldedCheckpoint'>``
+
+* If you call an asyncio function where a Trio function is expected: ``TypeError:
+  trio.run received unrecognized yield message <Future ...>.``
+
+Other errors are possible too.
+
+-----------------------
+ Flavor versus context
+-----------------------
+
+The concept of function flavor is distinct from the concept of
+"asyncio context" or "Trio context". You're in Trio context if you're
+(indirectly) inside a call to :func:`trio.run`. You're in asyncio
+context if :func:`asyncio.get_running_loop` returns a valid event
+loop. In a trio-asyncio program, you will frequently be in both Trio
+context and asyncio context at the same time, but each async function is
+either Trio-flavored or asyncio-flavored (not both).
+
+Most *synchronous* asyncio or Trio functions (:meth:`trio.Event.set`,
+:meth:`asyncio.StreamWriter.close`, etc) only require you to be in
+asyncio or Trio context, and work equally well regardless of the
+flavor of function calling them. The exceptions are functions that
+access the current task (:func:`asyncio.current_task`,
+:func:`trio.hazmat.current_task`, and anything that calls them),
+because there's only a meaningful concept of the current *foo* task
+when a *foo*-flavored function is executing.  For example, this means
+context managers that set a timeout on their body (``with
+async_timeout.timeout(N):``, ``with trio.move_on_after(N):``) must be
+run from within the correct flavor of function.
+
+---------------------------------
+ Flavor transitions are explicit
+---------------------------------
+
+As mentioned above, trio-asyncio does not generally allow you to
+transparently call ``await trio.something()`` from asyncio code, nor
+vice versa; you need to use :func:`aio_as_trio` or
+:func:`trio_as_aio` when calling a function whose flavor is
+different than yours. This is certainly more frustrating than having
+it "just work". Unfortunately, semantic differences between Trio and
+asyncio (such as how to signal cancellation) need to be resolved at
+each boundary between asyncio and Trio, and we haven't found a way to
+do this with acceptable performance and robustness unless those
+boundaries are marked.
+
+If you insist on living on the wild side, trio-asyncio does provide
+:func:`allow_asyncio` which allows *limited,
+experimental, and slow* mixing of Trio-flavored and asyncio-flavored
+calls in the same Trio-flavored function.
+
+-------------------------------------------
+ trio-asyncio's place in the asyncio stack
+-------------------------------------------
+
+At its base, asyncio doesn't know anything about futures or
+coroutines, nor does it have any concept of a task. All of these
+features are built on top of the simpler interfaces provided by the
+event loop. The event loop itself has little functionality beyond executing
+synchronous functions submitted with :meth:`~asyncio.loop.call_soon`
+and :meth:`~asyncio.loop.call_later`
+and invoking I/O availability callbacks registered using
+:meth:`~asyncio.loop.add_reader` and :meth:`~asyncio.loop.add_writer`
+at the appropriate times.
+
+trio-asyncio provides an asyncio event loop implementation which
+performs these basic operations using Trio APIs. Everything else in
+asyncio (futures, tasks, cancellation, and so on) is ultimately
+implemented in terms of calls to event loop methods, and thus works
+"magically" with Trio once the trio-asyncio event loop is
+installed. This strategy provides a high level of compatibility with
+asyncio libraries, but it also means that asyncio-flavored code
+running under trio-asyncio doesn't benefit much from Trio's more
+structured approach to concurrent programming: cancellation,
+causality, and exception propagation in asyncio-flavored code are just
+as error-prone under trio-asyncio as they are under the default
+asyncio event loop. (Of course, your Trio-flavored code will still
+benefit from all the usual Trio guarantees.)
+
+If you look at a Trio task tree, you'll see only one Trio task for the
+entire asyncio event loop. The distinctions between different asyncio
+tasks are erased, because they've all been merged into a single pot of
+callback soup by the time they get to trio-asyncio. Similarly, context
+variables will only work properly in asyncio-flavored code when
+running Python 3.7 or later (where they're supported natively), even
+though Trio supports them on earlier Pythons using a backport package.
+
+----------------------------
+ Event loop implementations
+----------------------------
+
+An asyncio event loop may generally be interrupted and restarted at any
+time, simply by making repeated calls to :meth:`run_until_complete()
+<asyncio.loop.run_until_complete>`.
+Trio, however, requires one long-running main loop. trio-asyncio bridges
+this gap by providing two event loop implementations.
+
+* The preferred option is to use an "async loop": inside a
+  Trio-flavored async function, write ``async with
+  trio_asyncio.open_loop() as loop:``.  Within the ``async with``
+  block (and anything it calls, and any tasks it starts, and so on),
+  :func:`asyncio.get_event_loop` and :func:`asyncio.get_running_loop`
+  will return *loop*. You can't manually start and stop an async
+  loop. Instead, it starts when you enter the ``async with`` block and
+  stops when you exit the block.
+
+* The other option is a "sync loop".
+  If you've imported trio-asyncio but aren't in Trio context, and you haven't
+  installed a custom event loop policy, calling :func:`asyncio.new_event_loop`
+  (including the implicit call made by the first :func:`asyncio.get_event_loop`
+  in the main thread) will give you an event loop that transparently runs
+  in a separate thread in order to support multiple
+  calls to :meth:`~asyncio.loop.run_until_complete`,
+  :meth:`~asyncio.loop.run_forever`, and :meth:`~asyncio.loop.stop`.
+  Sync loops are intended to allow trio-asyncio to run the existing
+  test suites of large asyncio libraries, which often call
+  :meth:`~asyncio.loop.run_until_complete` on the same loop multiple times.
+  Using them for other purposes is deprecated.