From 6e545328cade47ed0ea117209f5c04367ae37916 Mon Sep 17 00:00:00 2001 From: jakkdl Date: Tue, 14 May 2024 11:43:45 +0200 Subject: [PATCH 01/13] stray newline broke bullet list --- docs/rules.rst | 3 +-- 1 file changed, 1 insertion(+), 2 deletions(-) diff --git a/docs/rules.rst b/docs/rules.rst index 05d4ab57..a170e0f8 100644 --- a/docs/rules.rst +++ b/docs/rules.rst @@ -6,8 +6,7 @@ General rules ============= - **ASYNC100**: A ``with [trio/anyio].fail_after(...):`` or ``with [trio/anyio].move_on_after(...):`` context does not contain any ``await`` statements. This makes it pointless, as the timeout can only be triggered by a checkpoint. This check also allows ``yield`` statements, since checkpoints can happen in the caller we yield to. -- **ASYNC101**: ``yield`` inside a trio nursery, anyio/asyncio TaskGroup, or in a timeout/cancel scope is only safe when implementing a context manager - otherwise, it breaks exception handling. See `this thread `_ for discussion of a future PEP. -This has substantial overlap with :ref:`ASYNC119 `, which will warn on almost all instances of ASYNC101, but ASYNC101 is about a conceptually different problem that will not get resolved by `PEP 533 `_. +- **ASYNC101**: ``yield`` inside a trio nursery, anyio/asyncio TaskGroup, or in a timeout/cancel scope is only safe when implementing a context manager - otherwise, it breaks exception handling. See `this thread `_ for discussion of a future PEP. This has substantial overlap with :ref:`ASYNC119 `, which will warn on almost all instances of ASYNC101, but ASYNC101 is about a conceptually different problem that will not get resolved by `PEP 533 `_. - **ASYNC102**: It's unsafe to await inside ``finally:`` or ``except BaseException/trio.Cancelled/anyio.get_cancelled_exc_class()/asyncio.exceptions.CancelledError`` unless you use a shielded cancel scope with a timeout. This is currently not able to detect asyncio shields. - **ASYNC103**: ``except BaseException/trio.Cancelled/anyio.get_cancelled_exc_class()/asyncio.exceptions.CancelledError``, or a bare ``except:`` with a code path that doesn't re-raise. If you don't want to re-raise ``BaseException``, add a separate handler for ``trio.Cancelled``/``anyio.get_cancelled_exc_class()``/``asyncio.exceptions.CancelledError`` before. - **ASYNC104**: ``trio.Cancelled``/``anyio.get_cancelled_exc_class()``/``asyncio.exceptions.CancelledError``/``BaseException`` must be re-raised. The same as ASYNC103, except specifically triggered on ``return`` or a different exception being raised. From e63f8e9516bc1d3736a61b487f74b12456929ba9 Mon Sep 17 00:00:00 2001 From: jakkdl Date: Tue, 14 May 2024 12:43:54 +0200 Subject: [PATCH 02/13] wip rules doc improvements --- docs/rules.rst | 84 ++++++++++++++++++++++++++++++++++++++++++++++++++ 1 file changed, 84 insertions(+) diff --git a/docs/rules.rst b/docs/rules.rst index a170e0f8..0102fe8b 100644 --- a/docs/rules.rst +++ b/docs/rules.rst @@ -2,9 +2,93 @@ List of rules **************** +.. Esp when writing short descriptions it'd be very handy to link to a glossary, instead of saying stuff like ``except BaseException/trio.Cancelled/anyio.get_cancelled_exc_class()/asyncio.exceptions.CancelledError`` + it also allows easier use of library-specific terminology without forcing people to know all libraries by heart. + It should probably have it's own page in the long run + +Glossary +======== + +- timeout context + + - https://trio.readthedocs.io/en/stable/reference-core.html#cancellation-and-timeouts + - `trio.CancelScope`, `trio.move_on_after`, `trio.move_on_at`, `trio.fail_after`, `trio.fail_at` + - https://anyio.readthedocs.io/en/stable/cancellation.html + - `anyio.move_on_after`, `anyio.fail_after`, `anyio.CancelScope` + - `asyncio.timeout`, `asyncio.timeout_at` + General rules ============= + + +.. term wishlist: + nursery-or-cancelscope-or-timeout + sensitive-exception / cancelled + +.. TODO: + async100 does not list cancelscope or asyncio + reeeeally need intersphinx now + +.. list-table:: + :widths: 1 18 40 + :header-rows: 1 + + * - Code + - Name + - Message + * - ASYNC100 + - scope-no-checkpoint + - A ``with [trio/anyio].fail_after(...):`` or ``with [trio/anyio].move_on_after(...):`` context does not contain any ``await`` statements. This makes it pointless, as the timeout can only be triggered by a checkpoint. This check also allows ``yield`` statements, since checkpoints can happen in the caller we yield to. + * - ASYNC101 + - yield-in-nursery-or-scope + - ``yield`` inside a trio nursery, anyio/asyncio TaskGroup, or in a timeout/cancel scope is only safe when implementing a context manager - otherwise, it breaks exception handling. + * - ASYNC102 + - await-in-finally-or-cancelled + - ``await`` inside ``finally``/``except cancelled`` must have shielded cancelscope with timeout. + * - ASYNC103 + - no-reraise-cancelled + - cancelled-catching exception that does not reraise the exception. + * - ASYNC104 + - cancelled-not-raised + - cancelled-catching exception does not raise cancelled + * - ASYNC105 + - missing-await + - async trio function called without using ``await`` + * - ASYNC106 + - bad-async-library-import + - trio/anyio/asyncio must be imported with ``import xxx`` for the linter to work. + * - ASYNC109 + - async-function-with-timeout + - Async function definition with a ``timeout`` parameter. In structured concurrency the caller should instead use timeout context managers. + * - ASYNC110 + - busy-wait + - ``while : await [trio/anyio].sleep()`` should be replaced by a ``[trio/anyio].Event``. + * - ASYNC111 + - variable-from-cm-in-start-soon + - Variable, from context manager opened inside nursery, passed to ``start[_soon]`` might be invalidly accessed while in use, due to context manager closing before the nursery. This is usually a bug, and nurseries should generally be the inner-most context manager. + * - ASYNC112 + - useless-nursery + - Nursery body with only a call to ``nursery.start[_soon]`` and not passing itself as a parameter can be replaced with a regular function call. + * - ASYNC113 + - start-soon-in-aenter + - Using ``nursery.start_soon`` in ``__aenter__`` doesn't wait for the task to begin. Consider replacing with ``nursery.start``. + * - ASYNC114 + - startable-not-in-config + - Startable function (i.e. has a ``task_status`` keyword parameter) not in ``--startable-in-context-manager`` parameter list, please add it so ASYNC113 can catch errors when using it. + * - ASYNC115 + - sleep-zero + - Replace ``[trio/anyio].sleep(0)`` with the more suggestive ``[trio/anyio].lowlevel.checkpoint()``. + * - ASYNC116 + - long-sleep-not-forever + - ``[trio/anyio].sleep()`` with >24 hour interval should usually be ``[trio/anyio].sleep_forever()``. + * - ASYNC118 + - cancelled-class-saved + - Don't assign the value of ``anyio.get_cancelled_exc_class()`` to a variable, since that breaks linter checks and multi-backend programs. + * - ASYNC119 + - yield-in-cm-in-async-gen + - ``yield`` in context manager in async generator is unsafe, the cleanup may be delayed until ``await`` is no longer allowed. + - **ASYNC100**: A ``with [trio/anyio].fail_after(...):`` or ``with [trio/anyio].move_on_after(...):`` context does not contain any ``await`` statements. This makes it pointless, as the timeout can only be triggered by a checkpoint. This check also allows ``yield`` statements, since checkpoints can happen in the caller we yield to. - **ASYNC101**: ``yield`` inside a trio nursery, anyio/asyncio TaskGroup, or in a timeout/cancel scope is only safe when implementing a context manager - otherwise, it breaks exception handling. See `this thread `_ for discussion of a future PEP. This has substantial overlap with :ref:`ASYNC119 `, which will warn on almost all instances of ASYNC101, but ASYNC101 is about a conceptually different problem that will not get resolved by `PEP 533 `_. - **ASYNC102**: It's unsafe to await inside ``finally:`` or ``except BaseException/trio.Cancelled/anyio.get_cancelled_exc_class()/asyncio.exceptions.CancelledError`` unless you use a shielded cancel scope with a timeout. This is currently not able to detect asyncio shields. From 50f1d3df5a8d5294b2e9bbe6b3a98814a0e0f32a Mon Sep 17 00:00:00 2001 From: jakkdl Date: Thu, 16 May 2024 15:06:05 +0200 Subject: [PATCH 03/13] improve glossary, make use of it and intersphinx in blurbs --- docs/index.rst | 4 +-- docs/rules.rst | 87 ++++++++++++++++++++++++++++++++++++++------------ 2 files changed, 68 insertions(+), 23 deletions(-) diff --git a/docs/index.rst b/docs/index.rst index c610291c..b38f1a6a 100644 --- a/docs/index.rst +++ b/docs/index.rst @@ -8,7 +8,7 @@ flake8-async ############ -A highly opinionated flake8 plugin for problems related to `Trio `_, `AnyIO `_, or `asyncio `_. +A highly opinionated flake8 plugin for problems related to `Trio `__, `AnyIO `__, or `asyncio `__. This can include anything from outright bugs, to pointless/dead code, @@ -20,7 +20,7 @@ The plugin may well be too noisy or pedantic depending on your requirements or o Pairs well with flake8-bugbear. -Some rules are incorporated into `ruff `_. +Some rules are incorporated into `ruff `__. We previously maintained separate flake8-async and flake8-trio plugins, but merged both into this plugin under the more general "flake8-async" name after flake8-trio grew support for anyio and asyncio and became a superset of the former flake8-async. All flake8-trio error codes were renamed from TRIOxxx to ASYNCxxx and the flake8-trio package is now deprecated. diff --git a/docs/rules.rst b/docs/rules.rst index e885a8c6..12da7304 100644 --- a/docs/rules.rst +++ b/docs/rules.rst @@ -9,26 +9,71 @@ List of rules Glossary ======== -- timeout context +.. _timeout_context: + +timeout context +--------------- +Colloquially referred to as a "Timeout" or "CancelScope". A context manager that enforces a timeout on a block of code. Trio/AnyIO timeout functions are implemented with a ``CancelScope``, but you can also directly use ``CancelScope`` as a context manager. + +.. I find this to have excessive spacing before/after sublists. Probably requires CSS to fix? + +* Trio + + * `Documentation `__ + + * :class:`trio.CancelScope`, :func:`trio.move_on_after`, :func:`trio.move_on_at`, :func:`trio.fail_after`, :func:`trio.fail_at` + +* AnyIO + + * `Documentation `__ + + * :class:`anyio.CancelScope`, :func:`anyio.move_on_after`, :func:`anyio.fail_after`` + +* asyncio + + * `Documentation `__ + + * :func:`asyncio.timeout`, :func:`asyncio.timeout_at` + +.. _taskgroup_nursery: + +TaskGroup / Nursery +------------------- + +A collection of child Tasks that can run concurrently. + +* Trio + + * `Documentation `__ + * :class:`trio.Nursery`, created with :func:`trio.open_nursery` +* AnyIO + + * `Documentation `__ + * :class:`anyio.abc.TaskGroup`, created with :func:`anyio.create_task_group`. +* asyncio + + * `Documentation `__ + * :class:`asyncio.TaskGroup` (since python 3.11) + + +.. _cancelled: + +Cancelled / CancelledError +-------------------------- + + Handling cancellation is very sensitive, and you generally never want to catch a cancellation exception without letting it propagate to the library. + + * Trio: :class:`trio.Cancelled`. `Documentation `__ + * AnyIO: :func:`anyio.get_cancelled_exc_class`. `Documentation `__ + * asyncio: :class:`asyncio.CancelledError`. `Documentation `__ - - https://trio.readthedocs.io/en/stable/reference-core.html#cancellation-and-timeouts - - `trio.CancelScope`, `trio.move_on_after`, `trio.move_on_at`, `trio.fail_after`, `trio.fail_at` - - https://anyio.readthedocs.io/en/stable/cancellation.html - - `anyio.move_on_after`, `anyio.fail_after`, `anyio.CancelScope` - - `asyncio.timeout`, `asyncio.timeout_at` General rules ============= -.. term wishlist: - nursery-or-cancelscope-or-timeout - sensitive-exception / cancelled - -.. TODO: - async100 does not list cancelscope or asyncio - reeeeally need intersphinx now +.. For some reason using :ref:`timeout_context` fails to find the reference, but :ref:`timeout_context ` works. I have no clue why .. list-table:: :widths: 1 18 40 @@ -39,19 +84,19 @@ General rules - Message * - ASYNC100 - scope-no-checkpoint - - A ``with [trio/anyio].fail_after(...):`` or ``with [trio/anyio].move_on_after(...):`` context does not contain any ``await`` statements. This makes it pointless, as the timeout can only be triggered by a checkpoint. This check also allows ``yield`` statements, since checkpoints can happen in the caller we yield to. + - A :ref:`timeout_context ` does not contain any ``await`` statements. This makes it pointless, as the timeout can only be triggered by a checkpoint. This check also allows ``yield`` statements, since checkpoints can happen in the caller we yield to. * - ASYNC101 - - yield-in-nursery-or-scope - - ``yield`` inside a trio nursery, anyio/asyncio TaskGroup, or in a timeout/cancel scope is only safe when implementing a context manager - otherwise, it breaks exception handling. + - yield-in-taskgroup-or-scope + - ``yield`` inside a :ref:`TaskGroup/Nursery ` or :ref:`timeout_context ` is only safe when implementing a context manager - otherwise, it breaks exception handling. * - ASYNC102 - await-in-finally-or-cancelled - - ``await`` inside ``finally``/``except cancelled`` must have shielded cancelscope with timeout. + - ``await`` inside ``finally`` or :ref:`cancelled-catching ` ``except:`` must have shielded cancelscope with timeout. * - ASYNC103 - no-reraise-cancelled - - cancelled-catching exception that does not reraise the exception. + - :ref:`cancelled `-catching exception that does not reraise the exception. * - ASYNC104 - cancelled-not-raised - - cancelled-catching exception does not raise cancelled + - :ref:`cancelled `-catching exception does not raise the exception. Triggered on ``return`` or raising a different exception. * - ASYNC105 - missing-await - async trio function called without using ``await`` @@ -60,10 +105,10 @@ General rules - trio/anyio/asyncio must be imported with ``import xxx`` for the linter to work. * - ASYNC109 - async-function-with-timeout - - Async function definition with a ``timeout`` parameter. In structured concurrency the caller should instead use timeout context managers. + - Async function definition with a ``timeout`` parameter. In structured concurrency the caller should instead use :ref:`timeout context managers `. * - ASYNC110 - busy-wait - - ``while : await [trio/anyio].sleep()`` should be replaced by a ``[trio/anyio].Event``. + - ``while ...: await [trio/anyio].sleep()`` should be replaced by a :class:`trio.Event`/:class:`anyio.Event`. * - ASYNC111 - variable-from-cm-in-start-soon - Variable, from context manager opened inside nursery, passed to ``start[_soon]`` might be invalidly accessed while in use, due to context manager closing before the nursery. This is usually a bug, and nurseries should generally be the inner-most context manager. From 6b77295010d365ba0c454ad8913ba09a08f13997 Mon Sep 17 00:00:00 2001 From: John Litborn <11260241+jakkdl@users.noreply.github.com> Date: Fri, 17 May 2024 13:34:55 +0200 Subject: [PATCH 04/13] Update docs/rules.rst Co-authored-by: Zac Hatfield-Dodds --- docs/rules.rst | 15 ++++++++++++++- 1 file changed, 14 insertions(+), 1 deletion(-) diff --git a/docs/rules.rst b/docs/rules.rst index 12da7304..68e040a4 100644 --- a/docs/rules.rst +++ b/docs/rules.rst @@ -9,11 +9,24 @@ List of rules Glossary ======== +.. _cancel_scope: + +cancel scope +------------ +A cancel scope is a context manager which can request the library cancels +whatever task is executing in the body of the ``with`` (or ``async with``) +block. A cancel scope is the key component of a timeout context, and used in +TaskGroups / Nurseries to cancel any remaining child tasks if one raises an +exception. Trio and AnyIO have an explicit ``CancelScope`` type; in asyncio +they are implicit. + .. _timeout_context: timeout context --------------- -Colloquially referred to as a "Timeout" or "CancelScope". A context manager that enforces a timeout on a block of code. Trio/AnyIO timeout functions are implemented with a ``CancelScope``, but you can also directly use ``CancelScope`` as a context manager. +A context manager that enforces a timeout on a block of code, by cancelling it +after a specified duration or at a preset time. The timeout can also be +rescheduled after creation. .. I find this to have excessive spacing before/after sublists. Probably requires CSS to fix? From 3f0af4d471b19f2e983ab15efe371241d49c9012 Mon Sep 17 00:00:00 2001 From: jakkdl Date: Fri, 17 May 2024 16:08:36 +0200 Subject: [PATCH 05/13] moar doc improvements --- docs/rules.rst | 183 ++++++++++++++++++++++++++++++++----------------- docs/usage.rst | 26 ++++--- 2 files changed, 137 insertions(+), 72 deletions(-) diff --git a/docs/rules.rst b/docs/rules.rst index 68e040a4..049c13b6 100644 --- a/docs/rules.rst +++ b/docs/rules.rst @@ -15,18 +15,29 @@ cancel scope ------------ A cancel scope is a context manager which can request the library cancels whatever task is executing in the body of the ``with`` (or ``async with``) -block. A cancel scope is the key component of a timeout context, and used in -TaskGroups / Nurseries to cancel any remaining child tasks if one raises an +block. A cancel scope is the key component of a :ref:`timeout context `, and used in :ref:`TaskGroups / Nurseries ` to cancel any remaining child tasks if one raises an exception. Trio and AnyIO have an explicit ``CancelScope`` type; in asyncio they are implicit. +* Trio + + * `Documentation `__ + + * :class:`trio.CancelScope` + +* AnyIO + + * `Documentation `__ + + * :class:`anyio.CancelScope` + .. _timeout_context: timeout context --------------- A context manager that enforces a timeout on a block of code, by cancelling it after a specified duration or at a preset time. The timeout can also be -rescheduled after creation. +rescheduled after creation. They are internally implemented with a :ref:`cancel scope `, which in anyio & trio can be directly initialized with a deadline. .. I find this to have excessive spacing before/after sublists. Probably requires CSS to fix? @@ -34,13 +45,13 @@ rescheduled after creation. * `Documentation `__ - * :class:`trio.CancelScope`, :func:`trio.move_on_after`, :func:`trio.move_on_at`, :func:`trio.fail_after`, :func:`trio.fail_at` + * :func:`trio.move_on_after`, :func:`trio.move_on_at`, :func:`trio.fail_after`, :func:`trio.fail_at`, :class:`trio.CancelScope` * AnyIO * `Documentation `__ - * :class:`anyio.CancelScope`, :func:`anyio.move_on_after`, :func:`anyio.fail_after`` + * :func:`anyio.move_on_after`, :func:`anyio.fail_after`, :class:`anyio.CancelScope` * asyncio @@ -53,11 +64,12 @@ rescheduled after creation. TaskGroup / Nursery ------------------- -A collection of child Tasks that can run concurrently. +A collection of child Tasks that can run concurrently. Internally contains a :ref:`cancel scope ` for canceling any remaining child tasks if one raises an exception. * Trio * `Documentation `__ + * :class:`trio.Nursery`, created with :func:`trio.open_nursery` * AnyIO @@ -74,12 +86,26 @@ A collection of child Tasks that can run concurrently. Cancelled / CancelledError -------------------------- - Handling cancellation is very sensitive, and you generally never want to catch a cancellation exception without letting it propagate to the library. +Handling cancellation is very sensitive, and you generally never want to catch a cancellation exception without letting it propagate to the library. * Trio: :class:`trio.Cancelled`. `Documentation `__ * AnyIO: :func:`anyio.get_cancelled_exc_class`. `Documentation `__ * asyncio: :class:`asyncio.CancelledError`. `Documentation `__ +.. _checkpoint: + +Checkpoint +---------- +Checkpoints are ``await``, ``async for``, and ``async with`` (on one of enter/exit). TODO write more and link stuff + +.. _channel_stream_queue: + +Channel / Stream / Queue +------------------------ +* Trio: `channel `__ +* AnyIO: `stream `__ +* asyncio: `queue `__ + General rules ============= @@ -96,23 +122,25 @@ General rules - Name - Message * - ASYNC100 - - scope-no-checkpoint - - A :ref:`timeout_context ` does not contain any ``await`` statements. This makes it pointless, as the timeout can only be triggered by a checkpoint. This check also allows ``yield`` statements, since checkpoints can happen in the caller we yield to. + - cancel-scope-no-checkpoint + - .. _async100: + + A :ref:`timeout_context ` does not contain any :ref:`checkpoints ` statements. This makes it pointless, as the timeout can only be triggered by a checkpoint. This check also allows ``yield`` statements, since checkpoints can happen in the caller we yield to. * - ASYNC101 - - yield-in-taskgroup-or-scope - - ``yield`` inside a :ref:`TaskGroup/Nursery ` or :ref:`timeout_context ` is only safe when implementing a context manager - otherwise, it breaks exception handling. + - yield-in-cancel-scope + - ``yield`` inside a :ref:`TaskGroup/Nursery ` or :ref:`timeout_context ` is only safe when implementing a context manager - otherwise, it breaks exception handling. See `this thread `_ for discussion of a future PEP. This has substantial overlap with :ref:`ASYNC119 `, which will warn on almost all instances of ASYNC101, but ASYNC101 is about a conceptually different problem that will not get resolved by `PEP 533 `_. * - ASYNC102 - await-in-finally-or-cancelled - - ``await`` inside ``finally`` or :ref:`cancelled-catching ` ``except:`` must have shielded cancelscope with timeout. + - ``await`` inside ``finally`` or :ref:`cancelled-catching ` ``except:`` must have shielded :ref:`cancel scope ` with timeout. This is currently not able to detect asyncio shields. * - ASYNC103 - no-reraise-cancelled - - :ref:`cancelled `-catching exception that does not reraise the exception. + - :ref:`cancelled `-catching exception that does not reraise the exception. If you don't want to re-raise :class:`BaseException`, add a separate handler for :ref:`Cancelled ` before. * - ASYNC104 - cancelled-not-raised - :ref:`cancelled `-catching exception does not raise the exception. Triggered on ``return`` or raising a different exception. * - ASYNC105 - missing-await - - async trio function called without using ``await`` + - async trio function called without using ``await``. This is only supported with trio functions, but you can get similar functionality with a type-checker. * - ASYNC106 - bad-async-library-import - trio/anyio/asyncio must be imported with ``import xxx`` for the linter to work. @@ -127,81 +155,110 @@ General rules - Variable, from context manager opened inside nursery, passed to ``start[_soon]`` might be invalidly accessed while in use, due to context manager closing before the nursery. This is usually a bug, and nurseries should generally be the inner-most context manager. * - ASYNC112 - useless-nursery - - Nursery body with only a call to ``nursery.start[_soon]`` and not passing itself as a parameter can be replaced with a regular function call. + - :ref:`Nursery/TaskGroup ` body with only a call to ``.start[_soon]`` and not passing itself as a parameter can be replaced with a regular function call. * - ASYNC113 - start-soon-in-aenter - - Using ``nursery.start_soon`` in ``__aenter__`` doesn't wait for the task to begin. Consider replacing with ``nursery.start``. + - Using :meth:`~trio.Nursery.start_soon`/:meth:`~anyio.abc.TaskGroup.start_soon` in ``__aenter__`` doesn't wait for the task to begin. Consider replacing with :meth:`~trio.Nursery.start`/:meth:`~anyio.abc.TaskGroup.start`. * - ASYNC114 - startable-not-in-config - - Startable function (i.e. has a ``task_status`` keyword parameter) not in ``--startable-in-context-manager`` parameter list, please add it so ASYNC113 can catch errors when using it. + - Startable function (i.e. has a ``task_status`` keyword parameter) not in :ref:`--startable-in-context-manager <--startable-in-context-manager>` parameter list, please add it so ASYNC113 can catch errors when using it. * - ASYNC115 - sleep-zero - - Replace ``[trio/anyio].sleep(0)`` with the more suggestive ``[trio/anyio].lowlevel.checkpoint()``. + - Replace :func:`trio.sleep(0) `/:func:`anyio.sleep(0) ` with the more suggestive :func:`trio.lowlevel.checkpoint`/:func:`anyio.lowlevel.checkpoint`. * - ASYNC116 - long-sleep-not-forever - - ``[trio/anyio].sleep()`` with >24 hour interval should usually be ``[trio/anyio].sleep_forever()``. + - :func:`trio.sleep`/:func:`anyio.sleep` with >24 hour interval should usually be :func:`trio.sleep_forever`/:func:`anyio.sleep_forever`. * - ASYNC118 - cancelled-class-saved - - Don't assign the value of ``anyio.get_cancelled_exc_class()`` to a variable, since that breaks linter checks and multi-backend programs. + - Don't assign the value of :func:`anyio.get_cancelled_exc_class()` to a variable, since that breaks linter checks and multi-backend programs. * - ASYNC119 - yield-in-cm-in-async-gen - - ``yield`` in context manager in async generator is unsafe, the cleanup may be delayed until ``await`` is no longer allowed. - -- **ASYNC100**: A ``with [trio/anyio].fail_after(...):`` or ``with [trio/anyio].move_on_after(...):`` context does not contain any ``await`` statements. This makes it pointless, as the timeout can only be triggered by a checkpoint. This check also allows ``yield`` statements, since checkpoints can happen in the caller we yield to. - -- **ASYNC101**: ``yield`` inside a :class:`trio.Nursery`/:class:`anyio.abc.TaskGroup`/:py:class:`asyncio.TaskGroup`, or in a timeout/cancel scope is only safe when implementing a context manager - otherwise, it breaks exception handling. See `this thread `_ for discussion of a future PEP. This has substantial overlap with :ref:`ASYNC119 `, which will warn on almost all instances of ASYNC101, but ASYNC101 is about a conceptually different problem that will not get resolved by `PEP 533 `_. -- **ASYNC102**: It's unsafe to await inside ``finally:`` or ``except BaseException/trio.Cancelled/anyio.get_cancelled_exc_class()/asyncio.exceptions.CancelledError`` unless you use a shielded cancel scope with a timeout. This is currently not able to detect asyncio shields. -- **ASYNC103**: ``except`` :class:`BaseException`/:class:`trio.Cancelled`/:func:`anyio.get_cancelled_exc_class`/:class:`asyncio.CancelledError`, or a bare ``except:`` with a code path that doesn't re-raise. If you don't want to re-raise :class:`BaseException`, add a separate handler for :class:`trio.Cancelled`/:func:`anyio.get_cancelled_exc_class`/:class:`asyncio.CancelledError` before. -- **ASYNC104**: :class:`trio.Cancelled`/:func:`anyio.get_cancelled_exc_class`/:class:`asyncio.CancelledError`/:class:`BaseException` must be re-raised. The same as ASYNC103, except specifically triggered on ``return`` or a different exception being raised. -- **ASYNC105**: Calling a trio async function without immediately ``await``\ ing it. This is only supported with trio functions, but you can get similar functionality with a type-checker. -- **ASYNC106**: ``trio``/``anyio``/``asyncio`` must be imported with ``import trio``/``import anyio``/``import asyncio`` for the linter to work. -- **ASYNC109**: Async function definition with a ``timeout`` parameter - use ``[trio/anyio].[fail/move_on]_[after/at]`` instead. -- **ASYNC110**: ``while : await [trio/anyio].sleep()`` should be replaced by a ``[trio/anyio].Event``. -- **ASYNC111**: Variable, from context manager opened inside nursery, passed to ``start[_soon]`` might be invalidly accessed while in use, due to context manager closing before the nursery. This is usually a bug, and nurseries should generally be the inner-most context manager. -- **ASYNC112**: Nursery body with only a call to ``nursery.start[_soon]`` and not passing itself as a parameter can be replaced with a regular function call. -- **ASYNC113**: Using :meth:`trio.Nursery.start_soon` in ``__aenter__`` doesn't wait for the task to begin. Consider replacing with ``nursery.start``. -- **ASYNC114**: Startable function (i.e. has a ``task_status`` keyword parameter) not in ``--startable-in-context-manager`` parameter list, please add it so ASYNC113 can catch errors when using it. -- **ASYNC115**: Replace ``[trio/anyio].sleep(0)`` with the more suggestive ``[trio/anyio].lowlevel.checkpoint()``. -- **ASYNC116**: ``[trio/anyio].sleep()`` with >24 hour interval should usually be ``[trio/anyio].sleep_forever()``. -- **ASYNC118**: Don't assign the value of :func:`anyio.get_cancelled_exc_class` to a variable, since that breaks linter checks and multi-backend programs. + - .. _async119: - .. _async119: - -- **ASYNC119**: ``yield`` in context manager in async generator is unsafe, the cleanup may be delayed until ``await`` is no longer allowed. We strongly encourage you to read `PEP 533 `_ and use `async with aclosing(...) `_, or better yet avoid async generators entirely (see :ref:`ASYNC900 ` ) in favor of context managers which return an iterable `channel (trio) `_, `stream (anyio) `_, or `queue (asyncio) `_. - - .. TODO: use intersphinx(?) instead of having to specify full URL + ``yield`` in context manager in async generator is unsafe, the cleanup may be delayed until ``await`` is no longer allowed. We strongly encourage you to read `PEP 533 `_ and use `async with aclosing(...) `_, or better yet avoid async generators entirely (see :ref:`ASYNC900 ` ) in favor of context managers which return an iterable :ref:`channel/stream/queue `. Blocking sync calls in async functions ====================================== Note: 22X, 23X and 24X has not had asyncio-specific suggestions written. +.. list-table:: + :widths: 1 18 40 + :header-rows: 1 -- **ASYNC200**: User-configured error for blocking sync calls in async functions. Does nothing by default, see :ref:`async200-blocking-calls` for how to configure it. -- **ASYNC210**: Sync HTTP call in async function, use ``httpx.AsyncClient``. This and the other ASYNC21x checks look for usage of ``urllib3`` and ``httpx.Client``, and recommend using ``httpx.AsyncClient`` as that's the largest http client supporting anyio/trio. -- **ASYNC211**: Likely sync HTTP call in async function, use ``httpx.AsyncClient``. Looks for ``urllib3`` method calls on pool objects, but only matching on the method signature and not the object. -- **ASYNC212**: Blocking sync HTTP call on httpx object, use httpx.AsyncClient. -- **ASYNC220**: Sync process call in async function, use ``await nursery.start([trio/anyio].run_process, ...)``. ``asyncio`` users can use `asyncio.create_subprocess_[exec/shell] `_. -- **ASYNC221**: Sync process call in async function, use ``await [trio/anyio].run_process(...)``. ``asyncio`` users can use `asyncio.create_subprocess_[exec/shell] `_. -- **ASYNC222**: Sync ``os.*`` call in async function, wrap in ``await [trio/anyio].to_thread.run_sync()``. ``asyncio`` users can use `asyncio.loop.run_in_executor `_. -- **ASYNC230**: Sync IO call in async function, use ``[trio/anyio].open_file(...)``. ``asyncio`` users need to use a library such as `aiofiles `_, or switch to `anyio `_. -- **ASYNC231**: Sync IO call in async function, use ``[trio/anyio].wrap_file(...)``. ``asyncio`` users need to use a library such as `aiofiles `_, or switch to `anyio `_. -- **ASYNC232**: Blocking sync call on file object, wrap the file object in ``[trio/anyio].wrap_file()`` to get an async file object. -- **ASYNC240**: Avoid using ``os.path`` in async functions, prefer using ``[trio/anyio].Path`` objects. ``asyncio`` users should consider `aiopath `_ or `anyio `_. -- **ASYNC250**: Builtin ``input()`` should not be called from async function. Wrap in ``[trio/anyio].to_thread.run_sync()`` or ``asyncio.loop.run_in_executor()``. -- **ASYNC251**: ``time.sleep(...)`` should not be called from async function. Use ``[trio/anyio/asyncio].sleep(...)``. + * - Code + - Name + - Message + * - ASYNC200 + - blocking-call + - User-configured error for blocking sync calls in async functions. Does nothing by default, see :ref:`async200-blocking-calls` for how to configure it. + * - ASYNC210 + - blocking-http-call + - Sync HTTP call in async function, use ``httpx.AsyncClient``. This and the other ASYNC21x checks look for usage of ``urllib3`` and ``httpx.Client``, and recommend using ``httpx.AsyncClient`` as that's the largest http client supporting anyio/trio. + * - ASYNC211 + - blocking-http-call-pool + - Likely sync HTTP call in async function, use ``httpx.AsyncClient``. Looks for ``urllib3`` method calls on pool objects, but only matching on the method signature and not the object. + * - ASYNC212 + - blocking-sync-http-call-httpx + - Blocking sync HTTP call on httpx object, use httpx.AsyncClient. + * - ASYNC220 + - blocking-process-call-1 + - Sync process call in async function, use ``await nursery.start([trio/anyio].run_process, ...)``. ``asyncio`` users can use `asyncio.create_subprocess_[exec/shell] `_. + * - ASYNC221 + - blocking-process-call-2 + - Sync process call in async function, use ``await [trio/anyio].run_process(...)``. ``asyncio`` users can use `asyncio.create_subprocess_[exec/shell] `_. + * - ASYNC222 + - blocking-process-call-3 + - Sync ``os.*`` call in async function, wrap in ``await [trio/anyio].to_thread.run_sync()``. ``asyncio`` users can use `asyncio.loop.run_in_executor `_. + * - ASYNC230 + - blocking-io-call + - Sync IO call in async function, use ``[trio/anyio].open_file(...)``. ``asyncio`` users need to use a library such as `aiofiles `_, or switch to `anyio `_. + * - ASYNC231 + - blocking-io-call-wrap + - Sync IO call in async function, use ``[trio/anyio].wrap_file(...)``. ``asyncio`` users need to use a library such as `aiofiles `_, or switch to `anyio `_. + * - ASYNC232 + - blocking-file-call + - Blocking sync call on file object, wrap the file object in ``[trio/anyio].wrap_file()`` to get an async file object. + * - ASYNC240 + - blocking-path-usage + - Avoid using :mod:`os.path` in async functions, prefer using ``[trio/anyio].Path`` objects. ``asyncio`` users should consider `aiopath `_ or `anyio `_. + * - ASYNC250 + - blocking-input + - Builtin ``input()`` should not be called from async function. Wrap in ``[trio/anyio].to_thread.run_sync()`` or ``asyncio.loop.run_in_executor()``. + * - ASYNC251 + - blocking-sleep + - ``time.sleep(...)`` should not be called from async function. Use ``[trio/anyio/asyncio].sleep(...)``. Optional rules disabled by default ================================== -.. _async900: +.. list-table:: + :widths: 1 18 40 + :header-rows: 1 + + * - Code + - Name + - Message + * - ASYNC900 + - unsafe-async-generator + - .. _async900: + + Async generator without ``@asynccontextmanager`` not allowed. You might want to enable this on a codebase since async generators are inherently unsafe and cleanup logic might not be performed. See https://github.com/python-trio/flake8-async/issues/211 and https://discuss.python.org/t/using-exceptiongroup-at-anthropic-experience-report/20888/6 for discussion. + + * - ASYNC910 + - async-function-no-checkpoint + - .. _async910: + + Exit or ``return`` from async function with no guaranteed :ref:`checkpoint ` or exception since function definition. You might want to enable this on a codebase to make it easier to reason about checkpoints, and make the logic of ASYNC911 correct. + * - ASYNC911 + - async-generator-no-checkpoint + - .. _async911: -- **ASYNC900**: Async generator without ``@asynccontextmanager`` not allowed. You might want to enable this on a codebase since async generators are inherently unsafe and cleanup logic might not be performed. See https://github.com/python-trio/flake8-async/issues/211 and https://discuss.python.org/t/using-exceptiongroup-at-anthropic-experience-report/20888/6 for discussion. -- **ASYNC910**: Exit or ``return`` from async function with no guaranteed checkpoint or exception since function definition. You might want to enable this on a codebase to make it easier to reason about checkpoints, and make the logic of ASYNC911 correct. -- **ASYNC911**: Exit, ``yield`` or ``return`` from async iterable with no guaranteed checkpoint since possible function entry (yield or function definition) - Checkpoints are ``await``, ``async for``, and ``async with`` (on one of enter/exit). -- **ASYNC912**: A timeout/cancelscope has checkpoints, but they're not guaranteed to run. Similar to ASYNC100, but it does not warn on trivial cases where there is no checkpoint at all. It instead shares logic with ASYNC910 and ASYNC911 for parsing conditionals and branches. + Exit, ``yield`` or ``return`` from async iterable with no guaranteed :ref:`checkpoint ` since possible function entry (yield or function definition). + * - ASYNC912 + - cancel-scope-no-guaranteed-checkpoint + - A timeout/cancelscope has :ref:`checkpoints `, but they're not guaranteed to run. Similar to :ref:`ASYNC100 `, but it does not warn on trivial cases where there is no checkpoint at all. It instead shares logic with :ref:`ASYNC910 ` and :ref:`ASYNC911 ` for parsing conditionals and branches. Removed rules ================ diff --git a/docs/usage.rst b/docs/usage.rst index 78aba3bc..00399ce9 100644 --- a/docs/usage.rst +++ b/docs/usage.rst @@ -97,6 +97,8 @@ Note that when running ``flake8-async`` as a standalone it's not currently possi Selecting rules =============== +.. _valueerror_ignore: + ``ValueError`` when trying to ``ignore`` error codes in config file ------------------------------------------------------------------- @@ -106,8 +108,8 @@ configuration with a regex. We have decided not to conform to this, as it would be a breaking change for end-users requiring them to update ``noqa``\ s and configurations, we think the ``ASYNC`` code is much more readable than e.g. ``ASYxxx``, and ruff does not enforce such a limit. The easiest option -for users hitting this error is to instead use the ``--disable`` option as -documented `below <#--disable>`__. See further discussion and other +for users hitting this error is to instead use our :ref:`--disable` option. +See further discussion and other workarounds in https://github.com/python-trio/flake8-async/issues/230. @@ -121,7 +123,7 @@ Comma-separated list of error codes to enable, similar to flake8 --select but is ``--disable`` ------------- -Comma-separated list of error codes to disable, similar to flake8 ``--ignore`` but is additionally more performant as it will disable non-enabled visitors from running instead of just silencing their errors. It will also bypass errors introduced in flake8>=6, see above. +Comma-separated list of error codes to disable, similar to flake8 ``--ignore`` but is additionally more performant as it will disable non-enabled visitors from running instead of just silencing their errors. :ref:`It will also bypass errors introduced in flake8>=6 `. ``--autofix`` ------------- @@ -156,7 +158,8 @@ Comma-separated list of decorators to disable checkpointing checks for, turning Decorators-to-match must be identifiers or dotted names only (not PEP-614 expressions), and will match against the name only - e.g. ``foo.bar`` matches ``foo.bar``, ``foo.bar()``, and ``foo.bar(args, here)``, etc. -For example: +Example +^^^^^^^ :: @@ -166,12 +169,16 @@ For example: ign*, *.ignore, +.. _--startable-in-context-manager: + ``startable-in-context-manager`` -------------------------------- -Comma-separated list of methods which should be used with ``.start()`` when opening a context manager, -in addition to the default ``trio.run_process``, ``trio.serve_tcp``, ``trio.serve_ssl_over_tcp``, and -``trio.serve_listeners``. Names must be valid identifiers as per ``str.isidentifier()``. For example: +Comma-separated list of methods which should be used with :meth:`trio.Nursery.start`/:meth:`anyio.abc.TaskGroup.start` when opening a context manager, +in addition to the default :func:`trio.run_process`, :func:`trio.serve_tcp`, :func:`trio.serve_ssl_over_tcp`, and :func:`trio.serve_listeners`. Names must be valid identifiers as per :meth:`str.isidentifier`. + +Example +^^^^^^^ :: @@ -188,9 +195,10 @@ Comma-separated list of pairs of values separated by ``->`` (optional whitespace The format of the error message is ``User-configured blocking sync call {0} in async function, consider replacing with {1}.``, where ``{0}`` is the pattern the call matches and ``{1}`` is the suggested replacement. -Example: +Examples +^^^^^^^^ -:: +.. code-block:: none async200-blocking-calls = my_blocking_call -> async.alternative, From 076c5ad5c9b7f2aee8a7b53e946835587d59e209 Mon Sep 17 00:00:00 2001 From: jakkdl Date: Fri, 17 May 2024 16:53:12 +0200 Subject: [PATCH 06/13] move out glossary to its own page, link to rules in the readme --- README.md | 3 ++ docs/index.rst | 2 + docs/rules.rst | 99 -------------------------------------------------- 3 files changed, 5 insertions(+), 99 deletions(-) diff --git a/README.md b/README.md index 50508aeb..d0e16910 100644 --- a/README.md +++ b/README.md @@ -18,3 +18,6 @@ Pairs well with flake8-bugbear. Some checks are incorporated into [ruff](https://github.com/astral-sh/ruff). This plugin was previously known as flake8-trio, and there was a separate small plugin known as flake8-async for asyncio. But this plugin was a superset of the checks in flake8-async, and support for anyio was added, so it's now named flake8-async to more properly convey its usage. At the same time all error codes were renamed from TRIOxxx to ASYNCxxx, as was previously used by the old flake8-async. + +## Rules +https://flake8-async.readthedocs.io/en/latest/rules.html diff --git a/docs/index.rst b/docs/index.rst index b38f1a6a..092f5335 100644 --- a/docs/index.rst +++ b/docs/index.rst @@ -33,6 +33,7 @@ Contents: usage rules + glossary changelog contributing @@ -46,5 +47,6 @@ Indices and tables * :ref:`search` * :doc:`usage` * :doc:`rules` +* :doc:`glossary` * :doc:`changelog` * :doc:`contributing` diff --git a/docs/rules.rst b/docs/rules.rst index 049c13b6..b0dd562e 100644 --- a/docs/rules.rst +++ b/docs/rules.rst @@ -6,105 +6,6 @@ List of rules it also allows easier use of library-specific terminology without forcing people to know all libraries by heart. It should probably have it's own page in the long run -Glossary -======== - -.. _cancel_scope: - -cancel scope ------------- -A cancel scope is a context manager which can request the library cancels -whatever task is executing in the body of the ``with`` (or ``async with``) -block. A cancel scope is the key component of a :ref:`timeout context `, and used in :ref:`TaskGroups / Nurseries ` to cancel any remaining child tasks if one raises an -exception. Trio and AnyIO have an explicit ``CancelScope`` type; in asyncio -they are implicit. - -* Trio - - * `Documentation `__ - - * :class:`trio.CancelScope` - -* AnyIO - - * `Documentation `__ - - * :class:`anyio.CancelScope` - -.. _timeout_context: - -timeout context ---------------- -A context manager that enforces a timeout on a block of code, by cancelling it -after a specified duration or at a preset time. The timeout can also be -rescheduled after creation. They are internally implemented with a :ref:`cancel scope `, which in anyio & trio can be directly initialized with a deadline. - -.. I find this to have excessive spacing before/after sublists. Probably requires CSS to fix? - -* Trio - - * `Documentation `__ - - * :func:`trio.move_on_after`, :func:`trio.move_on_at`, :func:`trio.fail_after`, :func:`trio.fail_at`, :class:`trio.CancelScope` - -* AnyIO - - * `Documentation `__ - - * :func:`anyio.move_on_after`, :func:`anyio.fail_after`, :class:`anyio.CancelScope` - -* asyncio - - * `Documentation `__ - - * :func:`asyncio.timeout`, :func:`asyncio.timeout_at` - -.. _taskgroup_nursery: - -TaskGroup / Nursery -------------------- - -A collection of child Tasks that can run concurrently. Internally contains a :ref:`cancel scope ` for canceling any remaining child tasks if one raises an exception. - -* Trio - - * `Documentation `__ - - * :class:`trio.Nursery`, created with :func:`trio.open_nursery` -* AnyIO - - * `Documentation `__ - * :class:`anyio.abc.TaskGroup`, created with :func:`anyio.create_task_group`. -* asyncio - - * `Documentation `__ - * :class:`asyncio.TaskGroup` (since python 3.11) - - -.. _cancelled: - -Cancelled / CancelledError --------------------------- - -Handling cancellation is very sensitive, and you generally never want to catch a cancellation exception without letting it propagate to the library. - - * Trio: :class:`trio.Cancelled`. `Documentation `__ - * AnyIO: :func:`anyio.get_cancelled_exc_class`. `Documentation `__ - * asyncio: :class:`asyncio.CancelledError`. `Documentation `__ - -.. _checkpoint: - -Checkpoint ----------- -Checkpoints are ``await``, ``async for``, and ``async with`` (on one of enter/exit). TODO write more and link stuff - -.. _channel_stream_queue: - -Channel / Stream / Queue ------------------------- -* Trio: `channel `__ -* AnyIO: `stream `__ -* asyncio: `queue `__ General rules From bc1796c496b9c762685f4170f0a5f8fa12420199 Mon Sep 17 00:00:00 2001 From: jakkdl Date: Fri, 17 May 2024 17:52:16 +0200 Subject: [PATCH 07/13] actually add glossary.rst --- docs/glossary.rst | 100 ++++++++++++++++++++++++++++++++++++++++++++++ docs/rules.rst | 2 +- 2 files changed, 101 insertions(+), 1 deletion(-) create mode 100644 docs/glossary.rst diff --git a/docs/glossary.rst b/docs/glossary.rst new file mode 100644 index 00000000..e9d2fc55 --- /dev/null +++ b/docs/glossary.rst @@ -0,0 +1,100 @@ +******** +Glossary +******** + +.. _cancel_scope: + +cancel scope +------------ +A cancel scope is a context manager which can request the library cancels +whatever task is executing in the body of the ``with`` (or ``async with``) +block. A cancel scope is the key component of a :ref:`timeout context `, and used in :ref:`TaskGroups / Nurseries ` to cancel any remaining child tasks if one raises an +exception. Trio and AnyIO have an explicit ``CancelScope`` type; in asyncio +they are implicit. + +* Trio + + * `Documentation `__ + + * :class:`trio.CancelScope` + +* AnyIO + + * `Documentation `__ + + * :class:`anyio.CancelScope` + +.. _timeout_context: + +timeout context +--------------- +A context manager that enforces a timeout on a block of code, by cancelling it +after a specified duration or at a preset time. The timeout can also be +rescheduled after creation. They are internally implemented with a :ref:`cancel scope `, which in anyio & trio can be directly initialized with a deadline. + +.. I find this to have excessive spacing before/after sublists. Probably requires CSS to fix? + +* Trio + + * `Documentation `__ + + * :func:`trio.move_on_after`, :func:`trio.move_on_at`, :func:`trio.fail_after`, :func:`trio.fail_at`, :class:`trio.CancelScope` + +* AnyIO + + * `Documentation `__ + + * :func:`anyio.move_on_after`, :func:`anyio.fail_after`, :class:`anyio.CancelScope` + +* asyncio + + * `Documentation `__ + + * :func:`asyncio.timeout`, :func:`asyncio.timeout_at` + +.. _taskgroup_nursery: + +TaskGroup / Nursery +------------------- + +A collection of child Tasks that can run concurrently. Internally contains a :ref:`cancel scope ` for canceling any remaining child tasks if one raises an exception. + +* Trio + + * `Documentation `__ + + * :class:`trio.Nursery`, created with :func:`trio.open_nursery` +* AnyIO + + * `Documentation `__ + * :class:`anyio.abc.TaskGroup`, created with :func:`anyio.create_task_group`. +* asyncio + + * `Documentation `__ + * :class:`asyncio.TaskGroup` (since python 3.11) + + +.. _cancelled: + +Cancelled / CancelledError +-------------------------- + +Handling cancellation is very sensitive, and you generally never want to catch a cancellation exception without letting it propagate to the library. + + * Trio: :class:`trio.Cancelled`. `Documentation `__ + * AnyIO: :func:`anyio.get_cancelled_exc_class`. `Documentation `__ + * asyncio: :class:`asyncio.CancelledError`. `Documentation `__ + +.. _checkpoint: + +Checkpoint +---------- +Checkpoints are ``await``, ``async for``, and ``async with`` (on one of enter/exit). TODO write more and link stuff + +.. _channel_stream_queue: + +Channel / Stream / Queue +------------------------ +* Trio: `channel `__ +* AnyIO: `stream `__ +* asyncio: `queue `__ diff --git a/docs/rules.rst b/docs/rules.rst index b0dd562e..33877673 100644 --- a/docs/rules.rst +++ b/docs/rules.rst @@ -100,7 +100,7 @@ Note: 22X, 23X and 24X has not had asyncio-specific suggestions written. - blocking-http-call-pool - Likely sync HTTP call in async function, use ``httpx.AsyncClient``. Looks for ``urllib3`` method calls on pool objects, but only matching on the method signature and not the object. * - ASYNC212 - - blocking-sync-http-call-httpx + - blocking-http-call-httpx - Blocking sync HTTP call on httpx object, use httpx.AsyncClient. * - ASYNC220 - blocking-process-call-1 From b05247351ce61edfc1d7c9f3b90ed557400246a8 Mon Sep 17 00:00:00 2001 From: jakkdl Date: Mon, 20 May 2024 11:51:21 +0200 Subject: [PATCH 08/13] switch from table to definition list --- docs/index.rst | 2 +- docs/rules.rst | 234 ++++++++++++++++++++----------------------------- 2 files changed, 98 insertions(+), 138 deletions(-) diff --git a/docs/index.rst b/docs/index.rst index 092f5335..5346d8da 100644 --- a/docs/index.rst +++ b/docs/index.rst @@ -17,7 +17,7 @@ a misunderstanding. The plugin may well be too noisy or pedantic depending on your requirements or opinions, in which case you should consider :ref:`--disable` for those rules. -Pairs well with flake8-bugbear. +Pairs well with `flake8-bugbear `__. Some rules are incorporated into `ruff `__. diff --git a/docs/rules.rst b/docs/rules.rst index 33877673..e6740ade 100644 --- a/docs/rules.rst +++ b/docs/rules.rst @@ -2,164 +2,124 @@ List of rules **************** -.. Esp when writing short descriptions it'd be very handy to link to a glossary, instead of saying stuff like ``except BaseException/trio.Cancelled/anyio.get_cancelled_exc_class()/asyncio.exceptions.CancelledError`` - it also allows easier use of library-specific terminology without forcing people to know all libraries by heart. - It should probably have it's own page in the long run - - General rules ============= +_`ASYNC100` : cancel-scope-no-checkpoint + A :ref:`timeout_context` does not contain any :ref:`checkpoint` statements. This makes it pointless, as the timeout can only be triggered by a checkpoint. This check also allows ``yield`` statements, since checkpoints can happen in the caller we yield to. + +ASYNC101 : yield-in-cancel-scope + ``yield`` inside a :ref:`TaskGroup/Nursery ` or :ref:`timeout_context` is only safe when implementing a context manager - otherwise, it breaks exception handling. See `this thread `_ for discussion of a future PEP. This has substantial overlap with :ref::ref:`ASYNC119 `, which will warn on almost all instances of ASYNC101, but ASYNC101 is about a conceptually different problem that will not get resolved by `PEP 533 `_. + +ASYNC102 : await-in-finally-or-cancelled + ``await`` inside ``finally`` or :ref:`cancelled-catching ` ``except:`` must have shielded :ref:`cancel scope ` with timeout. This is currently not able to detect asyncio shields. + +ASYNC103 : no-reraise-cancelled + :ref:`cancelled`-catching exception that does not reraise the exception. If you don't want to re-raise :class:`BaseException`, add a separate handler for :ref:`Cancelled` before. + +ASYNC104 : cancelled-not-raised + :ref:`Cancelled`-catching exception does not raise the exception. Triggered on ``return`` or raising a different exception. + +ASYNC105 : missing-await + async trio function called without using ``await``. This is only supported with trio functions, but you can get similar functionality with a type-checker. + +ASYNC106 : bad-async-library-import + trio/anyio/asyncio must be imported with ``import xxx`` for the linter to work. + +ASYNC109 : async-function-with-timeout + Async function definition with a ``timeout`` parameter. In structured concurrency the caller should instead use :ref:`timeout context managers `. + +ASYNC110 : busy-wait + ``while ...: await [trio/anyio].sleep()`` should be replaced by a :class:`trio.Event`/:class:`anyio.Event`. + +ASYNC111 : variable-from-cm-in-start-soon + Variable, from context manager opened inside nursery, passed to ``start[_soon]`` might be invalidly accessed while in use, due to context manager closing before the nursery. This is usually a bug, and nurseries should generally be the inner-most context manager. + +ASYNC112 : useless-nursery + :ref:`Nursery/TaskGroup ` body with only a call to ``.start[_soon]`` and not passing itself as a parameter can be replaced with a regular function call. + +ASYNC113 : start-soon-in-aenter + Using :meth:`~trio.Nursery.start_soon`/:meth:`~anyio.abc.TaskGroup.start_soon` in ``__aenter__`` doesn't wait for the task to begin. Consider replacing with :meth:`~trio.Nursery.start`/:meth:`~anyio.abc.TaskGroup.start`. + +ASYNC114 : startable-not-in-config + Startable function (i.e. has a ``task_status`` keyword parameter) not in :ref:`--startable-in-context-manager <--startable-in-context-manager>` parameter list, please add it so ASYNC113 can catch errors when using it. + +ASYNC115 : sleep-zero + Replace :func:`trio.sleep(0) `/:func:`anyio.sleep(0) ` with the more suggestive :func:`trio.lowlevel.checkpoint`/:func:`anyio.lowlevel.checkpoint`. + +ASYNC116 : long-sleep-not-forever + :func:`trio.sleep`/:func:`anyio.sleep` with >24 hour interval should usually be :func:`trio.sleep_forever`/:func:`anyio.sleep_forever`. + +ASYNC118 : cancelled-class-saved + Don't assign the value of :func:`anyio.get_cancelled_exc_class()` to a variable, since that breaks linter checks and multi-backend programs. + +_`ASYNC119` : yield-in-cm-in-async-gen + ``yield`` in context manager in async generator is unsafe, the cleanup may be delayed until ``await`` is no longer allowed. We strongly encourage you to read `PEP 533 `_ and use `async with aclosing(...) `_, or better yet avoid async generators entirely (see `ASYNC900`_ ) in favor of context managers which return an iterable :ref:`channel/stream/queue `. + -.. For some reason using :ref:`timeout_context` fails to find the reference, but :ref:`timeout_context ` works. I have no clue why - -.. list-table:: - :widths: 1 18 40 - :header-rows: 1 - - * - Code - - Name - - Message - * - ASYNC100 - - cancel-scope-no-checkpoint - - .. _async100: - - A :ref:`timeout_context ` does not contain any :ref:`checkpoints ` statements. This makes it pointless, as the timeout can only be triggered by a checkpoint. This check also allows ``yield`` statements, since checkpoints can happen in the caller we yield to. - * - ASYNC101 - - yield-in-cancel-scope - - ``yield`` inside a :ref:`TaskGroup/Nursery ` or :ref:`timeout_context ` is only safe when implementing a context manager - otherwise, it breaks exception handling. See `this thread `_ for discussion of a future PEP. This has substantial overlap with :ref:`ASYNC119 `, which will warn on almost all instances of ASYNC101, but ASYNC101 is about a conceptually different problem that will not get resolved by `PEP 533 `_. - * - ASYNC102 - - await-in-finally-or-cancelled - - ``await`` inside ``finally`` or :ref:`cancelled-catching ` ``except:`` must have shielded :ref:`cancel scope ` with timeout. This is currently not able to detect asyncio shields. - * - ASYNC103 - - no-reraise-cancelled - - :ref:`cancelled `-catching exception that does not reraise the exception. If you don't want to re-raise :class:`BaseException`, add a separate handler for :ref:`Cancelled ` before. - * - ASYNC104 - - cancelled-not-raised - - :ref:`cancelled `-catching exception does not raise the exception. Triggered on ``return`` or raising a different exception. - * - ASYNC105 - - missing-await - - async trio function called without using ``await``. This is only supported with trio functions, but you can get similar functionality with a type-checker. - * - ASYNC106 - - bad-async-library-import - - trio/anyio/asyncio must be imported with ``import xxx`` for the linter to work. - * - ASYNC109 - - async-function-with-timeout - - Async function definition with a ``timeout`` parameter. In structured concurrency the caller should instead use :ref:`timeout context managers `. - * - ASYNC110 - - busy-wait - - ``while ...: await [trio/anyio].sleep()`` should be replaced by a :class:`trio.Event`/:class:`anyio.Event`. - * - ASYNC111 - - variable-from-cm-in-start-soon - - Variable, from context manager opened inside nursery, passed to ``start[_soon]`` might be invalidly accessed while in use, due to context manager closing before the nursery. This is usually a bug, and nurseries should generally be the inner-most context manager. - * - ASYNC112 - - useless-nursery - - :ref:`Nursery/TaskGroup ` body with only a call to ``.start[_soon]`` and not passing itself as a parameter can be replaced with a regular function call. - * - ASYNC113 - - start-soon-in-aenter - - Using :meth:`~trio.Nursery.start_soon`/:meth:`~anyio.abc.TaskGroup.start_soon` in ``__aenter__`` doesn't wait for the task to begin. Consider replacing with :meth:`~trio.Nursery.start`/:meth:`~anyio.abc.TaskGroup.start`. - * - ASYNC114 - - startable-not-in-config - - Startable function (i.e. has a ``task_status`` keyword parameter) not in :ref:`--startable-in-context-manager <--startable-in-context-manager>` parameter list, please add it so ASYNC113 can catch errors when using it. - * - ASYNC115 - - sleep-zero - - Replace :func:`trio.sleep(0) `/:func:`anyio.sleep(0) ` with the more suggestive :func:`trio.lowlevel.checkpoint`/:func:`anyio.lowlevel.checkpoint`. - * - ASYNC116 - - long-sleep-not-forever - - :func:`trio.sleep`/:func:`anyio.sleep` with >24 hour interval should usually be :func:`trio.sleep_forever`/:func:`anyio.sleep_forever`. - * - ASYNC118 - - cancelled-class-saved - - Don't assign the value of :func:`anyio.get_cancelled_exc_class()` to a variable, since that breaks linter checks and multi-backend programs. - * - ASYNC119 - - yield-in-cm-in-async-gen - - .. _async119: - - ``yield`` in context manager in async generator is unsafe, the cleanup may be delayed until ``await`` is no longer allowed. We strongly encourage you to read `PEP 533 `_ and use `async with aclosing(...) `_, or better yet avoid async generators entirely (see :ref:`ASYNC900 ` ) in favor of context managers which return an iterable :ref:`channel/stream/queue `. Blocking sync calls in async functions ====================================== Note: 22X, 23X and 24X has not had asyncio-specific suggestions written. -.. list-table:: - :widths: 1 18 40 - :header-rows: 1 - - * - Code - - Name - - Message - * - ASYNC200 - - blocking-call - - User-configured error for blocking sync calls in async functions. Does nothing by default, see :ref:`async200-blocking-calls` for how to configure it. - * - ASYNC210 - - blocking-http-call - - Sync HTTP call in async function, use ``httpx.AsyncClient``. This and the other ASYNC21x checks look for usage of ``urllib3`` and ``httpx.Client``, and recommend using ``httpx.AsyncClient`` as that's the largest http client supporting anyio/trio. - * - ASYNC211 - - blocking-http-call-pool - - Likely sync HTTP call in async function, use ``httpx.AsyncClient``. Looks for ``urllib3`` method calls on pool objects, but only matching on the method signature and not the object. - * - ASYNC212 - - blocking-http-call-httpx - - Blocking sync HTTP call on httpx object, use httpx.AsyncClient. - * - ASYNC220 - - blocking-process-call-1 - - Sync process call in async function, use ``await nursery.start([trio/anyio].run_process, ...)``. ``asyncio`` users can use `asyncio.create_subprocess_[exec/shell] `_. - * - ASYNC221 - - blocking-process-call-2 - - Sync process call in async function, use ``await [trio/anyio].run_process(...)``. ``asyncio`` users can use `asyncio.create_subprocess_[exec/shell] `_. - * - ASYNC222 - - blocking-process-call-3 - - Sync ``os.*`` call in async function, wrap in ``await [trio/anyio].to_thread.run_sync()``. ``asyncio`` users can use `asyncio.loop.run_in_executor `_. - * - ASYNC230 - - blocking-io-call - - Sync IO call in async function, use ``[trio/anyio].open_file(...)``. ``asyncio`` users need to use a library such as `aiofiles `_, or switch to `anyio `_. - * - ASYNC231 - - blocking-io-call-wrap - - Sync IO call in async function, use ``[trio/anyio].wrap_file(...)``. ``asyncio`` users need to use a library such as `aiofiles `_, or switch to `anyio `_. - * - ASYNC232 - - blocking-file-call - - Blocking sync call on file object, wrap the file object in ``[trio/anyio].wrap_file()`` to get an async file object. - * - ASYNC240 - - blocking-path-usage - - Avoid using :mod:`os.path` in async functions, prefer using ``[trio/anyio].Path`` objects. ``asyncio`` users should consider `aiopath `_ or `anyio `_. - * - ASYNC250 - - blocking-input - - Builtin ``input()`` should not be called from async function. Wrap in ``[trio/anyio].to_thread.run_sync()`` or ``asyncio.loop.run_in_executor()``. - * - ASYNC251 - - blocking-sleep - - ``time.sleep(...)`` should not be called from async function. Use ``[trio/anyio/asyncio].sleep(...)``. +ASYNC200 : blocking-call + User-configured error for blocking sync calls in async functions. Does nothing by default, see :ref:`async200-blocking-calls` for how to configure it. +ASYNC210 : blocking-http-call + Sync HTTP call in async function, use ``httpx.AsyncClient``. This and the other ASYNC21x checks look for usage of ``urllib3`` and ``httpx.Client``, and recommend using ``httpx.AsyncClient`` as that's the largest http client supporting anyio/trio. -Optional rules disabled by default -================================== +ASYNC211 : blocking-http-call-pool + Likely sync HTTP call in async function, use ``httpx.AsyncClient``. Looks for ``urllib3`` method calls on pool objects, but only matching on the method signature and not the object. + +ASYNC212 : blocking-http-call-httpx + Blocking sync HTTP call on httpx object, use httpx.AsyncClient. + +ASYNC220 : blocking-process-call-1 + Sync process call in async function, use ``await nursery.start([trio/anyio].run_process, ...)``. ``asyncio`` users can use `asyncio.create_subprocess_[exec/shell] `_. + +ASYNC221 : blocking-process-call-2 + Sync process call in async function, use ``await [trio/anyio].run_process(...)``. ``asyncio`` users can use `asyncio.create_subprocess_[exec/shell] `_. + +ASYNC222 : blocking-process-call-3 + Sync ``os.*`` call in async function, wrap in ``await [trio/anyio].to_thread.run_sync()``. ``asyncio`` users can use `asyncio.loop.run_in_executor `_. + +ASYNC230 : blocking-io-call + Sync IO call in async function, use ``[trio/anyio].open_file(...)``. ``asyncio`` users need to use a library such as `aiofiles `_, or switch to `anyio `_. -.. list-table:: - :widths: 1 18 40 - :header-rows: 1 +ASYNC231 : blocking-io-call-wrap + Sync IO call in async function, use ``[trio/anyio].wrap_file(...)``. ``asyncio`` users need to use a library such as `aiofiles `_, or switch to `anyio `_. - * - Code - - Name - - Message - * - ASYNC900 - - unsafe-async-generator - - .. _async900: +ASYNC232 : blocking-file-call + Blocking sync call on file object, wrap the file object in ``[trio/anyio].wrap_file()`` to get an async file object. +ASYNC240 : blocking-path-usage + Avoid using :mod:`os.path` in async functions, prefer using ``[trio/anyio].Path`` objects. ``asyncio`` users should consider `aiopath `__ or `anyio `__. + +ASYNC250 : blocking-input + Builtin ``input()`` should not be called from async function. Wrap in ``[trio/anyio].to_thread.run_sync()`` or ``asyncio.loop.run_in_executor()``. + +ASYNC251 : blocking-sleep + ``time.sleep(...)`` should not be called from async function. Use ``[trio/anyio/asyncio].sleep(...)``. + + +Optional rules disabled by default +================================== + +_`ASYNC900` : unsafe-async-generator Async generator without ``@asynccontextmanager`` not allowed. You might want to enable this on a codebase since async generators are inherently unsafe and cleanup logic might not be performed. See https://github.com/python-trio/flake8-async/issues/211 and https://discuss.python.org/t/using-exceptiongroup-at-anthropic-experience-report/20888/6 for discussion. - * - ASYNC910 - - async-function-no-checkpoint - - .. _async910: - Exit or ``return`` from async function with no guaranteed :ref:`checkpoint ` or exception since function definition. You might want to enable this on a codebase to make it easier to reason about checkpoints, and make the logic of ASYNC911 correct. - * - ASYNC911 - - async-generator-no-checkpoint - - .. _async911: +_`ASYNC910` : async-function-no-checkpoint + Exit or ``return`` from async function with no guaranteed :ref:`checkpoint` or exception since function definition. You might want to enable this on a codebase to make it easier to reason about checkpoints, and make the logic of ASYNC911 correct. + +_`ASYNC911` : async-generator-no-checkpoint + Exit, ``yield`` or ``return`` from async iterable with no guaranteed :ref:`checkpoint` since possible function entry (yield or function definition). - Exit, ``yield`` or ``return`` from async iterable with no guaranteed :ref:`checkpoint ` since possible function entry (yield or function definition). - * - ASYNC912 - - cancel-scope-no-guaranteed-checkpoint - - A timeout/cancelscope has :ref:`checkpoints `, but they're not guaranteed to run. Similar to :ref:`ASYNC100 `, but it does not warn on trivial cases where there is no checkpoint at all. It instead shares logic with :ref:`ASYNC910 ` and :ref:`ASYNC911 ` for parsing conditionals and branches. +ASYNC912 : cancel-scope-no-guaranteed-checkpoint + A timeout/cancelscope has :ref:`checkpoints `, but they're not guaranteed to run. Similar to `ASYNC100`_, but it does not warn on trivial cases where there is no checkpoint at all. It instead shares logic with `ASYNC910`_ and `ASYNC911`_ for parsing conditionals and branches. Removed rules ================ From c5298ffeadb3bd28129c359ccecda63441b50fd5 Mon Sep 17 00:00:00 2001 From: jakkdl Date: Mon, 20 May 2024 13:18:32 +0200 Subject: [PATCH 09/13] add more intersphinx links --- docs/glossary.rst | 2 +- docs/rules.rst | 94 ++++++++++++++++++++++++++++++----------------- 2 files changed, 61 insertions(+), 35 deletions(-) diff --git a/docs/glossary.rst b/docs/glossary.rst index e9d2fc55..258bfe87 100644 --- a/docs/glossary.rst +++ b/docs/glossary.rst @@ -89,7 +89,7 @@ Handling cancellation is very sensitive, and you generally never want to catch a Checkpoint ---------- -Checkpoints are ``await``, ``async for``, and ``async with`` (on one of enter/exit). TODO write more and link stuff +Checkpoints are ``await``, ``async for``, and ``async with`` (on at least one of enter/exit). TODO write more and link stuff .. _channel_stream_queue: diff --git a/docs/rules.rst b/docs/rules.rst index e6740ade..45e35cec 100644 --- a/docs/rules.rst +++ b/docs/rules.rst @@ -8,40 +8,51 @@ General rules _`ASYNC100` : cancel-scope-no-checkpoint - A :ref:`timeout_context` does not contain any :ref:`checkpoint` statements. This makes it pointless, as the timeout can only be triggered by a checkpoint. This check also allows ``yield`` statements, since checkpoints can happen in the caller we yield to. + A :ref:`timeout_context` does not contain any :ref:`checkpoints `. + This makes it pointless, as the timeout can only be triggered by a checkpoint. + This check also treats ``yield`` as a checkpoint, since checkpoints can happen in the caller we yield to. ASYNC101 : yield-in-cancel-scope - ``yield`` inside a :ref:`TaskGroup/Nursery ` or :ref:`timeout_context` is only safe when implementing a context manager - otherwise, it breaks exception handling. See `this thread `_ for discussion of a future PEP. This has substantial overlap with :ref::ref:`ASYNC119 `, which will warn on almost all instances of ASYNC101, but ASYNC101 is about a conceptually different problem that will not get resolved by `PEP 533 `_. + ``yield`` inside a :ref:`taskgroup_nursery` or :ref:`timeout_context` is only safe when implementing a context manager - otherwise, it breaks exception handling. + See `this thread `_ for discussion of a future PEP. + This has substantial overlap with :ref:`ASYNC119 `, which will warn on almost all instances of ASYNC101, but ASYNC101 is about a conceptually different problem that will not get resolved by `PEP 533 `_. ASYNC102 : await-in-finally-or-cancelled - ``await`` inside ``finally`` or :ref:`cancelled-catching ` ``except:`` must have shielded :ref:`cancel scope ` with timeout. This is currently not able to detect asyncio shields. + ``await`` inside ``finally`` or :ref:`cancelled-catching ` ``except:`` must have shielded :ref:`cancel scope ` with timeout. + This is currently not able to detect asyncio shields. ASYNC103 : no-reraise-cancelled - :ref:`cancelled`-catching exception that does not reraise the exception. If you don't want to re-raise :class:`BaseException`, add a separate handler for :ref:`Cancelled` before. + :ref:`cancelled`-catching exception that does not reraise the exception. + If you don't want to re-raise :class:`BaseException`, add a separate handler for :ref:`Cancelled` before. ASYNC104 : cancelled-not-raised - :ref:`Cancelled`-catching exception does not raise the exception. Triggered on ``return`` or raising a different exception. + :ref:`Cancelled`-catching exception does not raise the exception. + Triggered on ``return`` or raising a different exception. ASYNC105 : missing-await - async trio function called without using ``await``. This is only supported with trio functions, but you can get similar functionality with a type-checker. + async trio function called without using ``await``. + This is only supported with trio functions, but you can get similar functionality with a type-checker. ASYNC106 : bad-async-library-import trio/anyio/asyncio must be imported with ``import xxx`` for the linter to work. ASYNC109 : async-function-with-timeout - Async function definition with a ``timeout`` parameter. In structured concurrency the caller should instead use :ref:`timeout context managers `. + Async function definition with a ``timeout`` parameter. + In structured concurrency the caller should instead use :ref:`timeout context managers `. ASYNC110 : busy-wait ``while ...: await [trio/anyio].sleep()`` should be replaced by a :class:`trio.Event`/:class:`anyio.Event`. ASYNC111 : variable-from-cm-in-start-soon - Variable, from context manager opened inside nursery, passed to ``start[_soon]`` might be invalidly accessed while in use, due to context manager closing before the nursery. This is usually a bug, and nurseries should generally be the inner-most context manager. + Variable, from context manager opened inside :ref:`taskgroup_nursery`, passed to ``start[_soon]`` might be invalidly accessed while in use, due to context manager closing before the nursery. + This is usually a bug, and nurseries should generally be the inner-most context manager. ASYNC112 : useless-nursery - :ref:`Nursery/TaskGroup ` body with only a call to ``.start[_soon]`` and not passing itself as a parameter can be replaced with a regular function call. + :ref:`taskgroup_nursery` body with only a call to ``.start[_soon]`` and not passing itself as a parameter can be replaced with a regular function call. ASYNC113 : start-soon-in-aenter - Using :meth:`~trio.Nursery.start_soon`/:meth:`~anyio.abc.TaskGroup.start_soon` in ``__aenter__`` doesn't wait for the task to begin. Consider replacing with :meth:`~trio.Nursery.start`/:meth:`~anyio.abc.TaskGroup.start`. + Using :meth:`~trio.Nursery.start_soon`/:meth:`~anyio.abc.TaskGroup.start_soon` in ``__aenter__`` doesn't wait for the task to begin. + Consider replacing with :meth:`~trio.Nursery.start`/:meth:`~anyio.abc.TaskGroup.start`. ASYNC114 : startable-not-in-config Startable function (i.e. has a ``task_status`` keyword parameter) not in :ref:`--startable-in-context-manager <--startable-in-context-manager>` parameter list, please add it so ASYNC113 can catch errors when using it. @@ -56,70 +67,85 @@ ASYNC118 : cancelled-class-saved Don't assign the value of :func:`anyio.get_cancelled_exc_class()` to a variable, since that breaks linter checks and multi-backend programs. _`ASYNC119` : yield-in-cm-in-async-gen - ``yield`` in context manager in async generator is unsafe, the cleanup may be delayed until ``await`` is no longer allowed. We strongly encourage you to read `PEP 533 `_ and use `async with aclosing(...) `_, or better yet avoid async generators entirely (see `ASYNC900`_ ) in favor of context managers which return an iterable :ref:`channel/stream/queue `. + ``yield`` in context manager in async generator is unsafe, the cleanup may be delayed until ``await`` is no longer allowed. + We strongly encourage you to read `PEP 533 `_ and use `async with aclosing(...) `_, or better yet avoid async generators entirely (see `ASYNC900`_ ) in favor of context managers which return an iterable :ref:`channel/stream/queue `. Blocking sync calls in async functions ====================================== +.. _httpx.Client: https://www.python-httpx.org/api/#client +.. _httpx.AsyncClient: https://www.python-httpx.org/api/#asyncclient +.. _urllib3: https://github.com/urllib3/urllib3 + Note: 22X, 23X and 24X has not had asyncio-specific suggestions written. -ASYNC200 : blocking-call - User-configured error for blocking sync calls in async functions. Does nothing by default, see :ref:`async200-blocking-calls` for how to configure it. +ASYNC200 : blocking-configured-call + User-configured error for blocking sync calls in async functions. + Does nothing by default, see :ref:`async200-blocking-calls` for how to configure it. ASYNC210 : blocking-http-call - Sync HTTP call in async function, use ``httpx.AsyncClient``. This and the other ASYNC21x checks look for usage of ``urllib3`` and ``httpx.Client``, and recommend using ``httpx.AsyncClient`` as that's the largest http client supporting anyio/trio. + Sync HTTP call in async function, use `httpx.AsyncClient`_. + This and the other ASYNC21x checks look for usage of `urllib3`_ and `httpx.Client`_, and recommend using `httpx.AsyncClient`_ as that's the largest http client supporting anyio/trio. ASYNC211 : blocking-http-call-pool - Likely sync HTTP call in async function, use ``httpx.AsyncClient``. Looks for ``urllib3`` method calls on pool objects, but only matching on the method signature and not the object. + Likely sync HTTP call in async function, use `httpx.AsyncClient`_. + Looks for `urllib3`_ method calls on pool objects, but only matching on the method signature and not the object. ASYNC212 : blocking-http-call-httpx - Blocking sync HTTP call on httpx object, use httpx.AsyncClient. + Blocking sync HTTP call on httpx object, use `httpx.AsyncClient`_. -ASYNC220 : blocking-process-call-1 - Sync process call in async function, use ``await nursery.start([trio/anyio].run_process, ...)``. ``asyncio`` users can use `asyncio.create_subprocess_[exec/shell] `_. +ASYNC220 : blocking-create-subprocess + Sync call to :class:`subprocess.Popen` (or equivalent) in async function, use :func:`trio.run_process`/:func:`anyio.run_process` in a :ref:`taskgroup_nursery`. ``asyncio`` users can use `asyncio.create_subprocess_[exec/shell] `_. -ASYNC221 : blocking-process-call-2 - Sync process call in async function, use ``await [trio/anyio].run_process(...)``. ``asyncio`` users can use `asyncio.create_subprocess_[exec/shell] `_. +ASYNC221 : blocking-run-process + Sync call to :func:`subprocess.run` (or equivalent) in async function, use :func:`trio.run_process`/:func:`anyio.run_process`. ``asyncio`` users can use `asyncio.create_subprocess_[exec/shell] `_. -ASYNC222 : blocking-process-call-3 - Sync ``os.*`` call in async function, wrap in ``await [trio/anyio].to_thread.run_sync()``. ``asyncio`` users can use `asyncio.loop.run_in_executor `_. +ASYNC222 : blocking-process-wait + Sync call to :func:`os.wait` (or equivalent) in async function, wrap in :func:`trio.to_thread.run_sync`/:func:`anyio.to_thread.run_sync`. ``asyncio`` users can use `asyncio.loop.run_in_executor `_. -ASYNC230 : blocking-io-call - Sync IO call in async function, use ``[trio/anyio].open_file(...)``. ``asyncio`` users need to use a library such as `aiofiles `_, or switch to `anyio `_. +ASYNC230 : blocking-open-call + Sync call to :func:`open` in async function, use :func:`trio.open_file`/:func:`anyio.open_file`. ``asyncio`` users need to use a library such as `aiofiles `_, or switch to `anyio `_. -ASYNC231 : blocking-io-call-wrap - Sync IO call in async function, use ``[trio/anyio].wrap_file(...)``. ``asyncio`` users need to use a library such as `aiofiles `_, or switch to `anyio `_. +ASYNC231 : blocking-fdopen-call + Sync call to :func:`os.fdopen` in async function, use :func:`trio.wrap_file`/:func:`anyio.wrap_file`. ``asyncio`` users need to use a library such as `aiofiles `_, or switch to `anyio `_. ASYNC232 : blocking-file-call - Blocking sync call on file object, wrap the file object in ``[trio/anyio].wrap_file()`` to get an async file object. + Blocking sync call on file object, wrap the file object in :func:`trio.wrap_file`/:func:`anyio.wrap_file` to get an async file object. ASYNC240 : blocking-path-usage - Avoid using :mod:`os.path` in async functions, prefer using ``[trio/anyio].Path`` objects. ``asyncio`` users should consider `aiopath `__ or `anyio `__. + Avoid using :mod:`os.path` in async functions, prefer using :class:`trio.Path`/:class:`anyio.Path` objects. ``asyncio`` users should consider `aiopath `__ or `anyio `__. ASYNC250 : blocking-input - Builtin ``input()`` should not be called from async function. Wrap in ``[trio/anyio].to_thread.run_sync()`` or ``asyncio.loop.run_in_executor()``. + Builtin :func:`input` should not be called from async function. + Wrap in :func:`trio.to_thread.run_sync`/:func:`anyio.to_thread.run_sync` or :meth:`asyncio.loop.run_in_executor`. ASYNC251 : blocking-sleep - ``time.sleep(...)`` should not be called from async function. Use ``[trio/anyio/asyncio].sleep(...)``. + :func:`time.sleep` should not be called from async function. + Use :func:`trio.sleep`/:func:`anyio.sleep`/:func:`asyncio.sleep`. Optional rules disabled by default ================================== _`ASYNC900` : unsafe-async-generator - Async generator without ``@asynccontextmanager`` not allowed. You might want to enable this on a codebase since async generators are inherently unsafe and cleanup logic might not be performed. See https://github.com/python-trio/flake8-async/issues/211 and https://discuss.python.org/t/using-exceptiongroup-at-anthropic-experience-report/20888/6 for discussion. + Async generator without :func:`@asynccontextmanager ` not allowed. + You might want to enable this on a codebase since async generators are inherently unsafe and cleanup logic might not be performed. + See `#211 `__ and https://discuss.python.org/t/using-exceptiongroup-at-anthropic-experience-report/20888/6 for discussion. _`ASYNC910` : async-function-no-checkpoint - Exit or ``return`` from async function with no guaranteed :ref:`checkpoint` or exception since function definition. You might want to enable this on a codebase to make it easier to reason about checkpoints, and make the logic of ASYNC911 correct. + Exit or ``return`` from async function with no guaranteed :ref:`checkpoint` or exception since function definition. + You might want to enable this on a codebase to make it easier to reason about checkpoints, and make the logic of ASYNC911 correct. _`ASYNC911` : async-generator-no-checkpoint - Exit, ``yield`` or ``return`` from async iterable with no guaranteed :ref:`checkpoint` since possible function entry (yield or function definition). + Exit, ``yield`` or ``return`` from async iterable with no guaranteed :ref:`checkpoint` since possible function entry (``yield`` or function definition). ASYNC912 : cancel-scope-no-guaranteed-checkpoint - A timeout/cancelscope has :ref:`checkpoints `, but they're not guaranteed to run. Similar to `ASYNC100`_, but it does not warn on trivial cases where there is no checkpoint at all. It instead shares logic with `ASYNC910`_ and `ASYNC911`_ for parsing conditionals and branches. + A timeout/cancelscope has :ref:`checkpoints `, but they're not guaranteed to run. + Similar to `ASYNC100`_, but it does not warn on trivial cases where there is no checkpoint at all. + It instead shares logic with `ASYNC910`_ and `ASYNC911`_ for parsing conditionals and branches. Removed rules ================ From 5d2b925b0a59df47cbcb772b65d47b0f9486b5b0 Mon Sep 17 00:00:00 2001 From: jakkdl Date: Mon, 20 May 2024 16:04:50 +0200 Subject: [PATCH 10/13] glossary improvements, rename some rules, add some intersphinx links --- docs/glossary.rst | 38 +++++++++++++++++++++++++++++++------- docs/rules.rst | 30 +++++++++++++++--------------- 2 files changed, 46 insertions(+), 22 deletions(-) diff --git a/docs/glossary.rst b/docs/glossary.rst index 258bfe87..20b62a79 100644 --- a/docs/glossary.rst +++ b/docs/glossary.rst @@ -74,6 +74,7 @@ A collection of child Tasks that can run concurrently. Internally contains a :re * :class:`asyncio.TaskGroup` (since python 3.11) +.. _cancellation: .. _cancelled: Cancelled / CancelledError @@ -81,20 +82,43 @@ Cancelled / CancelledError Handling cancellation is very sensitive, and you generally never want to catch a cancellation exception without letting it propagate to the library. - * Trio: :class:`trio.Cancelled`. `Documentation `__ - * AnyIO: :func:`anyio.get_cancelled_exc_class`. `Documentation `__ - * asyncio: :class:`asyncio.CancelledError`. `Documentation `__ +General documentation on cancellation in the different async libraries: + +* `Trio `__ +* `AnyIO `__ +* `asyncio `__ + +Exception classes: + +* :class:`trio.Cancelled` +* :func:`anyio.get_cancelled_exc_class` +* :class:`asyncio.CancelledError` .. _checkpoint: Checkpoint ---------- -Checkpoints are ``await``, ``async for``, and ``async with`` (on at least one of enter/exit). TODO write more and link stuff +Checkpoints are points where the async backend checks for cancellation and invokes scheduling checks. Possible checkpoints are ``await``, ``async for`` (before each iteration, and when exhausting the iterator), and ``async with`` (on at least one of enter/exit). + +Trio has extensive and detailed documentation on the concept of :external+trio:ref:`checkpoints `, and guarantees that all trio async functions will checkpoint (unless they raised an exception). + +anyio does not currently have any documentation on checkpoints. + +asyncio will checkpoint... ??? + +To make it easier to reason about checkpoints the :ref:`ASYNC91x ` rules enforces the same rules as trio for your own project - i.e. all async functions must guarantee a checkpoint (or exception). To make it possible to reason the rules will also assume that all other async functions also adhere to those rules. This means you must be careful if you're using 3rd-party async libraries. + .. _channel_stream_queue: Channel / Stream / Queue ------------------------ -* Trio: `channel `__ -* AnyIO: `stream `__ -* asyncio: `queue `__ +Interfaces used for communicating between tasks, processes, the network, etc. + +.. anyio streams is a :doc: and not a :label:, so we can't link with intersphinx :( + +.. _anyio_streams: https://anyio.readthedocs.io/en/stable/streams.html#streams + +* Trio has :ref:`channels ` for python objects and :ref:`streams ` for bytes. +* AnyIO has ``byte`` and ``object`` `streams `_ +* asyncio has :ref:`queues ` for python objects and :ref:`streams ` for bytes. diff --git a/docs/rules.rst b/docs/rules.rst index 45e35cec..0f2329ef 100644 --- a/docs/rules.rst +++ b/docs/rules.rst @@ -1,6 +1,6 @@ -**************** -List of rules -**************** +***** +Rules +***** General rules @@ -40,7 +40,7 @@ ASYNC109 : async-function-with-timeout Async function definition with a ``timeout`` parameter. In structured concurrency the caller should instead use :ref:`timeout context managers `. -ASYNC110 : busy-wait +ASYNC110 : async-busy-wait ``while ...: await [trio/anyio].sleep()`` should be replaced by a :class:`trio.Event`/:class:`anyio.Event`. ASYNC111 : variable-from-cm-in-start-soon @@ -57,7 +57,7 @@ ASYNC113 : start-soon-in-aenter ASYNC114 : startable-not-in-config Startable function (i.e. has a ``task_status`` keyword parameter) not in :ref:`--startable-in-context-manager <--startable-in-context-manager>` parameter list, please add it so ASYNC113 can catch errors when using it. -ASYNC115 : sleep-zero +ASYNC115 : async-zero-sleep Replace :func:`trio.sleep(0) `/:func:`anyio.sleep(0) ` with the more suggestive :func:`trio.lowlevel.checkpoint`/:func:`anyio.lowlevel.checkpoint`. ASYNC116 : long-sleep-not-forever @@ -78,8 +78,8 @@ Blocking sync calls in async functions .. _httpx.Client: https://www.python-httpx.org/api/#client .. _httpx.AsyncClient: https://www.python-httpx.org/api/#asyncclient .. _urllib3: https://github.com/urllib3/urllib3 - -Note: 22X, 23X and 24X has not had asyncio-specific suggestions written. +.. _aiofiles: https://pypi.org/project/aiofiles/ +.. _anyio: https://github.com/agronholm/anyio ASYNC200 : blocking-configured-call User-configured error for blocking sync calls in async functions. @@ -87,9 +87,9 @@ ASYNC200 : blocking-configured-call ASYNC210 : blocking-http-call Sync HTTP call in async function, use `httpx.AsyncClient`_. - This and the other ASYNC21x checks look for usage of `urllib3`_ and `httpx.Client`_, and recommend using `httpx.AsyncClient`_ as that's the largest http client supporting anyio/trio. + This and the other :ref:`ASYNC21x ` checks look for usage of `urllib3`_ and `httpx.Client`_, and recommend using `httpx.AsyncClient`_ as that's the largest http client supporting anyio/trio. -ASYNC211 : blocking-http-call-pool +_`ASYNC211` : blocking-http-call-pool Likely sync HTTP call in async function, use `httpx.AsyncClient`_. Looks for `urllib3`_ method calls on pool objects, but only matching on the method signature and not the object. @@ -97,25 +97,25 @@ ASYNC212 : blocking-http-call-httpx Blocking sync HTTP call on httpx object, use `httpx.AsyncClient`_. ASYNC220 : blocking-create-subprocess - Sync call to :class:`subprocess.Popen` (or equivalent) in async function, use :func:`trio.run_process`/:func:`anyio.run_process` in a :ref:`taskgroup_nursery`. ``asyncio`` users can use `asyncio.create_subprocess_[exec/shell] `_. + Sync call to :class:`subprocess.Popen` (or equivalent) in async function, use :func:`trio.run_process`/:func:`anyio.run_process`/:ref:`asyncio.create_subprocess_[exec/shell] ` in a :ref:`taskgroup_nursery`. ASYNC221 : blocking-run-process - Sync call to :func:`subprocess.run` (or equivalent) in async function, use :func:`trio.run_process`/:func:`anyio.run_process`. ``asyncio`` users can use `asyncio.create_subprocess_[exec/shell] `_. + Sync call to :func:`subprocess.run` (or equivalent) in async function, use :func:`trio.run_process`/:func:`anyio.run_process`/:ref:`asyncio.create_subprocess_[exec/shell] `. ASYNC222 : blocking-process-wait - Sync call to :func:`os.wait` (or equivalent) in async function, wrap in :func:`trio.to_thread.run_sync`/:func:`anyio.to_thread.run_sync`. ``asyncio`` users can use `asyncio.loop.run_in_executor `_. + Sync call to :func:`os.wait` (or equivalent) in async function, wrap in :func:`trio.to_thread.run_sync`/:func:`anyio.to_thread.run_sync`/:meth:`asyncio.loop.run_in_executor`. ASYNC230 : blocking-open-call - Sync call to :func:`open` in async function, use :func:`trio.open_file`/:func:`anyio.open_file`. ``asyncio`` users need to use a library such as `aiofiles `_, or switch to `anyio `_. + Sync call to :func:`open` in async function, use :func:`trio.open_file`/:func:`anyio.open_file`. ``asyncio`` users need to use a library such as `aiofiles`_, or switch to `anyio`_. ASYNC231 : blocking-fdopen-call - Sync call to :func:`os.fdopen` in async function, use :func:`trio.wrap_file`/:func:`anyio.wrap_file`. ``asyncio`` users need to use a library such as `aiofiles `_, or switch to `anyio `_. + Sync call to :func:`os.fdopen` in async function, use :func:`trio.wrap_file`/:func:`anyio.wrap_file`. ``asyncio`` users need to use a library such as `aiofiles`_, or switch to `anyio`_. ASYNC232 : blocking-file-call Blocking sync call on file object, wrap the file object in :func:`trio.wrap_file`/:func:`anyio.wrap_file` to get an async file object. ASYNC240 : blocking-path-usage - Avoid using :mod:`os.path` in async functions, prefer using :class:`trio.Path`/:class:`anyio.Path` objects. ``asyncio`` users should consider `aiopath `__ or `anyio `__. + Avoid using :mod:`os.path` in async functions, prefer using :class:`trio.Path`/:class:`anyio.Path` objects. ``asyncio`` users should consider `aiopath `__ or `anyio`_. ASYNC250 : blocking-input Builtin :func:`input` should not be called from async function. From fa126a5fbf5719a28500b4c84752135a4a9cb9a8 Mon Sep 17 00:00:00 2001 From: jakkdl Date: Wed, 22 May 2024 13:02:27 +0200 Subject: [PATCH 11/13] improve the checkpoint glossary entry --- docs/glossary.rst | 15 ++++++++++----- 1 file changed, 10 insertions(+), 5 deletions(-) diff --git a/docs/glossary.rst b/docs/glossary.rst index 20b62a79..86d8347d 100644 --- a/docs/glossary.rst +++ b/docs/glossary.rst @@ -98,16 +98,21 @@ Exception classes: Checkpoint ---------- -Checkpoints are points where the async backend checks for cancellation and invokes scheduling checks. Possible checkpoints are ``await``, ``async for`` (before each iteration, and when exhausting the iterator), and ``async with`` (on at least one of enter/exit). +Checkpoints are points where the async backend checks for cancellation and invokes scheduling checks. Regular checkpoints are important to ensure timely behaviour, and to avoid deadlocks. -Trio has extensive and detailed documentation on the concept of :external+trio:ref:`checkpoints `, and guarantees that all trio async functions will checkpoint (unless they raised an exception). +Trio has extensive and detailed documentation on the concept of :external+trio:ref:`checkpoints `, and guarantees that all trio async functions will checkpoint (unless they raised an exception) when ``await``-ed. +``async for`` on Trio generators will checkpoint before each iteration, and when exhausting the iterator, and ``async with`` will checkpoint on at least one of enter/exit. -anyio does not currently have any documentation on checkpoints. +asyncio does not place any guarantees on if or when asyncio functions will checkpoint. This means that enabling and adhering to :ref:`ASYNC91x ` will still not guarantee checkpoints. -asyncio will checkpoint... ??? +For anyio it will depend on the current backend. -To make it easier to reason about checkpoints the :ref:`ASYNC91x ` rules enforces the same rules as trio for your own project - i.e. all async functions must guarantee a checkpoint (or exception). To make it possible to reason the rules will also assume that all other async functions also adhere to those rules. This means you must be careful if you're using 3rd-party async libraries. +When using Trio (or an AnyIO library that people might use on Trio), it can be very helpful to ensure that your own code adheres to the same guarantees as Trio. +For this we supply the :ref:`ASYNC91x ` rules. +To make it possible to reason the rules will also assume that all other async functions also adhere to those rules. +This means you must be careful if you're using 3rd-party async libraries. +To insert a checkpoint with no other side effects, you can use :func:`trio.lowlevel.checkpoint`/:func:`anyio.lowlevel.checkpoint`/:func:`asyncio.sleep(0) ` .. _channel_stream_queue: From e65dfd5d623d7bf7219731f31731fd4b65ee6e97 Mon Sep 17 00:00:00 2001 From: jakkdl Date: Wed, 22 May 2024 13:27:56 +0200 Subject: [PATCH 12/13] fix RTD --- docs/rules.rst | 10 +++++----- 1 file changed, 5 insertions(+), 5 deletions(-) diff --git a/docs/rules.rst b/docs/rules.rst index 1b4db1e1..9e0e68b4 100644 --- a/docs/rules.rst +++ b/docs/rules.rst @@ -10,7 +10,7 @@ _`ASYNC100` : cancel-scope-no-checkpoint A :ref:`timeout_context` does not contain any :ref:`checkpoints `. This makes it pointless, as the timeout can only be triggered by a checkpoint. This check also treats ``yield`` as a checkpoint, since checkpoints can happen in the caller we yield to. - See :ref:`ASYNC912` which will in addition guarantee checkpoints on every code path. + See :ref:`ASYNC912 ` which will in addition guarantee checkpoints on every code path. ASYNC101 : yield-in-cancel-scope ``yield`` inside a :ref:`taskgroup_nursery` or :ref:`timeout_context` is only safe when implementing a context manager - otherwise, it breaks exception handling. @@ -141,7 +141,7 @@ _`ASYNC910` : async-function-no-checkpoint _`ASYNC911` : async-generator-no-checkpoint Exit, ``yield`` or ``return`` from async iterable with no guaranteed :ref:`checkpoint` since possible function entry (``yield`` or function definition). -ASYNC912 : cancel-scope-no-guaranteed-checkpoint +_`ASYNC912` : cancel-scope-no-guaranteed-checkpoint A timeout/cancelscope has :ref:`checkpoints `, but they're not guaranteed to run. Similar to `ASYNC100`_, but it does not warn on trivial cases where there is no checkpoint at all. It instead shares logic with `ASYNC910`_ and `ASYNC911`_ for parsing conditionals and branches. @@ -151,9 +151,9 @@ ASYNC912 : cancel-scope-no-guaranteed-checkpoint Autofix support =============== The following rules support :ref:`autofixing `. -- :ref:`ASYNC100` -- :ref:`ASYNC910` -- :ref:`ASYNC911` +- :ref:`ASYNC100 ` +- :ref:`ASYNC910 ` +- :ref:`ASYNC911 ` Removed rules ================ From 6b03ff2a5030bd4efb2f9d1549013a5df7fb76fc Mon Sep 17 00:00:00 2001 From: Zac Hatfield-Dodds Date: Wed, 22 May 2024 13:43:42 -0400 Subject: [PATCH 13/13] checkpoint glossary tweaks + denser dot-points --- docs/glossary.rst | 102 +++++++++++++++++++++++----------------------- 1 file changed, 52 insertions(+), 50 deletions(-) diff --git a/docs/glossary.rst b/docs/glossary.rst index 86d8347d..7dde9b7b 100644 --- a/docs/glossary.rst +++ b/docs/glossary.rst @@ -8,21 +8,21 @@ cancel scope ------------ A cancel scope is a context manager which can request the library cancels whatever task is executing in the body of the ``with`` (or ``async with``) -block. A cancel scope is the key component of a :ref:`timeout context `, and used in :ref:`TaskGroups / Nurseries ` to cancel any remaining child tasks if one raises an -exception. Trio and AnyIO have an explicit ``CancelScope`` type; in asyncio -they are implicit. +block. A cancel scope is the key component of a :ref:`timeout context `, +and used in :ref:`TaskGroups / Nurseries ` to cancel any remaining child tasks if one raises an +exception. -* Trio +* Trio has an explicit :class:`trio.CancelScope` type, and `general documentation + `__ + about cancellation and timeouts. - * `Documentation `__ +* AnyIO similarly has :class:`anyio.CancelScope` and `documentation + `__ of cancellation handling. - * :class:`trio.CancelScope` +* asyncio does not have an explicit cancel-scope type, but incorporates similar semantics + in :func:`asyncio.timeout` and :class:`asyncio.TaskGroup` and has `some documentation + `__. -* AnyIO - - * `Documentation `__ - - * :class:`anyio.CancelScope` .. _timeout_context: @@ -30,48 +30,37 @@ timeout context --------------- A context manager that enforces a timeout on a block of code, by cancelling it after a specified duration or at a preset time. The timeout can also be -rescheduled after creation. They are internally implemented with a :ref:`cancel scope `, which in anyio & trio can be directly initialized with a deadline. - -.. I find this to have excessive spacing before/after sublists. Probably requires CSS to fix? - -* Trio - - * `Documentation `__ +rescheduled after creation. They are internally implemented with a :ref:`cancel scope `, +which in anyio & trio can be directly initialized with a deadline. - * :func:`trio.move_on_after`, :func:`trio.move_on_at`, :func:`trio.fail_after`, :func:`trio.fail_at`, :class:`trio.CancelScope` +* Trio has :func:`trio.move_on_after`, :func:`trio.move_on_at`, + :func:`trio.fail_after`, :func:`trio.fail_at`, and :class:`trio.CancelScope` + (`docs `__) -* AnyIO +* AnyIO has :func:`anyio.move_on_after`, :func:`anyio.fail_after`, and :class:`anyio.CancelScope` + (`docs `__) - * `Documentation `__ +* asyncio has :func:`asyncio.timeout` and :func:`asyncio.timeout_at` + (`docs `__) - * :func:`anyio.move_on_after`, :func:`anyio.fail_after`, :class:`anyio.CancelScope` - -* asyncio - - * `Documentation `__ - - * :func:`asyncio.timeout`, :func:`asyncio.timeout_at` .. _taskgroup_nursery: TaskGroup / Nursery ------------------- -A collection of child Tasks that can run concurrently. Internally contains a :ref:`cancel scope ` for canceling any remaining child tasks if one raises an exception. - -* Trio - - * `Documentation `__ +A collection of child Tasks that can run concurrently. Internally contains a +:ref:`cancel scope ` for canceling any remaining child tasks if +one raises an exception. - * :class:`trio.Nursery`, created with :func:`trio.open_nursery` -* AnyIO +* Trio has :class:`trio.Nursery`, created with :func:`trio.open_nursery` + (`docs `__) - * `Documentation `__ - * :class:`anyio.abc.TaskGroup`, created with :func:`anyio.create_task_group`. -* asyncio +* AnyIO has :class:`anyio.abc.TaskGroup`, created with :func:`anyio.create_task_group` + (`docs `__) - * `Documentation `__ - * :class:`asyncio.TaskGroup` (since python 3.11) +* asyncio has :class:`asyncio.TaskGroup` since python 3.11 + (`docs `__) .. _cancellation: @@ -80,7 +69,8 @@ A collection of child Tasks that can run concurrently. Internally contains a :re Cancelled / CancelledError -------------------------- -Handling cancellation is very sensitive, and you generally never want to catch a cancellation exception without letting it propagate to the library. +Handling cancellation is very sensitive, and you generally never want to catch a +cancellation exception without letting it propagate to the library. General documentation on cancellation in the different async libraries: @@ -98,21 +88,33 @@ Exception classes: Checkpoint ---------- -Checkpoints are points where the async backend checks for cancellation and invokes scheduling checks. Regular checkpoints are important to ensure timely behaviour, and to avoid deadlocks. +Checkpoints are points where the async backend checks for cancellation and +can switch which task is running, in an ``await``, ``async for``, or ``async with`` +expression. Regular checkpoints can be important for both performance and correctness. -Trio has extensive and detailed documentation on the concept of :external+trio:ref:`checkpoints `, and guarantees that all trio async functions will checkpoint (unless they raised an exception) when ``await``-ed. -``async for`` on Trio generators will checkpoint before each iteration, and when exhausting the iterator, and ``async with`` will checkpoint on at least one of enter/exit. +Trio has extensive and detailed documentation on the concept of +:external+trio:ref:`checkpoints `, and guarantees that all async +functions defined by Trio will either checkpoint or raise an exception when +``await``-ed. ``async for`` on Trio iterables will checkpoint before each +iteration, and when exhausting the iterator, and ``async with`` will checkpoint +on at least one of enter/exit. -asyncio does not place any guarantees on if or when asyncio functions will checkpoint. This means that enabling and adhering to :ref:`ASYNC91x ` will still not guarantee checkpoints. +asyncio does not place any guarantees on if or when asyncio functions will +checkpoint. This means that enabling and adhering to :ref:`ASYNC91x ` +will still not guarantee checkpoints. For anyio it will depend on the current backend. -When using Trio (or an AnyIO library that people might use on Trio), it can be very helpful to ensure that your own code adheres to the same guarantees as Trio. -For this we supply the :ref:`ASYNC91x ` rules. -To make it possible to reason the rules will also assume that all other async functions also adhere to those rules. -This means you must be careful if you're using 3rd-party async libraries. +When using Trio (or an AnyIO library that people might use on Trio), it can be +very helpful to ensure that your own code adheres to the same guarantees as +Trio. For this we supply the :ref:`ASYNC91x ` rules. To make it +possible to reason the rules will also assume that all other async functions +also adhere to those rules. This means you must be careful if you're using +3rd-party async libraries. -To insert a checkpoint with no other side effects, you can use :func:`trio.lowlevel.checkpoint`/:func:`anyio.lowlevel.checkpoint`/:func:`asyncio.sleep(0) ` +To insert a checkpoint with no other side effects, you can use +:func:`trio.lowlevel.checkpoint`/:func:`anyio.lowlevel.checkpoint`/:func:`asyncio.sleep(0) +` .. _channel_stream_queue: