From 6c82e34ca0e98e47c8d4aaf27f728d4316205b54 Mon Sep 17 00:00:00 2001 From: Zack Weinberg Date: Tue, 10 Jun 2025 13:34:47 -0400 Subject: [PATCH] gh-133465: Efficient signal checks with detached thread state. MIME-Version: 1.0 Content-Type: text/plain; charset=UTF-8 Content-Transfer-Encoding: 8bit Add new C-API functions `PyErr_CheckSignalsDetached` and `PyErr_AreSignalsPending`. `PyErr_CheckSignalsDetached` can *only* be called by threads that *don’t* have an attached thread state. It does the same thing as `PyErr_CheckSignals`, except that it guarantees it will only reattach the supplied thread state if necessary in order to run signal handlers. (Also, it never runs the cycle collector.) `PyErr_AreSignalsPending` can be called with or without an attached thread state. It reports to its caller whether signals are pending, but never runs any handlers itself. Rationale: Compiled-code modules that implement time-consuming operations that don’t require manipulating Python objects, are supposed to call PyErr_CheckSignals frequently throughout each such operation, so that if the user interrupts the operation with control-C, it is canceled promptly. In the normal case where no signals are pending, PyErr_CheckSignals is cheap (two atomic memory operations); however, callers must have an attached thread state, and compiled-code modules that implement time-consuming operations are also supposed to detach their thread state during each such operation. The overhead of re-attaching a thread state in order to call PyErr_CheckSignals, and then releasing it again, sufficiently often for reasonable user responsiveness, can be substantial, particularly in traditional (non-free-threaded) builds. These new functions permit compiled-code modules to avoid that extra overhead. --- Doc/c-api/exceptions.rst | 49 ++++++++++++- Include/pyerrors.h | 2 + ...-06-10-13-10-23.gh-issue-133465.PzxlaV.rst | 4 ++ Modules/signalmodule.c | 69 +++++++++++++++---- 4 files changed, 108 insertions(+), 16 deletions(-) create mode 100644 Misc/NEWS.d/next/C_API/2025-06-10-13-10-23.gh-issue-133465.PzxlaV.rst diff --git a/Doc/c-api/exceptions.rst b/Doc/c-api/exceptions.rst index 885dbeb75303d1..e3bf371f0aa87e 100644 --- a/Doc/c-api/exceptions.rst +++ b/Doc/c-api/exceptions.rst @@ -642,7 +642,7 @@ Signal Handling This function interacts with Python's signal handling. If the function is called from the main thread and under the main Python - interpreter, it checks whether a signal has been sent to the processes + interpreter, it checks whether a signal has been sent to the process and if so, invokes the corresponding signal handler. If the :mod:`signal` module is supported, this can invoke a signal handler written in Python. @@ -653,7 +653,11 @@ Signal Handling next :c:func:`PyErr_CheckSignals()` invocation). If the function is called from a non-main thread, or under a non-main - Python interpreter, it does nothing and returns ``0``. + Python interpreter, it does not check for pending signals, and always + returns ``0``. + + Regardless of calling context, this function may, as a side effect, + run the cyclic garbage collector (see :ref:`supporting-cycle-detection`). This function can be called by long-running C code that wants to be interruptible by user requests (such as by pressing Ctrl-C). @@ -662,6 +666,47 @@ Signal Handling The default Python signal handler for :c:macro:`!SIGINT` raises the :exc:`KeyboardInterrupt` exception. +.. c:function:: int PyErr_CheckSignalsDetached(PyThreadState *tstate) + + .. index:: + pair: module; signal + single: SIGINT (C macro) + single: KeyboardInterrupt (built-in exception) + + This function is similar to :c:func:`PyErr_CheckSignals`. However, unlike + that function, it must be called **without** an :term:`attached thread state`. + The ``tstate`` argument must be the thread state that was formerly attached to + the calling context (this is the value returned by :c:func:`PyEval_SaveThread`) + and it must be safe to re-attach the thread state briefly. + + If the ``tstate`` argument refers to the main thread and the main Python + interpreter, this function checks whether any signals have been sent to the + process, and if so, invokes the corresponding signal handlers. Otherwise it + does nothing. If signal handlers do need to be run, the supplied thread state + will be attached while they are run, then detached again afterward. + + The return value is the same as for :c:func:`PyErr_CheckSignals`, + i.e. ``-1`` if a signal handler raised an exception, ``0`` otherwise. + + Unlike :c:func:`PyErr_CheckSignals`, this function never runs the cyclic + garbage collector. + + This function can be called by long-running C code that wants to + be interruptible by user requests from within regions where it has + detached the thread state, while minimizing the overhead of the check + in the normal case of no pending signals. + +.. c:function:: int PyErr_AreSignalsPending(PyThreadState *tstate) + + This function returns a nonzero value if the execution context ``tstate`` + needs to process signals soon: that is, ``tstate`` refers to the main thread + and the main Python interpreter, and signals have been sent to the process, + and their handlers have not yet been run. Otherwise, it returns zero. It has + no side effects. + + .. note:: + This function may be called either with or without + an :term:`attached thread state`. .. c:function:: void PyErr_SetInterrupt() diff --git a/Include/pyerrors.h b/Include/pyerrors.h index 5d0028c116e2d8..1e7dc3ed255923 100644 --- a/Include/pyerrors.h +++ b/Include/pyerrors.h @@ -235,6 +235,8 @@ PyAPI_FUNC(void) PyErr_WriteUnraisable(PyObject *); /* In signalmodule.c */ PyAPI_FUNC(int) PyErr_CheckSignals(void); +PyAPI_FUNC(int) PyErr_CheckSignalsDetached(PyThreadState *state); +PyAPI_FUNC(int) PyErr_AreSignalsPending(PyThreadState *state); PyAPI_FUNC(void) PyErr_SetInterrupt(void); #if !defined(Py_LIMITED_API) || Py_LIMITED_API+0 >= 0x030A0000 PyAPI_FUNC(int) PyErr_SetInterruptEx(int signum); diff --git a/Misc/NEWS.d/next/C_API/2025-06-10-13-10-23.gh-issue-133465.PzxlaV.rst b/Misc/NEWS.d/next/C_API/2025-06-10-13-10-23.gh-issue-133465.PzxlaV.rst new file mode 100644 index 00000000000000..268f9719276ac9 --- /dev/null +++ b/Misc/NEWS.d/next/C_API/2025-06-10-13-10-23.gh-issue-133465.PzxlaV.rst @@ -0,0 +1,4 @@ +New functions :c:func:`PyErr_CheckSignalsDetached` and +:c:func:`PyErr_AreSignalsPending` for responding to signals +from within C extension modules that detach the thread state. +Patch by Zack Weinberg. diff --git a/Modules/signalmodule.c b/Modules/signalmodule.c index 54bcd3270ef31a..c295152eef9a89 100644 --- a/Modules/signalmodule.c +++ b/Modules/signalmodule.c @@ -1759,6 +1759,19 @@ _PySignal_Fini(void) Py_CLEAR(state->ignore_handler); } +/* used by PyErr_CheckSignalsDetached and _PyErr_CheckSignalsTstate */ +static int process_signals(PyThreadState *tstate); + +/* Declared in pyerrors.h */ +int +PyErr_AreSignalsPending(PyThreadState *tstate) +{ + if (!_Py_ThreadCanHandleSignals(tstate->interp)) { + return 0; + } + _Py_CHECK_EMSCRIPTEN_SIGNALS(); + return _Py_atomic_load_int(&is_tripped); +} /* Declared in pyerrors.h */ int @@ -1781,23 +1794,60 @@ PyErr_CheckSignals(void) _PyRunRemoteDebugger(tstate); #endif - if (!_Py_ThreadCanHandleSignals(tstate->interp)) { - return 0; + return _PyErr_CheckSignalsTstate(tstate); +} + +/* Declared in pyerrors.h */ +int +PyErr_CheckSignalsDetached(PyThreadState *tstate) +{ + int status = 0; + + /* Unlike PyErr_CheckSignals, we do not check whether the GC is + scheduled to run. This function can only be called from + contexts without an attached thread state, and contexts that + don't have an attached thread state cannot generate garbage. */ + +#if defined(Py_REMOTE_DEBUG) && defined(Py_SUPPORTS_REMOTE_DEBUG) + _PyRunRemoteDebugger(tstate); +#endif + + if (PyErr_AreSignalsPending(tstate)) { + PyEval_AcquireThread(tstate); + /* It is necessary to re-check whether any signals are pending + after re-attaching the thread state, because, while we were + waiting to acquire an attached thread state, the situation + might have changed. */ + if (PyErr_AreSignalsPending(tstate)) { + status = process_signals(tstate); + } + PyEval_ReleaseThread(tstate); } + return status; +} +/* Declared in cpython/pyerrors.h */ +int +_PyErr_CheckSignals(void) +{ + PyThreadState *tstate = _PyThreadState_GET(); return _PyErr_CheckSignalsTstate(tstate); } - /* Declared in cpython/pyerrors.h */ int _PyErr_CheckSignalsTstate(PyThreadState *tstate) { - _Py_CHECK_EMSCRIPTEN_SIGNALS(); - if (!_Py_atomic_load_int(&is_tripped)) { + if (!PyErr_AreSignalsPending(tstate)) { return 0; } + return process_signals(tstate); +} + +static int +process_signals(PyThreadState *tstate) +{ /* * The is_tripped variable is meant to speed up the calls to * PyErr_CheckSignals (both directly or via pending calls) when no @@ -1878,15 +1928,6 @@ _PyErr_CheckSignalsTstate(PyThreadState *tstate) } - -int -_PyErr_CheckSignals(void) -{ - PyThreadState *tstate = _PyThreadState_GET(); - return _PyErr_CheckSignalsTstate(tstate); -} - - /* Simulate the effect of a signal arriving. The next time PyErr_CheckSignals is called, the corresponding Python signal handler will be raised.