pythongh-133465: Efficient signal checks with detached thread state.

Add new C-API functions `PyErr_CheckSignalsDetached` and
`PyErr_AreSignalsPending`.

`PyErr_CheckSignalsDetached` can *only* be called by threads
that *don’t* have an attached thread state.  It does the same thing as
`PyErr_CheckSignals`, except that it guarantees it will only reattach
the supplied thread state if necessary in order to run signal
handlers.  (Also, it never runs the cycle collector.)

`PyErr_AreSignalsPending` can be called with or without an attached
thread state.  It reports to its caller whether signals are pending,
but never runs any handlers itself.

Rationale: Compiled-code modules that implement time-consuming
operations that don’t require manipulating Python objects, are
supposed to call PyErr_CheckSignals frequently throughout each such
operation, so that if the user interrupts the operation with
control-C, it is canceled promptly.

In the normal case where no signals are pending, PyErr_CheckSignals
is cheap (two atomic memory operations); however, callers must have
an attached thread state, and compiled-code modules that implement
time-consuming operations are also supposed to detach their thread
state during each such operation.  The overhead of re-attaching a
thread state in order to call PyErr_CheckSignals, and then releasing
it again, sufficiently often for reasonable user responsiveness, can
be substantial, particularly in traditional (non-free-threaded) builds.
These new functions permit compiled-code modules to avoid that extra
overhead.

Author: zackw

Doc/c-api/exceptions.rst

@@ -642,7 +642,7 @@ Signal Handling
    This function interacts with Python's signal handling.
 
    If the function is called from the main thread and under the main Python
-   interpreter, it checks whether a signal has been sent to the processes
+   interpreter, it checks whether a signal has been sent to the process
    and if so, invokes the corresponding signal handler.  If the :mod:`signal`
    module is supported, this can invoke a signal handler written in Python.
 
@@ -653,7 +653,11 @@ Signal Handling
    next :c:func:`PyErr_CheckSignals()` invocation).
 
    If the function is called from a non-main thread, or under a non-main
-   Python interpreter, it does nothing and returns ``0``.
+   Python interpreter, it does not check for pending signals, and always
+   returns ``0``.
+
+   Regardless of calling context, this function may, as a side effect,
+   run the cyclic garbage collector (see :ref:`supporting-cycle-detection`).
 
    This function can be called by long-running C code that wants to
    be interruptible by user requests (such as by pressing Ctrl-C).
@@ -662,6 +666,47 @@ Signal Handling
       The default Python signal handler for :c:macro:`!SIGINT` raises the
       :exc:`KeyboardInterrupt` exception.
 
+.. c:function:: int PyErr_CheckSignalsDetached(PyThreadState *tstate)
+
+   .. index::
+      pair: module; signal
+      single: SIGINT (C macro)
+      single: KeyboardInterrupt (built-in exception)
+
+   This function is similar to :c:func:`PyErr_CheckSignals`.  However, unlike
+   that function, it must be called **without** an :term:`attached thread state`.
+   The ``tstate`` argument must be the thread state that was formerly attached to
+   the calling context (this is the value returned by :c:func:`PyEval_SaveThread`)
+   and it must be safe to re-attach the thread state briefly.
+
+   If the ``tstate`` argument refers to the main thread and the main Python
+   interpreter, this function checks whether any signals have been sent to the
+   process, and if so, invokes the corresponding signal handlers.  Otherwise it
+   does nothing.  If signal handlers do need to be run, the supplied thread state
+   will be attached while they are run, then detached again afterward.
+
+   The return value is the same as for :c:func:`PyErr_CheckSignals`,
+   i.e. ``-1`` if a signal handler raised an exception, ``0`` otherwise.
+
+   Unlike :c:func:`PyErr_CheckSignals`, this function never runs the cyclic
+   garbage collector.
+
+   This function can be called by long-running C code that wants to
+   be interruptible by user requests from within regions where it has
+   detached the thread state, while minimizing the overhead of the check
+   in the normal case of no pending signals.
+
+.. c:function:: int PyErr_AreSignalsPending(PyThreadState *tstate)
+
+   This function returns a nonzero value if the execution context ``tstate``
+   needs to process signals soon: that is, ``tstate`` refers to the main thread
+   and the main Python interpreter, and signals have been sent to the process,
+   and their handlers have not yet been run.  Otherwise, it returns zero.  It has
+   no side effects.
+
+   .. note::
+      This function may be called either with or without
+      an :term:`attached thread state`.
 
 .. c:function:: void PyErr_SetInterrupt()
 

Include/pyerrors.h

@@ -235,6 +235,8 @@ PyAPI_FUNC(void) PyErr_WriteUnraisable(PyObject *);
 
 /* In signalmodule.c */
 PyAPI_FUNC(int) PyErr_CheckSignals(void);
+PyAPI_FUNC(int) PyErr_CheckSignalsDetached(PyThreadState *state);
+PyAPI_FUNC(int) PyErr_AreSignalsPending(PyThreadState *state);
 PyAPI_FUNC(void) PyErr_SetInterrupt(void);
 #if !defined(Py_LIMITED_API) || Py_LIMITED_API+0 >= 0x030A0000
 PyAPI_FUNC(int) PyErr_SetInterruptEx(int signum);

Misc/NEWS.d/next/C_API/2025-06-10-13-10-23.gh-issue-133465.PzxlaV.rst

@@ -0,0 +1,4 @@
+New functions :c:func:`PyErr_CheckSignalsDetached` and
+:c:func:`PyErr_AreSignalsPending` for responding to signals
+from within C extension modules that detach the thread state.
+Patch by Zack Weinberg.

Modules/signalmodule.c

@@ -1759,6 +1759,19 @@ _PySignal_Fini(void)
     Py_CLEAR(state->ignore_handler);
 }
 
+/* used by PyErr_CheckSignalsDetached and _PyErr_CheckSignalsTstate */
+static int process_signals(PyThreadState *tstate);
+
+/* Declared in pyerrors.h */
+int
+PyErr_AreSignalsPending(PyThreadState *tstate)
+{
+    if (!_Py_ThreadCanHandleSignals(tstate->interp)) {
+        return 0;
+    }
+    _Py_CHECK_EMSCRIPTEN_SIGNALS();
+    return _Py_atomic_load_int(&is_tripped);
+}
 
 /* Declared in pyerrors.h */
 int
@@ -1781,23 +1794,60 @@ PyErr_CheckSignals(void)
     _PyRunRemoteDebugger(tstate);
 #endif
 
-    if (!_Py_ThreadCanHandleSignals(tstate->interp)) {
-        return 0;
+    return _PyErr_CheckSignalsTstate(tstate);
+}
+
+/* Declared in pyerrors.h */
+int
+PyErr_CheckSignalsDetached(PyThreadState *tstate)
+{
+    int status = 0;
+
+    /* Unlike PyErr_CheckSignals, we do not check whether the GC is
+       scheduled to run.  This function can only be called from
+       contexts without an attached thread state, and contexts that
+       don't have an attached thread state cannot generate garbage.  */
+
+#if defined(Py_REMOTE_DEBUG) && defined(Py_SUPPORTS_REMOTE_DEBUG)
+    _PyRunRemoteDebugger(tstate);
+#endif
+
+    if (PyErr_AreSignalsPending(tstate)) {
+        PyEval_AcquireThread(tstate);
+        /* It is necessary to re-check whether any signals are pending
+           after re-attaching the thread state, because, while we were
+           waiting to acquire an attached thread state, the situation
+           might have changed.  */
+        if (PyErr_AreSignalsPending(tstate)) {
+            status = process_signals(tstate);
+        }
+        PyEval_ReleaseThread(tstate);
     }
+    return status;
+}
 
+/* Declared in cpython/pyerrors.h */
+int
+_PyErr_CheckSignals(void)
+{
+    PyThreadState *tstate = _PyThreadState_GET();
     return _PyErr_CheckSignalsTstate(tstate);
 }
 
-
 /* Declared in cpython/pyerrors.h */
 int
 _PyErr_CheckSignalsTstate(PyThreadState *tstate)
 {
-    _Py_CHECK_EMSCRIPTEN_SIGNALS();
-    if (!_Py_atomic_load_int(&is_tripped)) {
+    if (!PyErr_AreSignalsPending(tstate)) {
         return 0;
     }
 
+    return process_signals(tstate);
+}
+
+static int
+process_signals(PyThreadState *tstate)
+{
     /*
      * The is_tripped variable is meant to speed up the calls to
      * PyErr_CheckSignals (both directly or via pending calls) when no
@@ -1878,15 +1928,6 @@ _PyErr_CheckSignalsTstate(PyThreadState *tstate)
 }
 
 
-
-int
-_PyErr_CheckSignals(void)
-{
-    PyThreadState *tstate = _PyThreadState_GET();
-    return _PyErr_CheckSignalsTstate(tstate);
-}
-
-
 /* Simulate the effect of a signal arriving. The next time PyErr_CheckSignals
    is called,  the corresponding Python signal handler will be raised.