release: 0.23.4

.github/workflows/ci.yml

@@ -115,7 +115,7 @@ jobs:
             {
               os: "ubuntu-latest",
               python-architecture: "x64",
-              rust-target: "wasm32-wasi",
+              rust-target: "wasm32-wasip1",
             },
             {
               os: "windows-latest",
@@ -489,7 +489,7 @@ jobs:
           components: rust-src
       - uses: actions/setup-node@v4
         with:
-          node-version: 14
+          node-version: 18
       - run: python -m pip install --upgrade pip && pip install nox
       - uses: actions/cache@v4
         id: cache
@@ -656,14 +656,17 @@ jobs:
           # ubuntu x86_64 -> windows x86_64
           - os: "ubuntu-latest"
             target: "x86_64-pc-windows-gnu"
-            flags: "-i python3.12 --features abi3 --features generate-import-lib"
-            manylinux: off
+            flags: "-i python3.12 --features generate-import-lib"
           # macos x86_64 -> aarch64
           - os: "macos-13" # last x86_64 macos runners
             target: "aarch64-apple-darwin"
           # macos aarch64 -> x86_64
           - os: "macos-latest"
             target: "x86_64-apple-darwin"
+          # windows x86_64 -> aarch64
+          - os: "windows-latest"
+            target: "aarch64-pc-windows-msvc"
+            flags: "-i python3.12 --features generate-import-lib"
     steps:
       - uses: actions/checkout@v4
       - uses: actions/setup-python@v5
@@ -677,11 +680,18 @@ jobs:
       - name: Setup cross-compiler
         if: ${{ matrix.target == 'x86_64-pc-windows-gnu' }}
         run: sudo apt-get install -y mingw-w64 llvm
-      - uses: PyO3/maturin-action@v1
+      - name: Compile version-specific library
+        uses: PyO3/maturin-action@v1
         with:
           target: ${{ matrix.target }}
           manylinux: ${{ matrix.manylinux }}
           args: --release -m examples/maturin-starter/Cargo.toml ${{ matrix.flags }}
+      - name: Compile abi3 library
+        uses: PyO3/maturin-action@v1
+        with:
+          target: ${{ matrix.target }}
+          manylinux: ${{ matrix.manylinux }}
+          args: --release -m examples/maturin-starter/Cargo.toml --features abi3 ${{ matrix.flags }}
 
   test-cross-compilation-windows:
     needs: [fmt]

.github/workflows/coverage-pr-base.yml

@@ -19,9 +19,6 @@ jobs:
       - name: Set PR base on codecov
         run: |
           # fetch the merge commit between the PR base and head
-          BASE_REF=refs/heads/${{ github.event.pull_request.base.ref }}
-          MERGE_REF=refs/pull/${{ github.event.pull_request.number }}/merge
-
           git fetch -u --progress --depth=1 origin "+$BASE_REF:$BASE_REF" "+$MERGE_REF:$MERGE_REF"
           while [ -z "$(git merge-base "$BASE_REF" "$MERGE_REF")" ]; do
             git fetch -u -q --deepen="10" origin "$BASE_REF" "$MERGE_REF";
@@ -38,3 +35,8 @@ jobs:
             --slug PyO3/pyo3 \
             --token ${{ secrets.CODECOV_TOKEN }} \
             --service github
+        env:
+          # Don't put these in bash, because we don't want the expansion to
+          # risk code execution
+          BASE_REF: "refs/heads/${{ github.event.pull_request.base.ref }}"
+          MERGE_REF: "refs/pull/${{ github.event.pull_request.number }}/merge"

CHANGELOG.md

@@ -10,6 +10,34 @@ To see unreleased changes, please see the [CHANGELOG on the main branch guide](h
 
 <!-- towncrier release notes start -->
 
+## [0.23.4] - 2025-01-10
+
+### Added
+
+- Add `PyList::locked_for_each`, which uses a critical section to lock the list on the free-threaded build. [#4789](https://github.com/PyO3/pyo3/pull/4789)
+- Add `pyo3_build_config::add_python_framework_link_args` build script API to set rpath when using macOS system Python. [#4833](https://github.com/PyO3/pyo3/pull/4833)
+
+### Changed
+
+- Use `datetime.fold` to distinguish ambiguous datetimes when converting to and from `chrono::DateTime<Tz>` (rather than erroring). [#4791](https://github.com/PyO3/pyo3/pull/4791)
+- Optimize PyList iteration on the free-threaded build. [#4789](https://github.com/PyO3/pyo3/pull/4789)
+
+### Fixed
+
+- Fix unnecessary internal `py.allow_threads` GIL-switch when attempting to access contents of a `PyErr` which originated from Python (could lead to unintended deadlocks). [#4766](https://github.com/PyO3/pyo3/pull/4766)
+- Fix thread-unsafe access of dict internals in `BoundDictIterator` on the free-threaded build. [#4788](https://github.com/PyO3/pyo3/pull/4788)
+* Fix unnecessary critical sections in `BoundDictIterator` on the free-threaded build. [#4788](https://github.com/PyO3/pyo3/pull/4788)
+- Fix time-of-check to time-of-use issues with list iteration on the free-threaded build. [#4789](https://github.com/PyO3/pyo3/pull/4789)
+- Fix `chrono::DateTime<Tz>` to-Python conversion when `Tz` is `chrono_tz::Tz`. [#4790](https://github.com/PyO3/pyo3/pull/4790)
+- Fix `#[pyclass]` not being able to be named `Probe`. [#4794](https://github.com/PyO3/pyo3/pull/4794)
+- Fix not treating cross-compilation from x64 to aarch64 on Windows as a cross-compile. [#4800](https://github.com/PyO3/pyo3/pull/4800)
+- Fix missing struct fields on GraalPy when subclassing builtin classes. [#4802](https://github.com/PyO3/pyo3/pull/4802)
+- Fix generating import lib for PyPy when `abi3` feature is enabled. [#4806](https://github.com/PyO3/pyo3/pull/4806)
+- Fix generating import lib for python3.13t when `abi3` feature is enabled. [#4808](https://github.com/PyO3/pyo3/pull/4808)
+- Fix compile failure for raw identifiers like `r#box` in `derive(FromPyObject)`. [#4814](https://github.com/PyO3/pyo3/pull/4814)
+- Fix compile failure for `#[pyclass]` enum variants with more than 12 fields. [#4832](https://github.com/PyO3/pyo3/pull/4832)
+
+
 ## [0.23.3] - 2024-12-03
 
 ### Packaging
@@ -2026,7 +2054,8 @@ Yanked
 
 - Initial release
 
-[Unreleased]: https://github.com/pyo3/pyo3/compare/v0.23.3...HEAD
+[Unreleased]: https://github.com/pyo3/pyo3/compare/v0.23.4...HEAD
+[0.23.4]: https://github.com/pyo3/pyo3/compare/v0.23.3...v0.23.4
 [0.23.3]: https://github.com/pyo3/pyo3/compare/v0.23.2...v0.23.3
 [0.23.2]: https://github.com/pyo3/pyo3/compare/v0.23.1...v0.23.2
 [0.23.1]: https://github.com/pyo3/pyo3/compare/v0.23.0...v0.23.1

Cargo.toml

@@ -1,6 +1,6 @@
 [package]
 name = "pyo3"
-version = "0.23.3"
+version = "0.23.4"
 description = "Bindings to Python interpreter"
 authors = ["PyO3 Project and Contributors <https://github.com/PyO3>"]
 readme = "README.md"
@@ -21,10 +21,10 @@ memoffset = "0.9"
 once_cell = "1.13"
 
 # ffi bindings to the python interpreter, split into a separate crate so they can be used independently
-pyo3-ffi = { path = "pyo3-ffi", version = "=0.23.3" }
+pyo3-ffi = { path = "pyo3-ffi", version = "=0.23.4" }
 
 # support crates for macros feature
-pyo3-macros = { path = "pyo3-macros", version = "=0.23.3", optional = true }
+pyo3-macros = { path = "pyo3-macros", version = "=0.23.4", optional = true }
 indoc = { version = "2.0.1", optional = true }
 unindent = { version = "0.2.1", optional = true }
 
@@ -66,7 +66,7 @@ static_assertions = "1.1.0"
 uuid = {version = "1.10.0", features = ["v4"] }
 
 [build-dependencies]
-pyo3-build-config = { path = "pyo3-build-config", version = "=0.23.3", features = ["resolve-config"] }
+pyo3-build-config = { path = "pyo3-build-config", version = "=0.23.4", features = ["resolve-config"] }
 
 [features]
 default = ["macros"]

README.md

@@ -71,7 +71,7 @@ name = "string_sum"
 crate-type = ["cdylib"]
 
 [dependencies]
-pyo3 = { version = "0.23.3", features = ["extension-module"] }
+pyo3 = { version = "0.23.4", features = ["extension-module"] }
 ```
 
 **`src/lib.rs`**
@@ -140,7 +140,7 @@ Start a new project with `cargo new` and add  `pyo3` to the `Cargo.toml` like th
 
 ```toml
 [dependencies.pyo3]
-version = "0.23.3"
+version = "0.23.4"
 features = ["auto-initialize"]
 ```
 

build.rs

@@ -1,7 +1,9 @@
 use std::env;
 
 use pyo3_build_config::pyo3_build_script_impl::{cargo_env_var, errors::Result};
-use pyo3_build_config::{bail, print_feature_cfgs, InterpreterConfig};
+use pyo3_build_config::{
+    add_python_framework_link_args, bail, print_feature_cfgs, InterpreterConfig,
+};
 
 fn ensure_auto_initialize_ok(interpreter_config: &InterpreterConfig) -> Result<()> {
     if cargo_env_var("CARGO_FEATURE_AUTO_INITIALIZE").is_some() && !interpreter_config.shared {
@@ -42,6 +44,9 @@ fn configure_pyo3() -> Result<()> {
     // Emit cfgs like `invalid_from_utf8_lint`
     print_feature_cfgs();
 
+    // Make `cargo test` etc work on macOS with Xcode bundled Python
+    add_python_framework_link_args();
+
     Ok(())
 }
 

emscripten/Makefile

@@ -4,7 +4,7 @@ CURDIR=$(abspath .)
 BUILDROOT ?= $(CURDIR)/builddir
 PYMAJORMINORMICRO ?= 3.11.0
 
-EMSCRIPTEN_VERSION=3.1.13
+EMSCRIPTEN_VERSION=3.1.68
 
 export EMSDKDIR = $(BUILDROOT)/emsdk
 

examples/decorator/.template/pre-script.rhai

@@ -1,4 +1,4 @@
-variable::set("PYO3_VERSION", "0.23.3");
+variable::set("PYO3_VERSION", "0.23.4");
 file::rename(".template/Cargo.toml", "Cargo.toml");
 file::rename(".template/pyproject.toml", "pyproject.toml");
 file::delete(".template");

examples/maturin-starter/.template/pre-script.rhai

@@ -1,4 +1,4 @@
-variable::set("PYO3_VERSION", "0.23.3");
+variable::set("PYO3_VERSION", "0.23.4");
 file::rename(".template/Cargo.toml", "Cargo.toml");
 file::rename(".template/pyproject.toml", "pyproject.toml");
 file::delete(".template");

examples/plugin/.template/pre-script.rhai

@@ -1,4 +1,4 @@
-variable::set("PYO3_VERSION", "0.23.3");
+variable::set("PYO3_VERSION", "0.23.4");
 file::rename(".template/Cargo.toml", "Cargo.toml");
 file::rename(".template/plugin_api/Cargo.toml", "plugin_api/Cargo.toml");
 file::delete(".template");

examples/setuptools-rust-starter/.template/pre-script.rhai

@@ -1,4 +1,4 @@
-variable::set("PYO3_VERSION", "0.23.3");
+variable::set("PYO3_VERSION", "0.23.4");
 file::rename(".template/Cargo.toml", "Cargo.toml");
 file::rename(".template/setup.cfg", "setup.cfg");
 file::delete(".template");

examples/word-count/.template/pre-script.rhai

@@ -1,4 +1,4 @@
-variable::set("PYO3_VERSION", "0.23.3");
+variable::set("PYO3_VERSION", "0.23.4");
 file::rename(".template/Cargo.toml", "Cargo.toml");
 file::rename(".template/pyproject.toml", "pyproject.toml");
 file::delete(".template");

guide/src/building-and-distribution.md

@@ -144,7 +144,17 @@ rustflags = [
 ]
 ```
 
-Using the MacOS system python3 (`/usr/bin/python3`, as opposed to python installed via homebrew, pyenv, nix, etc.) may result in runtime errors such as `Library not loaded: @rpath/Python3.framework/Versions/3.8/Python3`. These can be resolved with another addition to `.cargo/config.toml`:
+Using the MacOS system python3 (`/usr/bin/python3`, as opposed to python installed via homebrew, pyenv, nix, etc.) may result in runtime errors such as `Library not loaded: @rpath/Python3.framework/Versions/3.8/Python3`.
+
+The easiest way to set the correct linker arguments is to add a `build.rs` with the following content:
+
+```rust,ignore
+fn main() {
+    pyo3_build_config::add_python_framework_link_args();
+}
+```
+
+Alternatively it can be resolved with another addition to `.cargo/config.toml`:
 
 ```toml
 [build]
@@ -153,16 +163,6 @@ rustflags = [
 ]
 ```
 
-Alternatively, one can include in `build.rs`:
-
-```rust
-fn main() {
-    println!(
-        "cargo:rustc-link-arg=-Wl,-rpath,/Library/Developer/CommandLineTools/Library/Frameworks"
-    );
-}
-```
-
 For more discussion on and workarounds for MacOS linking problems [see this issue](https://github.com/PyO3/pyo3/issues/1800#issuecomment-906786649).
 
 Finally, don't forget that on MacOS the `extension-module` feature will cause `cargo test` to fail without the `--no-default-features` flag (see [the FAQ](https://pyo3.rs/main/faq.html#i-cant-run-cargo-test-or-i-cant-build-in-a-cargo-workspace-im-having-linker-issues-like-symbol-not-found-or-undefined-reference-to-_pyexc_systemerror)).

guide/src/free-threading.md

@@ -156,20 +156,40 @@ freethreaded build, holding a `'py` lifetime means only that the thread is
 currently attached to the Python interpreter -- other threads can be
 simultaneously interacting with the interpreter.
 
-The main reason for obtaining a `'py` lifetime is to interact with Python
+You still need to obtain a `'py` lifetime is to interact with Python
 objects or call into the CPython C API. If you are not yet attached to the
 Python runtime, you can register a thread using the [`Python::with_gil`]
 function. Threads created via the Python [`threading`] module do not not need to
-do this, but all other OS threads that interact with the Python runtime must
-explicitly attach using `with_gil` and obtain a `'py` liftime.
-
-Since there is no GIL in the free-threaded build, releasing the GIL for
-long-running tasks is no longer necessary to ensure other threads run, but you
-should still detach from the interpreter runtime using [`Python::allow_threads`]
-when doing long-running tasks that do not require the CPython runtime. The
-garbage collector can only run if all threads are detached from the runtime (in
-a stop-the-world state), so detaching from the runtime allows freeing unused
-memory.
+do this, and pyo3 will handle setting up the [`Python<'py>`] token when CPython
+calls into your extension.
+
+### Global synchronization events can cause hangs and deadlocks
+
+The free-threaded build triggers global synchronization events in the following
+situations:
+
+* During garbage collection in order to get a globally consistent view of
+  reference counts and references between objects
+* In Python 3.13, when the first background thread is started in
+  order to mark certain objects as immortal
+* When either `sys.settrace` or `sys.setprofile` are called in order to
+  instrument running code objects and threads
+* Before `os.fork()` is called.
+
+This is a non-exhaustive list and there may be other situations in future Python
+versions that can trigger global synchronization events.
+
+This means that you should detach from the interpreter runtime using
+[`Python::allow_threads`] in exactly the same situations as you should detach
+from the runtime in the GIL-enabled build: when doing long-running tasks that do
+not require the CPython runtime or when doing any task that needs to re-attach
+to the runtime (see the [guide
+section](parallelism.md#sharing-python-objects-between-rust-threads) that
+covers this). In the former case, you would observe a hang on threads that are
+waiting on the long-running task to complete, and in the latter case you would
+see a deadlock while a thread tries to attach after the runtime triggers a
+global synchronization event, but the spawning thread prevents the
+synchronization event from completing.
 
 ### Exceptions and panics for multithreaded access of mutable `pyclass` instances
 
@@ -183,10 +203,10 @@ mutability](./class.md#bound-and-interior-mutability),) but now in free-threaded
 Python there are more opportunities to trigger these panics from Python because
 there is no GIL to lock concurrent access to mutably borrowed data from Python.
 
-The most straightforward way to trigger this problem to use the Python
+The most straightforward way to trigger this problem is to use the Python
 [`threading`] module to simultaneously call a rust function that mutably borrows a
-[`pyclass`]({{#PYO3_DOCS_URL}}/pyo3/attr.pyclass.html). For example,
-consider the following implementation:
+[`pyclass`]({{#PYO3_DOCS_URL}}/pyo3/attr.pyclass.html) in multiple threads. For
+example, consider the following implementation:
 
 ```rust
 # use pyo3::prelude::*;
@@ -240,7 +260,7 @@ RuntimeError: Already borrowed
 We plan to allow user-selectable semantics for mutable pyclass definitions in
 PyO3 0.24, allowing some form of opt-in locking to emulate the GIL if that is
 needed. For now you should explicitly add locking, possibly using conditional
-compilation or using the critical section API to avoid creating deadlocks with
+compilation or using the critical section API, to avoid creating deadlocks with
 the GIL.
 
 ### Cannot build extensions using the limited API
@@ -252,8 +272,8 @@ PyO3 will print a warning and ignore that setting when building extensions using
 the free-threaded interpreter.
 
 This means that if your package makes use of the ABI forward compatibility
-provided by the limited API to uploads only one wheel for each release of your
-package, you will need to update and tooling or instructions to also upload a
+provided by the limited API to upload only one wheel for each release of your
+package, you will need to update your release procedure to also upload a
 version-specific free-threaded wheel.
 
 See [the guide section](./building-and-distribution/multiple-python-versions.md)