0.22 release

[davidhewitt]

Now that 0.21 and 0.21.1 are out the door, I think we can let main start merging breaking changes heading towards the 0.22 release. I will deal with any patch releases as cherry picks from here on out.

There's a couple of must-do items for 0.22:

• The cleanups for the GIL Refs migration for 0.22, listed in GIL Refs API Removal - Tracking Issue #3960
• The TLS soundness fixes in Close TLS-based soundness holes by running closure on a newly created thread. #3646

I also left a few items in the milestone which I thought made sense to look at soon-ish. I'll be looking at some of those - https://github.com/PyO3/pyo3/milestone/14. If anyone is interested in taking on some of these, please ping here, on those issues, or on Discord and I can gladly share any thoughts / notes with you and be a reviewer instead of the implementer 😄

Finally, I'm going to start writing some more high-level issues summarising topics where potential contributors might want to get involved. There are many areas where I think PyO3 can improve and just needs someone with the time and interest.

Along that line, these are some of my nice-to-haves for 0.22:

• I would love to see experimental-declarative-modules declared stable, I think there's not a huge amount remaining to do this.
• MSRV bump (cc @alex who I think had one prepped). Was it 1.62 or 1.63 we agreed as the target? Update MSRV to 1.63 #4129
• I'm planning to figure out a roadmap to get to Python 3.13's "freethreaded mode" support. I think this will consume the vast majority of my available time over the coming weeks.

[alex]

1.63 (this should let us drop parking_lot)

…

On Tue, Apr 2, 2024, 4:51 PM David Hewitt ***@***.***> wrote: Now that 0.21 and 0.21.1 are out the door, I think we can let main start merging breaking changes heading towards the 0.22 release. I will deal with any patch releases as cherry picks from here on out. There's a couple of must-do items for 0.22: - The cleanups for the GIL Refs migration for 0.22, listed in #3960 <#3960> - The TLS soundness fixes in #3646 <#3646> I also left a few items in the milestone which I thought made sense to look at soon-ish. I'll be looking at some of those - https://github.com/PyO3/pyo3/milestone/14. If anyone is interested in taking on some of these, please ping here, on those issues, or on Discord and I can gladly share any thoughts / notes with you and be a reviewer instead of the implementer 😄 Finally, I'm going to start writing some more high-level issues summarising topics where potential contributors might want to get involved. There are many areas where I think PyO3 can improve and just needs someone with the time and interest. Along that line, these are some of my nice-to-haves for 0.22: - I would love to see experimental-declarative-modules declared stable, I think there's not a huge amount remaining to do this. - MSRV bump (cc @alex <https://github.com/alex> who I think had one prepped). Was it 1.62 or 1.63 we agreed as the target? - I'm planning to figure out a roadmap to get to Python 3.13's "freethreaded mode" support. I think this will consume the vast majority of my available time over the coming weeks. — Reply to this email directly, view it on GitHub <#4038>, or unsubscribe <https://github.com/notifications/unsubscribe-auth/AAAAGBGG5X3ZBAJDM6WBANLY3MK55AVCNFSM6AAAAABFUCWSJ6VHI2DSMVQWIX3LMV43ASLTON2WKOZSGIZDCNBSGM2TSNQ> . You are receiving this because you were mentioned.Message ID: ***@***.***>

[wyfo]

I would also be nice to have the remaining async features merged for 0.22

[hugovk]

The Python 3.13 beta is out and the release manager has issued a call to action:

We strongly encourage maintainers of third-party Python projects to test with 3.13 during the beta phase and report issues found to the Python bug tracker as soon as possible. While the release is planned to be feature complete entering the beta phase, it is possible that features may be modified or, in rare cases, deleted up until the start of the release candidate phase (Tuesday 2024-07-30). Our goal is to have no ABI changes after beta 4 and as few code changes as possible after 3.13.0rc1, the first release candidate. To achieve that, it will be extremely important to get as much exposure for 3.13 as possible during the beta phase.

To help test projects which use PyO3 (and to test projects which depend on those), it would be very helpful to add support for 3.13.

Free-threaded support is experimental and can come later.

[davidhewitt]

Thanks @hugovk for the ping! This is on my radar to do asap and perhaps we can even ship support for 3.13 beta 1 in 0.21.3 release so that downstream can begin testing 👍

[holmanb]

Thanks @hugovk for the ping! This is on my radar to do asap and perhaps we can even ship support for 3.13 beta 1 in 0.21.3 release so that downstream can begin testing 👍

@davidhewitt That would be great, and should hopefully fix jsonschema's latest dependencies for python3.13: crate-py/rpds#72

[davidhewitt]

I think with the huge amount of work done mostly by @Icxolu and @alex (thanks both and all others who helped and reviewed!) we are complete with the GIL Ref migration work for 0.22, so in my head we can release whenever. I think there's a few various PRs and bug fixes kicking around which I hope to help review and merge in, and then maybe we can get this shipped within a week or two with 3.13 support.

I think most important is #3646 which we haven't found an alternative solution to and so it feels right to me that we move ahead with that change now.

[adamreichold]

I think most important is #3646 which we haven't found an alternative solution to and so it feels right to me that we move ahead with that change now.

I very much agree. I am bit pressed for time at the moment to actually carry it over the finish line though. I also think we should integrate #4097 into that PR which should be simple enough (if we do not insist on not spawning threads if the GIL is not held which could be a follow-up optimization).

[andy-maier]

I would like to put my 2 cents in for Python 3.13 support.

We are trying to test our own projects on the 3.13 beta and that fails when installing dependent packages due to py03 not supporting 3.13 (with the same symptom as described in crate-py/rpds#72).

In our case, "rpds-py" comes in through the packages "jsonschema" and "referencing", and these are so basic that I'm likely not the only one suffering from this.

A temporary circumvention for users is to pin the packages that pull in "rpds-py" to older versions that still use "pyrsistent". In my case, these are:

referencing>=...,<0.24.0
jsonschema>=...,<4.18.0

In fact, "referencing" is pulled in only via "jsonschema-specifications" which starts to get used with "jsonschema" 4.18.0, so the pinning of "jsonschema" was sufficient for my case.

Update: The lack of Python 3.13 support also seems to affect installation of any Python package that uses the "maturin" build backend. The version of maturin and its dependents cannot be controlled by my requirements files, but in my case, "pydantic-core" was the package using maturin which was used only by "safety" and I could circumvent the issue by pinning "safety" to a version before it started using "pydantic":

safety>=...,<3.0.0

[davidhewitt]

@adamreichold - as much as I think we'd both like to see the TLS defenses shipped in this release, we are both struggling to finish that PR right now and additionally we now have the changes to Py::clone which is potentially already one frustrating migration for users.

Shall we just allow that PR to slip again and publish 0.22 now, to unblock 3.13 migrations?

[musicinmybrain]

Shall we just allow that PR to slip again and publish 0.22 now, to unblock 3.13 migrations?

This would be helpful for us in Fedora, as we now have Python 3.13 as the system Python in Rawhide, and it can be hard to get upstreams involved in fixing Python 3.13 issues that they can’t yet reproduce in a virtualenv.