PEP 793: PyModExport: A new entry point for C extension modules #4435

encukou · 2025-05-23T16:39:04Z

Basic requirements (all PEP Types)

Standards Track requirements

📚 Documentation preview 📚: https://pep-previews--4435.org.readthedocs.build/pep-0793/

peps/pep-0793.rst

AA-Turner

Thanks!

General points:

I think the PEP would benefit from more examples. Perhaps a full example of a module only using PyModExport_*, an example on porting, and an example on using both export hooks at once, e.g. for a module supporting Python 3.11--3.15?
The text is very heavy on parenthetical asides. I've suggested removing some, but in general I think where full sentences are in brackets, the text would be stronger by properly incorporating the thought and removing the brackets.
Please replace typographical quotation marks with "/', Sphinx/Docutils insert them automatically.
Naming (sorry!): Currently, PyMod* is only used by PyModInitFunction. Would PyModuleExport be considered?

A

AA-Turner · 2025-05-24T02:26:37Z

peps/pep-0793.rst

General: I think the PEP would benefit from a full worked example, perhaps in How to Teach This. It's been noted that one of the problems with multi-phase is a lack of guidance on how to do a migration. A simple example of a 'spam' module would help readers understand what the end state is meant to look like, and could be repurposed for the planned documentation.

peps/pep-0793.rst

AA-Turner · 2025-05-24T02:47:03Z

peps/pep-0793.rst

+
+On failure, the export hook must return NULL with an exception set.
+This will cause the import to fail.
+(Python will not fall back to ``PyInit_*`` on error.)


Perhaps explicitly mention that defining both PyModExport_<NAME> and PyInit_<NAME> is allowed, and indeed expected for modules bridging Python versions?

AA-Turner · 2025-05-24T02:48:51Z

peps/pep-0793.rst

+
+If found, Python will call the hook with the appropriate
+``importlib.machinery.ModuleSpec`` object as *spec*.
+To support duck-typing, extensions should not type-check this object, and


Suggested change

To support duck-typing, extensions should not type-check this object, and

To support duck-typing, extension modules should not type-check this object, and

“Extension” could mean the extension author, extension code, the shared library, the module object -- take your pick.
In this instance, any of that is appropriate, so I don't want to use a more specific term.

AA-Turner · 2025-05-24T02:50:42Z

peps/pep-0793.rst

+(We expect functions to export a static constant, or one of several constants
+chosen depending on, for example, ``Py_Version``. Dynamic behaviour should
+generally happen in the ``Py_mod_create`` and ``Py_mod_exec`` functions.)


Suggested change

(We expect functions to export a static constant, or one of several constants

chosen depending on, for example, ``Py_Version``. Dynamic behaviour should

generally happen in the ``Py_mod_create`` and ``Py_mod_exec`` functions.)

We expect functions to export a static constant, or one of several constants

chosen depending on, for example, ``Py_Version``. Dynamic behaviour should

generally happen in the ``Py_mod_create`` and ``Py_mod_exec`` functions.

AA-Turner · 2025-05-24T03:15:40Z

peps/pep-0793.rst

+
+.. code-block:: c
+
+    PyObject *PyModule_FromSlotsAndSpec(PyModuleDef_Slot *, PyObject *spec)


The export hook is part of the API, but the functions are defined in extension modules. Worth including?

Suggested change

PyObject *PyModule_FromSlotsAndSpec(PyModuleDef_Slot *, PyObject *spec)

PyModuleDef_Slot *PyModExport_<NAME>(PyObject *spec);

PyObject *PyModule_FromSlotsAndSpec(PyModuleDef_Slot *, PyObject *spec)

AA-Turner · 2025-05-24T03:21:24Z

peps/pep-0793.rst

+Backwards Compatibility
+=======================


I think there should be greater emphasis on compatibility code for extension modules supporting older versions, but wanting to use the better/newer mechanism for 3.15+. Either in Backwards Compatibility or How to Teach This.

For instance, the multiple Py_mod_exec is solvable with a new function that calls all existing functions sequentially. Any suggested mechanisms #if ... for defining both a PyInit and PyModExport export hook? Anything that could go into pythoncapi-compat to make transition easier?

peps/pep-0793.rst

AA-Turner · 2025-05-24T03:26:05Z

peps/pep-0793.rst

+  A workaround is to check ``Py_Version`` in the export function,
+  and return a slot array suitable for the current interpreter.


Or, use #ifdef Py_<slot> or #if PY_VERSION_HEX >= ....

peps/pep-0793.rst

Co-authored-by: Hugo van Kemenade <1324225+hugovk@users.noreply.github.com> Co-authored-by: Hood Chatham <roberthoodchatham@gmail.com> Co-authored-by: Adam Turner <9087854+AA-Turner@users.noreply.github.com>

encukou

Thank you for the reviews! I went through some of the comments, will get to the rest later.
kid is waking up, I left some replies terse; please imagine a most friendly & respectful tone :)

I think the PEP would benefit from more examples. Perhaps a full example of a module only using PyModExport_*, an example on porting, and an example on using both export hooks at once, e.g. for a module supporting Python 3.11--3.15?

Agree, but these should end up in docs and I don't want people reading them here after acceptance. I'll add them to the reference implementation and point the PEP there.

The text is very heavy on parenthetical asides. I've suggested removing some, but in general I think where full sentences are in brackets, the text would be stronger by properly incorporating the thought and removing the brackets.

I use them to separate the hard spec from “mere“ implications/explanations. They are very helpful for me, and I like to think they're not too distracting for the reader.

Please replace typographical quotation marks with "/', Sphinx/Docutils insert them automatically.

I don't see a reason to do that. Any future editor is free to use " instead, it's not like Python identifiers which you need to repeat exactly somewhere else.

Naming (sorry!): Currently, PyMod* is only used by PyModInitFunction. Would PyModuleExport be considered?

That's a point for the discussion :)

(Yes, I considered it, but my conclusion is not strong enough for a Rejected Ideas entry and the question is not important enough for Open Questions. I'm literally waiting for the discussion thread here.)

encukou · 2025-05-24T04:54:32Z

peps/pep-0793.rst

+The slot will be optional, but extensions will be strongly encouraged to
+include it for the benefit of future APIs, external tooling,
+and debugging/introspection.


or maybe

Suggested change

The slot will be optional, but extensions will be strongly encouraged to

include it for the benefit of future APIs, external tooling,

and debugging/introspection.

The slot will be optional, but extension authors are strongly encouraged to

include it for the benefit of future APIs, external tooling, debugging,

and introspection.

encukou · 2025-05-24T04:57:03Z

peps/pep-0793.rst

+All new slots -- these and ``Py_tp_token`` discussed above -- may not be
+repeated in the slots array, and may not be used in a
+``PyModuleDef.m_slots`` array.
+They may not have a ``NULL`` value (instead, the slot can be omitted entirely).


I want to specifically point out that it's both the ones from the previous paragraph and Py_tp_token. This behaviour is “good“ for both (for reasons either obvious or mentioned in the Py_tp_token section), but the reviewer should think about them separately.

encukou · 2025-05-24T04:59:21Z

peps/pep-0793.rst

+For modules created without a ``PyModuleDef``, the ``Py_mod_create`` function
+will be called with ``NULL`` for the second argument.
+(In the future, if we find a use case for passing the input slots array, a new
+slot with an updated signature can be added.)


Suggested change

For modules created without a ``PyModuleDef``, the ``Py_mod_create`` function

will be called with ``NULL`` for the second argument.

(In the future, if we find a use case for passing the input slots array, a new

slot with an updated signature can be added.)

For modules created without a ``PyModuleDef``, the ``Py_mod_create`` function

will be called with ``NULL`` for the second argument (*def*).

(In the future, if we find a use case for passing the input slots array, a new

slot with an updated signature can be added.)

encukou · 2025-05-24T05:09:00Z

peps/pep-0793.rst

+To simplify the implementation, the slots arrays for both
+``PyModule_FromSlotsAndSpec`` and the new export hook will only allow up to one
+``Py_mod_exec`` slot.
+(Arrays in ``PyModuleDef.m_slots`` may have more; this will not change.)


Well, that is the main argument. I don't want to maintain the code for that :)
I can do what I did for other minor inconveniences:

Suggested change

To simplify the implementation, the slots arrays for both

``PyModule_FromSlotsAndSpec`` and the new export hook will only allow up to one

``Py_mod_exec`` slot.

(Arrays in ``PyModuleDef.m_slots`` may have more; this will not change.)

To simplify the implementation, the slots arrays for both

``PyModule_FromSlotsAndSpec`` and the new export hook will only allow up to one

``Py_mod_exec`` slot.

(Arrays in ``PyModuleDef.m_slots`` may have more; this will not change.)

This limitation might be lifted in the future.

FWIW, I'd like to lift the other limitations in this PEP -- in dedicated PRs with loads of tests. This limitation isn't worth it IMO.

encukou · 2025-05-24T05:16:08Z

peps/pep-0793.rst

+
+If found, Python will call the hook with the appropriate
+``importlib.machinery.ModuleSpec`` object as *spec*.
+To support duck-typing, extensions should not type-check this object, and


“Extension” could mean the extension author, extension code, the shared library, the module object -- take your pick.
In this instance, any of that is appropriate, so I don't want to use a more specific term.

encukou · 2025-05-24T05:17:37Z

peps/pep-0793.rst

+
+.. code-block:: c
+
+    PyModuleDef_Slot *PyModExport_<NAME>(PyObject *spec);


Thanks for the suggestion! I'll leave it for the API summary.

encukou · 2025-05-24T05:22:08Z

peps/pep-0793.rst

+Specification
+=============
+
+


The proper title for this subsection would be PyModExport: A new entry point for C extension modules. But it's already under that heading :)
This is the main thing; the rest is supporting API.

encukou · 2025-05-24T05:30:21Z

peps/pep-0793.rst

+
+New API summary
+---------------
+


Suggested change

Python will load a new module export hook, with two variants:

.. code-block:: c

PyModuleDef_Slot *PyModExport_<NAME>(PyObject *spec);

PyModuleDef_Slot *PyModExportU_<ENCODED_NAME>(PyObject *spec);

PEP 793: PyModExport: A new entry point for C extension modules

5604111

encukou requested a review from a team as a code owner May 23, 2025 16:39

Correct dates

fb46967

encukou commented May 23, 2025

View reviewed changes

peps/pep-0793.rst Show resolved Hide resolved

peps/pep-0793.rst Outdated Show resolved Hide resolved

peps/pep-0793.rst Outdated Show resolved Hide resolved

AA-Turner added the new-pep A new draft PEP submitted for initial review label May 23, 2025

hugovk reviewed May 23, 2025

View reviewed changes

peps/pep-0793.rst Outdated Show resolved Hide resolved

peps/pep-0793.rst Outdated Show resolved Hide resolved

hoodmane reviewed May 23, 2025

View reviewed changes

peps/pep-0793.rst Outdated Show resolved Hide resolved

hoodmane reviewed May 23, 2025

View reviewed changes

peps/pep-0793.rst Outdated Show resolved Hide resolved

hoodmane reviewed May 23, 2025

View reviewed changes

peps/pep-0793.rst Outdated Show resolved Hide resolved

peps/pep-0793.rst Outdated Show resolved Hide resolved

peps/pep-0793.rst Outdated Show resolved Hide resolved

peps/pep-0793.rst Outdated Show resolved Hide resolved

peps/pep-0793.rst Outdated Show resolved Hide resolved

AA-Turner reviewed May 24, 2025

View reviewed changes

Apply suggestions from code review

a27ae32

Co-authored-by: Hugo van Kemenade <1324225+hugovk@users.noreply.github.com> Co-authored-by: Hood Chatham <roberthoodchatham@gmail.com> Co-authored-by: Adam Turner <9087854+AA-Turner@users.noreply.github.com>

encukou commented May 24, 2025

View reviewed changes

	To support duck-typing, extensions should not type-check this object, and
	To support duck-typing, extension modules should not type-check this object, and


		.. code-block:: c

		PyObject PyModule_FromSlotsAndSpec(PyModuleDef_Slot , PyObject *spec)

	PyObject PyModule_FromSlotsAndSpec(PyModuleDef_Slot , PyObject *spec)
	PyModuleDef_Slot PyModExport_<NAME>(PyObject spec);
	PyObject PyModule_FromSlotsAndSpec(PyModuleDef_Slot , PyObject *spec)

		A workaround is to check ``Py_Version`` in the export function,
		and return a slot array suitable for the current interpreter.

		Backwards Compatibility
		=======================

		Specification
		=============


		New API summary
		---------------

Uh oh!

PEP 793: PyModExport: A new entry point for C extension modules #4435

Are you sure you want to change the base?

PEP 793: PyModExport: A new entry point for C extension modules #4435

Uh oh!

Conversation

encukou commented May 23, 2025 • edited Loading Uh oh! There was an error while loading. Please reload this page.

Uh oh!

Basic requirements (all PEP Types)

Standards Track requirements

Uh oh!

Uh oh!

Uh oh!

Uh oh!

Uh oh!

Uh oh!

Uh oh!

Uh oh!

Uh oh!

Uh oh!

Uh oh!

Uh oh!

Uh oh!

AA-Turner left a comment

Choose a reason for hiding this comment

Uh oh!

Choose a reason for hiding this comment

Uh oh!

Uh oh!

Choose a reason for hiding this comment

Uh oh!

Choose a reason for hiding this comment

Uh oh!

Choose a reason for hiding this comment

Uh oh!

Choose a reason for hiding this comment

Uh oh!

Choose a reason for hiding this comment

Uh oh!

Choose a reason for hiding this comment

Uh oh!

Uh oh!

Choose a reason for hiding this comment

Uh oh!

Uh oh!

encukou left a comment

Choose a reason for hiding this comment

Uh oh!

Choose a reason for hiding this comment

Uh oh!

Choose a reason for hiding this comment

Uh oh!

Choose a reason for hiding this comment

Uh oh!

Choose a reason for hiding this comment

Uh oh!

Choose a reason for hiding this comment

Uh oh!

Choose a reason for hiding this comment

Uh oh!

Choose a reason for hiding this comment

Uh oh!

Choose a reason for hiding this comment

Uh oh!

Uh oh!

encukou commented May 23, 2025 •

edited

Loading