-
-
Notifications
You must be signed in to change notification settings - Fork 1.6k
PEP 792: Project status markers in the simple index #4432
New issue
Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.
By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.
Already on GitHub? Sign in to your account
base: main
Are you sure you want to change the base?
Changes from all commits
ddf1f9f
e238ba3
ff972d7
3b28def
bae063d
b803be9
a253d94
3965134
11ddd9f
6f27d07
1b40e31
40a4245
File filter
Filter by extension
Conversations
Jump to
Diff view
Diff view
There are no files selected for viewing
Original file line number | Diff line number | Diff line change |
---|---|---|
@@ -0,0 +1,383 @@ | ||
PEP: 792 | ||
Title: Project status markers in the simple index | ||
Author: William Woodruff <william@yossarian.net>, | ||
Facundo Tuesca <facundo.tuesca@trailofbits.com>, | ||
Sponsor: Donald Stufft <donald@stufft.io> | ||
PEP-Delegate: Donald Stufft <donald@stufft.io> | ||
Discussions-To: Pending | ||
Status: Draft | ||
Type: Standards Track | ||
Topic: Packaging | ||
Created: 21-May-2025 | ||
Post-History: `03-Feb-2025 <https://discuss.python.org/t/79356/>`__, | ||
|
||
Abstract | ||
======== | ||
|
||
This PEP proposes a standardized set of index-supplied project status markers, | ||
as well as a mechanism for communicating those markers in the HTML and JSON | ||
simple indices. | ||
|
||
Rationale and Motivation | ||
======================== | ||
|
||
The "status" of a project is an important piece of metadata, made more important | ||
by growth in both the size and complexity of the Python packaging ecosystem. | ||
Project status (or proxies such as recent activity) is useful to know in | ||
determining whether a project is maintained or otherwise suitable for consumption. | ||
|
||
Python packaging has at least three different mechanisms for communicating | ||
the "status" of a project: | ||
|
||
1. Distribution packages can include *Trove classifiers* in their metadata, as | ||
originally specified in :pep:`301`. The list of supported classifiers is | ||
`maintained by the PyPA <https://github.com/pypa/trove-classifiers>`_, | ||
and includes the ``Development Status`` hierarchy. For example, a | ||
distribution can include the ``Development Status :: 7 - Inactive`` | ||
classifier to indicate that the distribution's project is inactive. | ||
|
||
Trove classifiers are flexible, but also come with significant limitations: | ||
they're machine-readable and are rendered on indices like PyPI, but | ||
they also require the maintainer to push one or more *new* distributions | ||
each time they wish to update their project's development status. | ||
Furthermore, because distributions are *de facto* immutable in the Python | ||
packaging ecosystem, older distributions can't have their classifiers | ||
updated to reflect the current status of the project. | ||
|
||
2. Indices can mark distributions and releases as "yanked", as originally | ||
specified in :pep:`592`. Yanked distributions are not considered | ||
eligible for dependency resolution. | ||
|
||
When a distribution has been yanked, it is marked with ``data-yanked`` | ||
in the HTML index and with ``yanked: bool | str`` in the JSON index. | ||
Additionally, indices like PyPI will hide yanked distributions by default | ||
and will render them with a notice when the user navigates directly to them. | ||
|
||
Yanking is machine-readable like Trove classifiers, but is single-purpose | ||
rather than general-purpose: users can specify a free-text reason for | ||
yanking a given distribution package, but the semantics of yanking are | ||
fixed, and no reliable inference of project status can be made by a machine | ||
based upon that free-text reason. | ||
|
||
3. PyPI itself has *project statuses*, which apply to the entire project | ||
(i.e., all releases and distributions). Project statuses have both | ||
maintainer- and index-admin-controllable states: | ||
|
||
* PyPI administrators can "quarantine" a project. Quarantine behaves like | ||
a strengthened yank: the entire project remains uninstallable while | ||
quarantined, and only an administrator can un-quarantine it. | ||
|
||
* Project owners can "archive" a project. Archiving a project | ||
disables new release and distribution uploads to that project, | ||
but otherwise has no effect on the ability to download a project. | ||
|
||
Project statuses are machine-readable *in principle*, but are not currently | ||
exposed via any of PyPI's APIs. Instead, PyPI renders project statuses on | ||
each project's user-facing (i.e. non-index) webpage. | ||
|
||
In summary, there are multiple ways to communicate the "status" of a project in | ||
Python packaging. However, none of them satisfy the four characteristics we | ||
desire. There is no current project status indicator that is machine-readable, | ||
general (i.e. conveys more than one possible state), index-agnostic, and applies | ||
to the entire project, instead of per-release or per-distribution. | ||
|
||
===================== ================ ======= ============== ============ | ||
Mechanism Machine-readable General Index-agnostic Project-wide | ||
===================== ================ ======= ============== ============ | ||
Trove classifiers ✅ ✅ ✅ ❌ | ||
Yanking ✅ ❌ ✅ ✅ | ||
PyPI project statuses ✅ ✅ ❌ ✅ | ||
===================== ================ ======= ============== ============ | ||
|
||
This PEP proposes adopting PyPI's project statuses as an index-agnostic | ||
mechanism, satisfying all four conditions. | ||
|
||
Specification | ||
============= | ||
|
||
This PEP specifies two aspects: a set of project status markers, | ||
as well as their presentation in the standard HTML and JSON indices. | ||
|
||
Project status markers | ||
---------------------- | ||
|
||
This PEP proposes the following project status markers. | ||
There was a problem hiding this comment. Choose a reason for hiding this commentThe reason will be displayed to describe this comment to others. Learn more. I can't find anywhere in Specification defining state transitions between status markers. E.g. an explicit note that one may un-deprecate a project, etc. It may also be good to note the expected interface for markers (e.g. project owners or admins can change at any time per-project), though this probably is a little overkill. There was a problem hiding this comment. Choose a reason for hiding this commentThe reason will be displayed to describe this comment to others. Learn more.
One of the follow-on discussions I had with the PyPI folks is that this PEP should probably avoid defining state transitions directly, since that begins to fall into permissions/index-specific behavior. But as a midway point I could note which statuses are conceptually "owner-level" versus "admin-level" (e.g. "archived" vs. "quarantined").
Yeah, I think this would push too far into index policy 😅 -- I can see a reasonable scenario in which a non-PyPI index might want admins to be able to archive or deprecate projects, while that's not something that PyPI itself would likely do. There was a problem hiding this comment. Choose a reason for hiding this commentThe reason will be displayed to describe this comment to others. Learn more. My intent here wasn't defining the actor, but just the state -- currently, it is unclear if an archived project can be un-archived, or a deprecated project un-deprecated, etc. I think a line saying that there are no restrictions in the PEP on changing status would help. You could maybe add a further line to note that indices may have further policies on the 'quarrantine' status as a special case? There was a problem hiding this comment. Choose a reason for hiding this commentThe reason will be displayed to describe this comment to others. Learn more. Sounds good, will do! There was a problem hiding this comment. Choose a reason for hiding this commentThe reason will be displayed to describe this comment to others. Learn more. I'm going to add some language that'll hopefully clarify that states can be set and un-set, but that the PEP leaves it up to the index to determine which "principals" can do what (i.e. whether maintainers or admins can deprecate, archive, etc.) |
||
|
||
A project always has exactly one status. If no status is explicitly noted, | ||
then the project is considered to be in the ``active`` state. | ||
|
||
Indices **MAY** implement any subset of the status markers specified in this | ||
PEP, as applicable. | ||
|
||
This PEP does not prescribe *which* principals (i.e. project maintainers, | ||
index administrators, etc.) are allowed to set and unset which statuses. | ||
|
||
``active`` | ||
~~~~~~~~~~ | ||
|
||
Description: The project is active. This is the default status for a project. | ||
|
||
Index semantics: | ||
|
||
* The index hosting the project **MUST** allow uploads of new distributions to | ||
the project. | ||
* The index **MUST** offer existing distributions of the project for download. | ||
|
||
Installer semantics: none. | ||
|
||
``archived`` | ||
~~~~~~~~~~~~ | ||
|
||
Description: The project does not expect to be updated in the future. | ||
|
||
Index semantics: | ||
|
||
* The index hosting the project **MUST NOT** allow uploads of new distributions to | ||
the project. | ||
* The index **MUST** offer existing distributions of the project for download. | ||
|
||
Installer semantics: | ||
|
||
* Installers **MAY** produce warnings about a project's archival. | ||
|
||
``quarantined`` | ||
~~~~~~~~~~~~~~~ | ||
|
||
Description: The project is considered generally unsafe for use, e.g. due to | ||
malware. | ||
|
||
Index semantics: | ||
|
||
* The index hosting the project **MUST NOT** allow uploads of new distributions to | ||
the project. | ||
* The index **MUST NOT** offer any distributions of the project for download. | ||
|
||
Installer semantics: | ||
|
||
* Installers **MAY** produce warnings about a project's quarantine, although | ||
doing so is effectively moot (as the index will not offer any distributions | ||
for installation). | ||
|
||
``deprecated`` | ||
~~~~~~~~~~~~~~ | ||
|
||
Description: The project is considered obsolete, and may have been superseded | ||
by another project. | ||
|
||
Index semantics: | ||
|
||
* This status shares the same semantics as ``active``. | ||
|
||
Installer semantics: | ||
|
||
* Installers **MAY** produce warnings about a project's deprecation. | ||
|
||
Status markers in the index APIs | ||
-------------------------------- | ||
|
||
This PEP defines version 1.4 of the index APIs. | ||
|
||
HTML index | ||
~~~~~~~~~~ | ||
|
||
The following changes are made to the | ||
:ref:`simple repository API <packaging:simple-repository-api-base>`: | ||
|
||
* The index **SHALL** define the ``pypi:repository-version`` as ``1.4``. | ||
* The index **SHOULD** add an appropriate ``pypi:project-status`` meta tag, with | ||
a ``content`` of the project's status marker. The index **MAY** choose to omit | ||
the ``pypi:project-status`` meta tag if the project is marked as ``active``. | ||
|
||
For example, the following would be a valid HTML index response for | ||
``sampleproject`` after is has been marked as ``quarantined``: | ||
|
||
.. code-block:: html | ||
:emphasize-lines: 5 | ||
|
||
<!DOCTYPE html> | ||
<html> | ||
<head> | ||
<meta name="pypi:repository-version" content="1.4"> | ||
<meta name="pypi:project-status" content="quarantined"> | ||
<title>Links for sampleproject</title> | ||
</head> | ||
<body> | ||
<h1>Links for sampleproject</h1> | ||
</body> | ||
</html> | ||
|
||
Observe that, per the ``quarantined`` semantics above, the index response | ||
contains no distribution links for the project. | ||
|
||
JSON index | ||
~~~~~~~~~~ | ||
|
||
The following changes are made to the | ||
:ref:`JSON simple index <packaging:simple-repository-api-json>`: | ||
|
||
* The index **SHALL** define the ``meta.api-version`` as ``1.4``. | ||
* The index **SHOULD** include a ``project-status`` key in the JSON response, | ||
with a value of the project's status marker. The index **MAY** choose to omit | ||
the ``project-status`` key if the project is marked as ``active``. | ||
|
||
For example, the following would be a valid JSON index response for | ||
``sampleproject`` after is has been marked as ``quarantined``: | ||
|
||
.. code-block:: json | ||
:emphasize-lines: 5 | ||
|
||
{ | ||
"meta": { | ||
"api-version": "1.4" | ||
}, | ||
"project-status": "quarantined", | ||
"alternate-locations": [], | ||
"files": [], | ||
"name": "sampleproject", | ||
"versions": [ | ||
"1.2.0", | ||
"1.3.0", | ||
"1.3.1", | ||
"2.0.0", | ||
"3.0.0", | ||
"4.0.0" | ||
] | ||
} | ||
|
||
Observe that, like with the HTML index, the JSON response contains no | ||
distribution links for the ``quarantined`` project. | ||
|
||
Future Considerations | ||
===================== | ||
|
||
This PEP defines only four project status markers: ``active``, ``archived``, | ||
``quarantined``, and ``deprecated``. | ||
|
||
Future PEPs (or PyPA standards processes) may define additional project status | ||
markers, as needed. Any future status markers may require a metadata version | ||
bump, unless a future metadata change is made to allow for "open-ended" status | ||
markers (i.e., where indices and installers do not necessarily share a single | ||
common list of allowed statuses). | ||
|
||
As specified in this PEP, project status markers are "bare," i.e. they | ||
convey no additional user-controlled metadata (such as an explanation | ||
for a project's archival). | ||
|
||
A future PEP may choose to extend the project | ||
status mechanism to include user-controlled metadata, in a manner similar | ||
to the free-form text allowed during release yanking. | ||
|
||
Security Implications | ||
===================== | ||
|
||
This PEP does not identify any positive or negative security implications | ||
associated with adding project status markers. | ||
|
||
How to Teach This | ||
================= | ||
|
||
Educating the Python community about this PEP has two aspects: | ||
|
||
* Ordinary package maintainers will need to be informed of their ability to | ||
set project status markers, e.g. to inform their downstreams that | ||
a project has been archived or deprecated. | ||
|
||
If this PEP is accepted, the authors of this PEP will coordinate with | ||
PyPI on appropriate maintainer-oriented documentation and communication, | ||
including feature announcement blog posts and updates to | ||
`PyPI's user documentation <https://docs.pypi.org>`_. | ||
|
||
* Installer and index maintainers will need to be informed of the new project | ||
status markers, and how to interpret them. | ||
|
||
If this PEP is accepted, the authors of this PEP will perform its | ||
implementation on PyPI, serving as a reference implementation for other | ||
indices. | ||
|
||
This PEP does not *mandate* any changes in installer behavior. However, | ||
if this PEP is accepted, the authors of this PEP will coordinate with | ||
the maintainers of popular installers (e.g. ``pip``) to help each determine | ||
the extent to which they wish to surface project statuses. | ||
|
||
Rejected Ideas | ||
============== | ||
|
||
Using "reserved" keys | ||
--------------------- | ||
|
||
One alternative to this PEP is to avoid standardizing project status | ||
markers directly, but instead use existing mechanisms within the standards | ||
to communicate them in a non-standard fashion. | ||
|
||
For example, the `JSON simple index <packaging:simple-repository-api-json>`_ | ||
says the following: | ||
|
||
Keys (at any level) with a leading underscore are reserved as private for | ||
index server use. No future standard will assign a meaning to any such key. | ||
|
||
In effect, this means that the following would be standards-compliant: | ||
|
||
.. code-block:: json | ||
:emphasize-lines: 5 | ||
|
||
{ | ||
"meta": { | ||
"api-version": "1.4" | ||
}, | ||
"_project-status": "quarantined", | ||
"alternate-locations": [], | ||
"files": [], | ||
"name": "sampleproject", | ||
"versions": [ | ||
"1.2.0", | ||
"1.3.0", | ||
"1.3.1", | ||
"2.0.0", | ||
"3.0.0", | ||
"4.0.0" | ||
] | ||
} | ||
|
||
However, this approach has several drawbacks: | ||
|
||
* Standards-aligned tools (such as ``pip``, ``pip-audit``, and ``uv``) | ||
may find it unacceptable to use a "reserved" key, since that key will | ||
have no standard semantics or compatibility properties. | ||
* The "reserved" approach is only suitable for the JSON simple index; | ||
no equivalent mechanism exists for the HTML simple index. | ||
This would disadvantage consumers of the HTML simple index, as well as | ||
mirror implementations that may consume the JSON index but only expose | ||
an HTML index. | ||
|
||
Project markers in PyPI's non-standard JSON API | ||
----------------------------------------------- | ||
|
||
Another standardization-avoidance alternative is to expose project status | ||
markers, but only in PyPI's | ||
`non-standard JSON API <https://docs.pypi.org/api/json/>`_. PyPI has full | ||
control over the layout of this API, and could include a ``project-status`` | ||
or similar key without needing a PEP or underscore prefix. | ||
|
||
This has similar drawbacks as the "reserved" keys approach above, | ||
and more generally deepens the differences between the standard | ||
and non-standard APIs. | ||
|
||
Multiple project status markers at once | ||
--------------------------------------- | ||
|
||
An earlier version of this PEP considered proposing support for | ||
multiple project markers at once. For example, a project could be marked | ||
as both ``archived`` and ``quarantined``. | ||
|
||
After consideration, this was rejected for complexity reasons: having multiple | ||
project status markers requires the PEP to specify a conflict resolution | ||
mechanism when merging their semantics, as well as as state machine for which | ||
markers are exclusive (for example, ``active`` is conceptually exclusive with | ||
all other markers, while ``archived`` and ``quarantined`` are conceptually | ||
compatible with each other). | ||
|
||
Copyright | ||
========= | ||
|
||
This document is placed in the public domain or under the CC0-1.0-Universal | ||
license, whichever is more permissive. |
There was a problem hiding this comment.
Choose a reason for hiding this comment
The reason will be displayed to describe this comment to others. Learn more.
@dstufft Please can you confirm you are sponsoring this PEP?