|
| 1 | +PEP: 799 |
| 2 | +Title: A dedicated ``profilers`` package for organizing Python profiling tools |
| 3 | +Author: Pablo Galindo <pablogsal@python.org>, |
| 4 | + László Kiss Kollár <kiss.kollar.laszlo@gmail.com>, |
| 5 | +Discussions-To: Pending |
| 6 | +Status: Draft |
| 7 | +Type: Standards Track |
| 8 | +Created: 21-Jul-2025 |
| 9 | +Python-Version: 3.15 |
| 10 | +Post-History: |
| 11 | + |
| 12 | +Abstract |
| 13 | +======== |
| 14 | + |
| 15 | +This PEP proposes the creation of a new standard library module named |
| 16 | +:mod:`!profilers` to organize Python's built-in profiling tools under a single, |
| 17 | +coherent namespace. |
| 18 | + |
| 19 | +This PEP also proposes the deprecation of the :mod:`profile` module, a legacy |
| 20 | +pure-Python tracing profiler. |
| 21 | + |
| 22 | +Motivation |
| 23 | +========== |
| 24 | + |
| 25 | +Python currently ships with two tracing profilers: ``profile`` and ``cProfile``. The |
| 26 | +``profile`` module is implemented in pure Python and is largely educational or useful for |
| 27 | +subclassing, but too slow for realistic use. ``cProfile``, by contrast, is implemented |
| 28 | +in C and more suitable for practical profiling scenarios, although it is not a drop-in |
| 29 | +replacement for ``profile`` due to some behavioral differences. |
| 30 | + |
| 31 | +Both of these are tracing profilers, meaning they instrument every function call and return. |
| 32 | +This methodology introduces significant runtime overhead and can disable certain interpreter |
| 33 | +optimizations, such as those introduced by :pep:`659`. Moreover, ``cProfile`` only observes the |
| 34 | +main thread, making it less useful in modern concurrent Python programs. Confusingly, the naming |
| 35 | +of these modules implies that ``profile`` is canonical, when in fact it is largely obsolete. |
| 36 | + |
| 37 | +With Python 3.15, a new sampling profiler was introduced under ``profile.sample``. Known as |
| 38 | +**tachyon**, this tool uses statistical sampling to infer performance characteristics, which |
| 39 | +introduces much lower overhead and works better with the modern Python interpreter. It also supports |
| 40 | +multiple threads, async functions, and attaching to running processes. Despite these strengths, |
| 41 | +the placement of tachyon under ``profile.sample`` is misleading and obscures its importance. |
| 42 | + |
| 43 | +Currently, the organization of profiling tools lacks a consistent, discoverable structure. |
| 44 | +The proposed reorganization is meant to guide users more effectively toward appropriate tools, |
| 45 | +clarify methodological distinctions between profilers, and lay the groundwork for future extensions. |
| 46 | + |
| 47 | +Rationale |
| 48 | +========= |
| 49 | + |
| 50 | +This reorganization addresses several long-standing issues with Python’s profiling tools. |
| 51 | + |
| 52 | +First, it improves **discoverability** by collecting all built-in profilers |
| 53 | +under a single, clearly named namespace. Currently, profiling tools are |
| 54 | +scattered across modules with inconsistent naming and unclear relationships. By |
| 55 | +introducing the ``profilers`` module, users will have a well-defined and |
| 56 | +intuitive location to explore and access profiling functionality. |
| 57 | + |
| 58 | +Second, the proposal enhances **clarity** by naming the submodules according to |
| 59 | +their underlying methodology -- ``profilers.tracing`` for deterministic tracing |
| 60 | +profilers and ``profilers.sampling`` for statistical sampling profilers. This |
| 61 | +explicit categorization makes it easier to understand the behavior and |
| 62 | +limitations of each tool and aligns the API with the mental model users are |
| 63 | +encouraged to adopt. |
| 64 | + |
| 65 | +Third, it provides **guidance** to developers by making the recommended tools |
| 66 | +easier to find and use. The current structure can mislead users into relying on |
| 67 | +outdated or less performant modules like ``profile``, simply because of naming |
| 68 | +precedence. With the ``profilers`` namespace and its clearer semantics, users |
| 69 | +are more likely to choose the appropriate profiler for their specific use case, |
| 70 | +whether it involves low-overhead sampling or detailed tracing. |
| 71 | + |
| 72 | +Finally, this structure promotes **extensibility**. By consolidating profiling |
| 73 | +tools under a unified namespace, it becomes straightforward to introduce future |
| 74 | +profiling capabilities -- such as memory profilers, I/O profilers, or hybrid |
| 75 | +tools -- without overloading unrelated modules or adding redundant top-level names. |
| 76 | +The ``profilers`` module provides a natural home for this. |
| 77 | + |
| 78 | +Specification |
| 79 | +============= |
| 80 | + |
| 81 | +New Module Structure |
| 82 | +-------------------- |
| 83 | + |
| 84 | +This PEP introduces a new ``profilers`` module containing: |
| 85 | + |
| 86 | +- ``profilers.tracing``: an alias for ``cProfile``, providing deterministic function-call tracing. |
| 87 | +- ``profilers.sampling``: an alias for ``profilers.tachyon``, improving visibility and semantic clarity. |
| 88 | + |
| 89 | +Additionally the **code** for the ``tachyon`` profiler will be also relocated to |
| 90 | +the module as ``profilers.tachyon``. |
| 91 | + |
| 92 | +Deprecation of ``profile`` |
| 93 | +-------------------------- |
| 94 | + |
| 95 | +The ``profile`` module will be deprecated starting in Python 3.15. |
| 96 | + |
| 97 | +- In Python 3.15: importing ``profile`` emits a ``DeprecationWarning``. |
| 98 | +- In Python 3.16: all uses of ``profile`` emit a ``DeprecationWarning``. |
| 99 | +- In Python 3.17: the module will be removed from the standard library. |
| 100 | + |
| 101 | +From Python 3.15, :mod:`!profilers.tracing` will be preferred to :mod:`!cProfile` & :mod:`!profile`. |
| 102 | + |
| 103 | +The code that as the time of writing this is in :mod:`!profile.sampling` |
| 104 | +module no longer exist and will only exist in the :mod:`!profilers` package. |
| 105 | + |
| 106 | +Documentation |
| 107 | +============= |
| 108 | + |
| 109 | +The Python documentation will use the new :mod:`!profilers` module as the canonical |
| 110 | +entry point for profiling functionality. It will also describe the distinction between |
| 111 | +tracing and sampling profilers, and include guidance on when each type is most appropriate. |
| 112 | + |
| 113 | +Documentation for ``cProfile`` and ``profile.sample`` will remain available but will link to |
| 114 | +the new ``profilers`` equivalents. |
| 115 | + |
| 116 | +Backwards Compatibility |
| 117 | +======================= |
| 118 | + |
| 119 | +The only backwards incompatible aspect of this PEP is the future removal of the ``profile`` module |
| 120 | +but this will be made following the :pep:`387` procedure. |
| 121 | + |
| 122 | +Security Implications |
| 123 | +===================== |
| 124 | + |
| 125 | +None. |
| 126 | + |
| 127 | +Rejected Alternatives |
| 128 | +===================== |
| 129 | + |
| 130 | +Renaming ``cProfile`` |
| 131 | +--------------------- |
| 132 | + |
| 133 | +Renaming ``cProfile`` to ``profile.tracing`` was considered, but this change would impact a |
| 134 | +large amount of existing code. Maintaining the original name while aliasing it under |
| 135 | +``profilers.tracing`` strikes a balance between compatibility and clarity. |
| 136 | + |
| 137 | +Top-Level ``tachyon`` Module |
| 138 | +---------------------------- |
| 139 | + |
| 140 | +Introducing ``import tachyon`` as a new top-level module was rejected. Grouping tachyon under |
| 141 | +``profilers`` helps establish a logical structure and prevents proliferation of top-level modules |
| 142 | +and also minimizes the usage of global namespace as requested by the Python Steering council |
| 143 | + |
| 144 | +Copyright |
| 145 | +========= |
| 146 | + |
| 147 | +This document is placed in the public domain or under the CC0-1.0-Universal |
| 148 | +license, whichever is more permissive. |
0 commit comments