Profiling and Optimizing Mypy

Overview

The performance of mypy is important, and we appreciate contributions that improve the performance of mypy, either overall, or in specific use cases. Contributions that regress performance significantly may be rejected. Generally if a new feature or fix regresses performance by over 1%, it may be a cause of concern, especially if this is a relatively minor change.

Mypy is compiled using mypyc to C extension modules. This means that using the stdlib cProfile or profile module isn't going to be effective, unless you use a non-compiled mypy -- and in this case the results may not be indicative of performance when using a compiled mypy. Also when measuring performance improvements, it's important to use a compiled mypy. All of this is documented below.

Tracking the performance of mypy

We have a daily job that measures the time needed to type check an older version of mypy: mypy self check benchmark results. The level of noise in the results is about 0.5%, but improvements or regressions of 1% or more tend to be visible in the results.

If the above data hasn't been updated for some time but there have been recent commits to the mypy repository, feel free to create a mypy GitHub issue about the benchmark job being potentially down.

Measuring the performance of mypy locally

Use misc/perf_compare.py to measure the performance impact of a branch or commit. For example, if you have some optimization in branch my-opt, you can measure the performance like this:

python misc/perf_compare.py master my-opt

A change of 0.5% or less can be noise. You can try using --num-runs 100 or something to improve precision, but even this isn't reliable, since trivial changes to the codebase can cause a repeatable 0.5% deviation in results.

To measure smaller improvements, you have a few options:

• Rebase your branch on top of 3-4 master commits from the last week or two and measure each separately. If you see a small performance improvement consistently, it's likely that it's real.
• Implement 2 or more optimizations in the same branch, until the overall performance impact is more than 0.5% (preferably 1.0% or more). Then you can create a PR with multiple optimizations, or create separate PRs but mention in each that you've measured the impact of a set of optimizations together.
• Use "trace logging" (discussed below) to measure the number of certain operations/events precisely. This doesn't help with all kinds of optimizations, but for certain micro-optimizations this can reliably show even quite small improvements in a convincing way.

Using perf to profile compiled mypy on Linux

TODO

Using event trace logging to analyze performance

TODO

Using py-spy for profiling compiled mypy

Note

As of Jul 2025, running py-spy on Python 3.12 and later may not work reliably. Python 3.11 can be used as a workaround. You can also try perf on Linux (discussed elsewhere).

py-spy is a profiling tool that works with compiled mypy (at least in Linux).

Use it like this to profile mypy (replace -c 'import os' with your command line arguments):

$ pip install py-spy
$ pip install mypy
$ py-spy record --native -f speedscope -o profile.dat -- mypy -c 'import os'

Now open https://www.speedscope.app/, click Browse and import the profile.dat file you generated above. You can click 'Sandwich' to get a flat profile.

If the mypy run is relatively quick (less than a few seconds), consider using -r 500 with py-spy to increase the sampling rate (but high sample rates may not work reliably).

Note: To get repeatable results, disable incremental mode by using mypy --no-incremental, or delete the .mypy_cache directory before each run.

If you are reporting a mypy performance issue or regression, feel free to add a link to the collected profile.dat (after you've verified with speedscope that it contains useful information) so that mypy developers can also analyze the profile.