Wheel cache concurrency issue when temp dir is not on the same filesystem as the cache dir

[sbidoul]

Description

At work we are investigating an issue of a corrupted wheel at the end of a pip wheel operation. With the bad wheel ending up polluting the cache. Our current analysis of the problem goes like this:

When building a wheel to the cache of locally built wheels, pip builds to a temporary directory then issues a shutil.move() to move the result to the cache.

pip/src/pip/_internal/wheel_builder.py

Line 234 in b73fc04

shutil.move(wheel_path, dest_path)

When the temporary directory and the cache are on the same filesystem, all is well as shutil.move() does a os.rename() which is atomic. Otherwise it defaults to shutil.copy2() which leads to concurrency issues if two pip processes are building the same wheel concurrently.

In our case it manifests itself in concurrent Dockerfile builds on a builder machine where the pip cache is mounted with RUN --mount=type=cache.

Expected behavior

No response

pip version

any

Python version

any

OS

linux

How to Reproduce

Run two concurrent pip wheel of the same large wheel. It ends with an error like so ERROR: Wheel '***' located at /root/.cache/pip/wheels/af/d2/b2/206e3ff539bc87defafe3a4d91f689eec5bd178b6143461fa0/***-16.0-py3-none-any.whl is invalid. The underlying exception is InvalidWheel which itself comes from a BadZipFile exception.

Output

No response

Code of Conduct

• I agree to follow the PSF Code of Conduct.