How to have a full python environment?

[youkaichao]

Question

I'm installing a new environment with uv venv ~/uv_envs/test --python 3.12 , it uses my system's python as the interpreter.

However, i find that my system intepreter is not complete, it does not have /usr/include/python3.12 , but python -c "from sysconfig import get_paths; print(get_paths()['include'])" still reports that directory.

Can I somehow tell uv to download a full python environment?

I can use sudo apt install python3.12-dev as a workaround, but then the environment is not reproducible, i.e. it depends on the initial python interpreter i installed.

can i make uv take the full control, downloading a full python regardless of my initial python environment?

Platform

Ubuntu 20.04, arm64

Version

uv 0.5.11

[pjrulez]

EDIT & TLDR:
'uv self update' and then I could install the latest python 3.12 version.

I can't download python 3.12.10 with uv:
uv python install 3.12.10
error: No download found for request: cpython-3.12.10-linux-aarch64-gnu

Additionally, I'm using this image:
FROM ghcr.io/astral-sh/uv:python3.12-bookworm-slim
but it was updated from 3.12.9 which worked with my project to 3.12.10 which now doesn't work during the build process because the image now comes with 3.12.10.

For a moment I thought this was relevant. Now, I'm not so sure.

[charliermarsh]

@youkaichao -- You can try passing --python-preference only-managed -- that will force uv to use a uv-downloaded Python.

[j178]

--python-preference is now hidden from user, the new convenient equivalent is the --managed-python flag.

[youkaichao]

@charliermarsh thanks, I'm using uv venv uv_envs/test_0414 --python 3.12 --managed-python --seed now, is --managed-python similar to --python-preference only-managed ?

[youkaichao]

well, it seems it still links to ~/.local/share/uv/python .

The context here, is I'm in a cluster, which has a shared file system at /sharefs , and I want to put uv there, without storing anything locally to that node.

Somehow uv links ~/.local/share/uv/python to /sharefs/uv_envs/test_0414 , and this link is broken on another node.

[charliermarsh]

Yeah, virtual environments on Linux and macOS will always link to a Python by default (unlike on Windows). The standard library python -m venv supports a --copies flag which lets you copy the distribution, I think? See #2103.

[youkaichao]

is it possible to have a fresh new, standalone environment, without any linking?

[youkaichao]

I try to add --link-mode copy to uv venv , but the include path is still not copied into this new environment.

ultimately, what i want is sysconfig.get_paths()["include"] should only contain the files in the uv env, so that i can place the full environment in the shared filesystem, and all nodes in the cluster can use it.

[youkaichao]

after fighting for a while, i end up with this:

export ROOT_DIR=/sharedfs/somewhere
export UV_HOME=$ROOT_DIR/uv_home
export UV_INSTALL_DIR="$ROOT_DIR/uv"
export UV_CACHE_DIR="$ROOT_DIR/uv_cache"
export XDG_DATA_HOME=$ROOT_DIR
export PATH="$PATH:$UV_INSTALL_DIR"
curl -LsSf https://astral.sh/uv/install.sh | sh

uv venv $ROOT_DIR/uv_envs/test_0416 --python 3.12 --managed-python --link-mode copy --seed

and then it seems all files are in the /sharedfs now.

[liquidcarbon]

As an alternative, for the last year I've been using Pixi for base python install, and wrapping uv in pixi run uv.
This way I know exactly which python is being used, despite ongoing changes in uv.

Here's a pet project that wraps this: https://github.com/liquidcarbon/puppy