Skip to content

Conversation

samypr100
Copy link
Collaborator

@samypr100 samypr100 commented Oct 6, 2025

Summary

Follow up to #15563
Closes #13485

This is a first-pass at adding support for conditional support for Git LFS between git sources, initial feedback welcome.

e.g.

[tool.uv.sources]
test-lfs-repo = { git = "https://github.com/zanieb/test-lfs-repo.git", lfs = true }

For context previously a user had to set UV_GIT_LFS to have uv fetch lfs objects on git sources. This env var was all or nothing, meaning you must always have it set to get consistent behavior and it applied to all git sources. If you fetched lfs objects at a revision and then turned off lfs (or vice versa), the git db, corresponding checkout lfs artifacts would not be updated properly. Similarly, when git source distributions were built, there would be no distinction between sources with lfs and without lfs. Hence, it could corrupt the git, sdist, and archive caches.

In order to support some sources being LFS enabled and other not, this PR adds a stateful layer roughly similar to how subdirectory works but for lfs since the git database, the checkouts and the corresponding caching layers needed to be LFS aware (requested vs installed). The caches also had to isolated and treated entirely separate when handling LFS sources.

Summary

  • Adds lfs = true or lfs = false (default) to git sources in pyproject.toml
  • Added lfs=true query param / fragments to most relevant url structs
    • In the case of uv add, --lfs is supported instead
  • direct-url.json now has an custom git_lfs entry under VcsInfo (note, this is not in the spec currently -- see caveats).
  • git database and checkouts have and isolated _lfs suffix identity as the sources should be treated effectively different for the same rev.
  • sdists cache also has the "_lfs" suffix to the directory of a built distribution if it was built using LFS enabled revisions to distinguish between non-LFS same revisions. This ensures the strong assumption for archive-v0 that an unpacked revision "doesn't change sources" stays valid.

Caveats

  • pylock.toml import support has not been added via lfs=true, going through the spec it wasn't clear to me it's something we'd support outside of the env var (for now).
  • direct-url struct was modified by adding a non-standard git_lfs field under VcsInfo which may be undersirable although the PEP 610 does say Additional fields that would be necessary to support such VCS SHOULD be prefixed with the VCS command name which could be interpret this change as ok.
  • There will be a slight lockfile and cache churn for users that use UV_GIT_LFS as all git lockfile entries will get a lfs=true fragment. The cache version does not need an update, but LFS sources will get their own namespace under git-v0 and sdist-v9/git hence a cache-miss will occur once.

Test Plan

Some initial tests were added. More tests likely to follow as we reach consensus on a final approach. Some existing tests still need to be updated.

For IT test, a repo under astral with LFS enabled would be great. I can keep using the one from @zanieb, but we'll want more LFS commits to test changing commits with different LFS artifacts.

Manual testing was done for common pathological cases like killing LFS fetch mid-way, uninstalling LFS after installing an sdist with it and reinstalling, fetching LFS artifacts in different commits, etc.

@samypr100 samypr100 force-pushed the conditional_lfs_support branch 5 times, most recently from 5960db0 to 3702bfa Compare October 12, 2025 20:16
@samypr100 samypr100 force-pushed the conditional_lfs_support branch 2 times, most recently from 6580395 to b34deec Compare October 13, 2025 19:01
@samypr100 samypr100 force-pushed the conditional_lfs_support branch from b34deec to 52cbd5c Compare October 13, 2025 19:37
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

None yet

Projects

None yet

Development

Successfully merging this pull request may close these issues.

Make UV_GIT_LFS active by default on some url

2 participants