|
| 1 | +- Feature Name: precise-pre-release-cargo-update |
| 2 | +- Start Date: 2023-09-20 |
| 3 | +- RFC PR: [rust-lang/rfcs#3493](https://github.com/rust-lang/rfcs/pull/3493) |
| 4 | +- Tracking Issue: [rust-lang/cargo#13290](https://github.com/rust-lang/cargo/issues/13290) |
| 5 | + |
| 6 | +# Summary |
| 7 | +[summary]: #summary |
| 8 | + |
| 9 | +This RFC proposes extending `cargo update` to allow updates to pre-release versions when requested with `--precise`. |
| 10 | +For example, a `cargo` user would be able to call `cargo update -p dep --precise 0.1.1-pre.0` as long as the version of `dep` requested by their project and its dependencies are semver compatible with `0.1.1`. |
| 11 | +This effectively splits the notion of compatibility in `cargo`. |
| 12 | +A pre-release version may be considered compatible when the version is explicitly requested with `--precise`. |
| 13 | +Cargo will not automatically select that version via a basic `cargo update`. |
| 14 | + |
| 15 | +One way to think of this is that we are changing from the version |
| 16 | +requirements syntax requiring opt-in to match pre-release of higher versions to |
| 17 | +the resolver ignoring pre-releases like yanked packages, with an override flag. |
| 18 | + |
| 19 | +# Motivation |
| 20 | +[motivation]: #motivation |
| 21 | + |
| 22 | +Today, version requirements ignore pre-release versions by default, |
| 23 | +so `1.0.0` cannot be used with `1.1.0-alpha.1`. |
| 24 | +Specifying a pre-release in a version requirement has two affects |
| 25 | +- Specifies the minimum compatible pre-release. |
| 26 | +- Opts-in to matching version requirements (within a version) |
| 27 | + |
| 28 | +However, coupling these concerns makes it difficult to try out pre-releases |
| 29 | +because every dependency in the tree has to opt-in. |
| 30 | +For example, a maintainer asks one of their users to try new API additions in |
| 31 | +`dep = "0.1.1-pre.0"` in a large project so that the user can give feedback on |
| 32 | +the release before the maintainer stabilises the new parts of the API. |
| 33 | +Unfortunately, since `dep = "0.1.0"` is a transitive dependency of several dependencies of the large project, `cargo` refuses the upgrade, stating that `0.1.1-pre.0` is incompatible with `0.1.0`. |
| 34 | +The user is left with no upgrade path to the pre-release unless they are able to convince all of their transitive uses of `dep` to release pre-releases of their own. |
| 35 | + |
| 36 | +# Guide-level explanation |
| 37 | +[guide-level-explanation]: #guide-level-explanation |
| 38 | + |
| 39 | +When Cargo considers `Cargo.toml` requirements for dependencies it always favours selecting stable versions over pre-release versions. |
| 40 | +When the specification is itself a pre-release version, Cargo will always select a pre-release. |
| 41 | +Cargo is unable to resolve a project with a `Cargo.toml` specification for a pre-release version if any of its dependencies request a stable release. |
| 42 | + |
| 43 | +If a user does want to select a pre-release version they are able to do so by explicitly requesting Cargo to update to that version. |
| 44 | +This is done by passing the `--precise` flag to Cargo. |
| 45 | +Cargo will refuse to select pre-release versions that are "incompatible" with the requirement in the projects `Cargo.toml`. |
| 46 | +A pre-release version is considered compatible for a precise upgrade if its major, minor, and patch versions are compatible with the requirement, ignoring its pre-release version. |
| 47 | +`x.y.z-pre.0` is considered compatible with `a.b.c` when requested `--precise`ly if `x.y.z` is semver compatible with `a.b.c` and `a.b.c` `!=` `x.y.z`. |
| 48 | + |
| 49 | +Consider a `Cargo.toml` with this `[dependencies]` section |
| 50 | + |
| 51 | +``` |
| 52 | +[dependencies] |
| 53 | +example = "1.0.0" |
| 54 | +``` |
| 55 | + |
| 56 | +It is possible to update to `1.2.0-pre.0` because `1.2.0` is semver compatible with `1.0.0` |
| 57 | + |
| 58 | +``` |
| 59 | +> cargo update -p example --precise 1.2.0-pre.0 |
| 60 | + Updating crates.io index |
| 61 | + Updating example v1.0.0 -> v1.2.0-pre.0 |
| 62 | +``` |
| 63 | + |
| 64 | +It is not possible to update to `2.0.0-pre.0` because `2.0.0` is not semver compatible with `1.0.0` |
| 65 | + |
| 66 | +``` |
| 67 | +> cargo update -p example --precise 2.0.0-pre.0 |
| 68 | + Updating crates.io index |
| 69 | +error: failed to select a version for the requirement `example = "^1"` |
| 70 | +candidate versions found which didn't match: 2.0.0-pre.0 |
| 71 | +location searched: crates.io index |
| 72 | +required by package `tmp-oyyzsf v0.1.0 (/home/ethan/.cache/cargo-temp/tmp-OYyZsF)` |
| 73 | +``` |
| 74 | + |
| 75 | +| Cargo.toml | Cargo.lock | Desired | Selectable w/o `--precise` | Selectable w/ `--precise` | |
| 76 | +| ---------------| --------------| --------------| -------------------------- | ---------------------------| |
| 77 | +| `^1.0.0` | `1.0.0` | `1.0.0-pre.0` | ❌ | ❌ | |
| 78 | +| `^1.0.0` | `1.0.0` | `1.2.0` | ✅ | ✅ | |
| 79 | +| `^1.0.0` | `1.0.0` | `1.2.0-pre.0` | ❌ | ✅ | |
| 80 | +| `^1.0.0` | `1.2.0-pre.0` | `1.2.0-pre.1` | ❌ | ✅ | |
| 81 | +| `^1.2.0-pre.0` | `1.2.0-pre.0` | `1.2.0-pre.1` | ✅¹ | ✅ | |
| 82 | +| `^1.2.0-pre.0` | `1.2.0-pre.0` | `1.3.0` | ✅¹ | ✅ | |
| 83 | + |
| 84 | +✅: Will upgrade |
| 85 | + |
| 86 | +❌: Will not upgrade |
| 87 | + |
| 88 | +¹This behaviour is considered by some to be undesirable and may change as proposed in [RFC: Precise Pre-release Deps](https://github.com/rust-lang/rfcs/pull/3263). |
| 89 | +This RFC preserves this behaviour to remain backwards compatible. |
| 90 | +Since this RFC is concerned with the behaviour of `cargo update --precise` changes to bare `cargo update` made in future RFCs should have no impact on this proposal. |
| 91 | + |
| 92 | +# Reference-level explanation |
| 93 | +[reference-level-explanation]: #reference-level-explanation |
| 94 | + |
| 95 | +## Version requirements |
| 96 | + |
| 97 | +Version requirement operator semantics will change to encompass pre-release versions, compared to before where the presence of pre-release would change matching modes. |
| 98 | +This will also better align with the mathematical properties associated with some of the operators (see the closed [RFC 3266](https://github.com/rust-lang/rfcs/pull/3266)). |
| 99 | + |
| 100 | +So before, |
| 101 | +``` |
| 102 | +1.2.3 -> ^1.2.3 -> >=1.2.3, <2.0.0 (with implicit holes excluding pre-release versions) |
| 103 | +``` |
| 104 | +would become |
| 105 | +``` |
| 106 | +1.2.3 -> ^1.2.3 -> >=1.2.3, <2.0.0-0 |
| 107 | +``` |
| 108 | +Note that the old syntax implicitly excluded `2.0.0-<prerelease>` which we have have to explicitly exclude by referencing the smallest possible pre-release version of `-0`. |
| 109 | + |
| 110 | +This change applies to all operators. |
| 111 | + |
| 112 | +## Dependency Resolution |
| 113 | + |
| 114 | +The intent is to mirror the behavior of yanked today. |
| 115 | + |
| 116 | +When parsing a `Cargo.lock`, any pre-release version would be tracked in an allow-list. |
| 117 | +When resolving, we would exclude from consideration any pre-release version unless: |
| 118 | +- It is in the allow-list |
| 119 | +- It matches the version requirement under the old pre-release version requirement semantics. |
| 120 | + |
| 121 | +## `cargo update` |
| 122 | + |
| 123 | +The version passed in via `--precise` would be added to the allow-list. |
| 124 | + |
| 125 | +**Note:** overriding of yanked via this mechanism is not meant to be assumed to be a part of this proposal. |
| 126 | +Support for selecting yanked with `--precise` should be decided separately from this RFC, instead see [rust-lang/cargo#4225](https://github.com/rust-lang/cargo/issues/4225) |
| 127 | + |
| 128 | +## [`semver`](https://crates.io/crates/semver) |
| 129 | + |
| 130 | +`cargo` will need both the old and new behavior exposed. |
| 131 | +To reduce risk of tools in the ecosystem unintentionally matching pre-releases (despite them still needing an opt-in), |
| 132 | +it might be reasonable for the |
| 133 | +`semver` |
| 134 | +package to offer this new matching behavior under a different name |
| 135 | +(e.g. `VersionReq::matches_prerelease` in contrast to the existing `VersionReq::matches`) |
| 136 | +(also avoiding a breaking change). |
| 137 | +However, we leave the exact API details to the maintainer of the `semver` package. |
| 138 | + |
| 139 | +# Drawbacks |
| 140 | +[drawbacks]: #drawbacks |
| 141 | + |
| 142 | +- Pre-release versions are not easily auditable when they are only specified in the lock file. |
| 143 | + A change that makes use of a pre-release version may not be noticed during code review as reviewers don't always check for changes in the lock file. |
| 144 | +- Library crates that require a pre-release version are not well supported since their lock files are ignored by their users (see [future-possibilities]) |
| 145 | +- This is an invasive change to cargo with a significant risk for bugs. This also extends out to packages in the ecosystem that deal with dependency versions. |
| 146 | +- There is a risk that error messages from the resolver may be negatively affected and we might be limited in fixes due to the resolver's current design. |
| 147 | + |
| 148 | +# Rationale and alternatives |
| 149 | +[rationale-and-alternatives]: #rationale-and-alternatives |
| 150 | + |
| 151 | +## One-time opt-in |
| 152 | + |
| 153 | +With this proposal, pre-release is like yanked: |
| 154 | +having `foo@1.2.3-alpha.0` in your `Cargo.lock` does not implicitly mean you can use `foo@1.2.3-alpha.1`. |
| 155 | +To update within pre-releases, you'll have to use `--precise` again. |
| 156 | + |
| 157 | +Instead, the lockfile could identify that `foo@1.2.3-alpha.0` is a pre-release allow updating to any `1.2.3` pre-release. |
| 158 | +`cargo update` focuses on compatible updates and pre-releases aren't necessarily compatible with each other |
| 159 | +(see also [RFC: Precise Pre-release Deps](https://github.com/rust-lang/rfcs/pull/3263)). |
| 160 | +Alternatively, in [future-possibilities] is `cargo update -p foo --allow-prerelease` which would be an explicit way to update. |
| 161 | + |
| 162 | +## Use overrides |
| 163 | + |
| 164 | +Cargo overrides can be used instead using `[patch]`. |
| 165 | +These provide a similar experience to pre-releases, however, they require that the library's code is somehow vendored outside of the registry, usually with git. |
| 166 | +This can cause issues particularly in CI where jobs may have permission to fetch from a private registry but not from private git repositories. |
| 167 | +Resolving issues around not being able to fetch pre-releases from the registry usually wastes a significant amount of time. |
| 168 | + |
| 169 | +## Extend `[patch]` |
| 170 | + |
| 171 | +It could be possible to build upon `[patch]` to [allow it to use crates published in the registry](https://github.com/rust-lang/cargo/issues/9227). |
| 172 | +This could be combined with [version overrides](https://github.com/rust-lang/cargo/issues/5640) to pretend that the pre-release crate is a stable version. |
| 173 | + |
| 174 | +My concern with this approach is that it doesn't introduce the concept of compatible pre-releases. |
| 175 | +This would allow any version to masquerade as another. |
| 176 | +Without the concept of compatible pre-releases there would be no path forward towards being able to express pre-release requirements in library crates. |
| 177 | +This is explored in [future-possibilities]. |
| 178 | + |
| 179 | +## Change the version in `Cargo.toml` rather than `Cargo.lock` when using `--precise` |
| 180 | + |
| 181 | +This [accepted proposal](https://github.com/rust-lang/cargo/issues/12425) allows cargo to update a projects `Cargo.toml` when the version is incompatible. |
| 182 | + |
| 183 | +The issue here is that cargo will not unify a pre-release version with a stable version. |
| 184 | +If the crate being updated is used pervasively this will more than likely cause a resolver error. |
| 185 | +This makes this alternative unfit for our [motivation]. |
| 186 | + |
| 187 | +The [accepted proposal](https://github.com/rust-lang/cargo/issues/12425) is affected by this RFC, |
| 188 | +insofar as it will not update the `Cargo.toml` in cases when the pre-release can be considered compatible for upgrade in `Cargo.lock`. |
| 189 | + |
| 190 | +## Pre-releases in `Cargo.toml` |
| 191 | + |
| 192 | +Another alternative would be to resolve pre-release versions in `Cargo.toml`s even when another dependency specifies a stable version. |
| 193 | +This is explored in [future-possibilities]. |
| 194 | +This would require significant changes to the resolver since the latest compatible version would depend on the versions required by other parts of the dependency tree. |
| 195 | +This RFC may be a stepping stone in that direction since it lays the groundwork for pre-release compatibility rules, however, I consider detailing such a change outside of the scope of this RFC. |
| 196 | + |
| 197 | +# Prior art |
| 198 | +[prior-art]: #prior-art |
| 199 | + |
| 200 | +[RFC: Precise Pre-release Deps](https://github.com/rust-lang/rfcs/pull/3263) aims to solve a similar but different issue where `cargo update` opts to upgrade |
| 201 | +pre-release versions to new pre-releases when one is released. |
| 202 | + |
| 203 | +Implementation-wise, this is very similar to how yanked packages work. |
| 204 | +- Not selected under normal conditions |
| 205 | +- Once its in the lockfile, that gets respected and stays in the lockfile |
| 206 | + |
| 207 | +The only difference being that `--precise` does not allow overriding the "ignore yank" behavior |
| 208 | +(though [it is desired by some](https://github.com/rust-lang/cargo/issues/4225)). |
| 209 | + |
| 210 | +For `--precise` forcing a version through, we have precedence in |
| 211 | +[an approved-but-not-implemented proposal](https://github.com/rust-lang/cargo/issues/12425) |
| 212 | +for `cargo update --precise` for incompatible versions to force its way |
| 213 | +through by modifying `Cargo.toml`. |
| 214 | + |
| 215 | +# Unresolved questions |
| 216 | +[unresolved-questions]: #unresolved-questions |
| 217 | + |
| 218 | +# Version ranges with pre-release upper bounds |
| 219 | + |
| 220 | +[As stated earlier](#reference-level-explanation), this RFC proposes that semver version semantics change to encompass pre-release versions. |
| 221 | + |
| 222 | +``` |
| 223 | +^1.2.3 -> >=1.2.3, <2.0.0-0 |
| 224 | +``` |
| 225 | + |
| 226 | +The addition of an implicit `-0` excludes `2.0.0-<prerelease>` releases. |
| 227 | +This transformation will also be made when the user explicitly specifies a multiple-version requirement range. |
| 228 | + |
| 229 | +``` |
| 230 | +>=1.2.3, <2.0.0 -> >=1.2.3, <2.0.0-0 |
| 231 | +``` |
| 232 | + |
| 233 | +This leaves a corner case for `>=0.14-0, <0.14.0`. |
| 234 | +Intuitively the user may expect this range to match all `0.14` pre-releases, however, due to the implicit `-0` on the second version requirement, this range will not match any versions. |
| 235 | +There are two ways we could go about solving this. |
| 236 | + |
| 237 | +- Only add the implicit `-0` to the upper bound if there is no pre-release on the lower bound. |
| 238 | +- Accept the existence of this unexpected behaviour. |
| 239 | + |
| 240 | +Either way, it may be desirable to introduce a dedicated warning for this case. |
| 241 | + |
| 242 | +# Future possibilities |
| 243 | +[future-possibilities]: #future-possibilities |
| 244 | + |
| 245 | +## Pre-release dep "allows" pre-release everywhere |
| 246 | + |
| 247 | +It would be nice if cargo could unify pre-release version requirements with stable versions. |
| 248 | + |
| 249 | +Take for example this dependency tree. |
| 250 | + |
| 251 | +``` |
| 252 | +example |
| 253 | +├── a ^0.1.0 |
| 254 | +│ └── b =0.1.1-pre.0 |
| 255 | +└── b ^0.1.0 |
| 256 | +``` |
| 257 | + |
| 258 | +Since crates ignore the lock files of their dependencies there is no way for `a` to communicate with `example` that it requires features from `b = 0.1.1-pre.0` without breaking `example`'s direct dependency on `b`. |
| 259 | +To enable this we could use the same concept of compatible pre-releases in `Cargo.toml`, not just `Cargo.lock`. |
| 260 | +This would require that pre-releases are specified with `=` and would allow pre-release versions to be requested anywhere within the dependency tree without causing the resolver to throw an error. |
| 261 | + |
| 262 | +## `--allow-prerelease` |
| 263 | + |
| 264 | +Instead of manually selecting a version with `--precise`, we could support `cargo update --package foo --allow-prerelease`. |
| 265 | + |
| 266 | +If we made this flag work without `--package`, we could the extend it also to `cargo generate-lockfile`. |
0 commit comments