Skip to content

Commit c0d183c

Browse files
jdnojdno
committed
Document the new release process for Rustup
The Rustup release process has historically been a manual process that involved copying files from S3 to the local machine and back to S3. This introduced a high risk of human error. When modifications to the existing release script became necessary, the decision was made to automate the release process (see rust-lang#3819 for details). The documentation in the dev-guide has been updated to cover the new release process, which is fully automated to produce `beta` releases using GitHub Actions and the [promote-release] tooling. [promote-release]: https://github.com/rust-lang/promote-release
1 parent 66b7a44 commit c0d183c

File tree

1 file changed

+207
-54
lines changed

1 file changed

+207
-54
lines changed

‎doc/dev-guide/src/release-process.md

Lines changed: 207 additions & 54 deletions
Original file line numberDiff line numberDiff line change
@@ -1,66 +1,219 @@
1-
# Making a release
1+
# Release Process
22

3-
Before making a release, ensure that `rustup-init.sh` is behaving correctly,
3+
This document describes the process for making a new release of Rustup. It is
4+
split into the two sections. The first section explains the release process,
5+
while the second section documents for maintainers how to make a release.
6+
7+
- [About the Release Process](#about-the-release-process)
8+
- [Historic Context](#historic-context)
9+
- [Beta and Stable Releases](#beta-and-stable-releases)
10+
- [Release Artifacts](#release-artifacts)
11+
- [Automation](#automation)
12+
- [How to Cut a Release](#how-to-cut-a-release)
13+
- [1. Test `rustup-init.sh`](#1-test-rustup-initsh)
14+
- [2. Update Version](#2-update-version)
15+
- [3. Update Checksum](#3-update-checksum)
16+
- [4. Sync `master` to `stable`](#4-sync-master-to-stable)
17+
- [5. Test the `beta` Release](#5-test-the-beta-release)
18+
- [6. Prepare Blog Post for `stable` Release](#6-prepare-blog-post-for-stable-release)
19+
- [7. Request the `stable` Release](#7-request-the-stable-release)
20+
- [8. Tag the `stable` Release](#8-tag-the-stable-release)
21+
22+
## About the Release Process
23+
24+
This section explains the release process for Rustup.
25+
26+
### Historic Context
27+
28+
The Rustup release process has evolved over time. In the past, the process
29+
involved more manual steps and large parts of it were executed locally. This
30+
required careful execution and coordination to avoid accidentally overwriting
31+
exiting releases, since the tooling did not implement sanity checks that would
32+
prevent destructive actions.
33+
34+
A full step-by-step guide for the old process can be found in
35+
the [old developer guide](https://github.com/rust-lang/rustup/blob/1cf1b5a6d80c978e0dcaabbce5f10b3861612425/doc/dev-guide/src/release-process.md).
36+
The following summarizes how release artifacts were created and copied around.
37+
38+
In the past, both beta and stable releases were started by merging a new commit
39+
into the `stable` branch in [rust-lang/rustup]. This started a new build on
40+
GitHub Actions that produced release
41+
artifacts ([source](https://github.com/rust-lang/rustup/blob/1cf1b5a6d80c978e0dcaabbce5f10b3861612425/.github/workflows/ci.yaml#L144-L151)),
42+
which were uploaded to `s3://dev-static-rust-lang-org/rustup/dist`. As new
43+
commits were merged into `stable`, they would overwrite the artifacts from prior
44+
builds.
45+
46+
The release artifacts were then copied to the final location by running a script
47+
on a local machine. This script would download the artifacts from the
48+
`dev-static` bucket on S3 and upload them to the `static` bucket. The script
49+
also generated a new manifest with the given version, and uploaded a copy of the
50+
files to an archive directory.
51+
52+
Given that the process was manual, involved copying files to the local machine,
53+
and waiting between creating a `beta` and a `stable` release, there was a risk
54+
of human error. When we had to update the script after four years without a
55+
change, we [decided](https://github.com/rust-lang/rustup/pull/3819) to redesign
56+
and automate the release process.
57+
58+
### Beta and Stable Releases
59+
60+
Rustup can be released to two different environments: `beta` and `stable`. The
61+
main difference between the two is that they use different values for the
62+
`RUSTUP_UPDATE_ROOT` environment variable:
63+
64+
- A beta release is deployed on `https://dev-static.rust-lang.org/rustup`.
65+
- An official release is deployed on `https://static.rust-lang.org/rustup`.
66+
67+
By switching between those two values, Rustup effectively provides two
68+
"self update channels", making beta testing possible with `rustup self update`.
69+
70+
### Release Artifacts
71+
72+
The release artifacts are produced by CI when a new commit is merged into the
73+
`stable` branch, and they are uploaded to the `dev-static` bucket on S3. There,
74+
they are put into a folder named after their commit hash, for example the
75+
artifacts for commit `1cf1b5a` would be uploaded to
76+
`s3://dev-static-rust-lang-org/rustup/builds/1cf1b5a`.
77+
78+
When a new `beta` release is cut, the artifacts are copied to two new locations
79+
within the same bucket:
80+
81+
- One copy is put into an archive named after the version, e.g.
82+
`/rustup/archive/1.0.0`.
83+
- Another copy is put into the `/rustup/dist` folder, which is where clients
84+
look for new versions.
85+
86+
When a new `stable` release is cut, the artifacts are copied to the `static`
87+
bucket on S3, following the same process as the `beta` release:
88+
89+
- One copy is archived into `/rustup/archive/1.0.0`.
90+
- Another copy is put into `/rustup/dist`.
91+
92+
This ensures backwards compatibility with the old release process, while also
93+
ensuring that release artifacts are not overwritten by new builds.
94+
95+
### Automation
96+
97+
The interaction with the release artifacts is fully automated.
98+
99+
First, artifacts are produced automatically by
100+
the [`CI`](https://github.com/rust-lang/rustup/blob/master/.github/workflows/ci.yaml)
101+
job on GitHub Actions when a new commit is merged into the `stable` branch, and
102+
are then automatically uploaded to their respective S3 bucket by the action as
103+
well.
104+
105+
Second, when making a release, the artifacts are copied to their final locations
106+
by the [promote-release] tool. This reduces the risk of human error and ensures
107+
that the release process is consistent and reliable. The tool is also run in a
108+
secure environment on AWS CodeBuild, reducing the risk of leaking sensitive
109+
credentials that would give write access (past) releases.
110+
111+
For a `beta` release, `promote-release` performs the following actions:
112+
113+
1. Query GitHub's API to get the latest commit on the `stable` branch
114+
2. Confirm that `/rustup/builds/${commit}` exists in the `dev-static` bucket
115+
3. Get the new version number from the `stable` branch
116+
1. Download `Cargo.toml` from `stable`
117+
2. Parse the file and read the `version` field
118+
4. Confirm that `/rustup/archive/${version}` does not exist yet
119+
5. Copy `/rustup/builds/${commit}` to `/rustup/archive/${version}`
120+
6. Copy `/rustup/builds/${commit}` to `/rustup/dist`
121+
7. Generate a new manifest and upload it to `/rustup/dist`
122+
123+
For a new `stable` release, the process is the same. The only difference is that
124+
the steps 4-6 copy the artifacts to the `static` bucket.
125+
126+
## How to Cut a Release
127+
128+
This section documents the steps that a maintainer should follow to cut a new
129+
release of Rustup.
130+
131+
### 1. Test `rustup-init.sh`
132+
133+
Before cutting a release, ensure that `rustup-init.sh` is behaving correctly,
4134
and that you're satisfied that nothing in the ecosystem is breaking because
5135
of the update. A useful set of things to check includes verifying that
6136
real-world toolchains install okay, and that `rust-analyzer` isn't broken by
7137
the release. While it's not our responsibility if they depend on non-stable
8138
APIs, we should behave well if we can.
9139

10-
As a maintainer, you have two options to choose from when cutting a new
11-
release: a beta release or an official release.
12-
The main difference between the two is that they use different values for
13-
the `RUSTUP_UPDATE_ROOT` environment variable:
14-
- A beta release is deployed on `https://dev-static.rust-lang.org/rustup`.
15-
- An official release is deployed on `https://static.rust-lang.org/rustup`.
140+
### 2. Update Version
16141

17-
By switching between those two values, Rustup effectively provides two "self
18-
update channels", making beta testing possible with `rustup self update`.
19-
20-
Producing the final release artifacts is a bit involved because of the way
21-
Rustup is distributed.
22-
Below is a list of things to be done in order to cut a new [b]eta release
23-
or an official [r]elease:
24-
25-
1. [b/r] In a separate PR:
26-
1. If the version strings in `Cargo.toml`s haven't been updated:
27-
- Decide what the new version number `$VER_NUM` should be.
28-
> **Note:** We always increment the *minor* number unless:
29-
> - A major incompatibility has been introduced in this release:
30-
> increment the *major* number instead.
31-
> - This release is a hotfix because the last one had a defect:
32-
> increment the *patch* number instead.
33-
- Update `Cargo.toml` and `download/Cargo.toml` to have that same new
34-
version number, then run `cargo build` and review `Cargo.lock` changes.
142+
The release process gets metadata from the `Cargo.toml` file, so ensure that
143+
the version number in `Cargo.toml` is correct.
144+
145+
1. If the version strings in `Cargo.toml`s haven't been updated:
146+
- Decide what the new version number `$VER_NUM` should be.
147+
> **Note:** We always increment the *minor* number unless:
148+
> - A major incompatibility has been introduced in this release:
149+
> increment the *major* number instead.
150+
> - This release is a hotfix because the last one had a defect:
151+
> increment the *patch* number instead.
152+
- Update `Cargo.toml` and `download/Cargo.toml` to have that same new
153+
version number, then run `cargo build` and review `Cargo.lock` changes.
35154
If all looks well, make a commit.
36-
2. Update `CHANGELOG.md` accordingly if necessary.
37-
2. [b/r] After merging the PR made in step 1, in a separate PR:
38-
1. Update the commit shasum in `rustup-init.sh` to match the latest commit
39-
on `master`.
40-
3. [b/r] After merging the PR made in step 2, sync `master` to `stable` using
41-
`--ff-only`:
42-
- `git fetch origin master:master`
43-
- `git checkout stable && git merge --ff-only master`
44-
- `git push origin HEAD:stable`
45-
4. [b/r] While you wait for green CI on `stable`, double-check the
46-
functionality of `rustup-init.sh` and `rustup-init` just in case.
47-
5. [b/r] Ensure all of CI is green on the `stable` branch.
48-
Once it is, check through a representative proportion of the builds looking
49-
for the reported version statements to ensure that
50-
we definitely built something cleanly which reports as the right version
51-
number when run `--version`.
52-
6. [r] Make a new PR to the [Rust Blog] adding a new release announcement post.
53-
7. [b/r] Ping someone in the release team to perform the actual release.
54-
They can find instructions in `ci/sync-dist.py`.
55-
> **Note:** Some manual testing occurs here, so hopefully they'll catch
56-
anything egregious in which case abort the change and roll back.
57-
8. [b] Once the beta release has happened, post a new topic named "Seeking beta
58-
testers for Rustup $VER_NUM" on the [Internals Forum].
59-
9. [r] Once the official release has happened, prepare and push a tag on the
60-
latest `stable` commit.
61-
- `git tag -as $VER_NUM -m $VER_NUM` (optionally without `-s` if not GPG
62-
signing the tag)
63-
- `git push origin $VER_NUM`
155+
2. Update `CHANGELOG.md` accordingly if necessary.
156+
157+
Submit the changes in a PR and merge it.
158+
159+
### 3. Update Checksum
160+
161+
After merging the PR in the previous step, update the commit SHA checksum in
162+
`rustup-init.sh` to match the latest commit on `master`. Submit the change in a
163+
PR and merge it.
164+
165+
### 4. Sync `master` to `stable`
166+
167+
After merging the PR in the previous step, sync `master` to `stable` using
168+
`--ff-only`:
169+
170+
```shell
171+
git fetch origin master:master
172+
git checkout stable && git merge --ff-only master
173+
git push origin HEAD:stable
174+
```
175+
176+
This will kick off a new build on GitHub Actions, which will produce the release
177+
artifacts, upload them to `dev-static`, and make the new `beta` release
178+
available at `RUSTUP_UPDATE_ROOT=https://dev-static.rust-lang.org/rustup`.
179+
180+
### 5. Test the `beta` Release
181+
182+
While you wait for green CI on `stable`, double-check the functionality of
183+
`rustup-init.sh` and `rustup-init` just in case.
184+
185+
Ensure all of CI is green on the `stable` branch. Once it is, check through a
186+
representative proportion of the builds looking for the reported version
187+
statements to ensure that we definitely built something cleanly which reports as
188+
the right version number when run `--version`.
189+
190+
Once the beta release has happened, post a new topic named "Seeking beta testers
191+
for Rustup $VER_NUM" on the [Internals Forum].
192+
193+
### 6. Prepare Blog Post for `stable` Release
194+
195+
Make a new PR to the [Rust Blog] adding a new release announcement post.
196+
197+
### 7. Request the `stable` Release
198+
199+
Ping someone in the release team to perform the `stable` release.
200+
201+
They will have to start a new CodeBuild job on AWS to run the `promote-release`
202+
tool for Rustup.
203+
204+
### 8. Tag the `stable` Release
205+
206+
Once the official release has happened, prepare and push a tag on the latest
207+
`stable` commit.
208+
209+
- `git tag -as $VER_NUM -m $VER_NUM` (optionally without `-s` if not GPG
210+
signing the tag)
211+
- `git push origin $VER_NUM`
212+
213+
[promote-release]: https://github.com/rust-lang/promote-release
214+
215+
[rust-lang/rustup]:https://github.com/rust-lang/rustup
64216

65217
[Rust Blog]: https://github.com/rust-lang/blog.rust-lang.org
218+
66219
[Internals Forum]: https://internals.rust-lang.org

0 commit comments

Comments
 (0)