@@ -7,14 +7,142 @@ Rust toolchain (the Rust compiler, and also C++/Rust FFI tools like
77[ TOC]
88
99
10- ## Rolling Rust compiler and other Rust tools
10+ ## Background
1111
12- Steps to roll the Rust compiler (and other Rust tools like ` rustfmt ` ) to
13- a new version:
14- - Locally, update ` RUST_REVISION ` in ` update_rust.py ` .
15- (Update ` RUST_SUB_REVISION ` when the build or packaging is changed, but
16- the upstream Rust revision we build from is not changed.)
17- - Follow the general roll process below
12+ Like with Clang, Chromium uses bleeding edge Rust tooling. We track the upstream
13+ projects' latest development as closely as possible. However, Chromium cannot
14+ use official Rust builds for various reasons which require us to match the Rust
15+ LLVM backend version with the Clang we use.
16+
17+ It would not be reasonable to build the tooling for every Chromium build, so we
18+ build it centrally (with the scripts here) and distribute it for all to use
19+ (also fetched with the scripts here).
20+
21+
22+ ## Rust build overview
23+
24+ Each Rust package is built from an official Rust nightly source release and a
25+ corresponding LLVM revision. Hence a new Rust package must be built whenever
26+ either Rust or Clang is updated.
27+
28+ Chromium's Clang build process leaves behind several artifacts needed for the
29+ Rust build. Each Rust build begins after a Clang build and uses these artifacts,
30+ which include ` clang ` , LLVM libraries, etc.
31+
32+ A special CI job is used to build Clang and Rust from the revisions specified in
33+ the Chromium source tree. These are uploaded to a storage bucket. After being
34+ manually blessed by a developer, they can then be fetched by Chromium checkouts.
35+
36+ Scripts are provided in tree to fetch the packages for the specified revisions.
37+
38+ Whenever a Chromium checkout is updated, ` gclient ` hooks will update the
39+ toolchain packages to match the revisions listed in tree.
40+
41+
42+ ## Updating Rust
43+
44+ ### Set the revision
45+
46+ First, pick a new instance of the [ CIPD ` rust_src `
47+ package] ( https://chrome-infra-packages.appspot.com/p/chromium/third_party/rust_src/+/ ) .
48+ Click it and copy the version ID (e.g. ` version:2@2022-01-01 ` ) to the ` //DEPS ` entry for
49+ ` src/third_party/rust_src/src ` . For example (though don't change the other parts
50+ of the entry):
51+
52+ ```
53+ 'src/third_party/rust_src/src': {
54+ 'packages': [
55+ {
56+ 'package': 'chromium/third_party/rust_src',
57+ 'version': 'version:2@2022-01-01',
58+ },
59+ ],
60+ 'dep_type': 'cipd',
61+ 'condition': 'checkout_rust_toolchain_deps or use_rust',
62+ },
63+ ```
64+
65+ Similarly, update the ` RUST_REVISION ` named in ` //tools/rust/update_rust.py ` ,
66+ removing dashes from the date (e.g. ` version:2@2022-01-01 ` becomes
67+ ` RUST_REVISION = '20220101' ` ). Reset ` RUST_SUB_REVISION = 1 ` .
68+
69+ Run the following to check for changes to Rust's ` src/stage0.json ` , which
70+ contains revisions of upstream binaries to be fetched and used in the Rust
71+ build:
72+
73+ ```
74+ tools/rust/build_rust.py --verify-stage0-hash
75+ ```
76+
77+ If it exists without printing anything, the stage0 hash is up-to-date and
78+ nothing needs to be done. Otherwise, it will print the actual hash like so:
79+
80+ ```
81+ ...
82+ Actual hash: 6b1c61d494ad447f41c8ae3b9b3239626eecac00e0f0b793b844e0761133dc37
83+ ...
84+ ```
85+
86+ ...in which case you should check the ` stage0.json ` changes for trustworthiness
87+ (criteria TBD) and then update ` STAGE0_JSON_SHA256 ` in update_rust.py with the
88+ new hash. Re-run the above and confirm it succeeds.
89+
90+
91+ ### Optional: build locally and run tests
92+
93+ This step is not strictly necessary since the CI tooling will catch any errors.
94+ But the CI build process is slow and this can save some time.
95+
96+ Running this will do a full build and provide a local toolchain that works for
97+ the host machine, albeit not the same as the CI-provided one:
98+
99+ ```
100+ tools/rust/build_rust.py --fetch-llvm-libs --use-final-llvm-build-dir
101+ ```
102+
103+ To do a full build, first build Clang locally (TODO: provide instructions) then
104+ simply run ` tools/rust/build_rust.py ` with no arguments.
105+
106+ However, for most cases simply doing
107+
108+ ```
109+ tools/rust/build_rust.py --fetch-llvm-libs --use-final-llvm-build-dir --run-xpy -- build --stage 1 library/std
110+ ```
111+
112+ will catch most errors and will be fast.
113+
114+ ### Upload CL and build package
115+
116+ Upload a CL with the changes, which in the simplest case will only have two
117+ changes: one line in ` //DEPS ` , one line in ` //tools/rust/update_rust.py ` . Add the
118+ following line to the end of the CL description, which ensures the new toolchain
119+ will be tested on appropriate Rust tryjobs:
120+
121+ ```
122+ Cq-Include-Trybots: luci.chromium.try:android-rust-arm-dbg,android-rust-arm-rel,linux-rust-x64-dbg,linux-rust-x64-rel
123+ ```
124+
125+ From Gerrit run the ` linux_upload_clang ` tryjob on the CL and wait for it to
126+ finish. Check that it's successful; it is ** not** sufficient to check the
127+ result in Gerrit as a Rust failure will not surface here. Check the build page
128+ (e.g.
129+ https://ci.chromium.org/ui/p/chromium/builders/try/linux_upload_clang/2611/overview )
130+ and confirm the "package rust" step succeeded. If it did not, further
131+ investigation is needed.
132+
133+ After the package is built, a developer with permissions must bless the package
134+ for use. As of writing this is anyone in [ Clang
135+ OWNERS] ( /tools/clang/scripts/OWNERS ) or collinbaker@chromium.org .
136+
137+
138+ ### Submit CL
139+
140+ Once the package has been uploaded and blessed, it is ready to be fetched from
141+ any Chromium checkout.
142+
143+ Submit the CL. CQ tryjobs will use the specified toolchain package
144+ version. Any build failures will need to be investigated, as these indicate
145+ breaking changes to Rust.
18146
19147
20148## Rolling Crubit tools
@@ -35,35 +163,6 @@ to a new version:
35163 be made obsolete once Rust-specific tryjobs cover Crubit
36164 tests.
37165
38- - Follow the general roll process below
39-
40-
41- ## General roll process
42-
43- - Author a CL that updates ` CRUBIT_REVISION ` and/or ` RUST_REVISION `
44- (see one of the sections above for details).
45-
46- - Upload the CL to Gerrit.
47- Ask [ CQ] ( //docs/infra/cq.md ) for Rust-specific tryjobs in the CL description:
48- ` Cq-Include-Trybots: luci.chromium.try:linux-rust-x64-rel,linux-rust-x64-dbg,android-rust-arm-rel,android-rust-arm-dbg ` .
49-
50- - Run ` linux_upload_clang ` tryjob. This will run ` package_rust.py ` which will
51- ` build_rust.py ` and then upload a new version of the Rust toolchain package to
52- the staging bucket. This step runs ` build_crubit.py ` but doesn't (yet) package
53- the built binaries.
54- TODO(https://crbug.com/1329611 ): Update the docs once Crubit also gets packaged.
55-
56- - Move the new toolchain package from staging to prod.
57- (A small set of people have the permission for this. There are
58- some Google-internal docs with more details.)
59-
60- - Run CQ (see also ` Cq-Include-Trybots ` suggested above).
61- This step will test that ` gclient sync ` can fetch the new Rust toolchain
62- package + that the new package works fine in Chromium build.
63-
64- - Land the CL. The new package won't be used outside of this CL
65- until this step.
66-
67166
68167## Building and testing the tools locally
69168
0 commit comments