|
| 1 | +--- |
| 2 | +layout: post |
| 3 | +title: "Using rustc_codegen_cranelift for debug builds" |
| 4 | +author: Joshua Nelson |
| 5 | +team: The Compiler Team <https://www.rust-lang.org/governance/teams/compiler> |
| 6 | +--- |
| 7 | + |
| 8 | +## What is `rustc_codegen_cranelift`? |
| 9 | + |
| 10 | +[`rustc_codegen_cranelift`], or just `cg_clif` for short, is a new experimental |
| 11 | +codegen backend for the Rust compiler. The existing backend is LLVM, which is very |
| 12 | +good at producing fast, highly optimized code, but is not very good at |
| 13 | +compiling code quickly. `cg_clif`, which uses the [Cranelift] project, would |
| 14 | +provide a fast backend which greatly improves compile times, at the cost of |
| 15 | +performing very few optimizations. This is a great fit for debug builds, and the hope is |
| 16 | +that `cg_clif` will eventually be the default backend in debug mode. |
| 17 | + |
| 18 | +## What is the progress of using `rustc_codegen_cranelift` for debug builds? |
| 19 | + |
| 20 | +There has been a [Major Change Proposal][MCP] open for some time for making |
| 21 | +`cg_clif` part of the main Rust repository. Recently, [the MCP was |
| 22 | +accepted][compiler-team#270] and the compiler team [merged][#77975] |
| 23 | +`rustc_cranelift_codegen` [into the main Rust git repository][#77975]. |
| 24 | +`cg_clif` is not yet distributed with `rustup`, but this means you can now |
| 25 | +build it from source in-tree! |
| 26 | + |
| 27 | +## How do I use `rustc_codegen_cranelift`? |
| 28 | + |
| 29 | +In this section, I'll walk through step-by-step how to build the new backend from source, then use it on your own projects. All code is copy/paste-able, and each step is explained. |
| 30 | + |
| 31 | +First, let's build `cg_clif` from source. |
| 32 | + |
| 33 | +```sh |
| 34 | +$ git clone https://github.com/bjorn3/rustc_codegen_cranelift.git |
| 35 | +$ ./prepare.sh |
| 36 | +$ ./build.sh |
| 37 | +``` |
| 38 | + |
| 39 | +Now, we can start using it to compile a project. For demonstration purposes, |
| 40 | +I'll be be using `cargo`, but you can use any Rust project supported by |
| 41 | +`cg_clif`. |
| 42 | + |
| 43 | +``` |
| 44 | +$ cd .. |
| 45 | +$ git clone https://github.com/rust-lang/cargo/ |
| 46 | +$ cd cargo |
| 47 | +$ ../rustc_codegen_cranelift/build/cargo.sh build |
| 48 | +... |
| 49 | + Finished dev [unoptimized + debuginfo] target(s) in 49.93s |
| 50 | +``` |
| 51 | + |
| 52 | +It works! For comparison, let's see how long the equivalent LLVM backend would |
| 53 | +take. |
| 54 | + |
| 55 | +```sh |
| 56 | +$ rustup install nightly-2020-10-31 |
| 57 | +$ cargo +nightly-2020-10-31 build |
| 58 | +... |
| 59 | + Finished dev [unoptimized + debuginfo] target(s) in 54.64s |
| 60 | +``` |
| 61 | + |
| 62 | +LLVM takes a full 5 seconds longer for a full build. Next, let's try incremental builds: |
| 63 | + |
| 64 | +``` |
| 65 | +$ git apply <<EOF |
| 66 | +diff --git a/src/cargo/lib.rs b/src/cargo/lib.rs |
| 67 | +index bccb41121..703afa754 100644 |
| 68 | +--- a/src/cargo/lib.rs |
| 69 | ++++ b/src/cargo/lib.rs |
| 70 | +@@ -36,8 +36,8 @@ use anyhow::Error; |
| 71 | + use log::debug; |
| 72 | + use std::fmt; |
| 73 | + |
| 74 | +-pub use crate::util::errors::{InternalError, VerboseError}; |
| 75 | + pub use crate::util::{CargoResult, CliError, CliResult, Config}; |
| 76 | ++pub use crate::util::errors::{InternalError, VerboseError}; |
| 77 | + |
| 78 | + pub const CARGO_ENV: &str = "CARGO"; |
| 79 | +EOF |
| 80 | +$ ../rustc_codegen_cranelift/build/cargo.sh build |
| 81 | + Finished dev [unoptimized + debuginfo] target(s) in 7.98s |
| 82 | +$ cargo +nightly-2020-10-31 build |
| 83 | + Compiling cargo v0.50.0 (/home/joshua/cargo) |
| 84 | + Finished dev [unoptimized + debuginfo] target(s) in 5.48s |
| 85 | +``` |
| 86 | + |
| 87 | +LLVM is actually *faster* here: `serde_derive` took longer to run under cranelift, since it wasn't as optimized. Under cranelift it takes ~14% percent of the time, while under LLVM it takes less than 3%. |
| 88 | + |
| 89 | +## Building in-tree |
| 90 | + |
| 91 | +This section is mostly for compiler hackers, but feel free to follow along even |
| 92 | +if you're just interested! The reason this isn't the recommended way to build |
| 93 | +`cg_clif` is because the Rust compiler takes a very long time to build. |
| 94 | + |
| 95 | +First, download the Rust repository. |
| 96 | + |
| 97 | +```console |
| 98 | +git clone https://github.com/rust-lang/rust |
| 99 | +``` |
| 100 | + |
| 101 | +Now, let's set up the build system to use `cg_clif`. |
| 102 | + |
| 103 | +```text |
| 104 | +$ cat > config.toml <<EOF |
| 105 | +[rust] |
| 106 | +codegen-backends = ["cranelift"] |
| 107 | +EOF |
| 108 | +``` |
| 109 | + |
| 110 | +Finally, let's run the build. This can take a long time, over a half-hour in some cases. |
| 111 | + |
| 112 | +```console |
| 113 | +./x.py build |
| 114 | +``` |
| 115 | + |
| 116 | +## How can I help? |
| 117 | + |
| 118 | +You don't need to be a compiler developer to help improve `cg_clif`! The best |
| 119 | +way you can help is by testing `cg_clif` on different Rust crates across the |
| 120 | +ecosystem. Just while writing this article, I found [two][#1102] |
| 121 | +[bugs][#1101], so there's plenty of work left to be done. Please report any bugs you find |
| 122 | +to the [`rustc_codegen_cranelift` git repository][issue]. |
| 123 | + |
| 124 | +In the future, we hope to distribute `cg_clif` with Rustup, and if it matures sufficiently, eventually make it the default backend for debug builds. |
| 125 | + |
| 126 | +[`rustc_codegen_cranelift`]: https://github.com/bjorn3/rustc_codegen_cranelift |
| 127 | +[Cranelift]: https://github.com/bytecodealliance/wasmtime/tree/main/cranelift#cranelift-code-generator |
| 128 | +[#77975]: https://github.com/rust-lang/rust/pull/77975 |
| 129 | +[MCP]: https://forge.rust-lang.org/compiler/mcp.html |
| 130 | +[compiler-team#270]: https://github.com/rust-lang/compiler-team/issues/270 |
| 131 | +[`rustc-dev-guide`]: https://rustc-dev-guide.rust-lang.org/building/how-to-build-and-run.html#creating-a-rustup-toolchain |
| 132 | +[git worktree]: https://rustc-dev-guide.rust-lang.org/building/suggested.html#working-on-multiple-branches-at-the-same-time |
| 133 | +[#1102]: https://github.com/bjorn3/rustc_codegen_cranelift/issues/1102 |
| 134 | +[#1101]: https://github.com/bjorn3/rustc_codegen_cranelift/issues/1101 |
| 135 | +[issue]: https://github.com/bjorn3/rustc_codegen_cranelift/issues/new |
0 commit comments