Implement unstable `-Clinker-flavor=gcc:lld` for MCP 510 #96827

lqd · 2022-05-07T21:31:07Z

This PR is extracted from #96401 for easier review.

It implements half of MCP #510: adding a gcc:lld linker-flavor on the CLI, as a prerequisite for supporting rust-lld directly via a command similar to -Clinker-flavor=gcc:lld -Clink-self-contained=linker.

The option is unstable and requires -Z unstable-options to be used.

When used without -Clink-self-contained=linker, it's equivalent to -Clink-arg=-fuse-ld=lld.

r? @petrochenkov

rust-highfive · 2022-05-07T21:31:10Z

⚠️ Warning ⚠️

These commits modify compiler targets. (See the Target Tier Policy.)

compiler/rustc_codegen_ssa/src/back/link.rs

petrochenkov · 2022-05-14T20:18:31Z

I suddenly got busy and will get to this PR (and then to #96884) next weekend most likely.

petrochenkov · 2022-05-21T22:58:10Z

gcc linker-flavor accept a colon-separator to detect what to pass to cc. For example, -Clinker-flavor=gcc:$linker will be passed as -fuse-ld=$linker verbatim.

But that's not a linker flavor, that's literally -Zgcc-ld that you (we?) have been trying to eliminate.

If that was a linker flavor, then we'd add

"gcc-ld"
"gcc-wasm-ld"
"gcc-ld64.lld"
"gcc-ld.lld"
"gcc-lld-link"

to already existing

"ld"
"wasm-ld"
"ld64.lld"
"ld.lld"
"lld-link"

flavors.
No strings passed around verbatim, only "generalized classes" of linkers.

If we want to pass strings to -fuse-ld verbatim then it's better to keep a separate option like -Zgcc-ld.
At the same time, half of the concerns on the Zulip thread was about how "passing strings to linker is bad and we need something more high level".

EDIT:

a separate option like -Zgcc-ld

And that option already exists on stable and is called -Clink-arg=-fuse-ld=$linker, which makes -Zgcc-ld unnecessary and removable if we offload the "self-containedness" part to #96884.

petrochenkov · 2022-05-21T23:02:38Z

(I need to reread the zulip thread because I'm starting to forget all of the colors of this shed.)

bors · 2022-05-26T01:57:23Z

☔ The latest upstream changes (presumably #97409) made this pull request unmergeable. Please resolve the merge conflicts.

lqd · 2022-05-31T15:35:13Z

Vadim and I discussed this over zulip. He'll have some feedback about the PR, but in the meantime other issues were filed to track questions and various refactorings for these parts related to linkers.

I've rebased this PR over the -Zgcc-ld=lld simplifications of #97375.

@rustbot ready

petrochenkov · 2022-06-17T21:35:23Z

Sorry for the delay.
I'm now back from vacation and will return to this soon.

petrochenkov · 2022-06-27T08:56:22Z

Update: I'm still against having a free form component in -Clinker-flavor, but I don't have time to write a good motivation and finish my experiments in petrochenkov@836063a / petrochenkov@2227d27.
The timing has been really unlucky here, once I'm less busy and can allocate a few continuous hours of Rust work, this is the first thing I'm going to address.

bors · 2022-07-14T11:23:13Z

☔ The latest upstream changes (presumably #98975) made this pull request unmergeable. Please resolve the merge conflicts.

only the stable values for `-Clinker-flavor` can be used on stable until we have more feedback on the interface

petrochenkov · 2022-08-08T18:05:18Z

Sorry for the delay.

So I wanted to try documenting what "linker flavor" even means.
(I'll consider both -Clinker-flavor and target specs.)

All linkers have some kind of command line interfaces and rustc needs to know which commands to use with each of them.
So we clusterize all such interfaces into a (somewhat arbitrary) number of classes and call them "linker flavors".

Technically, it's not even necessary, we can nearly always infer the flavor from linker name and target properties like is_like_windows/is_like_osx/etc.
However, the PRs originally introducing -Zlinker-flavor (#40018 and friends) were aiming to reduce this kind of inference and provide something certain and explicitly specified instead.

The minimal set of linker flavors could look something like this (1)

// Unix-like linker with GNU extensions (both naked and compiler-wrapped forms)
// Besides similar "default" Linux/BSD linkers this also include Windows/GNU, which is somewhat different.
Gnu { cc: bool },
// Basic Unix-like linker, possibly with non-GNU extensions aka "any other Unix" (both naked and compiler-wrapped forms)
// Apple targets, Wasm, Solaris/illumos, l4re, msp430 - very different targets.
// Why this exists separately from GNU - the `linker_is_gnu` target property (which I will include into linker flavor in this post)
// is annoying to infer using other target properties.
Unix { cc: bool },
// MSVC linker has a unique interface
Msvc,
// Other linker-like tools with unique interfaces for exotic targets
EmCc,
Bpf,
Ptx,

Then LLD appears (#48125) and real life forces our hand to increase the number of clusters to be a bit less minimal.
LLD linker is self-sufficent only if its executable (argv[0]) is named in specific way, otherwise it doesn't work and requires a -flavor LLD_FLAVOR argument to choose which logic to use.
Our shipped rust-lld in particular is not named in such specific way, maybe it was a mistake to ship it like this, but it's too late now.
So it needs the flavor option, so let's also make it "certain and explicit" and not inferred from other target properties.
(I'll also assume we want to call LLD through a C compiler too, which is the point of this PR.)

This changes our set of linker flavors to something like this (2):

// Unix-like linker with GNU extensions (both naked and compiler-wrapped forms)
// Besides similar "default" Linux/BSD linkers this also include Windows/GNU, which is somewhat different.
Gnu { lld: bool, cc: bool },
// Unix-like linker for Apple targets (both naked and compiler-wrapped forms)
// Why it was extracted from "umbrella" Unix - due to the corresponding LLD flavor.
Darwin { lld: bool, cc: bool },
// Unix-like linker for Wasm targets (both naked and compiler-wrapped forms)
// Why it was extracted from "umbrella" Unix - due to the corresponding LLD flavor.
// Non-LLD version does not exist, so the lld flag here is currently hardcoded and not boolean.
WasmLld { cc: bool },
// Basic Unix-like linker, possibly with non-GNU extensions aka "any other Unix" (both naked and compiler-wrapped forms)
// Solaris/illumos, l4re, msp430 - very different targets.
// LLD doesn't support any of these.
Unix { cc: bool },
// MSVC linker has a unique interface
// LLD supports it
Msvc { lld: bool },
// Other linker-like tools with unique interfaces for exotic targets
EmCc,
Bpf,
Ptx,

Then we could increase the number of clusters even more, e.g. to cover most of conditions in compiler\rustc_codegen_ssa\src\back\linker.rs (3):

Gnu { lld: bool, cc: bool },
Mingw { lld: bool, cc: bool }, // split from GNU
Darwin { lld: bool, cc: bool },
WasmLld { cc: bool },
Solaris { cc: bool }, // split from Unix
L4Re { cc: bool }, // split from Unix
Unix { cc: bool }, // "other Unix" still needs to exist
Msvc { lld: bool },
EmCc,
Bpf,
Ptx,

We can realistically do this, but it may be too much for my taste. It would still not eliminate all the target-specific logic in the rustc_codegen_ssa though.
Anyway, then number of clusters is somewhat arbitrary, as I said above.

So what conclusions I draw from all of this.

I'd like to keep the original "something certain and explicitly specified" idea of linker flavor, instead of inferring things from strings and other properties.
- This means no :arbitrary-string part for -Clinker-flavor, maybe this second component can be added to -Clinker instead in the future, although I don't think it's necessary now, because -Clink-arg=-fuse-ld=/mold/or/whatever still exists.
I'd like to keep the number of linker flavor "clusters" minimal under existing circumstances, and make the set (2) user-facing (both in -Clinker-flavor and in json target specs).
E.g. the (non-legacy) accepted string values will be
```
gnu
gnu-lld
gnu-cc
gnu-lld-cc
darwin
darwin-lld
darwin-cc
darwin-lld-cc
wasm-lld
wasm-lld-cc
unix
unix-cc
msvc
msvc-lld
em-cc
bpf
ptx
```
instead of the current
```
ld
ld.lld
gcc
ld64.lld
wasm-ld
msvc
lld-link
l4-bender
em
bpf-linker
ptx-linker
```
(I tried to keep the naming consistent.)
Proof-of-concept branch for this scheme (without any backward compatibility) - https://github.com/petrochenkov/rust/tree/flavor2.

petrochenkov · 2022-08-08T18:06:19Z

I'll review the code in this PR a bit later, I expect that we can mostly land it as is.

petrochenkov · 2022-08-09T21:53:22Z

Since the free form component is removed, gcc:lld can be renamed to gcc-lld (*) and added to enum LinkerFlavor.
fn add_link_args can then fill link args for this flavor if flags for gcc are filled.
LinkerFlavorCli is also no longer necessary, all flavors are well known.

(*) Although I'd prefer to use the scheme from #96827 (comment) eventually.

petrochenkov · 2022-08-09T21:59:27Z

compiler/rustc_codegen_ssa/src/back/link.rs

+            "Using `lld`, self-contained linking, and coverage or profile generation has known \
+            issues. See issue #79555 for more details, at \
+            https://github.com/rust-lang/rust/issues/79555",
+        );


Ugh.
Is it really a compiler's job to do this?

Same about other cases of copypasting issue tracker into the compiler's source code.

petrochenkov · 2022-08-09T22:04:51Z

compiler/rustc_codegen_ssa/src/back/link.rs

+        cmd.arg(format!("-Wl,-rustc-lld-flavor={}", sess.target.lld_flavor.as_str()));
+    } else {
+        // Otherwise, we were asked to use `lld` but not `rust-lld`.
+        cmd.arg("-fuse-ld=lld");


Maybe this PR should wait for #100200, in addition to fixing the regression it basically makes the "self-contained" and "lld" aspects of -Zgcc-ld orthogonal and removes useless checks like

// - checking the `lld-wrapper`s exist in the sysroot

above, that check something that is not necessarily true, and will prevent finding system lld instead.

petrochenkov · 2022-08-09T22:08:31Z

src/test/ui/linkers/gcc_ld_mismatch.rs

+// compile-flags: -Zgcc-ld=lld -Clinker-flavor=em -Zunstable-options
+
+// Test ensuring that until the unstable flag is removed (if ever), if both the linker-flavor and
+// `gcc-ld` flags are used, they ask for the same linker.


If making these two options incompatible helps to remove at least some code or conditions in fn handle_cli_linker_flavors then please let's do that instead.

rustc_target: Add a compatibility layer to separate internal and user-facing linker flavors I want to do some refactorings in `rustc_target` - merge `lld_flavor` and `linker_is_gnu` into `linker_flavor`, support combination gcc+lld (rust-lang#96827). This PR adds some compatibility infra that makes that possible without making any changes to user-facing interfaces - `-Clinker-flavor` values and json target specs. (For json target specs this infra may eventually go away since they are not very stable.) The second commit does some light refactoring of internal linker flavors (applies changes from petrochenkov@53eca42 that don't require mass-editing target specs).

bors · 2022-09-02T13:52:35Z

☔ The latest upstream changes (presumably #101318) made this pull request unmergeable. Please resolve the merge conflicts.

In accordance with the design from rust-lang#96827 (comment)

rustc_target: Refactor internal linker flavors In accordance with the design from rust-lang#96827 (comment) `lld_flavor` and `linker_is_gnu` fields are removed from internal target specs, but still parsed from JSON specs using compatibility layer introduced in rust-lang#100552. r? `@lqd`

lqd · 2023-07-01T00:15:01Z

superseded by #112910

Implement most of MCP510 This implements most of what remains to be done for MCP510: - turns `-C link-self-contained` into a `+`/`-` list of components, like `-C link-self-contained=+linker,+crto,+libc,+unwind,+sanitizers,+mingw`. The scaffolding is present for all these expected components to be implemented and stabilized in the future on their own time. This PR only handles the `-Zgcc-ld=lld` subset of these link-self-contained components as `-Clink-self-contained=+linker` - handles `-C link-self-contained=y|n` as-is today, for compatibility with `rustc_codegen_ssa::back::link::self_contained`'s [explicit opt-in and opt-out](https://github.com/lqd/rust/blob/9eee230cd0a56bfba3ce65121798d9f9f4341cdd/compiler/rustc_codegen_ssa/src/back/link.rs#L1671-L1676). - therefore supports our plan to opt out of `rust-lld` (when it's enabled by default) even for current `-Clink-self-contained` users, with e.g. `-Clink-self-contained -Clink-self-contained=-linker` - turns `add_gcc_ld_path` into its expected final form, by using the `-C link-self-contained=+linker` CLI flag, and whether the `LinkerFlavor` has the expected `Cc::Yes` and `Lld::Yes` shape (this is not yet the case in practice for any CLI linker flavor) - makes the [new clean linker flavors](rust-lang#96827 (comment)) selectable in the CLI in addition to the legacy ones, in order to opt-in to using `cc` and `lld` to emulate `-Zgcc-ld=lld` - ensure the new `-C link-self-contained` components, and `-C linker-flavor`s are unstable, and require `-Z unstable-options` to be used The up-to-date set of flags for the future stable CLI version of `-Zgcc-ld=lld` is currently: `-Clink-self-contained=+linker -Clinker-flavor=gnu-lld-cc -Zunstable-options`. It's possible we'll also need to do something for distros that don't ship `rust-lld`, but maybe there are already no tool search paths to be added to `cc` in this situation anyways. r? `@petrochenkov`

Implement most of MCP510 This implements most of what remains to be done for MCP510: - turns `-C link-self-contained` into a `+`/`-` list of components, like `-C link-self-contained=+linker,+crto,+libc,+unwind,+sanitizers,+mingw`. The scaffolding is present for all these expected components to be implemented and stabilized in the future on their own time. This PR only handles the `-Zgcc-ld=lld` subset of these link-self-contained components as `-Clink-self-contained=+linker` - handles `-C link-self-contained=y|n` as-is today, for compatibility with `rustc_codegen_ssa::back::link::self_contained`'s [explicit opt-in and opt-out](https://github.com/lqd/rust/blob/9eee230cd0a56bfba3ce65121798d9f9f4341cdd/compiler/rustc_codegen_ssa/src/back/link.rs#L1671-L1676). - therefore supports our plan to opt out of `rust-lld` (when it's enabled by default) even for current `-Clink-self-contained` users, with e.g. `-Clink-self-contained -Clink-self-contained=-linker` - turns `add_gcc_ld_path` into its expected final form, by using the `-C link-self-contained=+linker` CLI flag, and whether the `LinkerFlavor` has the expected `Cc::Yes` and `Lld::Yes` shape (this is not yet the case in practice for any CLI linker flavor) - makes the [new clean linker flavors](rust-lang/rust#96827 (comment)) selectable in the CLI in addition to the legacy ones, in order to opt-in to using `cc` and `lld` to emulate `-Zgcc-ld=lld` - ensure the new `-C link-self-contained` components, and `-C linker-flavor`s are unstable, and require `-Z unstable-options` to be used The up-to-date set of flags for the future stable CLI version of `-Zgcc-ld=lld` is currently: `-Clink-self-contained=+linker -Clinker-flavor=gnu-lld-cc -Zunstable-options`. It's possible we'll also need to do something for distros that don't ship `rust-lld`, but maybe there are already no tool search paths to be added to `cc` in this situation anyways. r? `@petrochenkov`

Use lld by default on `x86_64-unknown-linux-gnu` stable This PR and stabilization report is joint work with `@Kobzol.` #### Use LLD on `x86_64-unknown-linux-gnu` by default, and stabilize `-Clinker-features=-lld` and `-Clink-self-contained=-linker` This PR proposes making LLD the default linker on the `x86_64-unknown-linux-gnu` target for the artifacts we distribute, and also stabilizing the `-Clinker-features=-lld` and `-Clink-self-contained=-linker` codegen options to make it possible to opt out. LLD has been used as the default linker on nightly and CI on this target since May 2024 ([PR](#124129), [blog post](https://blog.rust-lang.org/2024/05/17/enabling-rust-lld-on-linux.html)), and it seems like it is working fine, so we would like to propose stabilizing it. The main motivation for using LLD instead of the default BFD linker is improving [compilation times](https://perf.rust-lang.org/compare.html?start=b3e117044c7f707293edc040edb93e7ec5f7040a&end=baed03c51a68376c1789cc373581eea0daf89967&stat=instructions%3Au&tab=compile). For example, in the linked benchmark, it makes incremental recompilation of `ripgrep` in `debug` more than twice faster. Another benefit is that Rust compilation becomes more consistent and self-contained, because we will use a known version of the LLD linker, rather than "whatever GNU ld version is on the user's system". Due to the performance benefit being so huge, many people already opt into using LLD (or other fast linkers, such as mold) using various approaches ([1](https://github.com/search?type=code&q=%2Flinker-flavor%5B%3D+%5Dgnu-lld-cc%2F), [2](https://github.com/search?type=code&q=%2Flinker-features%5B%3D+%5D%5C%2Blld%2F), [3](https://github.com/search?type=code&q=language%3Atoml+%22-fuse-ld%3Dlld%22), [4](https://github.com/search?type=code&q=language%3Arust+%22-fuse-ld%3Dlld%22)). By making LLD the default linker on the `x86_64-unknown-linux-gnu` target, we will be able to speed up Rust compilation out of the box, without users having to opt in or know about it. > You can find an extended version of this stabilization report which includes analysis of crater results and more data [here](https://hackmd.io/tFDifkUcSLGoHPBRIl0z8w?view). ## What is being stabilized - `rust-lld` being used as the default linker on the `x86_64-unknown-linux-gnu` target. - Note that `rust-lld` is being enabled by default in the compiler artifacts distributed by our CI/rustup. It is still possible to use the system linker by default using `rust.lld = false` in `bootstrap.toml`, which can be useful e.g. for some Linux distros that might not want to use the LLD we distribute. - This is done by activating the LLD linker feature and using the self-contained linker on that target. Both of which are also usable on the CLI, if some opt outs are necessary, as described below. - `-Clinker-features=-lld` on the `x86_64-unknown-linux-gnu` target. This codegen option tells rustc to disable using the LLD linker. - Note that other options for this codegen flag (`cc`) remain unstable. - Note that only the opt-out is being stabilized, and only for `x86_64-unknown-linux-gnu`: opting in, or using the flag on other targets would still need to pass `-Zunstable-options`. - This flag is being stabilized so that users can opt out of LLD on stable, which would it turn also opt out of using the self-contained linker (since it's an LLD). - `-Clink-self-contained=-linker`. This codegen option tells rustc to use the self-contained linker. It's not particularly useful to turn it on by itself, but when enabled and combined with `-Clinker-features=+lld`, rustc will use the `rust-lld` linker wrapper shipped with the compiler toolchain, instead of some `lld` binary that the linker driver will find in the `PATH`. - Note that other options for this codegen flag (other than the previously stable `y/yes/n/no`). - Note that only the opt-out is being stabilized, and only for `x86_64-unknown-linux-gnu`: opting in, or using this flag on other targets would still need to pass `-Zunstable-options`. - This flag is being stabilized so that users can opt out of using self-contained linking on stable. Doing this would then fall back to using the system `lld`. To opt out of using LLD, `RUSTFLAGS="-Clinker-features=-lld"` would be used. To opt out of using `rust-lld`, falling back to the LLD installed on the system, `RUSTFLAGS="-Clink-self-contained=-linker"` would be used. ## Tests When enabling `rust-lld` on nightly, we also switched x64 linux to use it at stage >= 1, meaning that all tests have been running with lld since May 2024, on CI as well as contributors' machines. (Post opt-dist tests also had been using it when running their test subset earlier than that). There are also a few tests dedicated to the CLI behavior, or ensuring the default linker is indeed the one we expect: - [link-self-contained-consistency](https://github.com/rust-lang/rust/blob/1117bc1e6ce049495b0044dfe756afafc817d2d7/tests/ui/linking/link-self-contained-consistency.rs): Checks that `-Clink-self-contained` options are not inconsistent (i.e. that passing both `+linker` and `-linker` is an error). - [link-self-contained-unstable](https://github.com/rust-lang/rust/blob/1117bc1e6ce049495b0044dfe756afafc817d2d7/tests/ui/linking/link-self-contained-unstable.rs): Checks that only the `-linker` and `y/yes/n/no` options for `-Clink-self-contained` are stable. - [linker-features-unstable-cc](https://github.com/rust-lang/rust/blob/1117bc1e6ce049495b0044dfe756afafc817d2d7/tests/ui/linking/linker-features-unstable-cc.rs): Checks that only the non-lld options of `-Clinker-features` are unstable. - [linker-features-lld-disallowed](https://github.com/rust-lang/rust/blob/1117bc1e6ce049495b0044dfe756afafc817d2d7/tests/ui/linking/linker-features-lld-disallowed.rs): Checks that `-Clinker-features=-lld` is only stable on `x86_64-unknown-linux-gnu`. - [link-self-contained-linker-disallowed](https://github.com/rust-lang/rust/blob/1117bc1e6ce049495b0044dfe756afafc817d2d7/tests/ui/linking/link-self-contained-linker-disallowed.rs): Checks that `-Clink-self-contained=-linker` is only stable on `x86_64-unknown-linux-gnu`. - [no-gc-encapsulation-symbols](https://github.com/rust-lang/rust/blob/1117bc1e6ce049495b0044dfe756afafc817d2d7/tests/ui/linking/no-gc-encapsulation-symbols.rs): Checks that that linker encapsulation symbols are not garbage collected by LLD, so that crates like [linkme](https://github.com/dtolnay/linkme) still work. - [rust-lld](https://github.com/rust-lang/rust/blob/1117bc1e6ce049495b0044dfe756afafc817d2d7/tests/run-make/rust-lld): Checks that LLD is actually used when enabled with `-Clinker-features=+lld` and `-Clink-self-contained=+linker`. - [rust-lld-x86_64-unknown-linux-gnu](https://github.com/rust-lang/rust/blob/1117bc1e6ce049495b0044dfe756afafc817d2d7/tests/run-make/rust-lld-x86_64-unknown-linux-gnu): Checks that LLD is used by default on `x86_64-unknown-linux-gnu` when the bootstrap `rust.lld` config option is `true`. - [rust-lld-x86_64-unknown-linux-gnu-dist](https://github.com/rust-lang/rust/blob/1117bc1e6ce049495b0044dfe756afafc817d2d7/tests/run-make/rust-lld-x86_64-unknown-linux-gnu-dist): Dist test that checks that our distributed `x86_64-unknown-linux-gnu` archives actually use LLD by default. ## Ecosystem impact As already stated, LLD has been used as the default linker on x64 Linux on nightly for almost a year, and we haven't seen any blockers to stabilization in that time. There were a handful of issues reported, these are discussed later below. Furthermore, two crater runs ([November 2023](https://crater-reports.s3.amazonaws.com/pr-117684-2/index.html), [February 2025](https://crater-reports.s3.amazonaws.com/pr-137044-3/index.html)), were performed to test the impact of using LLD as the default linker. A triage of the earlier crater run was previously done [here](https://hackmd.io/OAJxlxc6Te6YUot9ftYSKQ), but all the important findings from both crater runs are reported below. Below is a list of compatibility differences between BFD and LLD that we have encountered. There is a more thorough list of differences in [this post](https://maskray.me/blog/2020-12-19-lld-and-gnu-linker-incompatibilities) from the current LLD maintainer. From that post, "99.9% pieces of software work with ld.lld without a change". --- ### `.ctors/.dtors` sections [#128286](#128286) reported an issue where LLD was unable to link certain CUDA library was using these sections that were using the `.ctors/.dtors` ELF sections. These were deprecated a long time ago (https://gcc.gnu.org/bugzilla/show_bug.cgi?id=46770), replaced with a more modern `.init_array/.fini_array` sections. LLD doesn't (and won't) support these sections ([1](llvm/llvm-project#68071), [2](llvm/llvm-project#30572)), so if they appear in input object files, the linked artifact might produce incorrect behavior, because e.g. some global variables might not get initialized properly. However, the usage of `.ctors/.dtors` should be very rare in practice. We have performed a [crater run](#137044) to test this. It has identified only 8 crates where the `.ctors/.dtors` section is occurring in the final linked artifact. It was caused by a few crates using the `.ctors` link section manually, and by using a very (~6 year) old version of the [ctor](https://crates.io/crates/ctor) crate. [Crater run analysis](https://hackmd.io/tFDifkUcSLGoHPBRIl0z8w?view#ctorsdtors-sections) **Possible workaround** It is possible to [detect](e5e2316) if `.ctors/.dtors` section is present in the final linked artifact (LLD will keep it there, but it won't be populated), and warn users about it. This check is very cheap and doesn't even appear on [perf](#112049 (comment)). We have benchmarked the check on a 240 MiB Chrome binary, where it took 0.8ms with page cache flushed, and 0.06ms with page cache primed (which should be the common case, as the linked artifact is written to disk just before the check is performed). In theory, this could be also solved with a linker script that moves `.ctors` to `.init_array`. We think that these sections should be so rare that it is not worth it to implement any workarounds for now. --- ### Different garbage collection behavior [#130397](#130397) reported an issue where LLD prunes a local symbol, so it is missing in the linked artifact. However, BFD keeps the same symbol, so it is a regression. This is caused by a difference in linker garbage collection. Rust uses `--gc-sections` and puts each function into a separate linker section, which prunes unused code. There is some code (specifically the somewhat popular [linkme](https://github.com/dtolnay/linkme) crate) that (arguably ab-)uses so called linker encapsulation symbols to achieve distributed slices. BFD (2.37+) uses a conservative linking mode that works as intended with this behavior, but it might slightly increase binary size of the linked artifact. LLD does not use this workaround by default, which causes the sections to be eliminated, but it can be made to use the conservative mode using [`-z nostart-stop-gc`](https://lld.llvm.org/ELF/start-stop-gc.html#z-start-stop-gc). To avoid this issue, we told LLD to use the [conservative mode](#137685), which maintains backwards compatibility with BFD. We found that it has [no effect](#112049 (comment)) on compilation performance and binary size in our benchmark suite. With this change, `linkme` works. Since then, #140872 removed `linkme` distributed slice's dependence on conservative GC behavior, so this PR also removes that conservative mode: no transition period is necessary, as the PR immediately fixed the crate with no source changes. [Crater run analysis](https://hackmd.io/tFDifkUcSLGoHPBRIl0z8w?view#Different-garbage-collection-behavior) --- ### Various uncommon issues A small number of issues that only occurred in a handful of instances were found in crater, and it is unclear if LLD is at fault or if there is some other issue that was not detected with BFD. You can examine these [here](https://hackmd.io/tFDifkUcSLGoHPBRIl0z8w?view#Various-uncommon-issues). --- ### Missing jobserver support LLD doesn't support the jobserver protocol for limiting the number of threads used, it simply defaults to using all available cores, and is one of the reasons why it's faster than BFD. However, this should mostly be a non-issue, because most of the linking done during high parallelism sections of `cargo build` is linking of build scripts and proc macros, which are typically very fast to link (e.g. ~50ms), and a potential oversubscription of cores thus doesn't hurt that much. When the final artifact is linked (which typically takes the most time), there should be no other sources of parallelism conflicts from compiling other code, so LLD should be able to use all available threads. That being said, it is a difference of behavior, where previously a `-j` flag was generally not using more cpu than the specified limit. It can be impactful in some resource-constrained systems, but to be clear that is already the case today due to [cargo parallelism](rust-lang/cargo#9157). This could be one reason to opt out of using `rust-lld` on some systems. LLD has support for limiting the number of threads to use, so in theory rustc could try to get all the jobserver tokens available and use that as lld's thread limit. It'd still be suboptimal as new tokens would not be dynamically detected, and we could be using less threads than available. We did a benchmark on a real-world crate that shows that using multiple LLD threads for intermediate artifacts doesn't seem to have a performance effect. You can find it [here](https://hackmd.io/tFDifkUcSLGoHPBRIl0z8w?view#Missing-jobserver-support). --- #### Opting out of LLD in the ecosystem We have also examined repositories where people opted out of LLD on nightly, using [this GitHub query](https://github.com/search?q=%22linker-features%3D-lld%22&type=code). The summary can be found below: <details> <summary>Summary of LLD opt outs</summary> > This examination was performed on 2025-03-09. Here we briefly examine the most common reasons why people use `-Zlinker-features=-lld`, based on comments and git history. - Nix/NixOS ([1](https://github.com/rszyma/vscode-kanata/blob/59d703dff5a238b14ab3759cac27f73fb34bbcfe/flake.nix#L33), [2](https://github.com/sbernauer/breakwater/blob/3cc3449fc126c5c99d4a971733fd32be589884e0/.cargo/config.toml#L4), [3](https://github.com/tiiuae/ebpf-firewall/blame/32bdb17cedd1c9bea1ab3482623de458d95da7d0/.cargo/config.toml#L2), [4](https://github.com/jules-sommer/wavetheme-gen/blob/f5f657d014d4a30607625afb70f810c229c0294e/Cargo.toml#L4), [5](https://github.com/LayerTwo-Labs/zside-rust/blob/e4266f5c5571a1b180a9c70cf0939c7070e410c7/.cargo/config.toml#L10), [6](https://github.com/przyjacielpkp/zkml/blob/22a4aef24e9d2c77789229d7c634fc67e9eb1184/README.md?plain=1#L78), [7](https://github.com/LayerTwo-Labs/thunder-rust/blob/2222d53474c8d2d0428b4c56f8157095dced6d5a/.cargo/config.toml#L2), [8](https://github.com/enesoztrk/nixos-tc-aya-test/blob/b2ffa59d3eba8b60fd04b0a4c8bbe047400eb981/.cargo/config.toml#L4), [9](https://github.com/lowRISC/container-hotplug/blob/3ead4ef9c7f79c303392178c99677dbecff1aea6/.cargo/config.toml#L2), [10](https://github.com/Eliah-Lakhin/ad-astra/blob/ca6b8c8a5dba7bb5e894f3f1013f17876962a021/work/examples/lsp-client/src/extension.ts#L94)) - There was an [issue](NixOS/nixpkgs#312661) with LLD, which seems to have been fixed with NixOS/nixpkgs#314268. It's unclear whether that fixed all the Nix issues though. - Issues with linkme ([1](https://github.com/0xPolygonZero/zk_evm/blob/ef388619ffbd5305209519a3a5bc0396185d68ac/.cargo/config.toml#L4), [2](https://github.com/conjure-cp/conjure-oxide/blob/be0fc5827ff90e8486d416cc184b6ce24f73bf01/README.md?plain=1#L20), [3](https://github.com/clchiou/garage/blob/c5d8444d56bb6ee24ca95e5c6b9880ed996f4918/rust/.cargo/config.toml#L6), [4](https://github.com/PonasKovas/craftflow/blob/5b4cc1a5196e08a975368399fefda4b71f3a2f6f/.cargo/config.toml#L3), [5](https://github.com/kezhuw/zookeeper-client-rust/blob/4e27c7de2a7cc5e709af012b791c8fea9bb47f1f/.github/workflows/ci.yml#L82), [6](https://github.com/niklasdewally/conjure-oxide/blob/8fe60c12bca7011a2f9eded4b7c95ad0e77b6f44/.github/workflows/code-coverage.yml#L48), [7](https://github.com/kezhuw/spawns/blob/c8b468379805de9df3287c01b94b4ed3db6b61ed/.github/workflows/ci.yml#L74)) - These should be resolved with the conservative garbage collection ([#137685](#137685)). - Bazel ([1](https://github.com/google-parfait/confidential-federated-compute/blob/1823f69ed8f5f4f819f7bfa21da1ca629fdc826b/.bazelrc#L71)), WASM ([1](https://github.com/Eliah-Lakhin/ad-astra/blob/ca6b8c8a5dba7bb5e894f3f1013f17876962a021/work/examples/wasm-build.sh#L37), [2](https://github.com/yacineb/pgrx-wasi-test/blob/2bf99037ca1b650b2cbc35f1257a87fb6ead0920/build.sh#L21)), uncategorized ([2](https://github.com/nbdd0121/r2vm/blob/5118be6b9e757c6fef2f019385873f403c23c548/.cargo/config.toml#L3), [3](https://github.com/Wyvern/Img/blame/45020c7e1dc4926c8129647014c708db0c13f463/.cargo/config.toml#L209), [4](https://github.com/arnaudpoullet/leptos-i18n-compile-error/blob/042eb835f7ca0dc36be67cf7fe65b35b22b6059f/README.md?plain=1#L89), [5](https://github.com/JonLeeCon/numerical-rust-cpu/blob/fd0b3006768ed81c56147044dc05c92b11b7b6f0/exercises/.cargo/config.toml#L13), [6](https://github.com/PonasKovas/shallowclone/blob/be65f2ec923cac6ceedbc8db520c89969ebfce7c/.github/workflows/rust.yml#L20)) - Reason unclear. </details> ## History The idea to use a faster linker by default has been on the radar for quite some time ([#39915](#39915), [#71515](#71515)). There were [very early attempts](#29974) to use the gold linker by default, but these had to be [reverted](#30913) because of compatibility issues. Support for LLD was implemented back in [2017](#40018), but it has not been made default yet, except for some more niche targets, such as [WASM](#48125), [ARM Cortex](#53648) or [RISC-V](#53822). It took quite some time to figure out how should the interface for selecting the linker (and the way it is invoked) look like, as it differs a lot between different platforms, linkers and compiler drivers. During that time, LLD has matured and achieved [almost perfect compatibility](https://maskray.me/blog/2020-12-19-lld-and-gnu-linker-incompatibilities) with the default Linux linker (BFD). - [#56351](#56351) stabilized `-Clinker-flavor`, which is used to determine how to invoke the linker. It is especially useful on targets where selecting the linker directly with `-Clinker` is not possible or is impractical. - December 2018, author `@davidtwco,` reviewer `@nagisa` - [#76158](#76158) stabilized `-Clink-self-contained=[y|n]`, which allows overriding the compiler's heuristic for deciding whether it should use self-contained or external tools (linker, sanitizers, libc, etc.). It only allowed using the self-contained mode either for everything (`y`) or nothing (`n`), but did not allow granular choice. - September 2020, author `@mati864,` reviewer `@petrochenkov` - [#85961](#85961) implemented the `-Zgcc-ld` flag, which was a hacky way of opting into LLD usage. - June 2021, author `@sledgehammervampire,` reviewer `@petrochenkov` - [MCP 510](rust-lang/compiler-team#510) proposed stabilizing the behavior of `-Zgcc-ld` using more granular flags (`-Clink-self-contained=linker -Clinker-flavor=gcc-lld`). - Initially implemented in [#96827](#96827), but `@petrochenkov` [suggested](#96827 (comment)) a slightly different approach. - The PR was split into [#96884](#96884), where it was decided what will be the individual components of `-Clink-self-contained=linker`. - And [#96401](#96401), which implemented the `-Clinker-flavor` part. - The MCP was finally implemented in [#112910](#112910). - [#116514](#116514) then removed `-Zgcc-ld`, as it was replaced by `-Clinker-flavor=gnu-lld-cc` + `-Clink-self-contained=linker`. - April 2022 - October 2023, author `@lqd,` reviewer `@petrochenkov` - Various linker handling refactorings were performed in the meantime: [#97375](#97375), [#98212](#98212), [#100126](#100126), [#100552](#100552), [#102836](#102836), [#110807](#110807), [#101988](#101988), [#116515](#116515) - The implementation of linker flavors with LLD was causing a sort of a combinatorial explosion of various options. [#119906](#119906) suggested a different approach for linker flavors (described [here](#119906 (comment))), where the individual flavors could be enabled separately using `+/-` (e.g. `+lld`). - After some back and forth, this idea was moved to `-Clinker-features` (see [comment 1](#119906 (comment)) and [comment 2](#119906 (comment))), which was implemented in [#123656](#123656). - April 2024, author `@lqd,` reviewer `@petrochenkov` - [#124129](#124129) enabled LLD by default on nightly. - April 2024, author `@lqd,` reviewer `@petrochenkov` - [#137685](#137685), [#137926](#137926) enabled the conservative gargage collection mode (`-znostart-stop-gc`) to improve compatibility with BFD. - February 2025, author `@lqd,` reviewer `@petrochenkov` (implementation), author `@kobzol,` reviewer `@lqd` (test) - [#96025](#96025) (April 2022), [#117684](#117684) (November 2023), [#137044](#137044) (February 2025): crater runs. ## Unresolved questions/concerns - Is changing the linker considered a breaking change? In (hopefully very rare) cases, it might break some existing code. It should mostly only affect the final linked artifact, so it should be easy to opt out. - Similarly, is the single-threaded behavior of such tools encompassed in our stability guarantee: it can be observed via the `-j` job limit (though I believe we have/had some open issues on sometimes using more CPU resources than the job count limit implied). As mentioned above, LLD does not support the jobserver protocol. - A concern [was raised](#71515 (comment)) about increased memory usage of LLD. We should probably let users know about the possibly increased memory usage, and jobserver incompatibility: we did so when announcing this landing on nightly. - LLD seems to produce [slightly larger](https://perf.rust-lang.org/compare.html?start=b3e117044c7f707293edc040edb93e7ec5f7040a&end=baed03c51a68376c1789cc373581eea0daf89967&stat=size%3Alinked_artifact&tab=compile) binary artifacts. This can be partially clawed back using Identical Code Folding (`-Clink-args=-Wl,--icf=all`). - Should we detect the outdated `.ctors/.dtors` sections to provide a better error message, even if that should be rare in practice? --- ### Next steps After the FCP completes: - we should land this PR at the beginning of a beta cycle, to maximize time for testing - keep an eye on the beta crater run results for possible linker issues (or do a dedicated beta crater run with only this change) - release a blog post announcing the change, and asking for testing feedback of the appropriate beta - depending on feedback, or if a period of testing of 6 weeks is not long enough, we could keep this change on beta for another cycle --- Development, testing, try builds were done in #138645. r? `@petrochenkov` `@rustbot` label +needs-fcp +T-compiler try-job: aarch64-gnu try-job: i686-gnu-*

Use lld by default on `x86_64-unknown-linux-gnu` stable This PR and stabilization report is joint work with `@Kobzol.` #### Use LLD on `x86_64-unknown-linux-gnu` by default, and stabilize `-Clinker-features=-lld` and `-Clink-self-contained=-linker` This PR proposes making LLD the default linker on the `x86_64-unknown-linux-gnu` target for the artifacts we distribute, and also stabilizing the `-Clinker-features=-lld` and `-Clink-self-contained=-linker` codegen options to make it possible to opt out. LLD has been used as the default linker on nightly and CI on this target since May 2024 ([PR](#124129), [blog post](https://blog.rust-lang.org/2024/05/17/enabling-rust-lld-on-linux.html)), and it seems like it is working fine, so we would like to propose stabilizing it. The main motivation for using LLD instead of the default BFD linker is improving [compilation times](https://perf.rust-lang.org/compare.html?start=b3e117044c7f707293edc040edb93e7ec5f7040a&end=baed03c51a68376c1789cc373581eea0daf89967&stat=instructions%3Au&tab=compile). For example, in the linked benchmark, it makes incremental recompilation of `ripgrep` in `debug` more than twice faster. Another benefit is that Rust compilation becomes more consistent and self-contained, because we will use a known version of the LLD linker, rather than "whatever GNU ld version is on the user's system". Due to the performance benefit being so huge, many people already opt into using LLD (or other fast linkers, such as mold) using various approaches ([1](https://github.com/search?type=code&q=%2Flinker-flavor%5B%3D+%5Dgnu-lld-cc%2F), [2](https://github.com/search?type=code&q=%2Flinker-features%5B%3D+%5D%5C%2Blld%2F), [3](https://github.com/search?type=code&q=language%3Atoml+%22-fuse-ld%3Dlld%22), [4](https://github.com/search?type=code&q=language%3Arust+%22-fuse-ld%3Dlld%22)). By making LLD the default linker on the `x86_64-unknown-linux-gnu` target, we will be able to speed up Rust compilation out of the box, without users having to opt in or know about it. > You can find an extended version of this stabilization report which includes analysis of crater results and more data [here](https://hackmd.io/tFDifkUcSLGoHPBRIl0z8w?view). ## What is being stabilized - `rust-lld` being used as the default linker on the `x86_64-unknown-linux-gnu` target. - Note that `rust-lld` is being enabled by default in the compiler artifacts distributed by our CI/rustup. It is still possible to use the system linker by default using `rust.lld = false` in `bootstrap.toml`, which can be useful e.g. for some Linux distros that might not want to use the LLD we distribute. - This is done by activating the LLD linker feature and using the self-contained linker on that target. Both of which are also usable on the CLI, if some opt outs are necessary, as described below. - `-Clinker-features=-lld` on the `x86_64-unknown-linux-gnu` target. This codegen option tells rustc to disable using the LLD linker. - Note that other options for this codegen flag (`cc`) remain unstable. - Note that only the opt-out is being stabilized, and only for `x86_64-unknown-linux-gnu`: opting in, or using the flag on other targets would still need to pass `-Zunstable-options`. - This flag is being stabilized so that users can opt out of LLD on stable, which would it turn also opt out of using the self-contained linker (since it's an LLD). - `-Clink-self-contained=-linker`. This codegen option tells rustc to use the self-contained linker. It's not particularly useful to turn it on by itself, but when enabled and combined with `-Clinker-features=+lld`, rustc will use the `rust-lld` linker wrapper shipped with the compiler toolchain, instead of some `lld` binary that the linker driver will find in the `PATH`. - Note that other options for this codegen flag (other than the previously stable `y/yes/n/no`). - Note that only the opt-out is being stabilized, and only for `x86_64-unknown-linux-gnu`: opting in, or using this flag on other targets would still need to pass `-Zunstable-options`. - This flag is being stabilized so that users can opt out of using self-contained linking on stable. Doing this would then fall back to using the system `lld`. To opt out of using LLD, `RUSTFLAGS="-Clinker-features=-lld"` would be used. To opt out of using `rust-lld`, falling back to the LLD installed on the system, `RUSTFLAGS="-Clink-self-contained=-linker"` would be used. ## Tests When enabling `rust-lld` on nightly, we also switched x64 linux to use it at stage >= 1, meaning that all tests have been running with lld since May 2024, on CI as well as contributors' machines. (Post opt-dist tests also had been using it when running their test subset earlier than that). There are also a few tests dedicated to the CLI behavior, or ensuring the default linker is indeed the one we expect: - [link-self-contained-consistency](https://github.com/rust-lang/rust/blob/1117bc1e6ce049495b0044dfe756afafc817d2d7/tests/ui/linking/link-self-contained-consistency.rs): Checks that `-Clink-self-contained` options are not inconsistent (i.e. that passing both `+linker` and `-linker` is an error). - [link-self-contained-unstable](https://github.com/rust-lang/rust/blob/1117bc1e6ce049495b0044dfe756afafc817d2d7/tests/ui/linking/link-self-contained-unstable.rs): Checks that only the `-linker` and `y/yes/n/no` options for `-Clink-self-contained` are stable. - [linker-features-unstable-cc](https://github.com/rust-lang/rust/blob/1117bc1e6ce049495b0044dfe756afafc817d2d7/tests/ui/linking/linker-features-unstable-cc.rs): Checks that only the non-lld options of `-Clinker-features` are unstable. - [linker-features-lld-disallowed](https://github.com/rust-lang/rust/blob/1117bc1e6ce049495b0044dfe756afafc817d2d7/tests/ui/linking/linker-features-lld-disallowed.rs): Checks that `-Clinker-features=-lld` is only stable on `x86_64-unknown-linux-gnu`. - [link-self-contained-linker-disallowed](https://github.com/rust-lang/rust/blob/1117bc1e6ce049495b0044dfe756afafc817d2d7/tests/ui/linking/link-self-contained-linker-disallowed.rs): Checks that `-Clink-self-contained=-linker` is only stable on `x86_64-unknown-linux-gnu`. - [no-gc-encapsulation-symbols](https://github.com/rust-lang/rust/blob/1117bc1e6ce049495b0044dfe756afafc817d2d7/tests/ui/linking/no-gc-encapsulation-symbols.rs): Checks that that linker encapsulation symbols are not garbage collected by LLD, so that crates like [linkme](https://github.com/dtolnay/linkme) still work. - [rust-lld](https://github.com/rust-lang/rust/blob/1117bc1e6ce049495b0044dfe756afafc817d2d7/tests/run-make/rust-lld): Checks that LLD is actually used when enabled with `-Clinker-features=+lld` and `-Clink-self-contained=+linker`. - [rust-lld-x86_64-unknown-linux-gnu](https://github.com/rust-lang/rust/blob/1117bc1e6ce049495b0044dfe756afafc817d2d7/tests/run-make/rust-lld-x86_64-unknown-linux-gnu): Checks that LLD is used by default on `x86_64-unknown-linux-gnu` when the bootstrap `rust.lld` config option is `true`. - [rust-lld-x86_64-unknown-linux-gnu-dist](https://github.com/rust-lang/rust/blob/1117bc1e6ce049495b0044dfe756afafc817d2d7/tests/run-make/rust-lld-x86_64-unknown-linux-gnu-dist): Dist test that checks that our distributed `x86_64-unknown-linux-gnu` archives actually use LLD by default. ## Ecosystem impact As already stated, LLD has been used as the default linker on x64 Linux on nightly for almost a year, and we haven't seen any blockers to stabilization in that time. There were a handful of issues reported, these are discussed later below. Furthermore, two crater runs ([November 2023](https://crater-reports.s3.amazonaws.com/pr-117684-2/index.html), [February 2025](https://crater-reports.s3.amazonaws.com/pr-137044-3/index.html)), were performed to test the impact of using LLD as the default linker. A triage of the earlier crater run was previously done [here](https://hackmd.io/OAJxlxc6Te6YUot9ftYSKQ), but all the important findings from both crater runs are reported below. Below is a list of compatibility differences between BFD and LLD that we have encountered. There is a more thorough list of differences in [this post](https://maskray.me/blog/2020-12-19-lld-and-gnu-linker-incompatibilities) from the current LLD maintainer. From that post, "99.9% pieces of software work with ld.lld without a change". --- ### `.ctors/.dtors` sections [#128286](#128286) reported an issue where LLD was unable to link certain CUDA library was using these sections that were using the `.ctors/.dtors` ELF sections. These were deprecated a long time ago (https://gcc.gnu.org/bugzilla/show_bug.cgi?id=46770), replaced with a more modern `.init_array/.fini_array` sections. LLD doesn't (and won't) support these sections ([1](llvm/llvm-project#68071), [2](llvm/llvm-project#30572)), so if they appear in input object files, the linked artifact might produce incorrect behavior, because e.g. some global variables might not get initialized properly. However, the usage of `.ctors/.dtors` should be very rare in practice. We have performed a [crater run](#137044) to test this. It has identified only 8 crates where the `.ctors/.dtors` section is occurring in the final linked artifact. It was caused by a few crates using the `.ctors` link section manually, and by using a very (~6 year) old version of the [ctor](https://crates.io/crates/ctor) crate. [Crater run analysis](https://hackmd.io/tFDifkUcSLGoHPBRIl0z8w?view#ctorsdtors-sections) **Possible workaround** It is possible to [detect](e5e2316) if `.ctors/.dtors` section is present in the final linked artifact (LLD will keep it there, but it won't be populated), and warn users about it. This check is very cheap and doesn't even appear on [perf](#112049 (comment)). We have benchmarked the check on a 240 MiB Chrome binary, where it took 0.8ms with page cache flushed, and 0.06ms with page cache primed (which should be the common case, as the linked artifact is written to disk just before the check is performed). In theory, this could be also solved with a linker script that moves `.ctors` to `.init_array`. We think that these sections should be so rare that it is not worth it to implement any workarounds for now. --- ### Different garbage collection behavior [#130397](#130397) reported an issue where LLD prunes a local symbol, so it is missing in the linked artifact. However, BFD keeps the same symbol, so it is a regression. This is caused by a difference in linker garbage collection. Rust uses `--gc-sections` and puts each function into a separate linker section, which prunes unused code. There is some code (specifically the somewhat popular [linkme](https://github.com/dtolnay/linkme) crate) that (arguably ab-)uses so called linker encapsulation symbols to achieve distributed slices. BFD (2.37+) uses a conservative linking mode that works as intended with this behavior, but it might slightly increase binary size of the linked artifact. LLD does not use this workaround by default, which causes the sections to be eliminated, but it can be made to use the conservative mode using [`-z nostart-stop-gc`](https://lld.llvm.org/ELF/start-stop-gc.html#z-start-stop-gc). To avoid this issue, we told LLD to use the [conservative mode](#137685), which maintains backwards compatibility with BFD. We found that it has [no effect](#112049 (comment)) on compilation performance and binary size in our benchmark suite. With this change, `linkme` works. Since then, #140872 removed `linkme` distributed slice's dependence on conservative GC behavior, so this PR also removes that conservative mode: no transition period is necessary, as the PR immediately fixed the crate with no source changes. [Crater run analysis](https://hackmd.io/tFDifkUcSLGoHPBRIl0z8w?view#Different-garbage-collection-behavior) --- ### Various uncommon issues A small number of issues that only occurred in a handful of instances were found in crater, and it is unclear if LLD is at fault or if there is some other issue that was not detected with BFD. You can examine these [here](https://hackmd.io/tFDifkUcSLGoHPBRIl0z8w?view#Various-uncommon-issues). --- ### Missing jobserver support LLD doesn't support the jobserver protocol for limiting the number of threads used, it simply defaults to using all available cores, and is one of the reasons why it's faster than BFD. However, this should mostly be a non-issue, because most of the linking done during high parallelism sections of `cargo build` is linking of build scripts and proc macros, which are typically very fast to link (e.g. ~50ms), and a potential oversubscription of cores thus doesn't hurt that much. When the final artifact is linked (which typically takes the most time), there should be no other sources of parallelism conflicts from compiling other code, so LLD should be able to use all available threads. That being said, it is a difference of behavior, where previously a `-j` flag was generally not using more cpu than the specified limit. It can be impactful in some resource-constrained systems, but to be clear that is already the case today due to [cargo parallelism](rust-lang/cargo#9157). This could be one reason to opt out of using `rust-lld` on some systems. LLD has support for limiting the number of threads to use, so in theory rustc could try to get all the jobserver tokens available and use that as lld's thread limit. It'd still be suboptimal as new tokens would not be dynamically detected, and we could be using less threads than available. We did a benchmark on a real-world crate that shows that using multiple LLD threads for intermediate artifacts doesn't seem to have a performance effect. You can find it [here](https://hackmd.io/tFDifkUcSLGoHPBRIl0z8w?view#Missing-jobserver-support). --- #### Opting out of LLD in the ecosystem We have also examined repositories where people opted out of LLD on nightly, using [this GitHub query](https://github.com/search?q=%22linker-features%3D-lld%22&type=code). The summary can be found below: <details> <summary>Summary of LLD opt outs</summary> > This examination was performed on 2025-03-09. Here we briefly examine the most common reasons why people use `-Zlinker-features=-lld`, based on comments and git history. - Nix/NixOS ([1](https://github.com/rszyma/vscode-kanata/blob/59d703dff5a238b14ab3759cac27f73fb34bbcfe/flake.nix#L33), [2](https://github.com/sbernauer/breakwater/blob/3cc3449fc126c5c99d4a971733fd32be589884e0/.cargo/config.toml#L4), [3](https://github.com/tiiuae/ebpf-firewall/blame/32bdb17cedd1c9bea1ab3482623de458d95da7d0/.cargo/config.toml#L2), [4](https://github.com/jules-sommer/wavetheme-gen/blob/f5f657d014d4a30607625afb70f810c229c0294e/Cargo.toml#L4), [5](https://github.com/LayerTwo-Labs/zside-rust/blob/e4266f5c5571a1b180a9c70cf0939c7070e410c7/.cargo/config.toml#L10), [6](https://github.com/przyjacielpkp/zkml/blob/22a4aef24e9d2c77789229d7c634fc67e9eb1184/README.md?plain=1#L78), [7](https://github.com/LayerTwo-Labs/thunder-rust/blob/2222d53474c8d2d0428b4c56f8157095dced6d5a/.cargo/config.toml#L2), [8](https://github.com/enesoztrk/nixos-tc-aya-test/blob/b2ffa59d3eba8b60fd04b0a4c8bbe047400eb981/.cargo/config.toml#L4), [9](https://github.com/lowRISC/container-hotplug/blob/3ead4ef9c7f79c303392178c99677dbecff1aea6/.cargo/config.toml#L2), [10](https://github.com/Eliah-Lakhin/ad-astra/blob/ca6b8c8a5dba7bb5e894f3f1013f17876962a021/work/examples/lsp-client/src/extension.ts#L94)) - There was an [issue](NixOS/nixpkgs#312661) with LLD, which seems to have been fixed with NixOS/nixpkgs#314268. It's unclear whether that fixed all the Nix issues though. - Issues with linkme ([1](https://github.com/0xPolygonZero/zk_evm/blob/ef388619ffbd5305209519a3a5bc0396185d68ac/.cargo/config.toml#L4), [2](https://github.com/conjure-cp/conjure-oxide/blob/be0fc5827ff90e8486d416cc184b6ce24f73bf01/README.md?plain=1#L20), [3](https://github.com/clchiou/garage/blob/c5d8444d56bb6ee24ca95e5c6b9880ed996f4918/rust/.cargo/config.toml#L6), [4](https://github.com/PonasKovas/craftflow/blob/5b4cc1a5196e08a975368399fefda4b71f3a2f6f/.cargo/config.toml#L3), [5](https://github.com/kezhuw/zookeeper-client-rust/blob/4e27c7de2a7cc5e709af012b791c8fea9bb47f1f/.github/workflows/ci.yml#L82), [6](https://github.com/niklasdewally/conjure-oxide/blob/8fe60c12bca7011a2f9eded4b7c95ad0e77b6f44/.github/workflows/code-coverage.yml#L48), [7](https://github.com/kezhuw/spawns/blob/c8b468379805de9df3287c01b94b4ed3db6b61ed/.github/workflows/ci.yml#L74)) - These should be resolved with the conservative garbage collection ([#137685](#137685)). - Bazel ([1](https://github.com/google-parfait/confidential-federated-compute/blob/1823f69ed8f5f4f819f7bfa21da1ca629fdc826b/.bazelrc#L71)), WASM ([1](https://github.com/Eliah-Lakhin/ad-astra/blob/ca6b8c8a5dba7bb5e894f3f1013f17876962a021/work/examples/wasm-build.sh#L37), [2](https://github.com/yacineb/pgrx-wasi-test/blob/2bf99037ca1b650b2cbc35f1257a87fb6ead0920/build.sh#L21)), uncategorized ([2](https://github.com/nbdd0121/r2vm/blob/5118be6b9e757c6fef2f019385873f403c23c548/.cargo/config.toml#L3), [3](https://github.com/Wyvern/Img/blame/45020c7e1dc4926c8129647014c708db0c13f463/.cargo/config.toml#L209), [4](https://github.com/arnaudpoullet/leptos-i18n-compile-error/blob/042eb835f7ca0dc36be67cf7fe65b35b22b6059f/README.md?plain=1#L89), [5](https://github.com/JonLeeCon/numerical-rust-cpu/blob/fd0b3006768ed81c56147044dc05c92b11b7b6f0/exercises/.cargo/config.toml#L13), [6](https://github.com/PonasKovas/shallowclone/blob/be65f2ec923cac6ceedbc8db520c89969ebfce7c/.github/workflows/rust.yml#L20)) - Reason unclear. </details> ## History The idea to use a faster linker by default has been on the radar for quite some time ([#39915](#39915), [#71515](#71515)). There were [very early attempts](#29974) to use the gold linker by default, but these had to be [reverted](#30913) because of compatibility issues. Support for LLD was implemented back in [2017](#40018), but it has not been made default yet, except for some more niche targets, such as [WASM](#48125), [ARM Cortex](#53648) or [RISC-V](#53822). It took quite some time to figure out how should the interface for selecting the linker (and the way it is invoked) look like, as it differs a lot between different platforms, linkers and compiler drivers. During that time, LLD has matured and achieved [almost perfect compatibility](https://maskray.me/blog/2020-12-19-lld-and-gnu-linker-incompatibilities) with the default Linux linker (BFD). - [#56351](#56351) stabilized `-Clinker-flavor`, which is used to determine how to invoke the linker. It is especially useful on targets where selecting the linker directly with `-Clinker` is not possible or is impractical. - December 2018, author `@davidtwco,` reviewer `@nagisa` - [#76158](#76158) stabilized `-Clink-self-contained=[y|n]`, which allows overriding the compiler's heuristic for deciding whether it should use self-contained or external tools (linker, sanitizers, libc, etc.). It only allowed using the self-contained mode either for everything (`y`) or nothing (`n`), but did not allow granular choice. - September 2020, author `@mati864,` reviewer `@petrochenkov` - [#85961](#85961) implemented the `-Zgcc-ld` flag, which was a hacky way of opting into LLD usage. - June 2021, author `@sledgehammervampire,` reviewer `@petrochenkov` - [MCP 510](rust-lang/compiler-team#510) proposed stabilizing the behavior of `-Zgcc-ld` using more granular flags (`-Clink-self-contained=linker -Clinker-flavor=gcc-lld`). - Initially implemented in [#96827](#96827), but `@petrochenkov` [suggested](#96827 (comment)) a slightly different approach. - The PR was split into [#96884](#96884), where it was decided what will be the individual components of `-Clink-self-contained=linker`. - And [#96401](#96401), which implemented the `-Clinker-flavor` part. - The MCP was finally implemented in [#112910](#112910). - [#116514](#116514) then removed `-Zgcc-ld`, as it was replaced by `-Clinker-flavor=gnu-lld-cc` + `-Clink-self-contained=linker`. - April 2022 - October 2023, author `@lqd,` reviewer `@petrochenkov` - Various linker handling refactorings were performed in the meantime: [#97375](#97375), [#98212](#98212), [#100126](#100126), [#100552](#100552), [#102836](#102836), [#110807](#110807), [#101988](#101988), [#116515](#116515) - The implementation of linker flavors with LLD was causing a sort of a combinatorial explosion of various options. [#119906](#119906) suggested a different approach for linker flavors (described [here](#119906 (comment))), where the individual flavors could be enabled separately using `+/-` (e.g. `+lld`). - After some back and forth, this idea was moved to `-Clinker-features` (see [comment 1](#119906 (comment)) and [comment 2](#119906 (comment))), which was implemented in [#123656](#123656). - April 2024, author `@lqd,` reviewer `@petrochenkov` - [#124129](#124129) enabled LLD by default on nightly. - April 2024, author `@lqd,` reviewer `@petrochenkov` - [#137685](#137685), [#137926](#137926) enabled the conservative gargage collection mode (`-znostart-stop-gc`) to improve compatibility with BFD. - February 2025, author `@lqd,` reviewer `@petrochenkov` (implementation), author `@kobzol,` reviewer `@lqd` (test) - [#96025](#96025) (April 2022), [#117684](#117684) (November 2023), [#137044](#137044) (February 2025): crater runs. ## Unresolved questions/concerns - Is changing the linker considered a breaking change? In (hopefully very rare) cases, it might break some existing code. It should mostly only affect the final linked artifact, so it should be easy to opt out. - Similarly, is the single-threaded behavior of such tools encompassed in our stability guarantee: it can be observed via the `-j` job limit (though I believe we have/had some open issues on sometimes using more CPU resources than the job count limit implied). As mentioned above, LLD does not support the jobserver protocol. - A concern [was raised](#71515 (comment)) about increased memory usage of LLD. We should probably let users know about the possibly increased memory usage, and jobserver incompatibility: we did so when announcing this landing on nightly. - LLD seems to produce [slightly larger](https://perf.rust-lang.org/compare.html?start=b3e117044c7f707293edc040edb93e7ec5f7040a&end=baed03c51a68376c1789cc373581eea0daf89967&stat=size%3Alinked_artifact&tab=compile) binary artifacts. This can be partially clawed back using Identical Code Folding (`-Clink-args=-Wl,--icf=all`). - Should we detect the outdated `.ctors/.dtors` sections to provide a better error message, even if that should be rare in practice? --- ### Next steps After the FCP completes: - we should land this PR at the beginning of a beta cycle, to maximize time for testing - keep an eye on the beta crater run results for possible linker issues (or do a dedicated beta crater run with only this change) - release a blog post announcing the change, and asking for testing feedback of the appropriate beta - depending on feedback, or if a period of testing of 6 weeks is not long enough, we could keep this change on beta for another cycle --- Development, testing, try builds were done in #138645. r? `@petrochenkov` `@rustbot` label +needs-fcp +T-compiler

rust-highfive assigned petrochenkov May 7, 2022

rustbot added the T-compiler Relevant to the compiler team, which will review and decide on the PR/issue. label May 7, 2022

rust-highfive added the S-waiting-on-review Status: Awaiting review from the assignee but also interested parties. label May 7, 2022

lqd mentioned this pull request May 7, 2022

Implement -Zgcc-ld=lld stabilization MCP #96401

Closed

bjorn3 reviewed May 8, 2022

View reviewed changes

compiler/rustc_codegen_ssa/src/back/link.rs Show resolved Hide resolved

lqd force-pushed the mcp_linker_flavor branch from 49a06ab to 0e60302 Compare May 8, 2022 06:56

This comment was marked as resolved.

Sign in to view

lqd force-pushed the mcp_linker_flavor branch from 3ced29e to b9a413d Compare May 21, 2022 17:11

lqd force-pushed the mcp_linker_flavor branch from b9a413d to aab6ab2 Compare May 31, 2022 15:23

nagisa mentioned this pull request Jun 3, 2022

contribute-nearcore: add info on ROCKSDB_LIB_DIR and other minor updates near/docs#1076

Merged

lqd added 7 commits July 25, 2022 10:48

add dedicated -C linker-flavor structures

3c08036

add tests for LinkerFlavorCli options parsing

8ef3d5b

use LinkerFlavorCli for -C linker-flavor

cc5ac27

implement linker-flavor enrichments for MCP

045cb0a

add sanity check about -Zgcc-ld and -Clinker-flavor=gcc:*

8d80814

require -Z unstable-options to use the new linker-flavor flag values

877fa6b

only the stable values for `-Clinker-flavor` can be used on stable until we have more feedback on the interface

test warning about known issue 79555 with lld

6e0bff9

lqd changed the title ~~Implement unstable -Clinker-flavor=gcc:* for MCP 510~~ Implement unstable -Clinker-flavor=gcc:lld for MCP 510 Jul 25, 2022

petrochenkov reviewed Aug 9, 2022

View reviewed changes

petrochenkov added S-waiting-on-author Status: This is awaiting some action (such as code changes or more information) from the author. and removed S-waiting-on-review Status: Awaiting review from the assignee but also interested parties. labels Aug 9, 2022

petrochenkov mentioned this pull request Aug 14, 2022

rustc_target: Add a compatibility layer to separate internal and user-facing linker flavors #100552

Merged

petrochenkov mentioned this pull request Sep 18, 2022

rustc_target: Refactor internal linker flavors #101988

Merged

petrochenkov added a commit to petrochenkov/rust that referenced this pull request Oct 6, 2022

rustc_target: Refactor internal linker flavors

572b6a9

In accordance with the design from rust-lang#96827 (comment)

pnkfelix mentioned this pull request Mar 16, 2023

-C linker-flavor=clang rust-lang/compiler-team#601

Closed

3 tasks

lqd mentioned this pull request Jun 21, 2023

Implement most of MCP510 #112910

Merged

lqd closed this Jul 1, 2023

lqd mentioned this pull request Apr 30, 2025

Use lld by default on x86_64-unknown-linux-gnu stable #140525

Merged

Implement unstable -Clinker-flavor=gcc:lld for MCP 510 #96827

Implement unstable -Clinker-flavor=gcc:lld for MCP 510 #96827

Uh oh!

Conversation

lqd commented May 7, 2022 • edited Loading Uh oh! There was an error while loading. Please reload this page.

Uh oh!

Uh oh!

rust-highfive commented May 7, 2022

Uh oh!

Uh oh!

petrochenkov commented May 14, 2022

Uh oh!

This comment was marked as resolved.

petrochenkov commented May 21, 2022 • edited Loading Uh oh! There was an error while loading. Please reload this page.

Uh oh!

Uh oh!

petrochenkov commented May 21, 2022

Uh oh!

bors commented May 26, 2022

Uh oh!

lqd commented May 31, 2022

Uh oh!

petrochenkov commented Jun 17, 2022

Uh oh!

petrochenkov commented Jun 27, 2022

Uh oh!

bors commented Jul 14, 2022

Uh oh!

petrochenkov commented Aug 8, 2022 • edited Loading Uh oh! There was an error while loading. Please reload this page.

Uh oh!

Uh oh!

petrochenkov commented Aug 8, 2022

Uh oh!

petrochenkov commented Aug 9, 2022

Uh oh!

petrochenkov Aug 9, 2022

Choose a reason for hiding this comment

Uh oh!

petrochenkov Aug 9, 2022

Choose a reason for hiding this comment

Uh oh!

petrochenkov Aug 9, 2022

Choose a reason for hiding this comment

Uh oh!

petrochenkov Aug 9, 2022

Choose a reason for hiding this comment

Uh oh!

bors commented Sep 2, 2022

Uh oh!

lqd commented Jul 1, 2023

Uh oh!

Uh oh!

Implement unstable `-Clinker-flavor=gcc:lld` for MCP 510 #96827

Implement unstable `-Clinker-flavor=gcc:lld` for MCP 510 #96827

lqd commented May 7, 2022 •

edited

Loading

petrochenkov commented May 21, 2022 •

edited

Loading

petrochenkov commented Aug 8, 2022 •

edited

Loading