Add compiler performance survey

[Kobzol]

As discussed at the All Hands, I would like to run a survey focused on compilation performance, as it is a recurring pain point for many Rust users, based on the results of our Annual Survey and also many online and offline conversations and experience reports.

The goal of the survey is to get more data on which kinds of compilation workflows are important for Rust users, where do they struggle with compilation performance and which workarounds do they currently use to improve their build times.

The survey also contains a few questions taken from the Annual survey, mostly for determining some basic information about the users (if they work at a company, if they use Cargo, which OS do they use).

[celinval]

I wonder if we can increase the scope of this survey to include a few other things that are cargo related, like usage of build.rs / proc_macro, usage of remote caching. @epage, might have some good suggestions.

[Kobzol]

Thanks everyone for the feedback! Did a pass through the survey and made some modifications based on the comments here.

One thing I haven't incorporated yet are more CI/CI caching/Docker questions. I'm not sure where to put them.

[the8472]

I think we should ask people about their hardware too. Being stingy about CI spending or doing things on an anemic laptop or over sshfs will be quite different compared to some beefy workstation or build servers.

[Kobzol]

I think we should ask people about their hardware too. Being stingy about CI spending or doing things on an anemic laptop or over sshfs will be quite different compared to some beefy workstation or build servers.

I was thinking about asking people to paste in their result of the Steam hardware survey 😆 Good point, although I don't want to spend many questions on this. Should we ask about the number of cores and available RAM? Or something else?

[programmerjake]

I was thinking about asking people to paste in their result of the Steam hardware survey 😆

please don't ask respondents to run proprietary software like Steam.

Good point, although I don't want to spend many questions on this. Should we ask about the number of cores and available RAM? Or something else?

imo we should ask what kind of cpu cores too, since a 1-core CI VM running on latest gen x86 hardware is waaay faster than 1 core on a raspberry pi or a pentium 4, to the point that 2 pentium 4 cores is probably much slower than 1 zen4 core.

[Kobzol]

please don't ask respondents to run proprietary software like Steam.

That was a joke, don't worry 😆

imo we should ask what kind of cpu cores too, since a 1-core CI VM running on latest gen x86 hardware is waaay faster than 1 core on a raspberry pi or a pentium 4, to the point that 2 pentium 4 cores is probably much slower than 1 zen4 core.

It would be interesting to see their hardware architecture, although just from the question about the used OS we will probably know almost everything (Windows = x64, macOs = ARM64, Linux = x64, maaybe ARM64). I'm not sure how exactly to ask that to be simple and understandable.

[Kobzol]

Thanks everyone for the feedback! I put the survey into SurveyHero, so that we can see it "in action": https://www.surveyhero.com/c/rust-compiler-performance-2025 Feel free to suggest more feedback!

I would like to start the survey in June, and let it run for 2/3 weeks. I'll also write an accompanying blog post on the main Rust blog. (I'll delete the answers before publishing the survey, so feel free to enter whatever data you want right now).

[programmerjake]

It would be interesting to see their hardware architecture, although just from the question about the used OS we will probably know almost everything (Windows = x64, macOs = ARM64, Linux = x64, maaybe ARM64). I'm not sure how exactly to ask that to be simple and understandable.

I've used a POWER9 machine running Linux, so not all Linux is x86 or Arm. Maybe if they're running Linux ask them to post the output of lscpu --json if they want? (I'm sure Windows and macOS have dialogs for the equivalent)

[Kobzol]

I heard an interesting idea about doing a "poor man's compiler performance metrics gathering", by essentially putting a free-form text field at the end, where people could copy-paste the output of some cargo command that we propose to them, which would build their crate and give us some profiling/metrics. We could then aggregate this data and get some (hopefully useful) real-world compilation statistics.

What do you think of this? And does anyone have a command in mind that produces useful information, but does not produce gigabytes of data? :) Maybe something like RUSTC_BOOTSTRAP=1 cargo build -Zunstable-options --timings=json or RUSTC_BOOTSTRAP=1 RUSTFLAGS="-Ztime-passes" cargo build?

[apiraino]

Maybe something like RUSTC_BOOTSTRAP=1 cargo build -Zunstable-options --timings=json or RUSTC_BOOTSTRAP=1 RUSTFLAGS="-Ztime-passes" cargo build?

is there anything that should be redacted or trimmed (like usernames, paths, etc) in such output?

[Kobzol]

Both outputs contain file paths for local crates, yeah. We could ask people to trim them manually, or provide some script for that, but it complicates things 🤔 @epage Do you know of a way to tell Cargo to not output thing like Compiling build_helper v0.1.0 (/projects/personal/rust/rust/src/build_helper) during a build?

[Kobzol]

Based on https://rust-lang.zulipchat.com/#narrow/channel/335408-foundation/topic/Privacy.2Flegal.20repercussions.20of.20running.20micro-surveys/with/522706782, I decided to not to add any PII questions from the survey, including contact questions and pasted outputs of Cargo/rustc commands, otherwise we would have to involve the Foundation and the survey would get more complicated (we can potentially include that in next iterations of the survey, we'll see how the first one goes).

I'd like to start the survey in exactly one week, 16th June. If anyone has more feedback, please write here until then.

[Kobzol]

Added question about CI caching solutions.

[Kobzol]

After going through the survey a few times, I added some finishing touches. Unless someone objects, I'm planning to start the survey today (https://www.surveyhero.com/c/rust-compiler-performance-2025).

I know that it's likely not perfect in many aspects, but I'd like to ship something, to get some data out of it. I expect/hope to run this survey multiple times, we can iterate on it progressively.