Limit toolchain version requirement to GPU code

[nazar-pc]

I've been looking at rust-gpu last two days and since toolchain upgrades are non-trivial and sometimes take a while to actually happen, I'm wondering if it would be possible to limit toolchain requirement to just the code that is compiled for GPU.

Specifically, SpirvBuilder calls cargo or rustc internally and could easily specify something like +nightly-whatever in CLI. This would result in backend matching what SPIR-V backend expects, while allowing the rest of user's workspace using whatever toolchain it uses by default.

Override with + has the highest priority, so this should work nicely.

Otherwise it requires much more elaborate setup with separate SPIR-V compilation and usage steps, probably separate workspaces, etc.

UPD: Looks like there is already toolchain_overwrite on the builder, now I'm wondering why its value doesn't default to this workspace's rust-toolchain.toml 🤔

UPD2: This is exactly what I'm talking about, it should just use correct toolchain by default and not check current toolchain/generate an error if there is a single version supported anyway:

   Compiling rustc_codegen_spirv v0.9.0 (https://github.com/Rust-GPU/rust-gpu?rev=e6d017d5504c4441a84edcc27f4eca61de6fc8cf#e6d017d5)
error: failed to run custom build command for `rustc_codegen_spirv v0.9.0 (https://github.com/Rust-GPU/rust-gpu?rev=e6d017d5504c4441a84edcc27f4eca61de6fc8cf#e6d017d5)`

Caused by:
  process didn't exit successfully: `/home/nazar-pc/.cache/cargo/target/debug/build/rustc_codegen_spirv-3c6c975d26047f08/build-script-build` (exit status: 1)
  --- stdout
  cargo:rerun-if-env-changed=RUSTGPU_SKIP_TOOLCHAIN_CHECK

  --- stderr
  error: wrong toolchain detected (found commit hash `c68340350c78eea402c4a85f8d9c1b7d3d607635`, expected `b19329a37cedf2027517ae22c87cf201f93d776e`).
  Make sure your `rust-toolchain.toml` file contains the following:
  -------------
  [toolchain]
  channel = "nightly-2024-11-22"
  components = ["rust-src", "rustc-dev", "llvm-tools"]
  -------------
Exit code 101

UPD3: This is even more problematic. Why does it check toolchain components if I explicitly used `` feature so it doesn't do that? I really don't want to add rustc-dev to the workspace since I'll not be using it anyway 😕

error: failed to run custom build command for `rustc_codegen_spirv v0.9.0 (https://github.com/Rust-GPU/rust-gpu?rev=e6d017d5504c4441a84edcc27f4eca61de6fc8cf#e6d017d5)`

Caused by:
  process didn't exit successfully: `/home/nazar-pc/.cache/cargo/target/debug/build/rustc_codegen_spirv-909afb669bc3840e/build-script-build` (exit status: 1)
  --- stdout
  cargo:rerun-if-env-changed=RUSTGPU_SKIP_TOOLCHAIN_CHECK

  --- stderr
  missing `rustc-dev` component from toolchain `nightly-2025-06-19-x86_64-unknown-linux-gnu` (at /home/nazar-pc/.rustup/toolchains/nightly-2025-06-19-x86_64-unknown-linux-gnu)
warning: build failed, waiting for other jobs to finish...
Exit code 101

UPD4: Looks like I hit #104 in the end.

[LegNeato]

It is totally possible to do this and what most folks do! While it is currently manual, we have an alpha tool that automates it: https://github.com/Rust-GPU/cargo-gpu.

[nazar-pc]

I'm not sure it does what I need here.

I'd like to build shaders in build.rs and I want to specify a toolchain to use in there (because toolchain of the workspace will certainly be different). Right now inclusion of rustc_codegen_spirv dependency triggers its build.rs, which expects a specific toolchain version. What I'd like here is to compile rustc_codegen_spirv itself for a custom toolchain too.

Probably something like rustc_codegen_spirv_builder that I can call programmatically and then set builder.rustc_codegen_spirv_location and other stuff manually.

[LegNeato]

cargo gpu's intention was to support the exact use-case you describe, but it is still alpha and may not be there. But the idea of "compile CPU code with a modern rust, GPU code with the specific nightly rust-gpu needs" is a supported workflow that most projects use. We haven't yet written it up, and cargo gpu is supposed to automate most if not all of it.

@Firestar99 , you use cargo-gpu in build.rs, right? It wasn't how it was intended to be used but I believe it works today. Can you give some insight here?

[nazar-pc]

Interesting! It would have been nice if it wasn't necessary to use a custom wrapper, especially since the project has a lot of crates and only a single one of them has GPU code. I also don't see if it could run things like clippy, nextest and others that I use regularly.

In the end I see the value for workspace that is primarily GPU-focused, but otherwise I'd strongly prefer for cargo build to be able to do the right thing. The more straightforward it is, the better. I'll experiment some more tomorrow and share if I come up with a satisfactory solution.

[Firestar99]

See Rust-GPU/cargo-gpu#71 for sample code :D

Otherwise, you should be able to run clippy and nextest just fine, with the target being your CPU. Maybe this isn't obvious: you can have your CPU crate depend on the shader crate, which allows you to reuse all your non-GPU specific structs and algorithms (Limitations: #[repr(C)] and don't use Vec3). Similarly, you can test non-GPU specific code with normal test and run them on the CPU, only the entry points are disabled and GPU intrinsics don't work.

[nazar-pc]

I meant that by simply depending on rustc_codegen_spirv my crate was not easy to compile with something like cargo clippy --all-targets in a workspace.

Rust-GPU/cargo-gpu#71 seems to be exactly what I need, will give it a try, thanks!

[Firestar99]

Note that clippy currently fails in projects with such a build script... Rust-GPU/cargo-gpu#77

[nazar-pc]

There has been some workarounds needed (including for Clippy), but in the end I came up with this build.rs using latest cargo-gpu library revision:

use cargo_gpu::spirv_builder::{MetadataPrintout, SpirvMetadata};
use std::path::PathBuf;
use std::{env, fs};

fn main() -> Result<(), Box<dyn std::error::Error>> {
    let target_arch = env::var("CARGO_CFG_TARGET_ARCH").expect("Always set by Cargo; qed");

    if target_arch != "spirv" {
        let out_dir = env::var("OUT_DIR").expect("Always set by Cargo; qed");

        // Skip compilation under Clippy, it doesn't work for some reason and isn't really needed
        // anyway
        if env::var("CLIPPY_ARGS").is_ok() {
            let empty_file = PathBuf::from(out_dir).join("empty.bin");
            fs::write(&empty_file, [])?;
            println!("cargo::rustc-env=SHADER_PATH={}", empty_file.display());

            return Ok(());
        }

        let cargo_manifest_dir = env::var("CARGO_MANIFEST_DIR").expect("Always set by Cargo; qed");
        let profile = env::var("PROFILE").expect("Always set by Cargo; qed");

        let shader_crate = PathBuf::from(cargo_manifest_dir);

        // TODO: Workaround for https://github.com/Rust-GPU/cargo-gpu/issues/90
        let cargo_target_dir = env::var("CARGO_TARGET_DIR").ok();
        // SAFETY: Single-threaded
        unsafe {
            env::remove_var("CARGO_TARGET_DIR");
        }

        let backend = cargo_gpu::Install::from_shader_crate(shader_crate.clone()).run()?;

        // TODO: Workaround for https://github.com/Rust-GPU/cargo-gpu/issues/90
        if let Some(cargo_target_dir) = cargo_target_dir {
            // SAFETY: Single-threaded
            unsafe {
                env::set_var("CARGO_TARGET_DIR", cargo_target_dir);
            }
        }

        let mut builder = backend.to_spirv_builder(shader_crate, "spirv-unknown-vulkan1.1");
        builder.print_metadata = MetadataPrintout::DependencyOnly;
        builder.spirv_metadata = if profile == "debug" {
            SpirvMetadata::NameVariables
        } else {
            SpirvMetadata::None
        };
        // Avoid Cargo deadlock
        builder.target_dir_path.replace(out_dir);

        let compile_result = builder.build()?;
        let path_to_spv = compile_result.module.unwrap_single();

        println!("cargo::rustc-env=SHADER_PATH={}", path_to_spv.display());
    }

    Ok(())
}

This crate is basically both shader and host conditionally compiled, and shader itself is embedded into the library at compile time:

const SHADER: ShaderModuleDescriptor<'static> = {
    assert!(
        u16::from_ne_bytes(1u16.to_le_bytes()) == 1u16,
        "Only little-endian platform supported"
    );

    #[repr(align(4))]
    struct ShaderBytes<T: ?Sized>(T);

    const SHADER_BYTES_INTERNAL: &ShaderBytes<[u8]> =
        &ShaderBytes(*include_bytes!(env!("SHADER_PATH")));

    assert!(align_of_val(SHADER_BYTES_INTERNAL) == align_of::<u32>());
    let shader_bytes = &SHADER_BYTES_INTERNAL.0;

    // SAFETY: Correctly aligned, all bit patterns are valid, lifetime is static before and after
    let shader_bytes = unsafe {
        slice::from_raw_parts(
            shader_bytes.as_ptr().cast::<u32>(),
            shader_bytes.len() / size_of::<u32>(),
        )
    };

    ShaderModuleDescriptor {
        label: Some(env!("CARGO_PKG_NAME")),
        source: ShaderSource::SpirV(Cow::Borrowed(shader_bytes)),
    }
};

Hopefully less boilerplate will be needed in the future.

[Firestar99]

Some notes on your scripts:

• Having the shader crate compile the shader crate works, but may be bad for compile times:
  • First, building your shader crate builds cargo gpu and build script, runs it, which in turn builds cargo gpu and build script again in the shader target directory, even if your build script just early returns
  • Second, it makes your compile process very sequential, as you first need to compile the shaders (where linking is often the slowest part), then compile the shader crate for the CPU and then your CPU crate. By having your CPU crate build your shader crate, you can parallelize the building of the shader crate for the spirv and CPU target.
• setting CARGO_TARGET_DIR will cause the shaders to be build directly in target and not in target/spirv-builder
  • If shader and cpu target dirs overlap, they can cause constant recompilation instead of proper caching. Can also be caused by multiple shader crates with a different set of capabilities or features.
  • From SpirvBuilder:
    

    rust-gpu/crates/spirv-builder/src/lib.rs

    Lines 432 to 436 in e6d017d

    	/// Set the target dir path to use for building shaders. Relative paths will be resolved
    	/// relative to the `target` dir of the shader crate, absolute paths are used as is.
    	/// Defaults to `spirv-builder`, resulting in the path `./target/spirv-builder`.
    	#[cfg_attr(feature = "clap", clap(skip))]
    	pub target_dir_path: Option<PathBuf>,

• wgpu already has some solutions mentioned in the docs on how to get a &[u32] properly included: https://docs.rs/wgpu/latest/wgpu/enum.ShaderSource.html#variant.SpirV

[nazar-pc]

First, building your shader crate builds cargo gpu and build script, runs it, which in turn builds cargo gpu and build script again in the shader target directory, even if your build script just early returns

Yes, in fact I tried to gate it for non-spirv target, but Cargo ignored that part, likely because build script is compiled for native platform (will try to add internal feature to exclude it).

Second, it makes your compile process very sequential, as you first need to compile the shaders (where linking is often the slowest part), then compile the shader crate for the CPU and then your CPU crate. By having your CPU crate build your shader crate, you can parallelize the building of the shader crate for the spirv and CPU target.

True, but it is probably fine in my case, I'll make a split if it ever becomes a bottleneck.

setting CARGO_TARGET_DIR will cause the shaders to be build directly in target and not in target/spirv-builder

Yes, but it is set in my ~/.profile globally, not just for this project, so ideally cargo-gpu would handle it more gracefully, hence Rust-GPU/cargo-gpu#90

If shader and cpu target dirs overlap, they can cause constant recompilation instead of proper caching. Can also be caused by multiple shader crates with a different set of capabilities or features.

Yeah, this is why I'm setting target_dir_path for shader compilation, so it is isolated in its own small place.

wgpu already has some solutions mentioned in the docs on how to get a &[u32] properly included: https://docs.rs/wgpu/latest/wgpu/enum.ShaderSource.html#variant.SpirV

And I did reach out to them first, but neither function nor macro are const. My solution though will result in a desired data structure being embedded into the binary with compile-time checks. I was lazy to check for magic bytes (and wgpu wasn't nice enough to make the constant public), but it is good enough.

[nazar-pc]

This this before calling building shader:

env::set_var(
    "RUSTGPU_RUSTFLAGS",
    format!(
        "{} --cfg cargo_gpu_internal_skip",
        env::var("RUSTGPU_RUSTFLAGS").unwrap_or_default()
    ),
);

Together with this:

[target.'cfg(not(cargo_gpu_internal_skip))'.build-dependencies]
cargo-gpu = { workspace = true }

I expected it to not compile cargo-gpu, but it doesn't work: rust-lang/cargo#4423