Build cargoArtifacts (craneLib.buildDepsOnly) with the wasm32-unknown-unknown target can fail due to incompatible libs

[kutu-dev]

Describe the bug
Build cargoArtifacts (craneLib.buildDepsOnly) with the wasm32-unknown-unknown can fail due to incompatible libs, if in the workspace Cargo.toml there is listed a dependency not available on the WASM32 target (cmd_lib for example) given that there is no official workspace.target.*.dependencies key Crane will try to build it and fail in the process.

Making a system to flag a workspace dependency as not available on the WASM32 target may be required.

Reproduction
flake.nix

{
  description = "Build a cargo workspace";

  inputs = {
    nixpkgs.url = "github:NixOS/nixpkgs/nixpkgs-unstable";

    crane.url = "github:ipetkov/crane";

    flake-utils.url = "github:numtide/flake-utils";

    advisory-db = {
      url = "github:rustsec/advisory-db";
      flake = false;
    };
  };

  outputs =
    {
      self,
      nixpkgs,
      crane,
      flake-utils,
      advisory-db,
      ...
    }:
    flake-utils.lib.eachDefaultSystem (
      system:
      let
        pkgs = nixpkgs.legacyPackages.${system};

        inherit (pkgs) lib;

        craneLib = crane.mkLib pkgs;
        src = craneLib.cleanCargoSource ./.;

        commonArgs = {
           # >>> Here is the cross compilation to WASM32 set
           CARGO_BUILD_TARGET = "wasm32-unknown-unknown";

          inherit src;
          strictDeps = true;

          buildInputs =
            [
              # Add additional build inputs here
            ]
            ++ lib.optionals pkgs.stdenv.isDarwin [
              # Additional darwin specific inputs can be set here
              pkgs.libiconv
            ];
        };

        cargoArtifacts = craneLib.buildDepsOnly commonArgs;

        individualCrateArgs = commonArgs // {
          inherit cargoArtifacts;
          inherit (craneLib.crateNameFromCargoToml { inherit src; }) version;
          doCheck = false;
        };

        fileSetForCrate =
          crate:
          lib.fileset.toSource {
            root = ./.;
            fileset = lib.fileset.unions [
              ./Cargo.toml
              ./Cargo.lock
              (craneLib.fileset.commonCargoSources ./crates/my-common)
              (craneLib.fileset.commonCargoSources ./crates/my-workspace-hack)
              (craneLib.fileset.commonCargoSources crate)
            ];
          };

        my-cli = craneLib.buildPackage (
          individualCrateArgs
          // {
            pname = "my-cli";
            cargoExtraArgs = "-p my-cli";
            src = fileSetForCrate ./crates/my-cli;
          }
        );
      in
      {
        packages = {
          inherit my-cli;
        };
      }
    );
}

Cargo.toml

[workspace]
resolver = "2"
members = ["crates/*"]

[workspace.package]
version = "0.1.0"
edition = "2021"

[workspace.metadata.crane]
name = "my-workspace"

[workspace.dependencies]
cmd_lib = "1.9.6"

[ipetkov]

Hi @kutu-dev thanks for the report!

Could you please provide a full working example/reproduction of the issue? it can be a sample repo or even a gist, so long as all the files are present and I can run nix flake check ... on it

Build cargoArtifacts (craneLib.buildDepsOnly) with the wasm32-unknown-unknown can fail due to incompatible libs

Based on this bit of information I want to see a full reproduction to rule out any potential project misconfiguration. Normally individual crates should be defining platform-specific dependencies where appropriate (e.g. something like [target.'cfg(not(target_arch = "wasm32"))'.dependencies]). buildDepsOnly effectively delegates to running cargo {clippy,build,test} which should respect the dependency tree (i.e. not build crates for the "wrong" platform) but if the dependencies themselves aren't declared properly it could result to build errors

[kutu-dev]

Hi @kutu-dev thanks for the report!

Could you please provide a full working example/reproduction of the issue? it can be a sample repo or even a gist, so long as all the files are present and I can run nix flake check ... on it

Build cargoArtifacts (craneLib.buildDepsOnly) with the wasm32-unknown-unknown can fail due to incompatible libs

Based on this bit of information I want to see a full reproduction to rule out any potential project misconfiguration. Normally individual crates should be defining platform-specific dependencies where appropriate (e.g. something like [target.'cfg(not(target_arch = "wasm32"))'.dependencies]). buildDepsOnly effectively delegates to running cargo {clippy,build,test} which should respect the dependency tree (i.e. not build crates for the "wrong" platform) but if the dependencies themselves aren't declared properly it could result to build errors

Here is a minimal reproducible example: https://github.com/kutu-dev/reproduce-crane-issue-873

If you run either nix build .#myCli or nix build .#myLib things will work just fine, but if you run nix build .#myLibWasm32 you will end up with a weird error message about a missing OwnedFd type. This is caused by the crate cmd_lib, listed in the workspace.depedencies entry, not being available on the WASM32 target, even if it's not used by the myLib local package.

I would be useful, given that there is no way to gate workspace dependencies to only a set of targets, to have some sort of denyList on craneLib.buildDepsOnly to avoid issues with crates like windows-sys, cmd_lib, etc.

As a side note in the Rust RFC2906 it is explicitly written that:

The [workspace] table will not support other kinds of dependencies like dev-dependencies, build-dependencies, or target."...".dependencies. Only [workspace.dependencies] will be supported.

So we're out of luck of requesting this to the upstream. Thanks for this great library!

[ipetkov]

Ah thank you for the full reproduction!

So the solution here is what I hinted at above: using a platform-specific dependency in the crate which consumes the workspace-defined dependency; by applying this diff, all three derivations successfully build in your flake:

diff --git a/crates/my-cli/Cargo.toml b/crates/my-cli/Cargo.toml
index 9787a50..1f6a5d1 100644
--- a/crates/my-cli/Cargo.toml
+++ b/crates/my-cli/Cargo.toml
@@ -3,5 +3,5 @@ name = "my-cli"
 edition.workspace = true
 publish = false
 
-[dependencies]
+[target.'cfg(not(target_arch = "wasm32"))'.dependencies]
 cmd_lib.workspace = true

Even though workspace dependencies do not support per-platform granularity, the crates which have to consume them can define that conditional dependency. In this case cmd_lib does not build under wasm (and perhaps it isn't even intended to) so you have to make sure it is not part of your dependency tree when building on wasm.

I would be useful, given that there is no way to gate workspace dependencies to only a set of targets, to have some sort of denyList on craneLib.buildDepsOnly to avoid issues with crates like windows-sys, cmd_lib, etc.

crane is very much designed around embracing cargo so unless cargo has a way of opting out of building specific crates (e.g. via specific CLI arg) there isn't much we can do. But really the answer is carefully controlling which crates are required within your workspace.

Hope this helps!

[kutu-dev]

Oh I didn't know that the dependencies were resolved that way, thanks for the help!

Maybe It would be interesting to take a note about this on the docs.