Unable to build static binary for musl with C dependencies

[uulm-janbaudisch]

I am trying to build a static executable for the x86_64-unknown-linux-musl target.

Here is an example repo for this problem: https://github.com/uulm-janbaudisch/nix-rust-static
The following actions run shows the errors: https://github.com/uulm-janbaudisch/nix-rust-static/actions/runs/9208714825/job/25331530984

Flake

{
  description = "Building static binaries with musl";

  inputs = {
    nixpkgs.url = "github:NixOS/nixpkgs/nixos-23.11";
    flake-utils.url = "github:numtide/flake-utils";
    crane = {
      url = "github:ipetkov/crane/v0.17.1";
      inputs.nixpkgs.follows = "nixpkgs";
    };
    fenix = {
      url = "github:nix-community/fenix";
      inputs.nixpkgs.follows = "nixpkgs";
    };
  };

  outputs = { nixpkgs, crane, fenix, flake-utils, ... }:
    flake-utils.lib.eachDefaultSystem (system:
      let
        pkgs = import nixpkgs { inherit system; };

        lib = pkgs.lib;

        toolchain = with fenix.packages.${system};
          combine [
            stable.defaultToolchain
            targets.x86_64-unknown-linux-musl.stable.rust-std
          ];

        craneLib = (crane.mkLib pkgs).overrideToolchain toolchain;
        craneLibStatic = (crane.mkLib pkgs.pkgsStatic).overrideToolchain toolchain;

        crate = {
          src = craneLib.cleanCargoSource (craneLib.path ./.);
          strictDeps = true;
        };

        crate-static = crate // {
          CARGO_BUILD_TARGET = "x86_64-unknown-linux-musl";
          CARGO_BUILD_RUSTFLAGS = "-C target-feature=+crt-static";
        };

        cargoArtifacts = craneLib.buildDepsOnly crate;
        cargoArtifacts-static = craneLib.buildDepsOnly crate-static;
        cargoArtifacts-pkgsStatic = craneLibStatic.buildDepsOnly crate-static;
      in {
        formatter = pkgs.nixfmt;
        packages = {
          default = craneLib.buildPackage (crate // { inherit cargoArtifacts; });
          deps = cargoArtifacts;
          static = craneLib.buildPackage (crate-static // { cargoArtifacts = cargoArtifacts-static; });
          deps-static = cargoArtifacts-static;
          pkgsStatic = craneLibStatic.buildPackage (crate-static // { cargoArtifacts = cargoArtifacts-pkgsStatic; });
          deps-pkgsStatic = cargoArtifacts-pkgsStatic;
        };
      });
}

When not using dependencies that build C code, the approach from the documentation works fine.

When using a crate like mimalloc that builds C code this approach seems to not work for me.
In this case I get errors like the following:

       > error: linking with `cc` failed: exit status: 1
       >   |
       >   = note: LC_ALL="C" PATH="/nix/store/5494br3jzsv0c9w143dzf6w7v538bl1x-rust-mixed/lib/rustlib/x86_64-unknown-linux-gnu/bin:/nix/store/5494br3jzsv0c9w143dzf6w7v538bl1x-rust-mixed/lib/rustl
       >   = note: /nix/store/01bi48ga8vzbz4zgh7g3bsl620cj27kj-binutils-2.40/bin/ld: /build/source/target/x86_64-unknown-linux-musl/release/deps/liblibmimalloc_sys-b7b4aff6726784d7.rlib(98cfcaec7182b1d8-static.o): in function `mi_out_buf':
       >           static.c:(.text.mi_out_buf+0x73): undefined reference to `__memcpy_chk'
       >           /nix/store/01bi48ga8vzbz4zgh7g3bsl620cj27kj-binutils-2.40/bin/ld: /build/source/target/x86_64-unknown-linux-musl/release/deps/liblibmimalloc_sys-b7b4aff6726784d7.rlib(98cfcaec7182b1d8-static.o): in function `mi_out_buf_stderr':
       >           static.c:(.text.mi_out_buf_stderr+0x7b): undefined reference to `__memcpy_chk'
       >           /nix/store/01bi48ga8vzbz4zgh7g3bsl620cj27kj-binutils-2.40/bin/ld: /build/source/target/x86_64-unknown-linux-musl/release/deps/liblibmimalloc_sys-b7b4aff6726784d7.rlib(98cfcaec7182b1d8-static.o): in function `mi_manage_os_memory_ex2':
       >           static.c:(.text.mi_manage_os_memory_ex2+0x166): undefined reference to `__memset_chk'
       >           /nix/store/01bi48ga8vzbz4zgh7g3bsl620cj27kj-binutils-2.40/bin/ld: /build/source/target/x86_64-unknown-linux-musl/release/deps/liblibmimalloc_sys-b7b4aff6726784d7.rlib(98cfcaec7182b1d8-static.o): in function `mi_heap_realloc_aligned':
       >           static.c:(.text.mi_heap_realloc_aligned+0x9b): undefined reference to `__memcpy_chk'
       >           collect2: error: ld returned 1 exit status
       >           
       >   = note: some `extern` functions couldn't be found; some native libraries may need to be installed or have their path specified
       >   = note: use the `-l` flag to specify native libraries to link
       >   = note: use the `cargo:rustc-link-lib` directive to specify the native libraries to link with Cargo (see https://doc.rust-lang.org/cargo/reference/build-scripts.html#rustc-link-lib)
       >
       > error: could not compile `hello` (bin "hello") due to 1 previous error

These look like glibc internals. So I guess somehow for the build of the C code of mimalloc a glibc gcc is used.

When using pkgs.pkgsStatic instead of pkgs, I get the following error:

       > ++ command cargo check --release --locked --all-targets
       >    Compiling cc v1.0.98
       >    Compiling libc v0.2.155
       >    Compiling hello v0.1.0 (/build/source)
       > error: linker `cc` not found
       >   |
       >   = note: No such file or directory (os error 2)

However, when entering a development shell for the dependencies (nix develop .#deps-pkgsStatic in case of the example), a cargo build invocation does work and indeed produces a statically linked binary.

Am I missing some confiugration or setting up something wrong?

[ipetkov]

Hi @uulm-janbaudisch, I don't have a lot of experience with either musl or mimalloc so take this with a big grain of salt!

I'm not entirely certain if mimalloc itself supports being built against musl directly (perhaps it assumes certain glibc symbols?). According to https://www.tweag.io/blog/2023-08-10-rust-static-link-with-mimalloc/ it sounds like it might need some patches to be compatible?

[uulm-janbaudisch]

Thank you for your reply!
What surprises me is the fact that entering a development shell and running cargo build does work. It seems to be a problem of how the C code of mimalloc is built when doing it via the Nix derivation. It could be the wrong compiler being used.
I will try to find out what exactly goes wrong in the build process.

[uulm-janbaudisch]

So it seems the only this missing was TARGET_CC. The following got a sample project with mimalloc working:

TARGET_CC =
  let
    cc = pkgs.pkgsStatic.stdenv.cc;
  in
  "${cc}/bin/${cc.targetPrefix}cc";

[TristanCacqueray]

Thank you, that also works for liblzma!