crane application in the mixed host-embedded project

[glaeqen]

Hello!

This is really a cool project, thanks for all the work put into it.

I wonder if there's a way to use crane to build a project with presumably less standard structure than usually.

│ // uses `host-native` target
├── bins
│   └── app-1
│       ├── src
│       └── Cargo.toml // uses libs from libs/
├── Cargo.toml // (workspace) uses libs from libs/
├── embedded-stm32xxxx // uses `thumbv8m-main-none-eabi` target
│   ├── app
│   │   ├── src
│   │   └── Cargo.toml
│   ├── Cargo.toml // (workspace) uses libs from libs/
│   └── hal
│       ├── src
│       └── Cargo.toml
├── embedded-nrf51xxxx // uses `thumbv6m-none-eabi` target
│   ├── app
│   │   ├── src
│   │   └── Cargo.toml
│   ├── Cargo.toml // (workspace) uses libs from libs/
│   └── hal
│       ├── src
│       └── Cargo.toml
└── libs // shared lib-crates
    └── lib-1
        ├── src
        └── Cargo.toml

Main problem solved here is having both host-target built and embedded-target built crates within the same repository and having them share certain lib-crates from that repo. Otherwise, putting all the crates together breaks cargo commands run from the root workspace due to incompatible targets and crate-features conflicts between them.

Segregating crates into their nested disjoint workspaces solves the problem. In my example it'd be 3 workspaces:

• <root>
• embedded-stm32xxxx
• embedded-nrf51xxxx

per-package-target cargo feature is still and for foreseeable future unstable which makes it unavailable on a stable toolchain without hacks. However, even if it was available, heavy abuse of mutually-exclusive features by embedded lib-crates often demands a deeper workspace split based not only on the microcontroller's CPU architecture but the family it belongs to.

This means that every workspace has its own target directory and before running a cargo command one has to cd into the relevant workspace. This is how I normally interact with them.

And this where my problem with crane starts because my impression is that it requires all cargo commands to be runnable from the root of the src passed into it.

Normally one could base src in the root of these workspaces but my workspaces point to the libs at the root of a repository (like ../libs/<crate>). Thus, any attempt to build a project with eg. src = ./embedded-stm32xxxx would cause crash as ../libs does not exist.

Certain things seem to be fixable or at least workaround-able, for example

• one could provide cargoLock manually to fix vendoring from the nested workspaces
• invoke cargo with --manifest-path <path> --target <target> through *ExtraArgs
• shadow mkCargoDerivation via overrideScope and add pre* hooks that cd into a desired workspaces

However the biggest (that is non-trivial) issue I found so far is that mkDummySrc would have to be (at least) extended to support my use-case. Even then, it might be that there's even more things that would need to be fixed.

My main question boils down to basically:

Is there

• any new meta of structuring mixed host-embedded projects that would "just work" with crane or
• a way to coerce crane into cooperation that is not a bunch of brittle hacks

Or would crane just have to be heavily extended to support my use-case?

I appreciate help - unless there's some other thread that I missed I think this one could become a good resource for Rust Embedded devs looking into Nix-based project caching.

[ipetkov]

Hello!

I wonder if there's a way to use crane to build a project with presumably less standard structure than usually.

Generally speaking, yes it should be possible, though there may be some extra steps you'd need to take when configuring/handling the derivations. How much of that is necessary will depend on your workflow!

This means that every workspace has its own target directory and before running a cargo command one has to cd into the relevant workspace. This is how I normally interact with them.

And this where my problem with crane starts because my impression is that it requires all cargo commands to be runnable from the root of the src passed into it.

Yes changing the directory before performing the build is totally possible: all the various build methods we expose are built on top of mkCargoDerivation which itself is based on top of stdenv.mkDerivation, which means all of the nixpkgs standard environment configuration options and workflows should be supported (and if they don't seem to be, feel free to open an issue about it).

Particularly you may be interested in using either of sourceRoot or setSourceRoot to control where the rest of the build will proceed from after unpacking. Alternatively you can always add your own logic (e.g. via a postUnpack or preConfigure hook for example) that will cd to the appropriate place.

Hope this helps!

[glaeqen]

Somehow I missed this obvious entry in the FAQ that talks about this, I think I dismissed it because it felt too simple to be the answer 🙈 I have still a lot to learn about mkDerivation, clearly. I tried your suggestion and it seemed to work! Thanks a lot for a quick response. I hope it's the last time I got stuck but we'll see :P