cargoToml/cargoLock when absolute paths are not possible?

[charludo]

Hi, I am trying to build a project using crane from outside the project dir.

More specifically, our project depends on a rust project, which we need to compile both locally and in CI. We obviously can't guarantee that the rust project's source is available locally, so something like this:

cargoToml = ./src/agent/Cargo.toml;
cargoLock = ./src/agent/Cargo.lock;

is out of the question. (Also, the files are not situated in the root, as can be seen above.)

Instead, I have a derivation agent with agent.src is produced by fetching the project from github with fetchFromGithub, then applying our own patches to it via applyPatches.

Crane is indeed able to successfully build the project when specifying the above options as:

cargoToml = "${agent.src}/src/agent/Cargo.toml";
cargoLock = "${agent.src}/src/agent/Cargo.lock";

However, making changes to a patch triggers a full rebuild of all dependencies. I think the reason is that if a patch is added or changed, the store path to the source changes, including the path to the toml/lock files; crane assumes the files have changed, and initiates a complete rebuild of all deps.

(Please correct me if I'm wrong though!)

Is there a workaround for this situation?

[ipetkov]

Crane is indeed able to successfully build the project when specifying the above options as:

cargoToml = "${agent.src}/src/agent/Cargo.toml";
cargoLock = "${agent.src}/src/agent/Cargo.lock";

Using an invocation like this is effectively capturing the entirety of agent.src and like you mentioned, any change to the agent source will result in a new hash and therefore rebuild everything.

My advice is to try to one of the following and use what works best for you:

1. Try to scope the path to result in only the Cargo.{toml,lock} to get added to the store. The incantations for this are always a bit strange but it might be something like cargoToml = agent.src + /src/agent/Cargo.toml;
2. Try reading those files at evaluation time and pass in the contents directly, e.g. via cargoTomlContents = builtins.readFile ... and similarly for cargoLockContents
3. You can use a source or fileset filter to clean agent.src and only keep Cargo.{toml,lock} and use that as inputs to things (this may trigger IFD but its up to you whether that is a problem or not)
4. Change the fetchFromGithub derivation to use a preInstall hook (or equivalent) to delete all files except for Cargo.{toml,lock}, though this will not save you from having to rebuild any time the original hash/tarball changes

Hope this helps!