Switch to nix-eval-jobs for local evaluation
#473
Draft
Conversation
This file contains hidden or bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
Until now, failed tests would not be reported separately and simply shown in the "tests build" list.
This switches `nix.Attr` from using `path` as the unique package identifier to `drv_path`.
Re-evaluating all attributes can significantly increase the memory consumption of `nix build` by several GB during the entire runtime of the process. Therefore, simply give the derivations that have already been evaluated to the process.
| let | ||
| binPath = | ||
| [ | ||
| pkgs.nixVersions.stable or nix_2_4 |
There was a problem hiding this comment.
Suggested change
| pkgs.nixVersions.stable or nix_2_4 | |
| pkgs.nixVersions.stable |
I don't think that is necessary anymore
|
|
||
| inputs = { | ||
| nixpkgs.url = "github:NixOS/nixpkgs/nixpkgs-unstable"; | ||
|
|
| "--nix-path", | ||
| nix_path, | ||
| "--extra-experimental-features", | ||
| "nix-command" if allow.url_literals else "nix-command no-url-literals", |
There was a problem hiding this comment.
Do we need to remove allow.url_literals now completely?
Sign up for free
to join this conversation on GitHub.
Already have an account?
Sign in to comment
2 participants
Add this suggestion to a batch that can be applied as a single commit.
This suggestion is invalid because no changes were made to the code.
Suggestions cannot be applied while the pull request is closed.
Suggestions cannot be applied while viewing a subset of changes.
Only one suggestion per line can be applied in a batch.
Add this suggestion to a batch that can be applied as a single commit.
Applying suggestions on deleted lines is not supported.
You must change the existing code in this line in order to create a valid suggestion.
Outdated suggestions cannot be applied.
This suggestion has been applied or marked resolved.
Suggestions cannot be applied from pending reviews.
Suggestions cannot be applied on multi-line comments.
Suggestions cannot be applied while the pull request is queued to merge.
Suggestion cannot be applied right now. Please check back later.
This PR replaces all calls to
nix-env -qaPandnix evalwith calls to nix-eval-jobs. This allows the user to control the memory consumption ofnixpkgs-reviewvery accurately.Each commit in this PR can be tested and reviewed individually.
With
nix-eval-jobs, the user now has to options to control the memory usage ofnixpkgs-review:--num-parallel-evals: the same flag that existed before and controlled the number of parallelnix-envcalls, is now used for the--workerargument tonix-eval-jobs(default 1)--max-memory-size: this is passed directly tonix-eval-jobsand controls the amount of memory in MB, that each worker can consume (default 4096).Due to the internals of
nix-eval-jobs, a worker can shortly consume more memory than specified by--max-memory-size, if the evaluation of the current attribute is large (e.g. nixos tests). But once the evaluation of that attribute is finished, the worker restarts and the memory is freed.The
default.nixcurrently includes an unreleasednix-eval-jobsversion, because the--applyflag is not released yet.To reduce the memory usage further, the invoked
nix buildcommand will now receive the derivations to build directly, instead of evaluating all attributes itself again (which can also consume multiple GB of RAM).