workflows/merge-group: add eval #451926
workflows/merge-group: add eval #451926
Conversation
Running eval in the merge queue prevents eval failures after merging multiple PRs with semantic merge conflicts. It's also the base for allowing more sophisticated checks about rebuild counts in the merge queue later. When branches are directly pushed to, such as for periodic merges, this will not run Eval immediately. However, the next PR will run it as part of its Eval step and will cache the results. Any further PR will then benefit from the same caching again. This also avoids running Eval twice when pushing to staging-next or haskell-updates, where a PR is open at the same time. Here, the PR's Eval still runs on the push, of course.
This was only separate to work around possible delays from the target branch's eval workflow. With the switch to the merge queue, this delay is impossible - the relevant target commit will only appear once Eval has completed in the merge queue, so Eval will be guaranteed to have finished. By running attrpaths and outpaths in the same step, we share ~10 seconds of eval time, traversing through all of Nixpkgs.
This change runs Lix on the target commit and Nix on the merged commit. This does two things for us at once: - We test both Lix and Nix. - We ensure that both Lix and Nix produce the same output hashes. If Lix and Nix were to produce different output hashes at some point, this would show up as rebuilds in every PR.
nixpkgs-ci
bot
added
10.rebuild-darwin: 0
nix-owners
bot
requested review from
MattSturgeon,
Mic92,
philiptaron and
zowoq
These will be fixed in #451929. Exactly one of these cases that can be caught with this kind of Lix/Nix comparison test - Nixpkgs was using a builtin that was not available in all implementations and thus caused reproducibility issues. |
|
Tangent question: as you do |
I have never seen image / commit references or hashes in |
|
Unfortunate if that's the case since the docs don't say about the actions@someCommit either. |
2 participants
Running eval in the merge queue prevents eval failures after merging multiple PRs with semantic merge conflicts.
It's also the base for allowing more sophisticated checks about rebuild counts in the merge queue later.
When branches are directly pushed to, such as for periodic merges, this will not run Eval immediately. However, the next PR will run it as part of its Eval step and will cache the results. Any further PR will then benefit from the same caching again.
This also avoids running Eval twice when pushing to staging-next or haskell-updates, where a PR is open at the same time. Here, the PR's Eval still runs on the push, of course.
This PR also runs Lix on the target commit and Nix on the merged commit. This does two things for us at once:
If Lix and Nix were to produce different output hashes at some point, this would show up as rebuilds in every PR.
Running this in my fork also shows the first two differences between Lix and Nix:
tests.devShellTools.nixostests.devShellTools.unstructuredDerivationInputEnvBoth of these produce different hashes.
Things done
The Merge Queue part can't be tested in my fork, because regular forks don't support Merge Queues.
devShellTools: Revert "devShellTools.unstructuredDerivationInputEnv: Match passAsFile basename" #451929Add a 👍 reaction to pull requests you find important.