Refactor setup for generated nix files#1223
Merged
mergify[bot] merged 2 commits intomasterfrom Feb 20, 2020
Merged
Conversation
the reiterates on #1220. The motivation for the refactoring was that the generated files were not using `gitSource`, so I would get cache misses and dirty files in the repo. Fixing this was easier if all generated nix files are in one place, so I did the following changes: * All generated nix files are in `nix/generated`. This de-pollutes the normal working area with generated files, and keeps them out from sources (more precise derivations). * The local generated files now use `gitSource`. * The generator is now `nix/generate.nix`. * I moved complex logic from `nix/default.nix` to `nix/generate.nix`, this means everything is in one place, instead of spread over several nix files. * There is now only a single check to check if the generated files are upto date, and a single command to update the files. * This check is no longer marked `allowSubstitutes = false`. The check isn’t cheap (it requires `cabal2nix`) and we want the check to happen on builders, not hydra itself, so that the cache has `cabal2nix`. * The check is now included in `all-systems-go`.
basvandijk
reviewed
Feb 21, 2020
| winter = check ./winter; | ||
| }; | ||
| check-generated = nixpkgs.runCommandNoCC "check-generated" { | ||
| nativeBuildInputs = [ nixpkgs.diffutils ]; |
Contributor
There was a problem hiding this comment.
For inexpensive builds like this I would use:
preferLocalBuild = true;
allowSubstitutes = false;
To let CI build this locally instead of shipping inputs and outputs to and from remote builders / cache.
Contributor
Author
There was a problem hiding this comment.
But is it inexpensive? It still depends on cabal2nix, so you’d have to build that on hydra, rather than the builders… Maybe it is not a concern.
The other reason is that we want cabal2nix etc. to be available on the cache, and this is the derivation that uses it, and the cache is only fed by the builders, isn’t it?
This file contains hidden or bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
Sign up for free
to join this conversation on GitHub.
Already have an account?
Sign in to comment
3 participants
Add this suggestion to a batch that can be applied as a single commit.This suggestion is invalid because no changes were made to the code.Suggestions cannot be applied while the pull request is closed.Suggestions cannot be applied while viewing a subset of changes.Only one suggestion per line can be applied in a batch.Add this suggestion to a batch that can be applied as a single commit.Applying suggestions on deleted lines is not supported.You must change the existing code in this line in order to create a valid suggestion.Outdated suggestions cannot be applied.This suggestion has been applied or marked resolved.Suggestions cannot be applied from pending reviews.Suggestions cannot be applied on multi-line comments.Suggestions cannot be applied while the pull request is queued to merge.Suggestion cannot be applied right now. Please check back later.
the reiterates on #1220. The motivation for the refactoring was that the
generated files were not using
gitSource, so I would get cache missesand dirty files in the repo. Fixing this was easier if all generated nix
files are in one place, so I did the following changes:
nix/generated. This de-pollutes thenormal working area with generated files, and keeps them out from
sources (more precise derivations).
gitSource.nix/generate.nix.nix/default.nixtonix/generate.nix,this means everything is in one place, instead of spread over several
nix files.
upto date, and a single command to update the files.
allowSubstitutes = false. The checkisn’t cheap (it requires
cabal2nix) and we want the check to happenon builders, not hydra itself, so that the cache has
cabal2nix.all-systems-go.