make-derivation: Don't add host-suffix to fixed-output derivations names#33589
Conversation
There was a problem hiding this comment.
Ah so this part isn't actually true. Say we are building compiler-rt; it run-time depends on nothing, but cares about the host platform as much as any other platform.
In that thread, I instead meant that separately we should do a check like:
let
fixedOutputDrv = args ? outputHash;
noNonNativeDeps = builtins.length (depsBuildTarget ++ depsBuildTargetPropagated
++ depsHostHost ++ depsHostHostPropagated
++ buildInputs ++ propagatedBuildInputs
++ depsTargetTarget ++ depsTargetTargetPropagated) == 0;
in
assert fixedOutputDrv -> noNonNativeDeps;There was a problem hiding this comment.
Technically, even that isn't always necessarily true. Yes, any of those deps mean we now need host- or target-platform-specific hashes, but for pre-built stuff (bootstrapping, proprietary, etc) we do just that. So I suppose pre-built stuff could have such deps. But unless all transitive deps were also fixed-output, it would be a total pain to keep the hash correct.
In practice, for the platform-specific deps we do use, run-time deps are hooked up in a downstream deriving, e.g. with substituteAll or makeWrapper, so I don't there's a practical reason not to have that assertion. But it would be fine to leave it for a separate PR if you like, as it might require fixing miscategorized deps / is a separate issue from this one.
|
Oh also with this change, we can remove hacks in https://github.com/NixOS/nixpkgs/blob/master/pkgs/top-level/splice.nix#L34-L40 and knock off that todo. Sorry the conversation wasn't the clearest thing to follow, but thanks for tackling this! |
|
Lastly, not sure weather its worth it until we go back to staging mass rebuilds, but I've been trying to linearizing the history of my PRs under https://github.com/obsidiansystems/nixpkgs/tree/ericson2314-cross-master, for easy diffing and bisecting if anything goes wrong. I just rebased this on top of that. |
0072eb5 to
6fd63d0
Compare
|
|
|
@Ericson2314 sounds like a test that should be disabled on cross! Right? |
|
@dtzWill well I disabled fixed output derivations with run-time dependencies across the board. But those test drvs' deps shouldn't be |
cab7cd3 to
a16c912
Compare
|
I suppose while we decide whether or not to reflex the restriction, we can still clean up the deps that unequivocally need it #33681 . |
a16c912 to
1d94ed0
Compare
This confuses me. I don't think nix allows fixed-output derivations to retain any rutime references. EDIT: ah, eval-time check for |
53a4027 to
d11bc26
Compare
|
@vcunat what do you think now? Switched back to keeping hash with run-time deps, and just making a warning. |
|
Changes in laziness of derivation attributes #33057 conflicted this, and it makes me wonder why you chose to do this warning so strictly. I think it's just another way of broken-ness and would better be evaluated in the same "cases" (i.e. with the same "laziness"). Maybe it would even be worth to move it to |
|
(I'm more or less disowning this PR, looks like @Ericson2314 is leading the way, thank you! 👍) |
d11bc26 to
7f8c199
Compare
Not only does the suffix unnecessarily reduce sharing, but it also breaks unpacker setup hooks (e.g. that of `unzip`) which identify interesting tarballs using the file extension. This also means we can get rid of the splicing hacks for fetchers.
7f8c199 to
218d4dc
Compare
|
|
||
| , ... } @ attrs: | ||
|
|
||
| # TODO(@Ericson2314): Make this more modular, and not O(n^2). |
There was a problem hiding this comment.
I believe this about the hardening flags, but it's quite unclear so I just removed it.
|
I just skipped the warning for now, avoiding what @vcunat was concerned about entirely. |
|
What is the impact of this change on evaluation performance? (Asking because there has been a pretty awful slowdown in Nixpkgs evaluation speed in the last year, and I suspect it's due to all the stdenv changes.) |
|
@edolstra I did think about that here. There should be no effect on native, since the |
| # suffix. But we have some weird ones with run-time deps that are | ||
| # just used for their side-affects. Those might as well since the | ||
| # hash can't be the same. See #32986. | ||
| (stdenv.hostPlatform != stdenv.buildPlatform && runtimeSensativeIfFixedOutput) |
Not only does the suffix unnecessarily reduce sharing, but it also breaks
unpacker setup hooks (e.g. that of
unzip) which identify interesting tarballsusing the file extension.
Motivation for this change
Fixes #32986.
(another from #30882!)
Things done
build-use-sandboxinnix.confon non-NixOS)nix-shell -p nox --run "nox-review wip"./result/bin/)