Skip to content

haskell generic-builder: Use strictDeps always#41420

Merged
matthewbauer merged 21 commits intoNixOS:masterfrom
obsidiansystems:haskell-strict-deps
Jul 3, 2018
Merged

haskell generic-builder: Use strictDeps always#41420
matthewbauer merged 21 commits intoNixOS:masterfrom
obsidiansystems:haskell-strict-deps

Conversation

@Ericson2314
Copy link
Member

@Ericson2314 Ericson2314 commented Jun 3, 2018

Motivation for this change

This helps avoid the ARG_MAX issues we've been having, and is generally a good idea to ensure cross comparability anyways.

Things done
  • Tested using sandboxing (nix.useSandbox on NixOS, or option sandbox in nix.conf on non-NixOS)
  • Built on platform(s)
    • NixOS
    • macOS
    • other Linux distributions
  • Tested via one or more NixOS test(s) if existing and applicable for the change (look inside nixos/tests)
  • Tested compilation of all pkgs that depend on this change using nix-shell -p nox --run "nox-review wip"
  • Tested execution of all binary files (usually in ./result/bin/)
  • Fits CONTRIBUTING.md.

@Ericson2314 Ericson2314 requested a review from peti as a code owner June 3, 2018 16:55
@Ericson2314 Ericson2314 force-pushed the haskell-strict-deps branch from 6e0311a to b4c2a16 Compare June 3, 2018 16:57
@GrahamcOfBorg GrahamcOfBorg added 6.topic: haskell General-purpose, statically typed, purely functional programming language 10.rebuild-darwin: 501+ This PR causes many rebuilds on Darwin and should normally target the staging branches. 10.rebuild-linux: 501+ This PR causes many rebuilds on Linux and should normally target the staging branches. labels Jun 3, 2018
@peti
Copy link
Member

peti commented Jun 3, 2018

https://hydra.nixos.org/eval/1460999 is a build of the Haskell package set with this change applied.

@colonelpanic8
Copy link
Contributor

@peti @Ericson2314 Is there anything blocking merge here?

@peti
Copy link
Member

peti commented Jun 3, 2018

Yes, the test builds are not complete yet.

@colonelpanic8
Copy link
Contributor

@Ericson2314 It seems like strictDeps does not ameliorate this issue when it is encountered with stack builds (or maybe I'm doing something totally incorrectly). Do you have any idea how to fix the stack build of taffybar?

@colonelpanic8
Copy link
Contributor

@Ericson2314 Actually, it seems that what happens when you enable crossConfig/stictDeps with the stack builder is that all of the haskell-gi generated bindings fail with

setup: The program 'pkg-config' version >=0.9.0 is required but it could not

I suspect that this is probably because the paths that are passed to the stack build are not properly inherited in the builds for each haskell package, but I have to say that I'm pretty out of my depth with this haskell/stack/nix stuff.

Copy link
Member

@peti peti left a comment

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

This change breaks haskell.packages.ghc843.hspec-discover. See the "new failing jobs" tab at https://hydra.nixos.org/eval/1460999 for further details.

@Ericson2314
Copy link
Member Author

Weird! This is because GHC cannot find the testsuite from the same package to use a pre-processor. We have tests for this in Cabal https://github.com/haskell/cabal/tree/master/cabal-testsuite/PackageTests/BuildToolDepends; I'd know as I wrote them!

@Ericson2314 Ericson2314 force-pushed the haskell-strict-deps branch from 09e0c1d to da046de Compare June 19, 2018 02:29
@Ericson2314
Copy link
Member Author

@Ericson2314
Copy link
Member Author

Ericson2314 commented Jun 19, 2018

@Ericson2314
Copy link
Member Author

Looks like there's some lingering issues with build tools: hspec-discover despite my patch, and maybealex too. If anyone has any ideas I'd love to hear them.

@Ericson2314 Ericson2314 force-pushed the haskell-strict-deps branch from da046de to 8f353c8 Compare June 19, 2018 15:01
@Ericson2314
Copy link
Member Author

Rebased on #42247 which should work to keep things moving.

@Ericson2314 Ericson2314 force-pushed the haskell-strict-deps branch from 8f353c8 to 73ca88d Compare June 22, 2018 17:02
@GrahamcOfBorg GrahamcOfBorg added the 2.status: merge conflict This PR has merge conflicts with the target branch label Jun 22, 2018
@Ericson2314
Copy link
Member Author

https://hydra.nixos.org/eval/1465142. It will probably fail again, but I'd love to be wrong!

@Ericson2314 Ericson2314 requested a review from matthewbauer June 22, 2018 17:13
Copy link
Member

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

I think this needs to be done for hspec-core (and maybe other hspec stuff) as well:

ghc: could not execute: hspec-meta-discover

Copy link
Member Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

OK I'll try

Copy link
Member Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Ah ok fixed. didn't realize this was the executable from hspec-meta, not hspec-discover.

@Ericson2314
Copy link
Member Author

OK fixed this build, reevaluating. We'll see what fails.

@Ericson2314 Ericson2314 force-pushed the haskell-strict-deps branch from 73ca88d to a413506 Compare June 23, 2018 17:06
@GrahamcOfBorg GrahamcOfBorg added 8.has: clean-up This PR removes packages or removes other cruft and removed 2.status: merge conflict This PR has merge conflicts with the target branch labels Jun 23, 2018
@Ericson2314
Copy link
Member Author

Huh. My evaluation hydra evaluation is stuck pending, and just disappeared before that.

matthewbauer and others added 10 commits July 2, 2018 15:52
Needed for new ‘strictDeps = true;’ handling.

Including-

- slim
- string-conversions
- skeletal-test
- catamorphism
- unliftio
- logging-facade
- distributive
- doctest
- http-types
- interpolate
- mockery
- mime-mail
- http2
Weird errors otherwise - seems to need this to know it can skip the
bad test.
This is needed for picking up hspec-discover in strictDeps generic-builder.nix.
@matthewbauer matthewbauer force-pushed the haskell-strict-deps branch from 9abf866 to 4590a37 Compare July 2, 2018 19:56
@GrahamcOfBorg GrahamcOfBorg removed the 2.status: merge conflict This PR has merge conflicts with the target branch label Jul 2, 2018
@domenkozar
Copy link
Member

This change requires a lot of manual overrides. I think solution @angerman is proposing is less intrusive. Another way is to actually fix the commit that made MAX_ARGS explode: https://github.com/NixOS/nixpkgs/pull/41939/files#diff-360bed088faac7580cee137eeffb39e0R279

@Ericson2314
Copy link
Member Author

@domenkozar the big change motivating all the overrides here is we're being explicit for the first time about when we need a binary from another package or just its library. So I think these changes will be very good for multiple outputs (#32629). Also, Cabal 3.0 due at the end of the year will remove the non-new-build stuff, in which case being explicit in the Cabal file will also be required.

My guess is Cabal will need to come up with some sort of back-compat story, and we can replace most the overrides with that, probably implemented in Cabal2nix but who knows.

@domenkozar
Copy link
Member

Oh you mean stuff like this: sol/with-location#1

OK, but why is this needed in order to unbreak ARG_MAX? :)

@domenkozar
Copy link
Member

Could someone pull in domenkozar@4149778

@GrahamcOfBorg GrahamcOfBorg removed the 10.rebuild-darwin-stdenv This PR causes stdenv to rebuild on Darwin and must target a staging branch. label Jul 3, 2018
@matthewbauer matthewbauer force-pushed the haskell-strict-deps branch from 53596e2 to 157b597 Compare July 3, 2018 23:26
@matthewbauer
Copy link
Member

Merging for today because of the many breakages on macOS. There are still breakages for now but we should be below 400. strictDeps seems to be the only way to fix this without also breaking cross compilation of haskell.

@matthewbauer matthewbauer merged commit f682ff9 into NixOS:master Jul 3, 2018
@matthewbauer matthewbauer deleted the haskell-strict-deps branch July 3, 2018 23:32
@domenkozar
Copy link
Member

@matthewbauer btw adding build tool dependencies should go to configuration-nix as it's not version dependent :) Thanks!

@domenkozar
Copy link
Member

FYI: it was reverted and I agree with @peti we can autogenerate these: NixOS/cabal2nix#356

@Ericson2314 Ericson2314 mentioned this pull request Jul 15, 2018
9 tasks
timokau pushed a commit to timokau/nixpkgs that referenced this pull request Jul 15, 2018
…h not be non-empty.

This commit was originally introduced as part of NixOS#41420 and then
reverted with the rest of that PR. However there was no reason to revert
his particular commit.
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

6.topic: fetch Fetchers (e.g. fetchgit, fetchsvn, ...) 6.topic: haskell General-purpose, statically typed, purely functional programming language 8.has: clean-up This PR removes packages or removes other cruft 8.has: package (new) This PR adds a new package 10.rebuild-darwin: 501+ This PR causes many rebuilds on Darwin and should normally target the staging branches. 10.rebuild-linux: 501+ This PR causes many rebuilds on Linux and should normally target the staging branches.

Projects

None yet

Development

Successfully merging this pull request may close these issues.

7 participants