Perhaps ghcWithPackages should change the GHC.Paths module

[aristidb]

ghcWithPackages refers to development/compilers/ghc/with-packages.nix.

When a program uses GHC.Paths.ghc, it gets /nix/store/xcsn1rgn9m94m66wakardg0fnr26pq0y-ghc-7.4.2-wrapper/bin/ghc (the ghc-wrapper shell script) on my machine, but the actual GHC shell script that ghcWithPackages generates and is in my PATH is /nix/store/6s8zylpykahxwlxr3ar8bi1s6slpqbjx-haskell-env-ghc-7.4.2/bin/ghc.

This leads to problems with for example the doctests package. I've asked the doctests author to consider using GHC from PATH, but perhaps it would be possible to patch the GHC.Paths module instead?

[kosmikus]

It's an interesting idea. How exactly would you patch it though? To use some Nix-specific environment variable? If we do this, I would want a patch we can safely apply to all versions of GHC we install via Nix, whether withPackages is used or not.

[aristidb]

There seems to be a ghc-paths package that contains the module. Perhaps each GHC variation could get its own version of that package?

[aristidb]

I think it's also interesting that GHC.Paths.ghc points to the ghc-wrapper, NOT to ghc itself.

[kosmikus]

This seems to indicate that we could fix it by patching the wrapper(s) without touching ghc-paths in strange ways. I will look into it.

[aristidb]

Another problem seems to be that ghc --libdir and GHC.Paths.libdir do not agree with each other. The actual problem preventing me from running doctests does not, in fact, seem to be failure to invoke GHC, but rather its earlier usage of the GHC API. Doctests uses GHC.Paths.libdir there.

$ ghc --print-libdir
/nix/store/7nzjd22ynzfr20421l7imwrigx2jywpz-haskell-env-ghc-7.4.2/lib

$ ghci
GHCi, version 7.4.2: http://www.haskell.org/ghc/  :? for help
Loading package ghc-prim ... linking ... done.
Loading package integer-gmp ... linking ... done.
Loading package base ... linking ... done.
Prelude> GHC.Paths.libdir
Loading package ghc-paths-0.1.0.9 ... linking ... done.
"/nix/store/2qa339vlijw9qyp771c194y9y1ki8p40-ghc-7.4.2/lib/ghc-7.4.2"

[aristidb]

OK, I have investigated. The ghc-paths package is compiled like a normal package, and the default compiler for Haskell packages is... ghc-wrapper! From haskell-packages.nix:

ghc = callPackage ../development/compilers/ghc/wrapper.nix

This explains why those paths correspond to ghc-wrapper paths... I start to think the best option is really to compile one ghc-paths version for each wrapper-like. (Really I wonder about the sanity of putting compiler paths into a separately compiled package so that they can end up being compiled by a different but compatible compiler and things just break, but we can't change the GHC API here.)

[kosmikus]

Yes, I just figured out the same. I admit I currently don't see a much better solution than what you suggest, but it's horrible to have to do that. I'll try to think a bit more if I can somehow figure out some better solution, but currently I'm pessimistic.

An additional problem with this approach is that every package that depends on ghc-paths will have to be recompiled every time you recompile the wrapper, i.e., change anything in the set of packages. That's extremely annoying.

[aristidb]

Right, because then the actual paths in ghc-paths do change. To get around that we would need to move the actual variable definitions outside the module, somehow, I guess.

[kosmikus]

I think I'll try an approach that patches ghc-paths to allow taking paths from Nix-specific environment variables.

[kosmikus]

@aristidb I'd appreciate if you could check if my patch helps to improve things for you. If you can figure out what the proper value for docdir is, I can try to set that, too.

[aristidb]

Apart from the technical problem that prevents it from working on GHC <7.6, this patch requires setting these environment variables by hand whenever I try to run a program that uses GHC.Paths, right? Not exactly satisfying. :/

[kosmikus]

Why does it not work for older GHCs, and why do you have to set the variables by hand? I have nothing to test it right now, but I cannot see why it would not work. Can you tell me where and why it fails?

[kosmikus]

Oh, I see. Due to lookupEnv. I can fix this.

[kosmikus]

Pushed a version that I've at least tested to compile with 7.4.2 and 7.6.1.

[aristidb]

OK, it seems to all work but I need to set NIX_GHC and NIX_GHC_LIBDIR manually to get any doctests to run, as it invokes the GHC API (and the GHC binary, in addition to that) at runtime.

So I wouldn't actually consider this issue closed, as it's merely a rather unpleasant work-around.

[kosmikus]

If it invokes the GHC binary, then it should work, because the wrapper sets the env vars. I don't see how we could reliably convey the paths to GHC API invocations by other binaries. Any idea?

[aristidb]

Well, it invokes the GHC API before it invokes the binary. Also, it uses GHC.Paths.ghc to determine which ghc to use.

I see two "solutions" of varying horribleness:

1. Have a different ghc-paths package for each wrapper, with all the downsides you mentioned.
2. Somehow patch paths "later" (say, when finalising the wrapper). (No, I have not fully fleshed this out.)

[kosmikus]

I don't think (1) is a better solution than to manually set the environment variables, and I don't currently see a reliable way of doing something like (2) or better than envvars.

[aristidb]

Well (1) is a different trade-off, isn't it. You need to recompile things a lot more, but once you have an executable binary it will work.

[aristidb]

Thinking about it a bit more, the most realistic near-term option is just to create wrapper scripts for all nix-built executables using the GHC API, right? That doesn't really help for doctests though, as those are usually invoked by cabal test.

[kosmikus]

I still have no solution. One problem though is that I don't have a test case myself for what you'd like to work. Can you provide one? Then I'll think about it some more.

[aristidb]

My testcase would be: cabal install --enable-tests doctest

[shlevy]

@aristidb @kosmikus Are either of you still working on this? What about giving the ghc wrapper a setup hook to set the env vars properly?

[aristidb]

No, for now the current version is kind of good enough for my current
needs. It would still be nice though.

2014-04-04 3:15 GMT+02:00 Shea Levy [email protected]:

@aristidb https://github.com/aristidb @kosmikushttps://github.com/kosmikusAre either of you still working on this? What about giving the ghc wrapper
a setup hook to set the env vars properly?

Reply to this email directly or view it on GitHubhttps://github.com//issues/213#issuecomment-39523320
.

[peti]

ghc-mod is a wrapper script these days that figures out all necessary flags to pass to the real executable in order to make Haskell packages visible that are found in the same profile as ghc-mod itself.

Users of ghcWithPackages who don't want to rely on this kind of behind-the-scenes magic can solve these issues deterministically by adding the following snippet to their ~/.bashrc:

NIX_GHC=$(type -p ghc)
if [ -n "$NIX_GHC" ]; then
  eval $(grep export "$NIX_GHC")
fi

This will set the variables NIX_GHC_DOCDIR, NIX_GHCPKG, NIX_GHC_LIBDIR, and NIX_GHC properly so that the built-in ghc-paths module does the right thing.