Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Can not build projects on MacOS Mojave #884

Closed
alonsodomin opened this issue May 22, 2019 · 18 comments · Fixed by #893 or #1017
Closed

Can not build projects on MacOS Mojave #884

alonsodomin opened this issue May 22, 2019 · 18 comments · Fixed by #893 or #1017
Assignees

Comments

@alonsodomin
Copy link

Describe the bug
When trying to build a project using these rules, Bazel will attempt to download the necessary archives and packages to end up failing saying that it can't find the @io_tweag_rules_haskell_ghc_darwin_amd64// package.

Here is the standard out:

sed: 1: "mk/config.mk.in": invalid command code m
INFO: An error occurred during the fetch of repository 'io_tweag_rules_haskell_ghc_darwin_amd64'
INFO: Call stack for the definition of repository 'io_tweag_rules_haskell_ghc_darwin_amd64':
 - /private/var/tmp/_bazel_domingueza/cf40c2340e1460b987f62a06aab24e21/external/io_tweag_rules_haskell/haskell/ghc_bindist.bzl:364:5
 - /private/var/tmp/_bazel_domingueza/cf40c2340e1460b987f62a06aab24e21/external/io_tweag_rules_haskell/haskell/ghc_bindist.bzl:400:9
 - /Users/domingueza/Workspace/Projects/herd/WORKSPACE:35:1
INFO: Repository 'io_tweag_rules_haskell_ghc_darwin_amd64' used the following cache hits instead of downloading the corresponding file.
 * Hash 'cccb58f142fe41b601d73690809f6089f7715b6a50a09aa3d0104176ab4db09e' for https://downloads.haskell.org/~ghc/8.6.4/ghc-8.6.4-x86_64-apple-darwin.tar.xz
If the definition of 'io_tweag_rules_haskell_ghc_darwin_amd64' was updated, verify that the hashes were also updated.
ERROR: /private/var/tmp/_bazel_domingueza/cf40c2340e1460b987f62a06aab24e21/external/io_tweag_rules_haskell_ghc_darwin_amd64-toolchain/BUILD:4:1: no such package '@io_tweag_rules_haskell_ghc_darwin_amd64//': Traceback (most recent call last):
	File "/private/var/tmp/_bazel_domingueza/cf40c2340e1460b987f62a06aab24e21/external/io_tweag_rules_haskell/haskell/ghc_bindist.bzl", line 215
		_execute_fail_loudly(ctx, ["sed", "-i", "s/RelocatableB..."])
	File "/private/var/tmp/_bazel_domingueza/cf40c2340e1460b987f62a06aab24e21/external/io_tweag_rules_haskell/haskell/ghc_bindist.bzl", line 177, in _execute_fail_loudly
		fail("{0} failed, aborting creation o...)))
sed -i s/RelocatableBuild = NO/RelocatableBuild = YES/ mk/config.mk.in failed, aborting creation of GHC bindist and referenced by '@io_tweag_rules_haskell_ghc_darwin_amd64-toolchain//:toolchain-impl'
ERROR: Analysis of target '//protocol:protocol' failed; build aborted: no such package '@io_tweag_rules_haskell_ghc_darwin_amd64//': Traceback (most recent call last):
	File "/private/var/tmp/_bazel_domingueza/cf40c2340e1460b987f62a06aab24e21/external/io_tweag_rules_haskell/haskell/ghc_bindist.bzl", line 215
		_execute_fail_loudly(ctx, ["sed", "-i", "s/RelocatableB..."])
	File "/private/var/tmp/_bazel_domingueza/cf40c2340e1460b987f62a06aab24e21/external/io_tweag_rules_haskell/haskell/ghc_bindist.bzl", line 177, in _execute_fail_loudly
		fail("{0} failed, aborting creation o...)))
sed -i s/RelocatableBuild = NO/RelocatableBuild = YES/ mk/config.mk.in failed, aborting creation of GHC bindist
INFO: Elapsed time: 35.882s
INFO: 0 processes.
FAILED: Build did NOT complete successfully (21 packages loaded, 238 targets configured)

To Reproduce
Clone the repo at https://github.com/alonsodomin/herd using the bazel branch and then run the Bazel build from the root folder:

git clone https://github.com/alonsodomin/herd
cd herd
git checkout bazel
bazel build --incompatible_bzl_disallow_load_after_statement=false //protocol

Expected behavior
Bazel should finish the build successfully.

Environment

  • OS name + version: macOS Mojave - 10.14.5
  • Bazel version: 0.25.2-homebrew
  • Version of the rules: 0.9

Additional context
You may also notice that with given version of Bazel, I need to use the --incompatible_bzl_disallow_load_after_statement=false parameter to accept the current rules as they are defined.

@alonsodomin alonsodomin changed the title Can not build projects on MacOS Can not build projects on MacOS Mojave May 22, 2019
@mboes
Copy link
Member

mboes commented May 22, 2019

Looks like the bindists on Darwin are failing. That's the only square in the matrix that we're not currently testing in CI. @Profpatsch could you have a look?

@Profpatsch Profpatsch self-assigned this May 22, 2019
@Profpatsch
Copy link
Contributor

The io_tweag_rules_haskel_ghc_darwin_amd64 directory exists, but it has no WORKSPACE file:

[nix-shell:~/tmp/herd]$ ls /private/var/tmp/_bazel_tweag/7a2cea8ca5dea464bfb2c4f7c905e1da/external/io_tweag_rules_haskell
io_tweag_rules_haskell/                             io_tweag_rules_haskell_ghc_darwin_amd64/            io_tweag_rules_haskell_ghc_darwin_amd64-toolchain/  io_tweag_rules_haskell_ghc_linux_amd64-toolchain/   io_tweag_rules_haskell_ghc_windows_amd64-toolchain/

[nix-shell:~/tmp/herd]$ ls /private/var/tmp/_bazel_tweag/7a2cea8ca5dea464bfb2c4f7c905e1da/external/io_tweag_rules_haskell_ghc_darwin_amd64
INSTALL  Makefile  bin	       compiler      config.log     config.sub	doc   driver  ghc.mk	inplace     lib     libraries	 llvm-targets  packages  rules	   settings.in
LICENSE  README    bindist.mk  config.guess  config.status  configure	docs  ghc     includes	install-sh  libffi  llvm-passes  mk	       rts	 settings  utils

The …-toolchain rule tries to call it as a workspace.

@Profpatsch
Copy link
Contributor

After fighting with our Macbook (which is constantly running out of disk space), I prepared this very simple patch:

#893

@alonsodomin can you try it out? You can check out rules_haskell at the v0.9 tag and apply the patch, then use:

$ bazel clean --expunge
$ bazel build --incompatible_bzl_disallow_load_after_statement=false //protocol --override_repository io_tweag_rules_haskell=/path/to/rules_haskell/checkout

(the clean might be necessary to not re-use the old bindist)

@alonsodomin
Copy link
Author

@Profpatsch sorry for the delay testing this. The issue persists after checking out the tag and manually adding the additional line.

I even commented out the http_archive definition in my WORKSPACE replacing it by this one:

local_repository(
      name = "io_tweag_rules_haskell",
      path = "/path/to/rules_haskell/checkout",
  )

I may be doing something wrong though, I'm pretty new to Bazel tbh.

@Profpatsch
Copy link
Contributor

Did you do a bazel clean --expunge?

Is the error message the same? I’m getting a different error after applying the change:

protocol/src/Herd/Data/Text.hs:11:1: error:                                                                                                                                                                                                                                                 
    Could not load module _Data.Text_             
    It is a member of the hidden package _text-1.2.3.1_.        
    You can run _:set -package text_ to expose it.                                                                                                                                                                                                                                          
    (Note: this unloads all the modules in the current scope.)                                                                                                                                                                                                                              
    Use -v to see a list of the files searched for.                                                                                                                                                                                                                                     
   |                                           
11 | import           Data.Text        (Text)                                                                                                                                                                                                                                               
   | ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^        

Which is to be expected, because the text package is not imported in the BUILD.bazel file (similar to how you have to specify them in .cabal files usually).

@alonsodomin
Copy link
Author

alonsodomin commented May 25, 2019 via email

@Profpatsch
Copy link
Contributor

I’m gonna push a minor bug-fix version.

@Profpatsch
Copy link
Contributor

Hm, we don’t have any other fixes lined up, so it wouldn’t make sense to push a release for one line of code. Are you okay with using current master in the meantime?

@alonsodomin
Copy link
Author

@Profpatsch yeah, I'm ok with it, thanks for the effort on getting this fixed!

Profpatsch added a commit that referenced this issue Jun 3, 2019
@Profpatsch
Copy link
Contributor

I pushed a release, because another user ran into the same problem: https://github.com/tweag/rules_haskell/releases/tag/v0.9.1

@alonsodomin
Copy link
Author

Really appreciate that, I did try using this release and, sorry for being a PITA, but I'm still hitting the same issue, this is even after I cleaned (using --expunge), uninstall Bazel, deleted local caches (under /private/var/tmp) and reinstalled Bazel fresh again from Homebrew.

This new test I made it in a fresh folder which I initialized using the curl https://haskell.build/start | sh and then updated the WORKSPACE file to be like this:

load("@bazel_tools//tools/build_defs/repo:http.bzl", "http_archive")

http_archive(
  name = "io_tweag_rules_haskell",
  strip_prefix = "rules_haskell-0.9.1",
  urls = ["https://github.com/tweag/rules_haskell/archive/v0.9.1.tar.gz"],
)

load(
    "@io_tweag_rules_haskell//haskell:repositories.bzl",
    "haskell_repositories",
)

# `haskell_repositories()` sets up all bazel dependencies
# required by `rules_haskell`.
haskell_repositories()

load(
    "@io_tweag_rules_haskell//haskell:haskell.bzl",
	"haskell_register_toolchains",
	"haskell_register_ghc_bindists",
)

haskell_register_ghc_bindists(version = "8.6.4")

#haskell_register_toolchains()

then run from the root folder using bazel build //:example --incompatible_bzl_disallow_load_after_statement=false and same as original reported happened.

My Bazel version is 0.26, so not sure if this is due to version incompatibility:

$ bazel version
Build label: 0.26.0-homebrew
Build target: bazel-out/darwin-opt/bin/src/main/java/com/google/devtools/build/lib/bazel/BazelServer_deploy.jar
Build time: Tue May 28 14:16:39 2019 (1559052999)
Build timestamp: 1559052999
Build timestamp as int: 1559052999

Maybe the fact of having to use the --incompatible_bzl_disallow_load_after_statement=false flag is a clue into this, or just a red herring. As said I'm pretty new to Bazel.

Thanks anyway for all the effort on getting this sorted.

@Profpatsch
Copy link
Contributor

We haven’t tested rules_haskell with bazel 0.26 extensively yet, so it might well be incompatible still. Can you test with bazel 0.24 instead?

@alonsodomin
Copy link
Author

alonsodomin commented Jun 4, 2019

Yeah, sorry for not being more exhaustive on my tests. I did test this with Bazel 0.24 and got the same error.

Here is the whole output from a single session after a fresh install:

domingueza@C02RH3LCG8WN:~/Workspace/Projects/herd$ bazel version
Extracting Bazel installation...
Starting local Bazel server and connecting to it...
Build label: 0.24.1
Build target: bazel-out/darwin-opt/bin/src/main/java/com/google/devtools/build/lib/bazel/BazelServer_deploy.jar
Build time: Tue Apr 2 16:32:47 2019 (1554222767)
Build timestamp: 1554222767
Build timestamp as int: 1554222767
domingueza@C02RH3LCG8WN:~/Workspace/Projects/herd$ bazel clean --expunge
INFO: Starting clean.
domingueza@C02RH3LCG8WN:~/Workspace/Projects/herd$ bazel build //protocol
Starting local Bazel server and connecting to it...
INFO: SHA256 (https://github.com/tweag/rules_haskell/archive/v0.9.1.tar.gz) = 36c52c9709555a6c939b71c04fe0053ba89425f8d89c3c23c9b0ddd3ad91120e
DEBUG: Rule 'io_tweag_rules_haskell' indicated that a canonical reproducible form can be obtained by modifying arguments sha256 = "36c52c9709555a6c939b71c04fe0053ba89425f8d89c3c23c9b0ddd3ad91120e"
sed: 1: "mk/config.mk.in": invalid command code m
ERROR: /private/var/tmp/_bazel_domingueza/cf40c2340e1460b987f62a06aab24e21/external/io_tweag_rules_haskell_ghc_darwin_amd64-toolchain/BUILD:4:1: no such package '@io_tweag_rules_haskell_ghc_darwin_amd64//': Traceback (most recent call last):
	File "/private/var/tmp/_bazel_domingueza/cf40c2340e1460b987f62a06aab24e21/external/io_tweag_rules_haskell/haskell/ghc_bindist.bzl", line 215
		_execute_fail_loudly(ctx, ["sed", "-i", "s/RelocatableB..."])
	File "/private/var/tmp/_bazel_domingueza/cf40c2340e1460b987f62a06aab24e21/external/io_tweag_rules_haskell/haskell/ghc_bindist.bzl", line 177, in _execute_fail_loudly
		fail("{0} failed, aborting creation o...)))
sed -i s/RelocatableBuild = NO/RelocatableBuild = YES/ mk/config.mk.in failed, aborting creation of GHC bindist and referenced by '@io_tweag_rules_haskell_ghc_darwin_amd64-toolchain//:toolchain-impl'
ERROR: Analysis of target '//protocol:protocol' failed; build aborted: no such package '@io_tweag_rules_haskell_ghc_darwin_amd64//': Traceback (most recent call last):
	File "/private/var/tmp/_bazel_domingueza/cf40c2340e1460b987f62a06aab24e21/external/io_tweag_rules_haskell/haskell/ghc_bindist.bzl", line 215
		_execute_fail_loudly(ctx, ["sed", "-i", "s/RelocatableB..."])
	File "/private/var/tmp/_bazel_domingueza/cf40c2340e1460b987f62a06aab24e21/external/io_tweag_rules_haskell/haskell/ghc_bindist.bzl", line 177, in _execute_fail_loudly
		fail("{0} failed, aborting creation o...)))
sed -i s/RelocatableBuild = NO/RelocatableBuild = YES/ mk/config.mk.in failed, aborting creation of GHC bindist
INFO: Elapsed time: 117.954s
INFO: 0 processes.
FAILED: Build did NOT complete successfully (21 packages loaded, 227 targets configured)
domingueza@C02RH3LCG8WN:~/Workspace/Projects/herd$

Still saying:

no such package '@io_tweag_rules_haskell_ghc_darwin_amd64//'

@Profpatsch
Copy link
Contributor

Profpatsch commented Jun 4, 2019

I cannot reproduce, I get a different error with bazel 0.24.1:

ERROR: /Users/tweag/tmp/rh/BUILD.bazel:29:1: error executing shell command: '/nix/store/w5yp7fppsx34gfk6k12kjpkq8l9dar46-bash/bin/bash -c bazel-out/darwin-fastbuild/bin/ghc_wrapper_example_HaskellLinkBinary' failed (Exit 1) bash failed: error executing command /nix/store/w5yp7fppsx34gfk6k12kjpkq8l9dar46-bash/bin/bash -c bazel-out/darwin-fastbuild/bin/ghc_wrapper_example_HaskellLinkBinary

Use --sandbox_debug to see verbose messages from the sandbox
ld: library not found for -liconv
clang-5.0: error: linker command failed with exit code 1 (use -v to see invocation)
`osx_cc_wrapper' failed in phase `Linker'. (Exit code: 1)
Target //:example failed to build
Use --verbose_failures to see the command lines of failed build steps.
INFO: Elapsed time: 93.784s, Critical Path: 1.20s
INFO: 5 processes: 5 processwrapper-sandbox.
FAILED: Build did NOT complete successfully
FAILED: Build did NOT complete successfully

[nix-shell:~/tmp/rh]$ bazel version
Build label: 0.24.1- (@non-git)
Build target: bazel-out/darwin-opt/bin/src/main/java/com/google/devtools/build/lib/bazel/BazelServer_deploy.jar
Build time: Tue Jan 1 00:00:00 1980 (315532800)
Build timestamp: 315532800
Build timestamp as int: 315532800

[nix-shell:~/tmp/rh]$ uname -a
Darwin Tweags-MacBook-Pro.local 18.0.0 Darwin Kernel Version 18.0.0: Wed Aug 22 20:13:40 PDT 2018; root:xnu-4903.201.2~1/RELEASE_X86_64 x86_64 i386 MacBookPro12,1 Darwin

How did you install bazel?

@Profpatsch Profpatsch reopened this Jun 4, 2019
@alonsodomin
Copy link
Author

I downloaded the 0.24.1 version from GitHub as specified in this link: https://docs.bazel.build/versions/0.26.0/install-os-x.html

Basically by downloading an installer package and then run it.

@Profpatsch
Copy link
Contributor

Profpatsch commented Jun 6, 2019

Ok, using the bazel 0.26 version from homebrew I get the same error. Will try to debug it tomorrow.

Profpatsch added a commit that referenced this issue Jun 11, 2019
MacOS uses BSD coreutils, which have a different set of options and
different semantics (e.g. `sed`’s `-i` option).

We cannot assume GNU coreutils anymore, because we want to support
bindists on MacOS without dependence on nix.

Closes #884
@Profpatsch
Copy link
Contributor

Okay, the problem was that MacOS uses BSD coreutils.

Can you try out #937?

Profpatsch added a commit that referenced this issue Jun 11, 2019
MacOS uses BSD coreutils, which have a different set of options and
different semantics (e.g. `sed`’s `-i` option).

We cannot assume GNU coreutils anymore, because we want to support
bindists on MacOS without dependence on nix.

Closes #884
@alonsodomin
Copy link
Author

yeah, that worked using the c5c4e67474e8ec20a747eb9bcad00a21f6745069 commit. The build got to the point in which it fail when compiling my code, so the issue gets solved with that.

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Projects
None yet
3 participants