llvmPackages_11: 11.0.0rc5 -> 11.0.0, enable on Darwin

[ggreif]

Motivation for this change

• llvmPackages_11 doesn't compile on Darwin
• libcxx (11.0.0rc) doesn't compile on Linux
• port treewide: Get rid of cc.gcc #91293 to 11.0.0
• anticipate treewide: move fixDarwinDylibNames to nativeBuildInputs #100388 (@LnL7, please check)
• http://lists.llvm.org/pipermail/release-testers/2020-October/001382.html

Things done

• Added conditionals
• python3 now mandatory for building libcxx everywhere
• Bumped to v11.0.0 final

• Tested using sandboxing (nix.useSandbox on NixOS, or option sandbox in nix.conf on non-NixOS linux)
• Built on platform(s)
  • NixOS
  • macOS
  • other Linux distributions
• Tested via one or more NixOS test(s) if existing and applicable for the change (look inside nixos/tests)
• Tested compilation of all pkgs that depend on this change using nix-shell -p nixpkgs-review --run "nixpkgs-review wip"
• Tested execution of all binary files (usually in ./result/bin/)
• Determined the impact on package closure size (by running nix path-info -S before and after)
• Ensured that relevant documentation is up to date
• Fits CONTRIBUTING.md.

[ggreif]

@GrahamcOfBorg build llvm_11 lld_11 clang_11

[ggreif]

@DieGoldeneEnte this builds now for Darwin, but stopped working for Linux. I have no clue why.

[DieGoldeneEnte]

libc++ failed,because it coudn't find python, it build normally once python3 is added as native build input.
Could this be, because the llvm-source is added as input?

I need to take another look, why it suddenly needs python...

[ggreif]

Could this be, because the llvm-source is added as input?

Then it would fail on Darwin too, right?

[DieGoldeneEnte]

Then it would fail on Darwin too, right?

That's my problem. I couldn't find a relevant difference between the working package in nixpkgs and this.
If I read it correctly, python is required when building libc++ standalone (I don't think we are setting LIBCXX_STANDALONE_BUILD, but python should only be needed if it is set) and using cmake > 3.12. Both should be given for the current nixpkgs derivation.

[ggreif]

@DieGoldeneEnte on Darwin I get cmake-3.18.2 with nix-build -A cmake. Same on Linux?

[ggreif]

@GrahamcOfBorg build lldb_11

[ggreif]

@GrahamcOfBorg build llvmPackages_11

[ggreif]

@DieGoldeneEnte Maybe the cmake machinery now runs some tests that require python3. I included Linux now, let's see what OfBorg says.

[DieGoldeneEnte]

I just tried rebuilding the current master version of libcxx, but it is actually broken, because of the missing monorepo-layout :(
Is this the same problem as on darwin (before the changes here)?

Regarding LIBCXX_STANDALONE_BUILD, this is set by cmake (in libcxx/cmake/Modules/HandleOutOfTreeLLVM.cmake). So I don't think it should be set.
Reading the CMakeLists.txt file, it seems python is mandatory when building standalone (and cmake >3.12). I don't understand, why darwin doesn't need it, since the cmake code is (shortened):

if (LIBCXX_STANDALONE_BUILD)
  if(CMAKE_VERSION VERSION_LESS 3.12)
    [...]
  else()
    find_package(Python3 COMPONENTS Interpreter)
    if(NOT Python3_Interpreter_FOUND)
      find_package(Python2 COMPONENTS Interpreter REQUIRED)
      [...]
    endif()
  endif()
endif()

Are you building libc++ in a sandbox? Maybe cmake finds a python version from the system?

error:

libc++abi now requires being built in a monorepo layout with libcxx available

[ggreif]

@DieGoldeneEnte I am debugging this right now. I'll trigger you after having done the fix.

[ggreif]

Darwin also needs `Python3`, but finds a Python2 substitute in some framework. ~~Trying to figure out why the initial test for build dir equality goes wrong, so that `HandleOutOfTreeLLVM.cmake` gets called.~~ It should be called. The monorepo layout is orthogonal to standalone build.

…

On 10/14/20, Holger Wünsche ***@***.***> wrote: I just tried rebuilding the current master version of libcxx, but it is actually broken, because of the missing monorepo-layout :( Is this the same problem as on darwin (before the changes here)? Regarding `LIBCXX_STANDALONE_BUILD`, this is set by cmake (in libcxx/cmake/Modules/HandleOutOfTreeLLVM.cmake). So I don't think it should be set. Reading the CMakeLists.txt file, it seems python is mandatory when building standalone (and cmake >3.12). I don't understand, why darwin doesn't need it, since the cmake code is (shortened): ``` if (LIBCXX_STANDALONE_BUILD) if(CMAKE_VERSION VERSION_LESS 3.12) [...] else() find_package(Python3 COMPONENTS Interpreter) if(NOT Python3_Interpreter_FOUND) find_package(Python2 COMPONENTS Interpreter REQUIRED) [...] endif() endif() endif() ``` Are you building libc++ in a sandbox? Maybe cmake finds a python version. error: ``` libc++abi now requires being built in a monorepo layout with libcxx available ``` -- You are receiving this because you authored the thread. Reply to this email directly or view it on GitHub: #99984 (comment)

[DieGoldeneEnte]

shouldn't the test (CMAKE_SOURCE_DIR STREQUAL CMAKE_CURRENT_SOURCE_DIR) be true, because we run cmake in the libc++ directory. This would be false if the CMakeLists.txt file would be added via add_subdirectory/add_llvm_external_project (that's at least my understanding given my limited cmake knowledge).

[ggreif]

@DieGoldeneEnte yes that test should be true. It is true. So this is a standalone build in a monorepo-layout source.
Darwin looks for a Python3 and doesn't find it. But on my machine it finds some Python2 in a framework, thus it can proceed. It would break in a sandbox.

I am making python3 mandatory now.

[ggreif]

@DieGoldeneEnte things look good on my side!

[DieGoldeneEnte]

@ggreif Why is the llvm source needed for libc++*? Is it to simulate the monorepo?
Builds without problems on my machine :)

[ggreif]

@ggreif Why is the llvm source needed for libc++*? Is it to simulate the monorepo?

That's what I understand, too.

Builds without problems on my machine :)

THX! 🥇

[DieGoldeneEnte]

LGTM then!

I think we will need to rewrite the llvm package since building the projects standalone is less and less supported.

[DieGoldeneEnte]

Problem is with compilert-rt:

CMake Error at cmake/Modules/AddCompilerRT.cmake:318 (add_library):
  No SOURCES given to target: clang_rt.builtins-i686
Call Stack (most recent call first):
  lib/builtins/CMakeLists.txt:636 (add_compiler_rt_runtime)

fixed it (at least for i686) in my PR targeting this PR-branch.

[ggreif]

@volth @DieGoldeneEnte but this is not a regression vs. 11.0.0rcs, right? Is it a regression vs. 10.0.1?

Anyway, better open a PR against those separately.

[DieGoldeneEnte]

@ggreif It was introduced with 11.0.0-rc1, 10.0.1 does not have this change.

[ggreif]

@DieGoldeneEnte feel free to merge your fix then.

…

On 10/15/20, Holger Wünsche ***@***.***> wrote: @ggreif It was introduced with 11.0.0-rc1, 10.0.1 does not have this change. -- You are receiving this because you were mentioned. Reply to this email directly or view it on GitHub: #99984 (comment)

[primeos]

The diff LGTM :) Thanks @ggreif and @DieGoldeneEnte!

@ggreif is this ready to be merged now?
Edit: I just re-read any comments and everything seems to be resolved :) I'll go ahead and merge this then. Huge thanks again!