llvmPackages_14.{mlir,flang}: init

[dtzWill]

Description of changes

Add packages for MLIR and Flang.

MLIR: https://mlir.llvm.org/
Flang: https://flang.llvm.org/docs/

Mostly came for packaging MLIR, but flang is nice to have and helps
check that the produced MLIR package is functional.

Note that (for now) using flang to produce binaries requires gfortran
available as well.

Editor bits for MLIR are installed as well as mlir-lsp-server.

Includes some of the runners for MLIR which are at least neat and while don't
significantly increase closure size, might be a liability should this
end up a dependency of something lighter (size or portability).

Things done

• Built on platform(s)
  • x86_64-linux
  • aarch64-linux
  • x86_64-darwin
  • aarch64-darwin
• For non-Linux: Is sandbox = true set in nix.conf? (See Nix manual)
• Tested, as applicable:
  • NixOS test(s) (look inside nixos/tests)
  • and/or package tests
  • or, for functions and "core" functionality, tests in lib/tests or pkgs/test
  • made sure NixOS tests are linked to the relevant packages
• Tested compilation of all packages that depend on this change using nix-shell -p nixpkgs-review --run "nixpkgs-review rev HEAD". Note: all changes have to be committed, also see nixpkgs-review usage
• Tested basic functionality of all binary files (usually in ./result/bin/)
• 22.05 Release Notes (or backporting 21.11 Release notes)
  • (Package updates) Added a release notes entry if the change is major or breaking
  • (Module updates) Added a release notes entry if the change is significant
  • (Module addition) Added a release notes entry if adding a new NixOS module
  • (Release notes changes) Ran nixos/doc/manual/md-to-db.sh to update generated release notes
• Fits CONTRIBUTING.md.

[dtzWill]

Put these in 'tools' since while MLIR is largely a library here, it's not a runtime library like the others.

[dtzWill]

As for installing MLIR tools:

• Use LLVM_{BUILD,INSTALL}_TOOLS instead of LLVM_{BUILD,INSTALL}_UTILS to generate the build / installation targets. This helps but doesn't work as errors are produced attempting to install the tools into ${llvm.out}/bin.
• MLIR uses add_llvm_tool in cmake which is defined in AddLLVM.cmake and sets the destination to LLVM_TOOLS_INSTALL_DIR. For building LLVM this is fine (as it's llvm.out) but causes problems for any downstream consumers of that cmake code. Which grep'ing llvm-project mono-repo suggests it's mostly MLIR doing this presently (and maybe other out-of-tree projects).
• Flang manages to avoid this (while another MLIR-based project I'm trying has issues) by using their own add_flang_tool cmake method, although offhand would suspect this isn't the only reason for this construction/effort.

Idea 1: Current approach: manually enumerate, explicitly build, and install the various utilities (and possibly have issues in other MLIR projects or in the future)

Idea 2: Patch llvm's cmake to let users of add_llvm_tool install to their /own/ installation directories?

Idea 3: Patch (and attempt to upstream) in a new add_mlir_tool or similar, mostly copying add_llvm_tool for our purposes (as seems often done-- add_${project}_tool definitions exist in many LLVM projects.

---

EDIT: The manual installation we're doing works fine (if not my favorite) for MLIR, and is enough for flang (hence this PR). However as we pull in more projects using MLIR, these issues with how they try to install themselves will be more problematic (I've worked on two besides flang so far).

[dtzWill]

(rebasing+force-pushing for LLVM 14 release)

[7c6f434c]

Looka reasonable to me

[Technius]

I think the MLIRConfig.cmake might also need to be fixed up here or somewhere else. Since the output is split, the $lib/lib/cmake folder will be moved to $dev/lib/cmake, which might invalidate some variables like MLIR_CMAKE_DIR.

[bjornfor]

In LLVM git / v15 there is inverse check too, so we'd need to add this to avoid build breakage:

substituteInPlace "$x" --replace 'if(NOT TARGET ''${LLVM_NATIVE_ARCH})' 'if (0)'

[bjornfor]

globstar doesn't seem to be enabled in stdenv/bash, so the for loop doesn't process files recursively. We can use find -name CMakeLists.txt | while read f; do ...; done instead.

[bjornfor]

I tried to use MLIR at $dayjob, but found the way llvmPackages does multi-derivation and multi-output to cause too much trouble. I ended up making a simple LLVM "monobuild" expression which has working MLIR cmake files:

# Build a full LLVM in one derivation with one output -- in contrast to nixpkgs
# llvmPackages which splits the package into multiple derivations (and multiple
# outputs).
#
# Pros:
# * Easier to match upstream builds / make MLIR CMake files work.
# * No patching required.
#
# Cons:
# * Slower to build / iterate on.
# * Larger closure sizes.
# * Not useable as stdenv component as $CC etc. doesn't get wrapped.

{ lib, stdenv, fetchFromGitHub, cmake, python3, ncurses, zlib, libffi, libxml2 }:

stdenv.mkDerivation rec {
  pname = "llvm";
  version = "15.0.0";

  src = fetchFromGitHub {
    owner = "llvm";
    repo = "llvm-project";
    rev = "llvmorg-${version}";
    sha256 = "sha256-4yviNtiJJLE6JPNqwPwRBk9fZ9vXqsVIaNOucrePmy8=";
  };

  nativeBuildInputs = [
    cmake python3
  ];

  buildInputs = [
    libffi
    libxml2
  ];

  propagatedBuildInputs = [
    ncurses
    zlib
  ];

  sourceRoot = "source/llvm";

  cmakeFlags = [
    "-DLLVM_ENABLE_PROJECTS=mlir"
    "-DLLVM_ENABLE_RTTI=ON"
  ];

  requiredSystemFeatures = [ "big-parallel" ];

  meta = {
    description = "A collection of modular and reusable compiler and toolchain technologies";
    homepage = "https://llvm.org/";
    license = lib.licenses.ncsa;
  };
}

[ConnorBaker]

Given that Triton (https://github.com/openai/triton) needs MLIR and Torch 2.0 will make use of Triton for its torch.compile feature, this would be wonderful to have. I'm trying to package MLIR myself and just found this PR. I'll take a look at it tomorrow to see what I can do with it.

@rrbutani any thoughts from your recent updates to llvmPackages? I notice that the rocm folder provides a derivation to build MLIR -- it would be nice to have the ability to build the main derivation with CUDA support.

[rrbutani]

@ConnorBaker Thanks for the heads-up about rocm; I missed that they're building mlir now.

---

I've got an mlir package (that builds on @dtzWill's work here) for LLVM 15 here that I've been meaning to clean up and contribute to nixpkgs. I think I got it to the point where the libraries and the MLIR python bindings work and the docs build (and IIRC I ended up finding workarounds for some of the pain points this PR ran into) but it still needs work for CUDA support and I don't think I got all the tests to pass.

---

re: Triton: do you happen to know what their plan is for the version of LLVM/MLIR they will require? I'm not familiar with Triton (and I couldn't find an answer after quickly glancing at their repo; they make reference to LLVM 11 and their CMake scripts seem to accept any LLVM version 6.0+) but other projects I've worked with that use MLIR tend to track upstream very closely and generally don't seem to line up their releases with LLVM/MLIR releases.

EDIT: I think we're fine; they seem to use LLVM/MLIR releases (currently 14.0.6) instead of HEAD, albeit their own distributions.

[ConnorBaker]

@rrbutani They're working their way towards using HEAD; they have a PR open to use LLVM 15 here: triton-lang/triton#1070. Both their internal team and the team at Meta want to track HEAD more closely.

I didn't notice that they switched to using their own distribution; that's new!

I've been wallowing in Dockerfiles the past month or two building Magma, PyTorch, TorchVision, and Triton from source. Eventually I broke and have been trying to package it with Nix since. Please let me know if there's anything I can do to help get MLIR with CUDA support in Nixpkgs. I don't want to go back to docker :(

[SomeoneSerge]

Hi! Is there a reason to only target one llvm version?

I'm currently using mlir from llvmPackages_rocm (effectively llvm15?) for pytorch 2.0, but I understand that the situation is likely to start changing pretty fast

EDIT 2023-04-06: In fact we're going to need llvmPackages_17 with mlir in order to upgrade to https://github.com/ROCmSoftwarePlatform/triton/releases/tag/pytorch-triton-rocm-v2.0.1. Right now we're using LLVM-15 (llvmPackages_rocm): #222273

[SuperSandro2000]

I don't think we set -l anymore in stdenv since a few months

[hmenke]

Could someone please update this for LLVM 17? Flang has become a lot more useful in the meantime and can now build executables without passing experimental flags.

[SomeoneSerge]

Following on the previous ping, @dtzWill what are your plans wrt this? Do you plan to look into this again, do you need any help to push this further and get it merged, or should we mark this closed for now?

CC @NixOS/rocm-maintainers can you help assessing the situation? How does this interact with rocmPackages.llvm.mlir, etc.

[wegank]

Related: #280572