From 18e864415dc0d86505c8dda82934f28e54201591 Mon Sep 17 00:00:00 2001 From: "Lehr, Jan-Patrick" Date: Tue, 5 Apr 2022 13:59:52 +0200 Subject: [PATCH] Integrate final changes for release v0.4.0 * Updates the README to contain more info w.r.t. versions and dependencies. * Removes download of dependencies git repositories (now handled in CMake). * Adds copy of Extra-P include files to install tree (allows simplifying CMake?). * Updates docker file. * Adds Sebastian as author. --- .gitlab-ci.yml | 2 +- AUTHORS | 1 + README.md | 38 ++++++++++++++++++++++++++++---------- build_submodules.sh | 42 +++++++++++++++++------------------------- container/metacg | 2 +- 5 files changed, 48 insertions(+), 37 deletions(-) diff --git a/.gitlab-ci.yml b/.gitlab-ci.yml index 9ee65a7..b515c52 100644 --- a/.gitlab-ci.yml +++ b/.gitlab-ci.yml @@ -75,7 +75,7 @@ build-mcg: GIT_STRATEGY: none GIT_CLONE_PATH: $CI_BUILDS_DIR/$CI_COMMIT_SHA script: - - cmake -S . -B build -DCMAKE_BUILD_TYPE=Debug -DCMAKE_INSTALL_PREFIX=/tmp/metacg -DCUBE_LIB=$(dirname $(which cube_info))/../lib -DCUBE_INCLUDE=$(dirname $(which cube_info))/../include/cubelib -DEXTRAP_INCLUDE=./extern/src/extrap/extrap-3.0/include -DEXTRAP_LIB=./extern/install/extrap/lib -DSPDLOG_BUILD_SHARED=ON -DCMAKE_VERBOSE_MAKEFILE:BOOL=ON + - cmake -S . -B build -DCMAKE_BUILD_TYPE=Debug -DCMAKE_INSTALL_PREFIX=/tmp/metacg -DCUBE_LIB=$(dirname $(which cube_info))/../lib -DCUBE_INCLUDE=$(dirname $(which cube_info))/../include/cubelib -DEXTRAP_INCLUDE=./extern/install/extrap/include -DEXTRAP_LIB=./extern/install/extrap/lib -DSPDLOG_BUILD_SHARED=ON -DCMAKE_VERBOSE_MAKEFILE:BOOL=ON - cmake --build build --parallel test-cgc: diff --git a/AUTHORS b/AUTHORS index 92548bc..33a7574 100644 --- a/AUTHORS +++ b/AUTHORS @@ -15,5 +15,6 @@ Constantin Völter Peter Arzt Julian Hindelang Jonas Rickert +Sebastian Kreutzer Tim Heldmann diff --git a/README.md b/README.md index 69e0716..0f761bc 100644 --- a/README.md +++ b/README.md @@ -51,31 +51,38 @@ Examples are empirically determined performance models, runtime measurements, or ## Requirements / Building MetaCG consists of the graph library, a CG construction tool, and an example analysis tool. -Currently, all components are built. +The graph library is always built. +The CG construction tool `CGCollector` can be disabled via the CMake option `-DWITH_CGCOLLECTOR=OFF`. +The example analysis tool `PGIS` can be disabled via the CMake option `-DWITH_PGIS=OFF`. -**Build Requirements** +**Build Requirements (for full build)** - Clang/LLVM version 10. -- nlohmann/json library [github](https://github.com/nlohmann/json) - Cube 4.5 [scalasca.org](https://www.scalasca.org/software/cube-4.x/download.html) +- Extra-P 3.0 [.tar.gz](http://apps.fz-juelich.de/scalasca/releases/extra-p/extrap-3.0.tar.gz) +- nlohmann/json library [github](https://github.com/nlohmann/json) - spdlog [github](https://github.com/gabime/spdlog) - cxxopts [github](https://github.com/jarro2783/cxxopts) -- Extra-P 3.0 [.tar.gz](http://apps.fz-juelich.de/scalasca/releases/extra-p/extrap-3.0.tar.gz) - PyQt5 - matplotlib -### Building the package +### Building -Currently, one first needs to build the dependencies for PGIS. -This can be done by running the provided convenience script, which we use in a Ubuntu 20.04 container and on a RedHat 8.5 (with certain LMod modules). -This will build and install the required software into `./deps/src` and `./deps/install`, respectively. -No guarantees that the script works on your machine. +Clang/LLVM (v 10) and the cube library are assumed to be available on the system. +Extra-P can be built using the `build_submodules.sh` script provided in the repository, though it is not tested outside of our CI system. +It builds and installs Extra-P into `./deps/src` and `./deps/install`, respectively. + +The `nlohmann-json`, `spdlog` and `cxxopts` library are downloaded at configure time, unless `-DMETACG_USE_EXTERNAL_JSON=ON` or `-DMETACG_USE_EXTERNAL_SPDLOG=ON` is given. +This will search for the respective libraries in the common CMake locations. +While CMake looks for `nlohmann-json` version 3.10., MetaCG should work starting from version 3.9.2. +For spdlog, we rely on version 1.8.2 -- other versions *may* work. ```{.sh} -?> cd pgis ?> ./build_submodules.sh ``` Thereafter, the package can be configured and built from the top-level CMake. +Change the `CMAKE_INSTALL_PREFIX` to where you want your MetaCG installation to live. +Providing `SPDLOG_SHARED=ON` is necessary to build the shared object version of spdlog and prevent linker errors. ```{.sh} ?> cmake -S . -B build -DCMAKE_BUILD_TYPE=Debug -DCMAKE_INSTALL_PREFIX=/tmp/metacg -DCUBE_LIB=$(dirname $(which cube_info))/../lib -DCUBE_INCLUDE=$(dirname $(which cube_info))/../include/cubelib -DEXTRAP_INCLUDE=./extern/src/extrap/extrap-3.0/include -DEXTRAP_LIB=./extern/install/extrap/lib -DSPDLOG_BUILD_SHARED=ON -DCMAKE_VERBOSE_MAKEFILE:BOOL=ON @@ -84,6 +91,17 @@ Thereafter, the package can be configured and built from the top-level CMake. ?> cmake --install build ``` +### CMake Options + +- Path `CUBE_LIB`: Path to the libcube library directory +- Path `CUBE_INCLUDE`: Path to the libcube include directory +- Path `EXTRAP_LIB`: Path to the Extra-P library directory +- Path `EXTRAP_INCLUDE`: Path to the Extra-P include directory +- Bool `WITH_CGCOLLECTOR`: Whether to build call-graph construction tool +- Bool `WITH_PGIS`: Whether to build demo-analysis tool +- Bool `METACG_USE_EXTERNAL_JSON`: Search for installed version of nlohmann-json +- Bool `METACG_USE_EXTERNAL_SPDLOG`: Search for installed version of spdlog + ## Usage ### Graph Library diff --git a/build_submodules.sh b/build_submodules.sh index b68140a..3e431e0 100755 --- a/build_submodules.sh +++ b/build_submodules.sh @@ -1,8 +1,8 @@ #! /usr/bin/env bash #""" # File: build_submodules.sh -# License: Part of the PIRA project. Licensed under BSD 3 clause license. See LICENSE.txt file at https://github.com/jplehr/pira/LICENSE.txt -# Description: Helper script to build the git submodules useed in PIRA. +# License: Part of the MetaCG project. Licensed under BSD 3 clause license. See LICENSE.txt file at https://github.com/tudasc/MetaCG/LICENSE.txt +# Description: Helper script to build the git submodules useed in MetaCG. #""" scriptdir="$( @@ -20,12 +20,12 @@ parallel_jobs="$1" add_flags="$2" # Extra-P (https://www.scalasca.org/software/extra-p/download.html) -echo "[PIRA] Building Extra-P (for PIRA II modeling)" -echo "[PIRA] Getting prerequisites ..." +echo "[MetaCG] Building Extra-P (for PIRA II modeling)" +echo "[MetaCG] Getting prerequisites ..." pip3 install --user PyQt5 2>&1 >/dev/null pip3 install --user matplotlib 2>&1 >/dev/null if [ $? -ne 0 ]; then - echo "[PIRA] Installting Extra-P dependencies failed." + echo "[MetaCG] Installting Extra-P dependencies failed." exit 1 fi @@ -34,7 +34,7 @@ cd $extsourcedir/extrap # TODO check if extra-p is already there, if so, no download / no build? if [ ! -f "extrap-3.0.tar.gz" ]; then - echo "[PIRA] Downloading and building Extra-P" + echo "[MetaCG] Downloading and building Extra-P" wget http://apps.fz-juelich.de/scalasca/releases/extra-p/extrap-3.0.tar.gz fi tar xzf extrap-3.0.tar.gz @@ -46,35 +46,27 @@ mkdir build && cd build # but at least with python 3.9 on ubuntu it is a bit buggy and some distributions don't support it at all pythonheader=$(python3 -c "from sysconfig import get_paths; print(get_paths()[\"include\"])") if [ -z $pythonheader ]; then - echo "[PIRA] Python header not found." + echo "[MetaCG] Python header not found." exit 1 fi -echo "[PIRA] Found Python.h at " $pythonheader +echo "[MetaCG] Found Python.h at " $pythonheader ../configure --prefix=$extinstalldir/extrap CPPFLAGS=-I$pythonheader 2>&1 >/dev/null if [ $? -ne 0 ]; then - echo "[PIRA] Configuring Extra-P failed." + echo "[MetaCG] Configuring Extra-P failed." exit 1 fi + make -j $parallel_jobs 2>&1 >/dev/null if [ $? -ne 0 ]; then - echo "[PIRA] Building Extra-P failed." + echo "[MetaCG] Building Extra-P failed." exit 1 fi -make install 2>&1 >/dev/null -# CXX Opts -echo "[PIRA] Getting cxxopts library" -cd $extsourcedir -if [ ! -d "$extsourcedir/cxxopts" ]; then - git clone --branch 2_1 --depth 1 https://github.com/jarro2783/cxxopts cxxopts 2>&1 >/dev/null +make install 2>&1 >/dev/null +if [ $? -ne 0 ]; then + echo "[MetaCG] Installing Extra-P failed." + exit 1 fi -#cd cxxopts -#echo "[PIRA] Select release branch 2_1 for cxxopts." -#git checkout 2_1 2>&1 >/dev/null -# JSON library -echo "[PIRA] Getting json library" -cd $extsourcedir -if [ ! -d "$extsourcedir/json" ]; then - git clone --depth 1 --branch v3.9.1 https://github.com/nlohmann/json json 2>&1 >/dev/null -fi +mkdir $extinstalldir/extrap/include +cp $extsourcedir/extrap/extrap-3.0/include/* $extinstalldir/extrap/include diff --git a/container/metacg b/container/metacg index 749feca..1a5b853 100644 --- a/container/metacg +++ b/container/metacg @@ -8,5 +8,5 @@ COPY . /opt/metacg/metacg RUN cd metacg && \ PATH=/opt/metacg/extern/install/cubelib/bin:$PATH ./build_submodules.sh $(nproc) && \ - cmake -S . -B build -DCMAKE_BUILD_TYPE=Debug -DCMAKE_INSTALL_PREFIX=/tmp/metacg -DCUBE_LIB=/opt/metacg/extern/install/cubelib/lib -DCUBE_INCLUDE=/opt/metacg/extern/install/cubelib/include/cubelib -DEXTRAP_INCLUDE=./extern/src/extrap/extrap-3.0/include -DEXTRAP_LIB=./extern/install/extrap/lib -DSPDLOG_BUILD_SHARED=ON -DCMAKE_VERBOSE_MAKEFILE:BOOL=ON && \ + cmake -S . -B build -DCMAKE_BUILD_TYPE=Debug -DCMAKE_INSTALL_PREFIX=/tmp/metacg -DCUBE_LIB=/opt/metacg/extern/install/cubelib/lib -DCUBE_INCLUDE=/opt/metacg/extern/install/cubelib/include/cubelib -DEXTRAP_INCLUDE=./extern/install/extrap/include -DEXTRAP_LIB=./extern/install/extrap/lib -DSPDLOG_BUILD_SHARED=ON -DCMAKE_VERBOSE_MAKEFILE:BOOL=ON && \ cmake --build build --parallel