Skip to content

Commit

Permalink
Integrate final changes for release v0.4.0
Browse files Browse the repository at this point in the history
* Updates the README to contain more info w.r.t. versions and dependencies.
* Removes download of dependencies git repositories (now handled in CMake).
* Adds copy of Extra-P include files to install tree (allows simplifying CMake?).
* Updates docker file.
* Adds Sebastian as author.
  • Loading branch information
jplehr committed Apr 5, 2022
1 parent ce69511 commit 18e8644
Show file tree
Hide file tree
Showing 5 changed files with 48 additions and 37 deletions.
2 changes: 1 addition & 1 deletion .gitlab-ci.yml
Original file line number Diff line number Diff line change
Expand Up @@ -75,7 +75,7 @@ build-mcg:
GIT_STRATEGY: none
GIT_CLONE_PATH: $CI_BUILDS_DIR/$CI_COMMIT_SHA
script:
- cmake -S . -B build -DCMAKE_BUILD_TYPE=Debug -DCMAKE_INSTALL_PREFIX=/tmp/metacg -DCUBE_LIB=$(dirname $(which cube_info))/../lib -DCUBE_INCLUDE=$(dirname $(which cube_info))/../include/cubelib -DEXTRAP_INCLUDE=./extern/src/extrap/extrap-3.0/include -DEXTRAP_LIB=./extern/install/extrap/lib -DSPDLOG_BUILD_SHARED=ON -DCMAKE_VERBOSE_MAKEFILE:BOOL=ON
- cmake -S . -B build -DCMAKE_BUILD_TYPE=Debug -DCMAKE_INSTALL_PREFIX=/tmp/metacg -DCUBE_LIB=$(dirname $(which cube_info))/../lib -DCUBE_INCLUDE=$(dirname $(which cube_info))/../include/cubelib -DEXTRAP_INCLUDE=./extern/install/extrap/include -DEXTRAP_LIB=./extern/install/extrap/lib -DSPDLOG_BUILD_SHARED=ON -DCMAKE_VERBOSE_MAKEFILE:BOOL=ON
- cmake --build build --parallel

test-cgc:
Expand Down
1 change: 1 addition & 0 deletions AUTHORS
Original file line number Diff line number Diff line change
Expand Up @@ -15,5 +15,6 @@ Constantin Völter <Scientific Computing>
Peter Arzt <Scientific Computing>
Julian Hindelang <Scientific Computing>
Jonas Rickert <Scientific Computing>
Sebastian Kreutzer <Scientific Computing>
Tim Heldmann <Scientific Computing>

38 changes: 28 additions & 10 deletions README.md
Original file line number Diff line number Diff line change
Expand Up @@ -51,31 +51,38 @@ Examples are empirically determined performance models, runtime measurements, or
## Requirements / Building

MetaCG consists of the graph library, a CG construction tool, and an example analysis tool.
Currently, all components are built.
The graph library is always built.
The CG construction tool `CGCollector` can be disabled via the CMake option `-DWITH_CGCOLLECTOR=OFF`.
The example analysis tool `PGIS` can be disabled via the CMake option `-DWITH_PGIS=OFF`.

**Build Requirements**
**Build Requirements (for full build)**
- Clang/LLVM version 10.
- nlohmann/json library [github](https://github.com/nlohmann/json)
- Cube 4.5 [scalasca.org](https://www.scalasca.org/software/cube-4.x/download.html)
- Extra-P 3.0 [.tar.gz](http://apps.fz-juelich.de/scalasca/releases/extra-p/extrap-3.0.tar.gz)
- nlohmann/json library [github](https://github.com/nlohmann/json)
- spdlog [github](https://github.com/gabime/spdlog)
- cxxopts [github](https://github.com/jarro2783/cxxopts)
- Extra-P 3.0 [.tar.gz](http://apps.fz-juelich.de/scalasca/releases/extra-p/extrap-3.0.tar.gz)
- PyQt5
- matplotlib

### Building the package
### Building

Currently, one first needs to build the dependencies for PGIS.
This can be done by running the provided convenience script, which we use in a Ubuntu 20.04 container and on a RedHat 8.5 (with certain LMod modules).
This will build and install the required software into `./deps/src` and `./deps/install`, respectively.
No guarantees that the script works on your machine.
Clang/LLVM (v 10) and the cube library are assumed to be available on the system.
Extra-P can be built using the `build_submodules.sh` script provided in the repository, though it is not tested outside of our CI system.
It builds and installs Extra-P into `./deps/src` and `./deps/install`, respectively.

The `nlohmann-json`, `spdlog` and `cxxopts` library are downloaded at configure time, unless `-DMETACG_USE_EXTERNAL_JSON=ON` or `-DMETACG_USE_EXTERNAL_SPDLOG=ON` is given.
This will search for the respective libraries in the common CMake locations.
While CMake looks for `nlohmann-json` version 3.10., MetaCG should work starting from version 3.9.2.
For spdlog, we rely on version 1.8.2 -- other versions *may* work.

```{.sh}
?> cd pgis
?> ./build_submodules.sh
```

Thereafter, the package can be configured and built from the top-level CMake.
Change the `CMAKE_INSTALL_PREFIX` to where you want your MetaCG installation to live.
Providing `SPDLOG_SHARED=ON` is necessary to build the shared object version of spdlog and prevent linker errors.

```{.sh}
?> cmake -S . -B build -DCMAKE_BUILD_TYPE=Debug -DCMAKE_INSTALL_PREFIX=/tmp/metacg -DCUBE_LIB=$(dirname $(which cube_info))/../lib -DCUBE_INCLUDE=$(dirname $(which cube_info))/../include/cubelib -DEXTRAP_INCLUDE=./extern/src/extrap/extrap-3.0/include -DEXTRAP_LIB=./extern/install/extrap/lib -DSPDLOG_BUILD_SHARED=ON -DCMAKE_VERBOSE_MAKEFILE:BOOL=ON
Expand All @@ -84,6 +91,17 @@ Thereafter, the package can be configured and built from the top-level CMake.
?> cmake --install build
```

### CMake Options

- Path `CUBE_LIB`: Path to the libcube library directory
- Path `CUBE_INCLUDE`: Path to the libcube include directory
- Path `EXTRAP_LIB`: Path to the Extra-P library directory
- Path `EXTRAP_INCLUDE`: Path to the Extra-P include directory
- Bool `WITH_CGCOLLECTOR`: Whether to build call-graph construction tool
- Bool `WITH_PGIS`: Whether to build demo-analysis tool
- Bool `METACG_USE_EXTERNAL_JSON`: Search for installed version of nlohmann-json
- Bool `METACG_USE_EXTERNAL_SPDLOG`: Search for installed version of spdlog

## Usage

### Graph Library
Expand Down
42 changes: 17 additions & 25 deletions build_submodules.sh
Original file line number Diff line number Diff line change
@@ -1,8 +1,8 @@
#! /usr/bin/env bash
#"""
# File: build_submodules.sh
# License: Part of the PIRA project. Licensed under BSD 3 clause license. See LICENSE.txt file at https://github.com/jplehr/pira/LICENSE.txt
# Description: Helper script to build the git submodules useed in PIRA.
# License: Part of the MetaCG project. Licensed under BSD 3 clause license. See LICENSE.txt file at https://github.com/tudasc/MetaCG/LICENSE.txt
# Description: Helper script to build the git submodules useed in MetaCG.
#"""

scriptdir="$(
Expand All @@ -20,12 +20,12 @@ parallel_jobs="$1"
add_flags="$2"

# Extra-P (https://www.scalasca.org/software/extra-p/download.html)
echo "[PIRA] Building Extra-P (for PIRA II modeling)"
echo "[PIRA] Getting prerequisites ..."
echo "[MetaCG] Building Extra-P (for PIRA II modeling)"
echo "[MetaCG] Getting prerequisites ..."
pip3 install --user PyQt5 2>&1 >/dev/null
pip3 install --user matplotlib 2>&1 >/dev/null
if [ $? -ne 0 ]; then
echo "[PIRA] Installting Extra-P dependencies failed."
echo "[MetaCG] Installting Extra-P dependencies failed."
exit 1
fi

Expand All @@ -34,7 +34,7 @@ cd $extsourcedir/extrap

# TODO check if extra-p is already there, if so, no download / no build?
if [ ! -f "extrap-3.0.tar.gz" ]; then
echo "[PIRA] Downloading and building Extra-P"
echo "[MetaCG] Downloading and building Extra-P"
wget http://apps.fz-juelich.de/scalasca/releases/extra-p/extrap-3.0.tar.gz
fi
tar xzf extrap-3.0.tar.gz
Expand All @@ -46,35 +46,27 @@ mkdir build && cd build
# but at least with python 3.9 on ubuntu it is a bit buggy and some distributions don't support it at all
pythonheader=$(python3 -c "from sysconfig import get_paths; print(get_paths()[\"include\"])")
if [ -z $pythonheader ]; then
echo "[PIRA] Python header not found."
echo "[MetaCG] Python header not found."
exit 1
fi
echo "[PIRA] Found Python.h at " $pythonheader
echo "[MetaCG] Found Python.h at " $pythonheader
../configure --prefix=$extinstalldir/extrap CPPFLAGS=-I$pythonheader 2>&1 >/dev/null
if [ $? -ne 0 ]; then
echo "[PIRA] Configuring Extra-P failed."
echo "[MetaCG] Configuring Extra-P failed."
exit 1
fi

make -j $parallel_jobs 2>&1 >/dev/null
if [ $? -ne 0 ]; then
echo "[PIRA] Building Extra-P failed."
echo "[MetaCG] Building Extra-P failed."
exit 1
fi
make install 2>&1 >/dev/null

# CXX Opts
echo "[PIRA] Getting cxxopts library"
cd $extsourcedir
if [ ! -d "$extsourcedir/cxxopts" ]; then
git clone --branch 2_1 --depth 1 https://github.com/jarro2783/cxxopts cxxopts 2>&1 >/dev/null
make install 2>&1 >/dev/null
if [ $? -ne 0 ]; then
echo "[MetaCG] Installing Extra-P failed."
exit 1
fi
#cd cxxopts
#echo "[PIRA] Select release branch 2_1 for cxxopts."
#git checkout 2_1 2>&1 >/dev/null

# JSON library
echo "[PIRA] Getting json library"
cd $extsourcedir
if [ ! -d "$extsourcedir/json" ]; then
git clone --depth 1 --branch v3.9.1 https://github.com/nlohmann/json json 2>&1 >/dev/null
fi
mkdir $extinstalldir/extrap/include
cp $extsourcedir/extrap/extrap-3.0/include/* $extinstalldir/extrap/include
2 changes: 1 addition & 1 deletion container/metacg
Original file line number Diff line number Diff line change
Expand Up @@ -8,5 +8,5 @@ COPY . /opt/metacg/metacg

RUN cd metacg && \
PATH=/opt/metacg/extern/install/cubelib/bin:$PATH ./build_submodules.sh $(nproc) && \
cmake -S . -B build -DCMAKE_BUILD_TYPE=Debug -DCMAKE_INSTALL_PREFIX=/tmp/metacg -DCUBE_LIB=/opt/metacg/extern/install/cubelib/lib -DCUBE_INCLUDE=/opt/metacg/extern/install/cubelib/include/cubelib -DEXTRAP_INCLUDE=./extern/src/extrap/extrap-3.0/include -DEXTRAP_LIB=./extern/install/extrap/lib -DSPDLOG_BUILD_SHARED=ON -DCMAKE_VERBOSE_MAKEFILE:BOOL=ON && \
cmake -S . -B build -DCMAKE_BUILD_TYPE=Debug -DCMAKE_INSTALL_PREFIX=/tmp/metacg -DCUBE_LIB=/opt/metacg/extern/install/cubelib/lib -DCUBE_INCLUDE=/opt/metacg/extern/install/cubelib/include/cubelib -DEXTRAP_INCLUDE=./extern/install/extrap/include -DEXTRAP_LIB=./extern/install/extrap/lib -DSPDLOG_BUILD_SHARED=ON -DCMAKE_VERBOSE_MAKEFILE:BOOL=ON && \
cmake --build build --parallel

0 comments on commit 18e8644

Please sign in to comment.