Issue building claw during omni configure

[mathomp4]

A query for the developers. I am trying out CLAW for our climate model and thought I'd try out the examples. The current issue is I can't seem to even make it. My steps were:

git clone --recursive [email protected]:claw-project/claw-compiler.git
cd claw-compiler
mkdir build-GCC9
cd build-GCC9
module load comp/gcc/9.1.0
cmake .. -DCMAKE_INSTALL_PREFIX=../install-GCC9

And the CMake step seems to succeed and I seem to have all it wants:

...
-- Found Ant: /usr/bin/ant (Required is at least version "1.7.1") 
-- Found Java: /usr/bin/java (found suitable version "10.0.2", minimum required is "1.8") 
-- CLAW Compiler version f5acc929
-- OMNI Compiler version 772cf790
-- Configure preprocessor with GNU compiler
...

And here's where the fun begins when I issue make, it eventually errors out in the configure step of omni-compiler:

checking size of size_t... 8
checking size of void *... (cached) 8
checking print format for size_t... available.
configure: creating ./config.status
config.status: creating Driver/atool/mk/vars.mk
config.status: creating Driver/atool/Makefile
config.status: creating Driver/atool/src/lib/Makefile
config.status: creating Driver/atool/src/include/nata/Makefile
config.status: creating Driver/atool/src/include/nata/nata_platform.h
config.status: creating Driver/atool/src/cmd/Makefile
config.status: creating Driver/atool/src/cmd/paths.h
config.status: creating XcodeML-Common/Makefile
config.status: creating C-FrontEnd/src/Makefile
config.status: creating C-BackEnd/Makefile
config.status: creating F-FrontEnd/src/Makefile
config.status: creating F-FrontEnd/src/F-input-xmod.h
config.status: creating F-FrontEnd/src/fincludes/iso_c_binding.f90
config.status: creating F-FrontEnd/gnu_module/Makefile
config.status: creating F-BackEnd/Makefile
config.status: creating Driver/bin/Makefile
config.status: creating Driver/bin/C_Back
config.status: creating Driver/bin/F_Back
config.status: creating Makefile
config.status: creating include/config.h
config.status: creating Driver/atool/src/include/nata/nata_config.h
config.status: Driver/atool/src/include/nata/nata_config.h is unchanged
config.status: executing modify-config commands
config.status: executing includes commands
config.status: executing depend commands
Creating empty ./Driver/atool/src/cmd/.depend ... Done.
Creating empty ./Driver/atool/src/lib/.depend ... Done.
make[3]: Entering directory `/home/mathomp4/CLAWTest/claw-compiler/omni-compiler/Driver/atool'
Makefile:6: /Driver/atool/mk/vars.mk: No such file or directory
Makefile:12: /Driver/atool/mk/rules.mk: No such file or directory
make[3]: *** No rule to make target `/Driver/atool/mk/rules.mk'.  Stop.
make[3]: Leaving directory `/home/mathomp4/CLAWTest/claw-compiler/omni-compiler/Driver/atool'

The weird thing is...the build continues on:

make[3]: *** No rule to make target `/Driver/atool/mk/rules.mk'.  Stop.
make[3]: Leaving directory `/home/mathomp4/CLAWTest/claw-compiler/omni-compiler/Driver/atool'
[100%] Performing build step for 'omni-compiler'
yacc: 9 shift/reduce conflicts, 2 reduce/reduce conflicts.
[100%] Performing install step for 'omni-compiler'
[100%] Completed 'omni-compiler'
[100%] Built target omni-compiler
Scanning dependencies of target claw-init
Resolving dependencies for the build

BUILD SUCCESSFUL
Total time: 0 seconds
[100%] Built target claw-init
Scanning dependencies of target claw-cx2t
Building CLAW X2T libraries

BUILD SUCCESSFUL
Total time: 1 second
[100%] Built target claw-cx2t
Scanning dependencies of target claw-ut-init
Resolving dependencies for the build

BUILD SUCCESSFUL
Total time: 0 seconds
[100%] Built target claw-ut-init
Scanning dependencies of target claw-ut
Building CLAW XcodeML/F to XcodeML/F unit tests

BUILD SUCCESSFUL
Total time: 1 second
[100%] Built target claw-ut
Scanning dependencies of target xmod-ieee_exceptions
Generating .xmod file for ieee_exceptions.f90
[100%] Built target xmod-ieee_exceptions
Scanning dependencies of target xmod-ieee_arithmetic
Generating .xmod file for ieee_arithmetic.f90
[100%] Built target xmod-ieee_arithmetic
Scanning dependencies of target xmod-ieee_features
Generating .xmod file for ieee_features.f90
[100%] Built target xmod-ieee_features
Scanning dependencies of target xmod-openacc
Generating .xmod file for openacc_lib.f90
[100%] Built target xmod-openacc

[github-actions]

Thank you for making us aware of an issue with this repository. We will have a look at it as soon as possible.

[mathomp4]

I went a little deeper and the problem does seem to be in Omni. If you look inside their code, you'll see that after the configure step, the Makefile, etc. look broken. Let's look at omni-compiler/Driver/atool. If we look at the Makefile.in there we see:

# $Id: Makefile.in 86 2012-07-30 05:33:07Z m-hirano $

TOPDIR          = @TOPDIR@
MKRULESDIR      = @MKRULESDIR@

include $(MKRULESDIR)/vars.mk

DIRS    =       $(TOPDIR)/Driver/atool/src/lib \
                $(TOPDIR)/Driver/atool/src/include/nata \
                $(TOPDIR)/Driver/atool/src/cmd
...

Now the Makefile made by configure:

# $Id: Makefile.in 86 2012-07-30 05:33:07Z m-hirano $

TOPDIR          = @TOPDIR@
MKRULESDIR      = /Driver/atool/mk

include $(MKRULESDIR)/vars.mk

DIRS    =       $(TOPDIR)/Driver/atool/src/lib \
                $(TOPDIR)/Driver/atool/src/include/nata \
                $(TOPDIR)/Driver/atool/src/cmd

As we see, TOPDIR was never changed and MKRULESDIR is just wacky. Now let's look at configure.in and configure in the root directory. configure.in:

❯ grep -A2 -B2 TOPDIR configure.in
AC_SUBST(TARGET_ARCH)

MKRULESDIR=${TOPDIR}/Driver/atool/mk
AC_SUBST(MKRULESDIR)

and now the configure:

❯ grep -A2 -B3 TOPDIR configure



MKRULESDIR=${TOPDIR}/Driver/atool/mk

Because TOPDIR is never set inside configure, it seems like anything that refers to TOPDIR is now set to...nothing. Thus MKRULESDIR = @MKRULESDIR@ is transformed by AC_SUBST(MKRULESDIR) and becomes MKRULESDIR = /Driver/atool/mk instead of MKRULESDIR = /home/mathomp4/CLAWTest/claw-compiler/omni-compiler/Driver/atool/mk

[clementval]

Hi @mathomp4,

This is indeed related to the omni-compiler submodule. By the way, this shouldn't stop you to build and use clawfc.
You can post an issue on their repo omni-compiler/xcodeml-tools and we can update the submodule once this is fixed.

[mathomp4]

@clementval Actually, I am now not so sure it's the fault of omni-compiler but rather...CMake.

From reading your CMakeLists.txt it looks like you build omni-compiler or, rather, xcodeml-tools with:

configure --prefix=<path> --enable-gnu-extension

Now I do this:

❯ git clone [email protected]:omni-compiler/xcodeml-tools.git xcodeml-tools-772cf790
Cloning into 'xcodeml-tools-772cf790'...
X11 forwarding request failed on channel 0
remote: Enumerating objects: 31, done.
remote: Counting objects: 100% (31/31), done.
remote: Compressing objects: 100% (17/17), done.
remote: Total 29732 (delta 13), reused 20 (delta 8), pack-reused 29701
Receiving objects: 100% (29732/29732), 8.55 MiB | 28.80 MiB/s, done.
Resolving deltas: 100% (19989/19989), done.
❯ cd xcodeml-tools-772cf790/
❯ git checkout 772cf790
Note: checking out '772cf790'.

You are in 'detached HEAD' state. You can look around, make experimental
changes and commit them, and you can discard any commits you make in this
state without impacting any branches by performing another checkout.

If you want to create a new branch to retain commits you create, you may
do so (now or later) by using -b with the checkout command again. Example:

  git checkout -b <new-branch-name>

HEAD is now at 772cf790 Merge branch 'acc_pezy'
❯ ./configure --prefix=$HOME/doesntmatter/notinstalling --enable-gnu-extension
checking build system type... x86_64-pc-linux-gnu
checking host system type... x86_64-pc-linux-gnu
checking target system type... x86_64-pc-linux-gnu
checking for gcc... /ford1/local/gcc/gcc-9.1.0/bin/gcc
checking whether the C compiler works... yes
checking for C compiler default output file name... a.out
checking for suffix of executables... 
...
config.status: creating Driver/atool/src/include/nata/nata_config.h
config.status: executing modify-config commands
config.status: executing includes commands
config.status: executing depend commands

So the configure step was happy. Now we make:

❯ make
make[1]: Entering directory `/home/mathomp4/CLAWTest/xcodeml-tools-772cf790/C-FrontEnd/src'
yacc  c-parser.y 
mv -f y.tab.c c-parser.c
/usr/bin/perl c-exprcode.pl
Created c-exprcode.h, c-exprcode.c
/bin/bash ../.././buildutils/ylwrap c-parser.y y.tab.c c-parser.c y.tab.h \
`echo c-parser.c | sed -e s/cc$/hh/ -e s/cpp$/hpp/ -e s/cxx$/hxx/ -e s/c++$/h++/ -e s/c$/h/` \
y.output c-parser.output -- yacc -v -d
...
install -m 755 -d build
cd src; /usr/bin/jar  cf ../build/om-f-back.jar xcodeml/f/*/*.class
make[1]: Leaving directory `/home/mathomp4/CLAWTest/xcodeml-tools-772cf790/F-BackEnd'
make[1]: Entering directory `/home/mathomp4/CLAWTest/xcodeml-tools-772cf790/Driver/bin'
make[1]: Nothing to be done for `all'.
make[1]: Leaving directory `/home/mathomp4/CLAWTest/xcodeml-tools-772cf790/Driver/bin'
❯ echo $?
0

Well that's odd. Then I realized, the difference is I didn't do the touch you do. If I remove the touch bit from CMakeLists.txt, I get no errors:

...
configure: creating ./config.status
config.status: creating Driver/atool/mk/vars.mk
config.status: creating Driver/atool/Makefile
config.status: creating Driver/atool/src/lib/Makefile
config.status: creating Driver/atool/src/include/nata/Makefile
config.status: creating Driver/atool/src/include/nata/nata_platform.h
config.status: creating Driver/atool/src/cmd/Makefile
config.status: creating Driver/atool/src/cmd/paths.h
config.status: creating XcodeML-Common/Makefile
config.status: creating C-FrontEnd/src/Makefile
config.status: creating C-BackEnd/Makefile
config.status: creating F-FrontEnd/src/Makefile
config.status: creating F-FrontEnd/src/F-input-xmod.h
config.status: creating F-FrontEnd/src/fincludes/iso_c_binding.f90
config.status: creating F-FrontEnd/gnu_module/Makefile
config.status: creating F-BackEnd/Makefile
config.status: creating Driver/bin/Makefile
config.status: creating Driver/bin/C_Back
config.status: creating Driver/bin/F_Back
config.status: creating Makefile
config.status: creating include/config.h
config.status: creating Driver/atool/src/include/nata/nata_config.h
config.status: executing modify-config commands
config.status: executing includes commands
config.status: executing depend commands
[100%] Performing build step for 'omni-compiler'
Created c-exprcode.h, c-exprcode.c
updating c-parser.h
updating c-parser.output
Created c-token.c, c-token.h
yacc: 9 shift/reduce conflicts, 2 reduce/reduce conflicts.
src/xcodeml/util/XmToolFactory.java:138: warning: [deprecation] newInstance() in Class has been deprecated
                _property.getDecompilerClassName()).newInstance();
...

I'm not too sure why the touch causes this to happen. You do seem touch files that don't exist in a regular clone of xcodeml-tools: configure.ac, aclocal.m4, and Makefile.am. Perhaps there is no longer a reason to have the touch before the configure step for omni-compiler?

[clementval]

You are right! This might be an old trick when xcodeml-tools was part of the omni-compiler repo. Anyway this should not prevent you to use claw

[mathomp4]

@clementval It doesn't. I just thought you should know. I could make a PR if you like. It would be a simple one. :)