Building for Windows on Arm wiki page will need updating once LLVM17 is out

[everton1984]

Hi,
I wanted to update the wiki page How to build OpenBLAS for Windows on ARM64 because the instructions are deprecated but the edit button is gone. What is the proper way to get those instructions updated? There is a new flang version and you can now build LAPACK along with OpenBLAS on Windows on Arm and I wanted to mention that.

[brada4]

There was some link waterholing/spamming incidents, @martin-frbg can help

[martin-frbg]

Yes, please suggest additions and corrections here - we had to clamp down the wiki as having to create a throwaway github id was too weak a deterrent for spammers, and even with best effort there would always be a time window of at least a few hours in which any phishing or malware links edited into the wiki would stay up.

[martin-frbg]

@everton1984 Not sure how to proceed here without your input - we do not have a Windows-on-Arm system available for testing, and at least the Linaro tracker for "Arm LLVM Toolchain Enhancement" lists an open issue LLVM-760 as holding back OpenBLAS compilation with "flang-new". (This ticket has not been updated since last summer, but the download page at llvm.org has only sketchy flang release notes from 15.0.0 even with the recent release of LLVM16)

[everton1984]

@martin-frbg Yeah I thought I had something working but when I got the unit tests to compile one of them are failing so I had to hold on before sending the updated text. Sorry about this delay. I can close this and reopen when I get it solved if you want.

[martin-frbg]

I'd think it depends on how long you expect the delay to be - and if it is something we can help with (without having accesss to a Windows-on-Arm system). I notice the f_check script currently does not know flang-new as an executable name, which may be a sufficient drawback in some circumstances.

[everton1984]

I've been fighting against two issues, the first one is that the CMake command add_executable produces a incorrect command line (I'm guessing this is a cmake issue but couldn't pinpoint yet). When I use a add_custom_target with the proper compilation commands, all level two and three unit tests like sblas2, dblas2, etc are hanging (or at least taking and absurd ammount of time). I believe I could get you access to WoA machine if you are interested.

[martin-frbg]

Eeek... not too keen to debug the lovely CMAKE system... is this something in OpenBLAS' own CMakelists.txt ? (Assumed to work on both Arm64 and Windows, just not tested on WoA...). Things hanging could mean problems with multithreading, unless the hangs also happen in single-threaded mode.

[everton1984]

hahaha same feelings here, I guess this is a weird combination of cmake + WoA + flang-new, still doing some tests to properly find the culprit. How can I run the tests in single-threaded mode?

[martin-frbg]

setting OPENBLAS_NUM_THREADS=1 in the environment should hopefully do. Or if in doubt, build everything with `-DUSE_THREAD=0'

[martin-frbg]

Btw, is your flang executable still called flang-new ? While the CMAKE build files do no rely on the f_check I mentioned above, they use whatever CMAKE thinks CMAKE_Fortran_COMPILER is, and cmake/fc.cmake adds some compiler options (only) if the capitalized name is STREQUAL "FLANG". (Not sure if this plays a role for your problem, would mostly affect handling of complex numbers)

[everton1984]

Yeah, they are still releasing it as flang-new for some reason I think it is going to be fused with normal flang just later this year. I noticed that as well but it does not seem to make a difference for this particular problem. Thanks for the tips. I will let you know if I make any progress, if I find this to be too complex I will close this issue and re-open it once everything is working and the wiki can be updated.

Btw, I noticed that on the CI for Windows the BUILD_TESTING flag is disabled, is there any reason? Is it something we can help with?

[martin-frbg]

Btw, I noticed that on the CI for Windows the BUILD_TESTING flag is disabled, is there any reason? Is it something we can help with?

IIRC that was just to keep jobs from exceeding their time limit. Maybe I'll try to activate it on Cirrus (once I get the windows_container build working there)

[brada4]

flang(.exe) has messy history, it was different LLVM-based compiler, then wrapper around gfortran, now the flang-new rename would be 3rd compiler under same name.

[martin-frbg]

Generally curious about the state of flang(-new) in LLVM16 - is it considered stable on all major LLVM platforms ? I just noticed that at least the "Chocolatey" package for Windows x86_64 (as suggested for Cirrus CI jobs) does not appear to include it. (And as mentioned above, the LLVM download page has only a draft 15.0.0 release note for flang under the 16.0.0 heading, with an empty section entitled "Windows support")

[everton1984]

Generally curious about the state of flang(-new) in LLVM16 - is it considered stable on all major LLVM platforms ? I just noticed that at least the "Chocolatey" package for Windows x86_64 (as suggested for Cirrus CI jobs) does not appear to include it. (And as mentioned above, the LLVM download page has only a draft 15.0.0 release note for flang under the 16.0.0 heading, with an empty section entitled "Windows support")

Afaik flang-new is not being distributed on x86 by default, only for Windows on Arm, at least for now but I believe the plan is to make flang-new the 'main' flang. It's hard to track flang state, it is always a moving target.

[everton1984]

@martin-frbg Even with threads disabled *blasN (N >= 2) is hanging. Can I create an Issue for this or at least have a way to contact somebody if I need a bit of help?

[martin-frbg]

Well, you certainly can - but it looks like you are the flang-on-WoA expert here, there's very scant documentation for this compiler and at least I would not know better than you how to debug or trace a program on that system. Perhaps make sure that a pure C build (NOFORTRAN=1 which pulls in f2c-converted LAPACK sources) works first, if you haven't done that already. Not sure if @mmuetzel can help with his recent experience with flang-new/LLVMFlang on x86_64 Windows (PR #4016) ?

[everton1984]

@martin-frbg Trying a pure C build fails for me with a bunch of errors similar to

C:\*\OpenBLAS\lapack-netlib\SRC\slalsa.c(963,25): error: incompatible pointer to integer conversion passing 'integer *' (aka 'int *') to parameter of type 'integer' (aka 'int'); remove & [-Wint-conversion]
            lf = pow_ii(&c__2, &i__1);

is this known or something related to WoA? I am using tag v0.3.21 btw.

[martin-frbg]

Please update to 0.3.23 (or the current state of the develop branch) so that you do not have to rediscover all the bugs that were fixed since last summer. If you really really absolutely need to use the old version, apply PR #3765 to it.

[mmuetzel]

MSYS2 applies a couple of patches for their OpenBLAS binary packages:
https://github.com/msys2/MINGW-packages/tree/master/mingw-w64-openblas

In particular this one might be relevant for you:
https://github.com/msys2/MINGW-packages/blob/d0c310a8f99bcc677b458910a0794318ed7ebeb2/mingw-w64-openblas/004-aarch64-detection.patch

(Not sure why it hasn't been upstreamed to here.)

[mmuetzel]

If you prefer a binary distribution over compiling everything yourself locally, you might be able to use the CLANGARM64 environment of MSYS2. It includes binaries of OpenBLAS for Windows on ARM:
https://packages.msys2.org/package/mingw-w64-clang-aarch64-openblas?repo=clangarm64

I don't have access to such a platform though. So, I don't know how well it is working...

[martin-frbg]

Interesting - I'd hazard a guess that it is not built with flang-whatever though ?

[mmuetzel]

You're guessing correctly. It's built using your f2c version of LAPACK with LLVM Clang.

[everton1984]

Yeah that's part of the point, I am trying to make native OpenBLAS compile with flang-new with LAPACK everything + testing. I can make it compile with some cmake options tweaks, it's just the tests that are giving me headache.

[mmuetzel]

Fwiw, it's building for me locally with MSYS2's version of LLVM Flang 16 on Windows x86 64-bit.
And it passes all tests here.
The flags I used to configure:

cmake -DCMAKE_BUILD_TYPE=Release -DBUILD_SHARED_LIBS=ON -DBUILD_STATIC_LIBS=ON -DDYNAMIC_ARCH=ON -DUSE_THREAD=ON -DNUM_THREADS=64 -DTARGET=CORE2 -DC_LAPACK=OFF ..

Additionally, I modified fc.cmake like in #4016:
2c2306b

[everton1984]

Fwiw, it's building for me locally with MSYS2's version of LLVM Flang 16 on Windows x86 64-bit. And it passes all tests here. The flags I used to configure:

cmake -DCMAKE_BUILD_TYPE=Release -DBUILD_SHARED_LIBS=ON -DBUILD_STATIC_LIBS=ON -DDYNAMIC_ARCH=ON -DUSE_THREAD=ON -DNUM_THREADS=64 -DTARGET=CORE2 -DC_LAPACK=OFF ..

Additionally, I modified fc.cmake like in #4016: 2c2306b

Thanks thats a good data point, I'm guessing my problem is the mix cmake + flang-new on WoA but I need to pinpoint the problem to open up the proper issue on the LLVM team. Although with LAPACK disabled its works fine here as well.

[mmuetzel]

Although with LAPACK disabled its works fine here as well.

Not sure why you point that out. But please notice that it does build LAPACK with LLVM Flang with the above configuration. I only added -DC_LAPACK=OFF to make absolutely sure it doesn't pick the f2c version of LAPACK.

[everton1984]

Although with LAPACK disabled its works fine here as well.

Not sure why you point that out. But please notice that it does build LAPACK with LLVM Flang with the above configuration. I only added -DC_LAPACK=OFF to make absolutely sure it doesn't pick the f2c version of LAPACK.

I pointed that out because I can't read :) I thought it was -DC_LAPACK=ON

[everton1984]

Looks like this is really a flang-new + cmake bug, gonna close this issue for now. I will get this sorted and reopen it when it is ready. Thanks for all the comments.

[martin-frbg]

@everton1984 any news on the status of LLVM17 and especially its flang-new on WoA ?