Windows x86-64 Clang compiler fails to compile because of lacking -mavx2

[Rt39]

I'm compileing flac on Windows x86-64 with Clang version 17.0.1 and targeting at x86_64-pc-windows-msvc with Visual Studio 2022 installed. This is the command I used to configure:

cmake .. -G Ninja -DCMAKE_BUILD_TYPE=Release -DCMAKE_C_COMPILER=clang -DCMAKE_CXX_COMPILER=clang++ -DWITH_OGG=OFF -DBUILD_EXAMPLES=OFF -DBUILD_TESTING=OFF -DBUILD_DOCS=OFF -DINSTALL_MANPAGES=OFF -DINSTALL_PKGCONFIG_MODULES=OFF -DCMAKE_INSTALL_PREFIX="D:\flac"

and I got the error

C:/Users/Administrator/Downloads/Compressed/flac-master/src/libFLAC/stream_encoder_intrin_avx2.c:65:5: error: use of undeclared identifier '__m256i'
   65 |                                 __m256i sum256 = _mm256_setzero_si256();
      |                                 ^
C:/Users/Administrator/Downloads/Compressed/flac-master/src/libFLAC/stream_encoder_intrin_avx2.c:70:6: error: use of undeclared identifier '__m256i'
   70 |                                         __m256i res256 = _mm256_abs_epi32(_mm256_loadu_si256((const __m256i*)(const void*)(residual+residual_sample)));
      |

It seems you need to add compile flag -mavx2 to the compiler, where I add the -DCMAKE_CXX_FLAGS="-mavx2" -DCMAKE_C_FLAGS="-mavx2" to the command and everthing works well.

[ktmf01]

This is not really a solution, because adding -mavx2 breaks runtime detection. The resulting binary will probably not work on any system that does not have AVX2. This bit of configuration should handle this:

flac/src/libFLAC/CMakeLists.txt

Lines 21 to 23 in 72787c3

	if(WITH_AVX AND MSVC)
	set_source_files_properties(fixed_intrin_avx2.c lpc_intrin_avx2.c stream_encoder_intrin_avx2.c lpc_intrin_fma.c PROPERTIES COMPILE_FLAGS /arch:AVX2)
	endif()

But perhaps that code does not work with Clang+MSVC? I have no clue how that hybrid is supposed to work. Could you perhaps share a the CMakeCache.txt file that your configuration produces, and the cmake console output?

[Rt39]

It seems everything go well in the cmake console output

D:\cs\flac-master\build>cmake .. -G Ninja -DCMAKE_BUILD_TYPE=Release -DCMAKE_C_COMPILER=clang -DCMAKE_CXX_COMPILER=clang++ -DWITH_OGG=OFF -DBUILD_EXAMPLES=OFF -DBUILD_TESTING=OFF -DBUILD_DOCS=OFF -DINSTALL_MANPAGES=OFF -DINSTALL_PKGCONFIG_MODULES=OFF -DCMAKE_INSTALL_PREFIX="D:\flac" -DCMAKE_CXX_FLAGS="-mavx2" -DCMAKE_C_FLAGS="-mavx2"
-- The C compiler identification is Clang 17.0.1 with GNU-like command-line
-- The CXX compiler identification is Clang 17.0.1 with GNU-like command-line
-- Detecting C compiler ABI info
-- Detecting C compiler ABI info - done
-- Check for working C compiler: C:/Program Files/LLVM/bin/clang.exe - skipped
-- Detecting C compile features
-- Detecting C compile features - done
-- Detecting CXX compiler ABI info
-- Detecting CXX compiler ABI info - done
-- Check for working CXX compiler: C:/Program Files/LLVM/bin/clang++.exe - skipped
-- Detecting CXX compile features
-- Detecting CXX compile features - done
-- Performing Test HAVE_MBSTATE
-- Performing Test HAVE_MBSTATE - Success
-- Performing Test DODEFINE_EXTENSIONS
-- Performing Test DODEFINE_EXTENSIONS - Success
-- Looking for byteswap.h
-- Looking for byteswap.h - not found
-- Looking for inttypes.h
-- Looking for inttypes.h - found
-- Looking for stdint.h
-- Looking for stdint.h - found
-- Looking for stdbool.h
-- Looking for stdbool.h - found
-- Looking for arm_neon.h
-- Looking for arm_neon.h - not found
-- Looking for semaphore.h
-- Looking for semaphore.h - not found
-- Looking for x86intrin.h
-- Looking for x86intrin.h - found
-- Looking for fseeko
-- Looking for fseeko - not found
-- Performing Test HAVE_BSWAP16
-- Performing Test HAVE_BSWAP16 - Success
-- Performing Test HAVE_BSWAP32
-- Performing Test HAVE_BSWAP32 - Success
-- Performing Test HAVE_LANGINFO_CODESET
-- Performing Test HAVE_LANGINFO_CODESET - Failed
-- Performing Test HAVE_WERROR_FLAG
-- Performing Test HAVE_WERROR_FLAG - Success
-- Performing Test HAVE_DECL_AFTER_STMT_FLAG
-- Performing Test HAVE_DECL_AFTER_STMT_FLAG - Success
-- Performing Test HAVE_STACKREALIGN_FLAG
-- Performing Test HAVE_STACKREALIGN_FLAG - Success
-- Performing Test HAVE_WEFFCXX_FLAG
-- Performing Test HAVE_WEFFCXX_FLAG - Success
-- Performing Test HAVE_STACK_PROTECTOR_FLAG
-- Performing Test HAVE_STACK_PROTECTOR_FLAG - Success
-- Looking for cpuid.h
-- Looking for cpuid.h - found
-- Looking for sys/param.h
-- Looking for sys/param.h - not found
-- Looking for lround
-- Looking for lround - not found
-- Check CPU architecture is x64
-- Check CPU architecture is x64 - yes
-- Performing Test HAVE_ASSOC_MATH
-- Performing Test HAVE_ASSOC_MATH - Failed
-- Looking for string.h
-- Looking for string.h - found
-- Looking for sys/ioctl.h
-- Looking for sys/ioctl.h - not found
-- Looking for termios.h
-- Looking for termios.h - not found
-- Looking for clock_gettime
-- Looking for clock_gettime - not found
-- Configuring done (5.0s)
-- Generating done (0.0s)
-- Build files have been written to: D:/cs/flac-master/build

Here is the CMakeCache.txt
CMakeCache.txt

Also, I found out that if a user choose to pre-build libogg, you will encounter errors. If I choose to build with ogg -DWITH_OGG with libogg-1.3.5 building with the following command:

cmake .. -G Ninja -DCMAKE_BUILD_TYPE=Release -DCMAKE_C_COMPILER=clang -DCMAKE_CXX_COMPILER=clang++ -DINSTALL_DOCS=OFF -DINSTALL_PKG_CONFIG_MODULE=OFF -DINSTALL_CMAKE_PACKAGE_MODULE=ON -DCMAKE_INSTALL_PREFIX="D:\libogg"

And use flac linking to this pre-built library by assigning there positon:

cmake .. -G Ninja -DCMAKE_BUILD_TYPE=Release -DCMAKE_C_COMPILER=clang -DCMAKE_CXX_COMPILER=clang++ -DWITH_OGG=ON  -DBUILD_EXAMPLES=OFF -DBUILD_TESTING=OFF -DBUILD_DOCS=OFF -DINSTALL_MANPAGES=OFF -DINSTALL_PKGCONFIG_MODULES=OFF -DCMAKE_INSTALL_PREFIX="D:\flac" -DCMAKE_CXX_FLAGS="-mavx2" -DCMAKE_C_FLAGS="-mavx2" -DOGG_INCLUDE_DIR="D:\libogg\include" -DOGG_LIBRARY="D:\libogg\lib"

You will get linker errors, complaining symbols not found.

It turns out that it might be some mistakes in CMakeLists.txt:
https://github.com/xiph/flac/blob/72787c3fee472809fb3f819558c7176066f0ee44/src/libFLAC/CMakeLists.txt#L70C1-L73

You can't simply judge whether WITH_OGG is on to include source code of a library. When a user choose to build libogg first instead of put it in the subdirectory of the project (just like me), there will be NO files like ogg_decoder_aspect.c or such, therefore the build will certainly fail.

[ktmf01]

I can't seem to figure out what the problem is. There is a lot if-then-else stuff that assumes a compiler is either clang or msvc, not both, so there is probably something going wrong there. I'm currently not able to investigate this any further.

It turns out that it might be some mistakes in CMakeLists.txt: https://github.com/xiph/flac/blob/72787c3fee472809fb3f819558c7176066f0ee44/src/libFLAC/CMakeLists.txt#L70C1-L73

There is nothing wrong with that, those files belong to FLAC, not to ogg.

You can't simply judge whether WITH_OGG is on to include source code of a library. When a user choose to build libogg first instead of put it in the subdirectory of the project (just like me), there will be NO files like ogg_decoder_aspect.c or such, therefore the build will certainly fail.

WITH_OGG will look for a subdirectory called ogg. If it is there, it will include it as a subdirectory and build from source. If it doesn't find that directory, it will look for headers and binary libraries.

[ktmf01]

Okay, I finally looked into this. Why are you selecting Ninja as generator when building with CMake for Visual Studio? It seems to me you should be using Visual Studio's own build system?

If I use cmake -T"ClangCL" X:\Path\to\source\dir and then cmake --build . --config Release I still get a lot of errors, presumably because the Clang frontend has been heavily modified, but that seems a more reasonable way to use these build tools?

[Rt39]

Thank you for your patience.

Well, I don't think it's related to building systems. I think I could use any generator, no matter it is Ninja, mingw32-make or nmake and the result would not change.

The reason I'm not using Visual Studio building system is that I'm having nothing to do ;-) and be struck on a whim trying to build this awesome program with other compilers. Just in case someone as bored as me is going to build it with clang or gcc on Windows?

After through testing with clang 18.1.5, clang-cl 18.1.5 and MinGW-W64 with gcc 13.2.0, I found that we only need compiler flag -mavx2 if you are building using clang with MSVC headers (i.e. -target x86_64-pc-windows-msvc), I think that can be fixed in CMakeLists.txt inside directory libFLAC.

To build with clang-cl, I didn't succeed using cmake -T"ClangCL" X:\Path\to\source\dir, but succeed by first activating x64 Native Tools Command Prompt for VS 2022 or vcvars64.bat, and then with cmake .. -G "NMake Makefiles" X:\Path\to\source\dir -DCMAKE_BUILD_TYPE=Release -DCMAKE_C_COMPILER=clang-cl -DCMAKE_CXX_COMPILER=clang-cl and then cmake --build .. Ninja could also perform its roles.