Using conan with clang-cl and CMake on Windows

[dutow]

Clang provides the clang-cl binary, which is compatible with MSVC's cl. It can also be used with the Ninja CMake generator, for example, the following way:

call "C:\Program Files (x86)\Microsoft Visual Studio\2017\Professional\VC\Auxiliary\Build\vcvarsall.bat" x64

set CC=clang-cl
set CXX=clang-cl
set CFLAGS=-m64 -fmsc-version=1910
set CXXFLAGS=-m64 -fmsc-version=1910
cmake ../ -GNinja

CMake's internal scripts correctly detect the compiler to be cl-compatible and use the cl style command line parameters in this case.

Conan itself has no knowledge about this compiler, but I assumed that if I compile the dependencies with MSVC (which is the default), I should be able to use them in my CMake project.

While this approach mostly works, the conanbuildinfo.cmake script generated by conan detects the compiler as Clang instead of as MSVC, and displays an error, because of the following line:

if( (CONAN_COMPILER STREQUAL "Visual Studio" AND NOT CMAKE_CXX_COMPILER_ID MATCHES MSVC) OR

While the Ninja generator sets the compiler id to Clang, it also sets the MSVC variable to 1 and sets the MSVC_VERSION correctly to 1910.

I also know that ideally compilers shouldn't be mixed, but I also assume that adding fully working clang/clang-cl support for conan on windows would be a significant work.

As a simpler solution, this check could be more permissive, for example:

1. By changing that line to just if( (CONAN_COMPILER STREQUAL "Visual Studio" AND NOT MSVC)
2. By changing the MSVC version checks to use MSVC_VERSION instead of the CMAKE_CXX_COMPILER_VERSION variable. (currently, this step isn't necessary, because as I described in Missing MSVC version checks in conanbuildinfo.cmake #1838, the MSVC version checks are incomplete and pass even when they shouldn't)

This change would allow any compiler which claims to be compatible with the given MSVC version to be used. It would be also consistent with how most projects detect that they are compiled with MSVC.

[memsharded]

Thanks for your detailed feedback and suggestions.

@SSE4 has been trying recently to use Clang in Windows, but not sure if clang-cl. Any update on this @SSE4 ?

So, @dutow the idea is that binaries compiled with clang-cl are fully interchangeable and binary-compatible with binaries built with cl?

[SSE4]

I used both "clang.exe" and "clang-cl.exe". first one accepts GCC-style arguments, while seconds MSVC-style arguments.
LLVM toolchain uses clang-cl, so CMake-generated projects use clang-cl as well.
I was building all dependencies with Clang, so was passing "-s compiler=clang" to conan command line.
clang doc says "Clang tries to be compatible with MSVC, but work in progress". so I am not sure is it really safe to mix Clang-compiled and MSVC-compiled libraries ATM.

[dutow]

so I am not sure is it really safe to mix Clang-compiled and MSVC-compiled libraries ATM.

You will always mix binaries with clang on windows - for example, clang doesn't reimplement the MSVC runtime, it uses the libraries / DLLs provided by MSVC. Same for the entire Windows SDK, and so on. If you check the list in your link, most of the components are "complete" or "mostly complete".

The only difference between clang and clang-cl is the default way they accept command line arguments - otherwise, they are the same, so both commands will link with MSVC libs / dlls.

Or you used clang with MingW somehow?

I was building all dependencies with Clang, so was passing "-s compiler=clang"

That argument also requires passing a value for compiler.libcxx - and neither stdc++11 or libc++ is correct there. While it's possible that we can compile packages with those flags, in reality, clang on windows will use the c++ standard library provided by MSVC (or libcxx, if you built it yourself, but that's even more experimental)

(And in my quick experiment, it also tried to build packages with make, which I do not have installed, but I'll see if I can change that somehow)

[SSE4]

nope, didn't use Clang with MinGW.
I've passed "compiler.libcxx=libc++" to just silence conan error, however, still compiled with MSVC stdlib

[dutow]

just silence conan error

While it indeed gets a few more things working, I'm not 100% comfortable adding such "fixes", or "silencers" to my builds scripts - that's the main reason I'm looking for something without such additions.

I rechecked why I'm getting errors, based on the zlib-conan project, which has an open issue about clang support on windows, and turns out that I can indeed build it using clang, assuming that I do specify several environment variables before running conan - which probably should be done automatically in the background.

So maybe the issue is just that conan's clang support should be smarter on Windows, and provide better default parameters for CMake:

• If on windows, and using clang, then set the CXX, CC, CXXFLAGS and CFLAGS environment variables correctly
• If on windows, and using clang, then load the visual studio environment before invoking cmake commands

It's possible that the first is only required when using the Ninja generator I haven't tested the others.

[SSE4]

yep, that's not a real fix, just crutch to get build further.
Clang on Windows causes lots of errors everywhere - support of tools and libraries are still very poor, and they make incorrect assumptions that prevent building (like assuming Windows = Visual Studio or Clang = Linux). I've already filled numerous PRs and bug reports against boost, CMake, conan, clang itself, and it continues. I didn't really get far enough, because boost still has some issues to be fixed.
about CXX, CC, CXXFLAGS, CFLAGS - you can create custom profile and set them appropriately, then pass profile name to the conan. I am already using such approach for Android, since it requires special flags to be set.
summarising, it's still very far enough to be used in production, and significant amount of work has to be done against various build tools and libraries.

[memsharded]

Thanks @SSE4 for all the feedback

yes, @dutow there are a lot of PRs from @SSE4 related to this issue. We are on it.

I am leaving this issue open for further discussion, I think it is quite important to first define some settings.yml that would be useful for this use case, then proceed to update the build helpers, like CMake.

[zscgeek]

Very interested in this issue. Right now we are building with clang-cl, as our codebase does not play well with the default MS cl.exe but we need to link into the MSVC runtimes. Has there been any recent work on this?

[lasote]

I think not directly to this but many improvements have been released since then. Would be nice to collect the current status (bugs, detected features...). @SSE4 @dutow any updated feedback? we can try to take a look for the next release if we have some concrete actions to do.

[zscgeek]

The issue I am running into is I want to be able to use clang-cl as my compiler and the MSVC 2017 runtimes. If I setup my conan profile to have the MSVC 2017 info, and set CC/CXX=clang-cl it will fail to build with the checks here:

https://github.com/mpdelbuono/conan/blob/master/conans/client/generators/cmake_common.py#L311

My alternative, is I can setup my conan profile for clang, setting CC/CXX=clang-cl and I will get warnings about invalid command line options that don't apply that are getting injected by conan. (-stdlib=libstdc++ for example).

I also get into all sorts of issues when I try to pull in deps that are built with the full MSVC tool set because the conan compiler settings wont match. So far I have been able to manually override and work around this issue by passing things like -s zlib:compiler="Visual Studio" -s zlib:compiler.runtime="MT" -s zlib:compiler.version="15" for every dep I want to pull in. Really a pain in the rear end.

If anyone has suggestions on how to set this up, I'm totally game to try it. So far it's just been a sea of pain and suffering.

[zscgeek]

The other warning we end up with is LINK : warning LNK4044: unrecognized option '/m64'; ignored. I think this comes because of this bit:

conan/conans/client/build/compiler_flags.py

Line 35 in a63498a

return '-m64'

When using clang-cl you end up usually using the msvc link.exe.

Other bits that could be relevant is that we are running cmake using the ninja generator for these projects.

[SSE4]

@zscgeek this is tricky to handle
the problem is that Clang on Windows has two front-ends:

• clang.exe, which accepts GCC/Clang style arguments
• clang-cl.exe, which tries to mimic MSVC and accepts the same arguments as cl.exe
  also, different linkers might be in use:
• lld.exe (LLVM linker)
• link.exe (MSVC linker)
  this is hard to manage from conan's perspective, but I guess I have an idea how to distinguish em

[zscgeek]

At least for cmake, they pretty much have assumed you are going to use clang-cl when on windows + MSVC. I was never able to get cmake to output clang.exe style -foo options

[SSE4]

that depends on build system for sure. e.g. for boost b2, clang.exe is used, and all arguments are in GCC style

[SSE4]

I will try to add required changes into this PR: #3256
subscribe if you're interested
(it's about detecting compiler id and use appropriate flags)

[zscgeek]

I will keep an eye on it!

[zscgeek]

In the meantime, how are others dealing with something like this in the existing system? Or is really nobody else mixing in clang-cl + MSVC?

I often find myself resorting to killing off settings.compiler from the final hash to make sure things get correctly found in this type of setup. For the packages we control that are internal that becomes the easiest fix.

[memsharded]

Quick question, from my ignorance about this issue. Could be using the v140_clang_c2 toolset from Visual Studio an option? Or maybe the v140_clang_3_7 one?

From what I have heard from users is that most could be waiting to use the full clang compiler in Windows, not much interest in fighting the clang-cl system. I am willing to improve conan clang-cl support, but that might take some time. Probably we need to add something more the default settings.yml.

[SSE4]

related https://github.com/conan-community/community/issues/63

[trondhe]

I can now build on Windows using Clang, CMake and Ninja. But it seems that the clang setting expects the usage of a GNU-like cli for the compiler as it requires libcxx setting in settings.yml. This is not valid on Windows as CMake only supports the usage of clang-cl which expects a MSVC style cli which does not use libstd. It worked fine for me by deleting the libcxx setting under settings.yml from clang, as then clang-cl did not complain about unknown command for libstdc++. There should probably be a setting to differentiate clang with gnu cli and clang with msvc cli.

Example of clang profile

[env]
CC=D:/Programs/LLVM/bin/clang-cl.exe
CXX=D:/Programs/LLVM/bin/clang-cl.exe

[settings]
os=Windows
os_target=Windows
os_build=Windows
arch=x86_64
compiler=clang
compiler.version=7.0
build_type=Debug

This only works when settings.yml for clang looks like this:

compiler:
    Visual Studio:
        runtime: [MD, MT, MTd, MDd]
        version: ["8", "9", "10", "11", "12", "14", "15"]
        toolset: [None, v90, v100, v110, v110_xp, v120, v120_xp,
                  v140, v140_xp, v140_clang_c2, LLVM-vs2012, LLVM-vs2012_xp,
                  LLVM-vs2013, LLVM-vs2013_xp, LLVM-vs2014, LLVM-vs2014_xp,
                  LLVM-vs2017, LLVM-vs2017_xp, v141, v141_xp, v141_clang_c2]
    clang:
        version: ["3.3", "3.4", "3.5", "3.6", "3.7", "3.8", "3.9", "4.0",
                  "5.0", "6.0", "7.0"]
    apple-clang:
        version: ["5.0", "5.1", "6.0", "6.1", "7.0", "7.3", "8.0", "8.1", "9.0", "9.1", "10.0"]
        libcxx: [libstdc++, libc++]

Where I have removed the libcxx option, as the msvc interface of clang-cl does not use this option and without removing it conan complains that it needs to be set.
This is again with using the Ninja generator.

[detwiler]

We're also trying to build with Conan, Clang-cl, CMake and Ninja on Windows. This entire thread was very helpful. @trondhe, your post was especially helpful. We were able to work around the issue you described with Conan expecting a libcxx setting for the clang compiler, by adding None to the list of values for libcxx in settings.yml. We distribute a shared settings.yml across our team, and we need to be able to specify a libcxx value for clang on Mac and Linux. Here's the profile and settings.yml that are working for us.

Example clang profile:

[settings]
os=Windows
os_build=Windows
arch=x86_64
arch_build=x86_64
build_type=Debug
compiler=clang
compiler.version=7.0

[env]
CC=clang-cl
CXX=clang-cl
CONAN_CMAKE_GENERATOR=Ninja
CONAN_CMAKE_PROGRAM=C:/PROGRA~1/CMake/bin/cmake.exe

Example settings.yml:

compiler:
    Visual Studio:
        runtime: [MD, MT, MTd, MDd]
        version: ["8", "9", "10", "11", "12", "14", "15"]
        toolset: [None, v90, v100, v110, v110_xp, v120, v120_xp,
                  v140, v140_xp, v140_clang_c2, LLVM-vs2012, LLVM-vs2012_xp,
                  LLVM-vs2013, LLVM-vs2013_xp, LLVM-vs2014, LLVM-vs2014_xp,
                  LLVM-vs2017, LLVM-vs2017_xp, v141, v141_xp, v141_clang_c2, LLVM]
    clang:
        version: ["3.3", "3.4", "3.5", "3.6", "3.7", "3.8", "3.9", "4.0",
                  "5.0", "6.0", "7.0", "8"]
        libcxx: [None, libstdc++, libstdc++11, libc++]
    apple-clang:
        version: ["5.0", "5.1", "6.0", "6.1", "7.0", "7.3", "8.0", "8.1", "9.0", "9.1", "10.0"]
        libcxx: [libstdc++, libc++]

The problem we're running into now is that some third-party Conan package recipes that we use haven't accounted for the possibility of compiler=clang when os=Windows.

[dutow]

Re: GNU command line clang. I have a merge request for CMake to support it on windows - after that gets merged, both clang and clang-cl can be real use cases on windows. In the current version, CMake gets a new information variable (CMAKE_lang_COMPILER_FRONTEND_VARIANT) which lets users differentiate between these. (https://gitlab.kitware.com/cmake/cmake/merge_requests/2992)

[IRainman]

I have problems with pcre [package] pcre/8.45: Build failed with ClangCL on Visual Studio and expat
[package] expat/2.4.1: Build failed with ClangCL on Visual Studio and bzip2
[package] bzip2/1.0.8: Build failed with ClangCL on Visual Studio

[SSE4]

related discussion: conan-io/conan-center-index#9807

[memsharded]

This PR #11492, merged for next 1.53 contains a few changes to better support clang in Windows, mainly for the new CMakeToolchain integration, but some minor changes for others too.

There are some other pending issues about the Windows subsystems environment management, we are also trying to improve them in #12178, so if you are using Clang in some subsystem and depend on the environment, you might want to track this PR too.

Closing this issue now, but we know that there might still be some gaps, so please try to update to the new integration (this is necessary for 2.0 anyway), and report what might still be failing against this new integration. The best starting point would be the tests in https://github.com/conan-io/conan/blob/develop/conans/test/functional/toolchains/cmake/test_cmake_toolchain_win_clang.py, or using any of the predefined templates conan new hello/0.1 -m=cmake_lib|autotools_lib|msbuild_lib|meson_lib, and open a new issue. Many thanks!

[hoyhoy]

@memsharded Doesn't appear to work. I put ccache ahead of cl.exe in my path, and set it via my conan profile, but cmake is still is calling the compiler out of the visual studio directory.

PS C:\Users\tperr> (Get-Command cl.exe).Path
C:\ccache\cl.exe

I even changed the name of the binary in my conan profile...

PS C:\Users\tperr> cat C:\Users\tperr\.conan2\profiles\default
[settings]
os=Windows
arch=x86_64
compiler=msvc
compiler.cppstd=14
compiler.version=193
compiler.cppstd=14
compiler.runtime=dynamic
build_type=Debug

[conf]
tools.build:compiler_executables = { "c" : "ccache.exe", "cpp": "ccache.exe" }

I tried specifying a full path to tools.build:compiler_executables as well. Also doesn't work. cl.exe is still being called from...

Conan/cmake is still running cl.exe from...

 C:\Program Files\Microsoft Visual Studio\2022\Professional\VC\Tools\MSVC\14.38.33130\bin\HostX64\x64\CL.exe

[hoyhoy]

Apparently MSBuild automatically overrides the compiler paths....