Modernise existing cmake options

[enetheru]

This PR builds on #1595
In this PR, I hope to consolidate all the existing code into one style so that we can start adding things that exist in the scons build but are missing from cmake that people have requested.

The primary changes are

• Adding all the {platform}.cmake files
• The moving of all conditional target options/definitions to generator expressions.

Tests for Linux, Macos, and Windows are passing.

This style is cleaner and supports the multi-config generators like Visual Studio and Ninja-Multi-Config. Tho I find it slightly more annoying to debug personally, it is the better method.

I've been making sure these builds are Identical using this spreadsheet

Commit log

- Added to .gitignore CMakeUserPresets.json

### Configuration:
- Changed python command to use single quotes to make build output log more legible.
- Added GODOT_DEV_BUILD to allow differentiation of debug or Release builds.
- Added find logic for macos Cocoa library

### godot-cpp Changes
- godot-cpp-test is changed to be incorporated into the cmake build as a target.
- Duplicated godot-cpp target into [template_release, template_debug, editor]
- Created cmake/sources.cmake to collect all the pre-existing source files, because globing is a source of bugs.
- Created {platform}.cmake files mirroring the style of the SCons build.

CMake has a feature called generator expressions for its configuration variables that are evaluated at build time. This allows multi-configuration build systems to properly evaulate options. for msvc, xcode and nijna multi-config.

- Moved configuration options to generator expressions with the notable exclusion of OSX_ARCHITECTURES.
- Remove CMAKE_BUILD_TYPE from msvc CI target as Multi-Config generators ignore it

### godot-cpp-test Changes
- Removed majority of the cmake code, now that the godot-cpp project is setup, the majority of the flags will be propagated as transient dependencies
- Marked with EXCLUDE_FROM_ALL so that it isn't built as part of the 'all' target
- Updated ci to build the godot-cpp-test target from the root directory using cmake
- Tests passing for Windows, Linux, and Macos builds.

### Documentation
Updated with new information
Added Emscripten example
Added Android example

[enetheru]

Looks like I have to fix up the test case

[enetheru]

Because of the changes, the CI stopped working, so I needed to overhaul the test/CMakeLists.txt to consider the changes, and reduced the cmake code dramatically.

[enetheru]

Again to aid with verification, here is a google doc spreadsheet showing the differences in output.

If there's anything you'd like me to check please mention it.

[aaronfranke]

Overall the organization in this PR looks great. There's a lot missing still but as you stated that is expected for this first PR. I like how it lays out a plan for what's to come. When we get to implementing features such as architecture handling in a future PR I have a branch here, which enetheru already knows about, just linking for easy access.

I tried testing this PR. I was able to build godot-cpp by itself, but I was not able to get the test project to build. I tried running cmake --build . --verbose -j $(nproc) -t godot-cpp-test (copied from the CI script in this PR) on macOS and it fails silently, no compiled files showed up in the bin folder. Although I wasn't able to get the test project working on master either, with a lot of errors being printed on master.

I also tried compiling Godot Orchestrator with this PR, which is a big Godot project using CMake. I had to make two expected changes to Orchestrator to make it compile: replacing INCLUDE(GodotCompilerWarnings) with INCLUDE(common_compiler_flags) (it's fine to have to make changes, since we don't guarantee API stability with the CMake files) and changing the Godot minor version to 3 to work around an Orchestrator bug. However, it then failed to compile with the error c++: error: unsupported option '-static-libgcc'. I've already opened a bug report for this here #1522 but it's strange that the bug didn't show up with Orchestrator with master, and doesn't show up in godot-cpp by itself either with master or this PR. Anyway, worth looking into if we can fix #1522 - AFAICT we shouldn't ask clang to be static libgcc since it's not gcc (or, if there is an obscure reason why it is needed, then there needs to be a comment, and we need to fix whatever is causing compilation to fail). However, just removing this line doesn't fix it, it will then complain about -R being unrecognized. Anyway, if fixing #1522 is not in-scope then it can be fixed in a follow-up PR.

[enetheru]

I found some issues with my hasty conversion of the test project.

[enetheru]

So it looks both like RPATH is not relevant to windows, and that clang and gcc differ in their flags to set it. since I dont currently have a linux or mac computer, perhaps @aaronfranke you might be able to get some compiler and link output for me that I can see what is listed when you compile? I think CMAKE will automatically do it using the target properties feature, but since I am currently on windows I cant tell.. I can get a linux machine up and running soon, but it might not be before I go on holiday. But I have no access to a mac.

[enetheru]

I also tried compiling Godot Orchestrator with this PR, which is a big Godot project using CMake

I'll try to see whats up with orchestrator.

[enetheru]

Ivan has provided me some feedback: https://gist.github.com/IvanInventor/339f686883ea184b632bad1fcd183462#gistcomment-5199373

[enetheru]

After reviewing how orchestrator expected to include the headers, I made an adjustment which is in keeping with those expectations. The include path is now exported to consumers, rather than all the header files individually.
To fix my compilation of orchestrator I only needed to remove the include of GodotCompilerWarnings as they are now automatically propogated. And update the bindings to the godot version I was compiling against. Though I'm waiting on a compile of the engine without DEV_ENABLED so that I can load up the resulting dll. @aaronfranke

[enetheru]

@aaronfranke Success!, Yeah so line 115, remove the INCLUDE( GodotCompilerWarnings ) is the only change I had to make. I have orchestrator example project working on godot/4.3, godot-cpp/modernise

[enetheru]

My spreadsheet is updated, mainly for the msvc version, which is really annoying because I'm mostly a Linux guy. There are real problems that need to be addressed.
https://docs.google.com/spreadsheets/d/1aUjib11P9CfcMK7P5WhdRIs80jMO9mGPK4RbMlSccCM/edit?usp=sharing

[enetheru]

I just buttoned down the last few remaining items that were on my concerns list. I think I can now justify the tiny changes, like filenames matching the build config, compile definitions, and warnings as being slightly more correct than the master branch.

I'm very confident that this will improve the lives of everyone who chooses cmake, whether that is personal preference, or that their other dependencies require it.

[enetheru]

Pretty sure this is done..

[enetheru]

@dsnopek OK, so I am back from my holiday in Europe( which was great ) and I want to get back onto this.

I remember their being a request in the re-structure comments to update the docs/cmake.md with updated information. Is there anything else that is wanted from this?

Regarding the documentation, I've been trying to think(granted jetlag is still an issue) about the purpose of what is there and I think it is not coherant. Building the godot-cpp static library separately to an extension using cmake is not the cmake way.

I cant quite make an argument for the doc/cmake.md existence that would not have a more appropriate alternative.

[enetheru]

OK so I had run into some trouble verifying the OSX build which forced me to re-think the approach I was taking. I was trying to do things in piecemeal, small single issue changes, but that was actually complicating the code when it came to making multi-platform work for more than two setups.

I eventually needed to incorporate the individual platforms as separate files to cater to the multi platform setup, which i was trying to put off, but could no longer. So thats the delay.

I have verified Linux, Darwin, Windows builds pass the test project. Since my current main machine is Windows it also builds Android and Web builds though I dont know how to test. And I have tested multiple other mingw toochains with tests reporting success.

In its current form it is almost a direct mirror of SCons so should be easy for a superficial comparison.

Cheers.

[dsnopek]

Thanks!

Overall, I love the direction this PR is going! I've skimmed through the code and it's mostly looking good, although, I haven't had a chance yet to compare the cmake config line-by-line with scons (which I do intend to try and do).

However, this is failing CI due to two minor spelling issues:

And I've been unable to actually test this on my machine, because I get errors when doing:

mkdir build
cd build
cmake ../ -G "Ninja"

I get these errors:

CMake Error at test/CMakeLists.txt:5 (add_library):
  Target 'godot-cpp-test' OUTPUT_NAME depends on itself.


CMake Error at cmake/godotcpp.cmake:181 (add_library):
  Target 'template_debug' OUTPUT_NAME depends on itself.
Call Stack (most recent call first):
  CMakeLists.txt:56 (godotcpp_generate)


CMake Error at cmake/godotcpp.cmake:181 (add_library):
  Target 'template_release' OUTPUT_NAME depends on itself.
Call Stack (most recent call first):
  CMakeLists.txt:56 (godotcpp_generate)


CMake Error at cmake/godotcpp.cmake:181 (add_library):
  Target 'template_debug' OUTPUT_NAME depends on itself.
Call Stack (most recent call first):
  CMakeLists.txt:56 (godotcpp_generate)

And it doesn't successfully generate the build files for me to use.

I get different errors when I try to build the test project - doing:

cd test
mkdir build
cd build
cmake ../ -C "Ninja"

... gives this error:

CMake Error at CMakeLists.txt:27 (set_target_properties):
  set_target_properties called with incorrect number of arguments.

Am I doing something wrong?

[enetheru]

Thanks!

Overall, I love the direction this PR is going! I've skimmed through the code and it's mostly looking good, although, I haven't had a chance yet to compare the cmake config line-by-line with scons (which I do intend to try and do).

@dsnopek Cheers. keep in mind when doing the line-by-line is that I have purposely left out options that werent already present and not required for a platform. like thread support, and optimisation. but I can add them if required.

However, this is failing CI due to two minor spelling issues:

I really detest these spelling checks, but I understand why they need to be. I even have it setup on my build machine as a task I can run, but I forgot. These have been fixed.

And I've been unable to actually test this on my machine, because I get errors when doing:

mkdir build
cd build
cmake ../ -G "Ninja"

I get these errors:

<snip>

And it doesn't successfully generate the build files for me to use.

I can't reproduce, even when requiring cmake versions to the minimum. Can you let me know what your cmake version is? build platform etc?

I get different errors when I try to build the test project - doing:

cd test

<snip>

Am I doing something wrong?

CMake can't depend on directories above the project root like scons can, so the test project is part of the godot-cpp project as another target.

mkdir build
cd build
cmake ../
cmake --build . --target godot-cpp-test

[dsnopek]

I really detest these spelling checks, but I understand why they need to be. I even have it setup on my build machine as a task I can run, but I forgot. These have been fixed.

Thanks!

I can't reproduce, even when requiring cmake versions to the minimum. Can you let me know what your cmake version is? build platform etc?

I'm on Ubuntu 22.04, with cmake 3.22.1 (from the Ubuntu package).

Here's the full output.

UPDATE: I've tried doing some more testing. I thought that maybe the issue was caused by some files I have laying around from doing development in the same directory over a long period of time, so I downloaded a ZIP of your branch, and ran cmake there, but I got the same issue. It's really weird, because the CI seems to also be using Ubuntu 22.04 and the Ubuntu cmake package, yet it works there...

[enetheru]

cmake 3.22.1

I found the problem with my code:

  $<LIST:GET,list,index,...>
    Added in version 3.27.

cmake-documentation

I'll come up with a different way of doing that thing. Cheers for the help.

[enetheru]

but I got the same issue

@dsnopek I really appreciate you testing and providing the logs.

I've replaced the offending generator expressions. I've also downloaded and tested with the project minimim version 3.13, I believe it should work now.

[dsnopek]

Thanks! This is looking really, really great to me :-)

I did a bunch of testing, and everything seems to be working for me. The docs look fantastic, covering lots of platforms and configurations, and everything I tried out of there worked.

I did some amount of line-by-line comparison, but it's tricky, because so much is provided by CMake out-of-the-box. For example, there's practically nothing in android.cmake, but it's fine because all the features we have in android.py for SCons appear to be built-in to CMake itself.

So, after the couple of minor notes I made below are addressed, I think I'm personally leaning towards just merging this (but not cherry-picking right away) and letting other developers try it out and see what issues they do or don't run into.

[enetheru]

I did some amount of line-by-line comparison, but it's tricky, because so much is provided by CMake out-of-the-box. For example, there's practically nothing in android.cmake, but it's fine because all the features we have in android.py for SCons appear to be built-in to CMake itself.

I can provide more information in that file in the way of documentation if you like? It cant hurt. I'll review the differences and see if I can't document how it works.

[dsnopek]

@enetheru:

I can provide more information in that file in the way of documentation if you like?

If you just mean putting a comment in android.cmake saying something like "we don't have much here because CMake provides all the necessary Android options out-of-the-box", that'd probably be nice to have, but I don't think its essential.

EDIT: Ah, I see you already added it! It looks good to me :-)

[dsnopek]

The latest version of this is looking great to me!

Thanks so much for your hard work and persistence :-)

[Ivorforce]

Hi, glad to see you stepping up to modernise the CMake files! A lot of people will appreciate this. It might help us with one project as well :)

I just tested your branch and got an error for the default config on macOS (x86_64, building universal (default)):

cmake .

godot-cpp modernise* ≡
 ❯ cmake .
Auto-detected 8 CPU cores available for build parallelism.
Using  cores, You can override it at configure time by using -j <n> or --parallel <n> on the build command.
  eg. cmake --build . -j 7  ...
-- Configuring done (0.3s)
-- Generating done (0.6s)
-- Build files have been written to: /Users/lukas/dev/godot/godot-cpp

godot-cpp modernise* ≡
 ❯ cmake --build . --verbose -t godot-cpp-test
Change Dir: '/Users/lukas/dev/godot/godot-cpp'

Run Build Command(s): /nix/store/dlk5mzpxif2hjfw43if4wid0sckv6sx7-cmake-3.29.6/bin/cmake -E env VERBOSE=1 /usr/bin/make -f Makefile godot-cpp-test
/nix/store/dlk5mzpxif2hjfw43if4wid0sckv6sx7-cmake-3.29.6/bin/cmake -P /Users/lukas/dev/godot/godot-cpp/CMakeFiles/VerifyGlobs.cmake
/nix/store/dlk5mzpxif2hjfw43if4wid0sckv6sx7-cmake-3.29.6/bin/cmake -S/Users/lukas/dev/godot/godot-cpp -B/Users/lukas/dev/godot/godot-cpp --check-build-system CMakeFiles/Makefile.cmake 0
/Applications/Xcode.app/Contents/Developer/usr/bin/make  -f CMakeFiles/Makefile2 godot-cpp-test
/nix/store/dlk5mzpxif2hjfw43if4wid0sckv6sx7-cmake-3.29.6/bin/cmake -P /Users/lukas/dev/godot/godot-cpp/CMakeFiles/VerifyGlobs.cmake
/nix/store/dlk5mzpxif2hjfw43if4wid0sckv6sx7-cmake-3.29.6/bin/cmake -S/Users/lukas/dev/godot/godot-cpp -B/Users/lukas/dev/godot/godot-cpp --check-build-system CMakeFiles/Makefile.cmake 0
/nix/store/dlk5mzpxif2hjfw43if4wid0sckv6sx7-cmake-3.29.6/bin/cmake -E cmake_progress_start /Users/lukas/dev/godot/godot-cpp/CMakeFiles 33
/Applications/Xcode.app/Contents/Developer/usr/bin/make  -f CMakeFiles/Makefile2 test/CMakeFiles/godot-cpp-test.dir/all
/Applications/Xcode.app/Contents/Developer/usr/bin/make  -f CMakeFiles/template_debug.dir/build.make CMakeFiles/template_debug.dir/depend
cd /Users/lukas/dev/godot/godot-cpp && /nix/store/dlk5mzpxif2hjfw43if4wid0sckv6sx7-cmake-3.29.6/bin/cmake -E cmake_depends "Unix Makefiles" /Users/lukas/dev/godot/godot-cpp /Users/lukas/dev/godot/godot-cpp /Users/lukas/dev/godot/godot-cpp /Users/lukas/dev/godot/godot-cpp /Users/lukas/dev/godot/godot-cpp/CMakeFiles/template_debug.dir/DependInfo.cmake "--color="
/Applications/Xcode.app/Contents/Developer/usr/bin/make  -f CMakeFiles/template_debug.dir/build.make CMakeFiles/template_debug.dir/build
[  0%] Building CXX object CMakeFiles/template_debug.dir/src/classes/editor_plugin_registration.cpp.o
/Users/lukas/.nix-profile/bin/c++ -DDEBUG_ENABLED -DDEBUG_METHODS_ENABLED -DGDEXTENSION -DHOT_RELOAD_ENABLED -DMACOS_ENABLED -DUNIX_ENABLED -I/Users/lukas/dev/godot/godot-cpp/include -I/Users/lukas/dev/godot/godot-cpp/gen/include -I/Users/lukas/dev/godot/godot-cpp/gdextension -O3 -DNDEBUG -std=c++17 -arch x86_64 -arch arm64 -isysroot /Applications/Xcode.app/Contents/Developer/Platforms/MacOSX.platform/Developer/SDKs/MacOSX15.1.sdk -mmacosx-version-min=14.5 -fPIC -fvisibility=hidden -fno-exceptions -fno-gnu-unique -Wall -Wctor-dtor-privacy -Wextra -Wno-unused-parameter -Wnon-virtual-dtor -Wwrite-strings -Walloc-zero -Wduplicated-branches -Wduplicated-cond -Wno-misleading-indentation -Wplacement-new=1 -Wshadow-local -Wstringop-overflow=4 -Wattribute-alias=2 -Wlogical-op -Wno-return-type -MD -MT CMakeFiles/template_debug.dir/src/classes/editor_plugin_registration.cpp.o -MF CMakeFiles/template_debug.dir/src/classes/editor_plugin_registration.cpp.o.d -o CMakeFiles/template_debug.dir/src/classes/editor_plugin_registration.cpp.o -c /Users/lukas/dev/godot/godot-cpp/src/classes/editor_plugin_registration.cpp
g++: error: this compiler does not support ‘arm64’
make[3]: *** [CMakeFiles/template_debug.dir/src/classes/editor_plugin_registration.cpp.o] Error 1
make[2]: *** [CMakeFiles/template_debug.dir/all] Error 2
make[1]: *** [test/CMakeFiles/godot-cpp-test.dir/rule] Error 2
make: *** [godot-cpp-test] Error 2

It notably works in scons; I can't run arm64 files but I can compile them no problem.

When configuring for x86_64 it seems to work:

cmake -D GODOT_ARCH=x86_64 .

godot-cpp modernise* ≡
 ❯ cmake -D GODOT_ARCH=x86_64 .
Auto-detected 8 CPU cores available for build parallelism.
Using  cores, You can override it at configure time by using -j <n> or --parallel <n> on the build command.
  eg. cmake --build . -j 7  ...
-- Configuring done (0.4s)
-- Generating done (0.6s)
-- Build files have been written to: /Users/lukas/dev/godot/godot-cpp

godot-cpp modernise* ≡
 ❯ cmake --build . --verbose -t godot-cpp-test
Change Dir: '/Users/lukas/dev/godot/godot-cpp'

Run Build Command(s): /nix/store/dlk5mzpxif2hjfw43if4wid0sckv6sx7-cmake-3.29.6/bin/cmake -E env VERBOSE=1 /usr/bin/make -f Makefile godot-cpp-test
/nix/store/dlk5mzpxif2hjfw43if4wid0sckv6sx7-cmake-3.29.6/bin/cmake -P /Users/lukas/dev/godot/godot-cpp/CMakeFiles/VerifyGlobs.cmake
/nix/store/dlk5mzpxif2hjfw43if4wid0sckv6sx7-cmake-3.29.6/bin/cmake -S/Users/lukas/dev/godot/godot-cpp -B/Users/lukas/dev/godot/godot-cpp --check-build-system CMakeFiles/Makefile.cmake 0
/Applications/Xcode.app/Contents/Developer/usr/bin/make  -f CMakeFiles/Makefile2 godot-cpp-test
/nix/store/dlk5mzpxif2hjfw43if4wid0sckv6sx7-cmake-3.29.6/bin/cmake -P /Users/lukas/dev/godot/godot-cpp/CMakeFiles/VerifyGlobs.cmake
/nix/store/dlk5mzpxif2hjfw43if4wid0sckv6sx7-cmake-3.29.6/bin/cmake -S/Users/lukas/dev/godot/godot-cpp -B/Users/lukas/dev/godot/godot-cpp --check-build-system CMakeFiles/Makefile.cmake 0
/nix/store/dlk5mzpxif2hjfw43if4wid0sckv6sx7-cmake-3.29.6/bin/cmake -E cmake_progress_start /Users/lukas/dev/godot/godot-cpp/CMakeFiles 33
/Applications/Xcode.app/Contents/Developer/usr/bin/make  -f CMakeFiles/Makefile2 test/CMakeFiles/godot-cpp-test.dir/all
/Applications/Xcode.app/Contents/Developer/usr/bin/make  -f CMakeFiles/template_debug.dir/build.make CMakeFiles/template_debug.dir/depend
cd /Users/lukas/dev/godot/godot-cpp && /nix/store/dlk5mzpxif2hjfw43if4wid0sckv6sx7-cmake-3.29.6/bin/cmake -E cmake_depends "Unix Makefiles" /Users/lukas/dev/godot/godot-cpp /Users/lukas/dev/godot/godot-cpp /Users/lukas/dev/godot/godot-cpp /Users/lukas/dev/godot/godot-cpp /Users/lukas/dev/godot/godot-cpp/CMakeFiles/template_debug.dir/DependInfo.cmake "--color="
/Applications/Xcode.app/Contents/Developer/usr/bin/make  -f CMakeFiles/template_debug.dir/build.make CMakeFiles/template_debug.dir/build
[  0%] Building CXX object CMakeFiles/template_debug.dir/src/classes/editor_plugin_registration.cpp.o
...

[Ivorforce]

I looked a bit further at my previous issues. It looks like this is a g++13 issue that is resolved in later versions. The reason my cmake config is encountering this while my scons config isn't is that scons uses builtin clang while my nix cmake uses g++13:

 ❯ /Users/lukas/.nix-profile/bin/c++ --version
g++ (GCC) 13.3.0
Copyright (C) 2023 Free Software Foundation, Inc.
This is free software; see the source for copying conditions.  There is NO
warranty; not even for MERCHANTABILITY or FITNESS FOR A PARTICULAR PURPOSE.


godot-cpp modernise* ≡
 ❯ clang++ --version
Apple clang version 16.0.0 (clang-1600.0.26.4)
Target: x86_64-apple-darwin23.5.0
Thread model: posix
InstalledDir: /Applications/Xcode.app/Contents/Developer/Toolchains/XcodeDefault.xctoolchain/usr/bin

So yeah, probably not related to your PR!

[enetheru]

scons uses builtin clang while my nix cmake uses g++13:

That's really interesting, I will try installing gcc on my mac mini and see if it has similar detection issues.

[Ivorforce]

You may not have the same experience. I used nix cmake specifically. Nix is very big on encapsulation, and if it used the system clang instead of shipping its own (g++ or clang++) i would have been very surprised.

[enetheru]

You may not have the same experience. I used nix cmake specifically. Nix is very big on encapsulation, and if it used the system clang instead of shipping its own (g++ or clang++) i would have been very surprised.

yeah installing gcc through brew doesnt at all clobber the builtin compiler detection. I woud have to create a toolchain, or alter environment variables to change that. I'm thinking thta nix is sort of like how msys on windows cretes its own environment.