Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

[vcpkg scripts] Crosscompiling fixes #26617

Merged
merged 55 commits into from
Oct 13, 2022
Merged
Show file tree
Hide file tree
Changes from 44 commits
Commits
Show all changes
55 commits
Select commit Hold shift + click to select a range
cff0492
test osx stuff
Neumann-A Aug 31, 2022
3f0bf20
deactivate most of ci
Neumann-A Sep 1, 2022
ce3fd60
get trace output
Neumann-A Sep 1, 2022
43553d9
add required flags to the linker flags
Neumann-A Sep 1, 2022
b8f6974
also look for --sysroot
Neumann-A Sep 1, 2022
c411a33
Link_args are always shared
Neumann-A Sep 2, 2022
5578e67
fix ar flags
Neumann-A Sep 2, 2022
ac5c8f9
setup FOR_BUILD vars.
Neumann-A Sep 2, 2022
98e8b82
try tricking the build.... but man configure scripts are stupid somet…
Neumann-A Sep 3, 2022
4b5f803
try 2
Neumann-A Sep 3, 2022
cf7adeb
major meson rewrite
Neumann-A Sep 3, 2022
d905202
fix duplicates
Neumann-A Sep 4, 2022
8c6339f
revert --trace-expand
Neumann-A Sep 4, 2022
6757d04
update port version
Neumann-A Sep 4, 2022
0bc3956
Merge remote-tracking branch 'upstream/master' into fix_cpp_osx_and_o…
Neumann-A Sep 4, 2022
332329f
revert pipeline changes
Neumann-A Sep 4, 2022
8b311a4
v db
Neumann-A Sep 4, 2022
886beb6
bump version
Neumann-A Sep 4, 2022
8ee8423
v db
Neumann-A Sep 4, 2022
002f6ba
set flags for rc compilers
Neumann-A Sep 5, 2022
a5b9bc8
a few more ios fixes.
Neumann-A Sep 6, 2022
133b0a0
v db
Neumann-A Sep 6, 2022
176edfb
fix osx/ios logic
Neumann-A Sep 8, 2022
22564ed
documentation and don't fatal_error in build/host machine entry calcu…
Neumann-A Sep 9, 2022
b4a45aa
Merge branch 'fix_cpp_osx_and_others' of https://github.com/Neumann-A…
Neumann-A Sep 9, 2022
1570c35
add android/compiler target stuff from #26570
Neumann-A Sep 14, 2022
1673027
remove extra endif
Neumann-A Sep 14, 2022
d347f83
Merge branch 'master' of https://github.com/microsoft/vcpkg into fix_…
Sep 16, 2022
357b504
Merge remote-tracking branch 'upstream/master' into fix_cpp_osx_and_o…
Neumann-A Sep 17, 2022
5b30569
v db
Neumann-A Sep 17, 2022
7e4ffa9
add target param to linker flags.
Neumann-A Sep 18, 2022
4acc2d0
target in cpp flags
Neumann-A Sep 18, 2022
005ccbe
Make vcpkg-cmake-get-vars more ios compatible
m-kuhn Sep 18, 2022
ad5ae3b
Merge pull request #10 from opengisch/opengisch/misc-osx-ios-android
Neumann-A Sep 18, 2022
6aca335
Update scripts/get_cmake_vars/CMakeLists.txt
Neumann-A Sep 18, 2022
ceb7bfe
More ios fixes
m-kuhn Sep 18, 2022
af8f8ea
Merge pull request #11 from m-kuhn/moreios
Neumann-A Sep 18, 2022
6f18956
fix target adding
Neumann-A Sep 18, 2022
4b8384e
Merge branch 'fix_cpp_osx_and_others' of https://github.com/Neumann-A…
Neumann-A Sep 18, 2022
21553e5
Fix broken loop
m-kuhn Sep 18, 2022
c144ba1
Merge pull request #12 from m-kuhn/fix_broken_loop
Neumann-A Sep 18, 2022
d902437
v db
Neumann-A Sep 18, 2022
342d37f
Merge branch 'fix_cpp_osx_and_others' of https://github.com/Neumann-A…
Neumann-A Sep 18, 2022
0475e5d
v db
Neumann-A Sep 18, 2022
a333006
Merge remote-tracking branch 'upstream/master' into fix_cpp_osx_and_o…
Neumann-A Sep 26, 2022
051c678
bump date
Neumann-A Sep 26, 2022
e422921
v db
Neumann-A Sep 26, 2022
96b5807
v db
Neumann-A Sep 26, 2022
184abe3
Merge branch 'microsoft:master' into fix_cpp_osx_and_others
Neumann-A Sep 28, 2022
f27f316
Merge branch 'microsoft:master' into fix_cpp_osx_and_others
Neumann-A Oct 1, 2022
999c02e
Merge branch 'microsoft:master' into fix_cpp_osx_and_others
Neumann-A Oct 4, 2022
0bf1edb
Merge remote-tracking branch 'upstream/master' into fix_cpp_osx_and_o…
Neumann-A Oct 9, 2022
7292d87
v db
Neumann-A Oct 9, 2022
6ae069e
remove port-version 1
Neumann-A Oct 9, 2022
29d5b6d
v db
Neumann-A Oct 9, 2022
File filter

Filter by extension

Filter by extension

Conversations
Failed to load comments.
Loading
Jump to
Jump to file
Failed to load files.
Loading
Diff view
Diff view
5 changes: 5 additions & 0 deletions docs/users/triplets.md
Original file line number Diff line number Diff line change
Expand Up @@ -100,6 +100,11 @@ This field is optional.

Also available as build-type specific `VCPKG_MESON_CONFIGURE_OPTIONS_DEBUG` and `VCPKG_MESON_CONFIGURE_OPTIONS_RELEASE` variables.

### VCPKG_MESON_(CROSS|NATIVE)_FILE(_RELEASE|_DEBUG)
Provide an additional (configuration dependent) file as a meson cross/native file. Can be used to override settings provided by vcpkg since it will be passed after vcpkg's generated cross/native files are passed.

Especially usefull to provide your own build_machine and host_machine entries.

### VCPKG_CMAKE_CONFIGURE_OPTIONS
Set additional CMake configure options that are appended to the configure command (in [`vcpkg_cmake_configure`](../maintainers/vcpkg_cmake_configure.md)).

Expand Down
33 changes: 18 additions & 15 deletions ports/vcpkg-cmake-get-vars/cmake_get_vars/CMakeLists.txt
Original file line number Diff line number Diff line change
Expand Up @@ -12,7 +12,7 @@ list(APPEND VCPKG_DEFAULT_VARS_TO_CHECK CMAKE_CROSSCOMPILING
CMAKE_SIZEOF_VOID_P
MSVC
MSVC_VERSION)
if(CMAKE_SYSTEM_NAME MATCHES "Darwin")
if(APPLE)
list(APPEND VCPKG_DEFAULT_VARS_TO_CHECK CMAKE_OSX_DEPLOYMENT_TARGET
CMAKE_OSX_SYSROOT)
endif()
Expand All @@ -33,7 +33,8 @@ foreach(_lang IN LISTS VCPKG_LANGUAGES)
list(APPEND VCPKG_DEFAULT_VARS_TO_CHECK CMAKE_${_lang}_STANDARD)
list(APPEND VCPKG_DEFAULT_VARS_TO_CHECK CMAKE_${_lang}_COMPILE_FEATURES)
list(APPEND VCPKG_DEFAULT_VARS_TO_CHECK CMAKE_${_lang}_EXTENSION)
# Probably never required since implicit.
list(APPEND VCPKG_DEFAULT_VARS_TO_CHECK CMAKE_${_lang}_COMPILER_TARGET)
# Probably never required since implicit.
#list(APPEND VCPKG_DEFAULT_VARS_TO_CHECK CMAKE_${_lang}_IMPLICIT_LINK_FRAMEWORK_DIRECTORIES)
#list(APPEND VCPKG_DEFAULT_VARS_TO_CHECK CMAKE_${_lang}_IMPLICIT_INCLUDE_DIRECTORIES)
#list(APPEND VCPKG_DEFAULT_VARS_TO_CHECK CMAKE_${_lang}_IMPLICIT_LINK_DIRECTORIES)
Expand Down Expand Up @@ -101,18 +102,18 @@ if(APPLE)
string(APPEND EXTRA_FLAGS " -arch ${arch}")
endforeach()
if(CMAKE_OSX_DEPLOYMENT_TARGET)
if(CMAKE_SYSTEM_NAME STREQUAL "iOS")
string(APPEND EXTRA_FLAGS " -mios-version-min=${CMAKE_OSX_DEPLOYMENT_TARGET}")
elseif(CMAKE_SYSTEM_NAME STREQUAL "Darwin")
string(APPEND EXTRA_FLAGS " -mmacosx-version-min=${CMAKE_OSX_DEPLOYMENT_TARGET}")
endif()
list(GET VCPKG_LANGUAGES 0 lang)
string(APPEND EXTRA_FLAGS " ${CMAKE_${lang}_OSX_DEPLOYMENT_TARGET_FLAG}${CMAKE_OSX_DEPLOYMENT_TARGET}")
unset(lang)
endif()
endif()
if(CMAKE_CXX_COMPILER_TARGET)
if(CMAKE_CXX_COMPILER_ID STREQUAL "Clang")
string(APPEND EXTRA_FLAGS " \"${CMAKE_CXX_COMPILE_OPTIONS_TARGET}${CMAKE_CXX_COMPILER_TARGET}\"")
elseif(CMAKE_CXX_COMPILE_OPTIONS_TARGET)
string(APPEND EXTRA_FLAGS " ${CMAKE_CXX_COMPILE_OPTIONS_TARGET} \"${CMAKE_CXX_COMPILER_TARGET}\"")
set(target_param "")
list(GET VCPKG_LANGUAGES 0 lang)
if(CMAKE_${lang}_COMPILER_TARGET)
if (CMAKE_${lang}_COMPILER_ID STREQUAL Clang)
set(target_param "${CMAKE_${lang}_COMPILE_OPTIONS_TARGET}${CMAKE_${lang}_COMPILER_TARGET}")
elseif(CMAKE_${lang}_COMPILE_OPTIONS_TARGET)
set(target_param "${CMAKE_${lang}_COMPILE_OPTIONS_TARGET} ${CMAKE_${lang}_COMPILER_TARGET}")
endif()
endif()
Neumann-A marked this conversation as resolved.
Show resolved Hide resolved
if(CMAKE_CXX_COMPILER_EXTERNAL_TOOLCHAIN)
Expand All @@ -126,12 +127,14 @@ if(CMAKE_SYSROOT AND CMAKE_CXX_COMPILE_OPTIONS_SYSROOT)
string(APPEND EXTRA_FLAGS " \"${CMAKE_CXX_COMPILE_OPTIONS_SYSROOT}${CMAKE_SYSROOT}\"")
endif()

foreach(flags CXX C SHARED_LINKER EXE_LINKER)
set(${flags}_FLAGS "${CMAKE_${flags}_FLAGS} ${CMAKE_${flags}_FLAGS_${VCPKG_CONFIGS}} ${EXTRA_FLAGS}")
foreach(flags ${VCPKG_LANGUAGES} SHARED_LINKER EXE_LINKER STATIC_LINKER MODULE_LINKER)
set(${flags}_FLAGS "${target_param} ${CMAKE_${flags}_FLAGS} ${CMAKE_${flags}_FLAGS_${VCPKG_CONFIGS}} ${EXTRA_FLAGS}")
string(STRIP ${flags}_FLAGS "${${flags}_FLAGS}")
endforeach()

foreach(flags STATIC_LINKER RC)
set(${flags}_FLAGS "${CMAKE_${flags}_FLAGS} ${CMAKE_${flags}_FLAGS_${VCPKG_CONFIGS}}")
string(STRIP ${flags}_FLAGS "${${flags}_FLAGS}")
endforeach()

foreach(incdir IN LISTS CMAKE_CXX_STANDARD_INCLUDE_DIRECTORIES)
Expand All @@ -142,7 +145,7 @@ foreach(incdir IN LISTS CMAKE_C_STANDARD_INCLUDE_DIRECTORIES)
string(APPEND C_FLAGS " \"${CMAKE_INCLUDE_FLAG_C}${incdir}\"")
endforeach()

foreach(flag CXX C SHARED_LINKER EXE_LINKER STATIC_LINKER RC)
foreach(flag ${VCPKG_LANGUAGES} SHARED_LINKER EXE_LINKER STATIC_LINKER MODULE_LINKER RC)
# When using MSVC, maybe transform /flags to -flags.
# When cross compiling, "/flags" may be an absolute path starting with /, so don't transform.
# Otherwise, transform to improve compatibility with scripts interpreting "/flags" as a path.
Expand Down
2 changes: 1 addition & 1 deletion ports/vcpkg-cmake-get-vars/vcpkg.json
Original file line number Diff line number Diff line change
@@ -1,6 +1,6 @@
{
"name": "vcpkg-cmake-get-vars",
"version-date": "2022-05-10",
"version-date": "2022-09-04",
"port-version": 1,
"documentation": "https://vcpkg.io/en/docs/README.html",
"license": "MIT",
Expand Down
2 changes: 1 addition & 1 deletion ports/vcpkg-cmake/vcpkg.json
Original file line number Diff line number Diff line change
@@ -1,6 +1,6 @@
{
"name": "vcpkg-cmake",
"version-date": "2022-07-18",
"version-date": "2022-09-04",
"documentation": "https://vcpkg.io/en/docs/maintainers/ports/vcpkg-cmake.html",
"license": "MIT"
}
42 changes: 42 additions & 0 deletions scripts/buildsystems/meson/meson.template.in
Original file line number Diff line number Diff line change
@@ -0,0 +1,42 @@
[binaries]
cmake = ['@CMAKE_COMMAND@']
python = ['@PYTHON3@']
ninja = ['@NINJA@']
pkgconfig= ['@PKGCONFIG@']
@MESON_MT@
@MESON_AR@
@MESON_RC@
@MESON_C@
@MESON_C_LD@
@MESON_CXX@
@MESON_CXX_LD@
@MESON_OBJC@
@MESON_OBJC_LD@
@MESON_OBJCPP@
@MESON_OBJCPP_LD@
@MESON_FC@
@MESON_FC_LD@
@MESON_WINDRES@
@MESON_ADDITIONAL_BINARIES@
[properties]
cmake_toolchain_file = '@SCRIPTS@/buildsystems/vcpkg.cmake'
[cmake]
VCPKG_TARGET_TRIPLET = '@TARGET_TRIPLET@'
VCPKG_HOST_TRIPLET = '@_HOST_TRIPLET@'
VCPKG_CHAINLOAD_TOOLCHAIN_FILE = '@VCPKG_CHAINLOAD_TOOLCHAIN_FILE@'
VCPKG_CRT_LINKAGE = '@VCPKG_CRT_LINKAGE@'
_VCPKG_INSTALLED_DIR = '@_VCPKG_INSTALLED_DIR@'
@MESON_HOST_MACHINE@
@MESON_BUILD_MACHINE@
[built-in options]
default_library = '@MESON_DEFAULT_LIBRARY@'
werror = false
@MESON_CFLAGS@
@MESON_CXXFLAGS@
@MESON_FCFLAGS@
@MESON_OBJCFLAGS@
@MESON_OBJCPPFLAGS@
# b_vscrt
@MESON_VSCRT_LINKAGE@
# c_winlibs/cpp_winlibs
@MESON_WINLIBS@
21 changes: 16 additions & 5 deletions scripts/cmake/vcpkg_configure_make.cmake
Original file line number Diff line number Diff line change
Expand Up @@ -74,8 +74,8 @@ macro(z_vcpkg_determine_autotools_target_arch_mac out_var)
endmacro()

macro(z_vcpkg_extract_cpp_flags_and_set_cflags_and_cxxflags flag_suffix)
string(REGEX MATCHALL "( |^)-D[^ ]+" CPPFLAGS_${flag_suffix} "${VCPKG_DETECTED_CMAKE_C_FLAGS_${flag_suffix}}")
string(REGEX MATCHALL "( |^)-D[^ ]+" CXXPPFLAGS_${flag_suffix} "${VCPKG_DETECTED_CMAKE_CXX_FLAGS_${flag_suffix}}")
string(REGEX MATCHALL "( |^)(-D|-isysroot|--sysroot=|-isystem|-m?[Aa][Rr][Cc][Hh]|--target=|-target) ?[^ ]+" CPPFLAGS_${flag_suffix} "${VCPKG_DETECTED_CMAKE_C_FLAGS_${flag_suffix}}")
string(REGEX MATCHALL "( |^)(-D|-isysroot|--sysroot=|-isystem|-m?[Aa][Rr][Cc][Hh]|--target=|-target) ?[^ ]+" CXXPPFLAGS_${flag_suffix} "${VCPKG_DETECTED_CMAKE_CXX_FLAGS_${flag_suffix}}")
Comment on lines +77 to +78
Copy link
Contributor

@dg0yt dg0yt May 3, 2023

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

@Neumann-A Do you remember why you remove --target from CFLAGSCFPFLAGS here?
It is explicitly injected into the flags by scripts/get_cmake_vars/CMakeLists.txt.
Removing this flag breaks android toolchains. (CMake selects clang for compiler, so --target=x86_64-none-linux-android21 etc. is needed to pick the right includes.) LDFLAGS still carries the option.

Copy link
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

From building luajit (WIP):

ld: error: undefined symbol: _IO_putc
>>> referenced by luajit.c:150 (/home/dg0yt/Projekte/vcpkg/vcpkg/buildtrees/luajit/x64-android-dbg/src/luajit.c:150)
>>>               luajit.o:(print_jit_status)
>>> referenced by luajit.c:153 (/home/dg0yt/Projekte/vcpkg/vcpkg/buildtrees/luajit/x64-android-dbg/src/luajit.c:153)
>>>               luajit.o:(print_jit_status)
>>> referenced by stdio.h:81 (/usr/lib/gcc/x86_64-linux-gnu/7.5.0/../../../../x86_64-linux-gnu/include/bits/stdio.h:81)
>>>               lib_base.o:(lj_cf_print) in archive libluajit.a
>>> referenced 1 more times

some time after:

/<android-ndk-r25c>/toolchains/llvm/prebuilt/linux-x86_64/bin/clang  -O2 -fomit-frame-pointer -Wall   -D_FILE_OFFSET_BITS=64 -D_LARGEFILE_SOURCE -U_FORTIFY_SOURCE  -DLUA_ROOT=\"/home/dg0yt/Projekte/vcpkg/vcpkg/installed/x64-android/debug\" -DLUA_MULTILIB=\"lib\" -fno-stack-protector -DLUAJIT_UNWIND_EXTERNAL  -g -fdata-sections -ffunction-sections -funwind-tables -fstack-protector-strong -no-canonical-prefixes -Wformat -Werror=format-security -fPIC -fno-limit-debug-info -c -o luajit.o luajit.c

Copy link
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Ignore. I found the right track now.
TLDR: There is CFLAGS and there is CPPFLAGS, and I should use both...

Copy link
Contributor Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Some people (meson) would argue all flags have to go into CC / CXX instead.

list(JOIN CXXPPFLAGS_${flag_suffix} "|" CXXREGEX)
if(CXXREGEX)
list(FILTER CPPFLAGS_${flag_suffix} INCLUDE REGEX "(${CXXREGEX})")
Expand Down Expand Up @@ -384,7 +384,7 @@ function(vcpkg_configure_make)
# Currently needed for arm because objdump yields: "unrecognised machine type (0x1c4) in Import Library Format archive"
list(APPEND arg_OPTIONS lt_cv_deplibs_check_method=pass_all)
endif()
elseif(NOT VCPKG_TARGET_IS_OSX)
else()
# Because OSX dosn't like CMAKE_C(XX)_COMPILER (cc) in CC/CXX and rather wants to have gcc/g++
function(z_vcpkg_make_set_env envvar cmakevar)
set(prog "${VCPKG_DETECTED_CMAKE_${cmakevar}} ${ARGN}")
Expand Down Expand Up @@ -432,7 +432,7 @@ function(vcpkg_configure_make)
endif()

# macOS - cross-compiling support
if(VCPKG_TARGET_IS_OSX)
if(VCPKG_TARGET_IS_OSX OR VCPKG_TARGET_IS_IOS)
if (requires_autoconfig AND NOT arg_BUILD_TRIPLET OR arg_DETERMINE_BUILD_TRIPLET)
z_vcpkg_determine_autotools_host_arch_mac(BUILD_ARCH) # machine you are building on => --build=
z_vcpkg_determine_autotools_target_arch_mac(TARGET_ARCH)
Expand All @@ -441,7 +441,7 @@ function(vcpkg_configure_make)
# --target: the machine that CC will produce binaries for
# https://stackoverflow.com/questions/21990021/how-to-determine-host-value-for-configure-when-using-cross-compiler
# Only for ports using autotools so we can assume that they follow the common conventions for build/target/host
if(NOT "${TARGET_ARCH}" STREQUAL "${BUILD_ARCH}") # we don't need to specify the additional flags if we build natively.
if(NOT "${TARGET_ARCH}" STREQUAL "${BUILD_ARCH}" OR NOT VCPKG_TARGET_IS_OSX) # we don't need to specify the additional flags if we build natively.
set(arg_BUILD_TRIPLET "--host=${TARGET_ARCH}-apple-darwin") # (Host activates crosscompilation; The name given here is just the prefix of the host tools for the target)
endif()
debug_message("Using make triplet: ${arg_BUILD_TRIPLET}")
Expand Down Expand Up @@ -754,10 +754,14 @@ function(vcpkg_configure_make)

# Setup environment
set(ENV{CPPFLAGS} "${CPPFLAGS_${current_buildtype}}")
set(ENV{CPPFLAGS_FOR_BUILD} "${CPPFLAGS_${current_buildtype}}")
set(ENV{CFLAGS} "${CFLAGS_${current_buildtype}}")
set(ENV{CFLAGS_FOR_BUILD} "${CFLAGS_${current_buildtype}}")
set(ENV{CXXFLAGS} "${CXXFLAGS_${current_buildtype}}")
#set(ENV{CXXFLAGS_FOR_BUILD} "${CXXFLAGS_${current_buildtype}}") -> doesn't exist officially
set(ENV{RCFLAGS} "${VCPKG_DETECTED_CMAKE_RC_FLAGS_${current_buildtype}}")
set(ENV{LDFLAGS} "${LDFLAGS_${current_buildtype}}")
set(ENV{LDFLAGS_FOR_BUILD} "${LDFLAGS_${current_buildtype}}")
if(ARFLAGS_${current_buildtype} AND NOT (arg_USE_WRAPPERS AND VCPKG_TARGET_IS_WINDOWS))
# Target windows with wrappers enabled cannot forward ARFLAGS since it breaks the wrapper
set(ENV{ARFLAGS} "${ARFLAGS_${current_buildtype}}")
Expand All @@ -767,6 +771,13 @@ function(vcpkg_configure_make)
# and libtool tries to avoid versioning for shared libraries and no symbolic links are created.
if(VCPKG_TARGET_IS_ANDROID)
set(ENV{LDFLAGS} "-avoid-version $ENV{LDFLAGS}")
set(ENV{LDFLAGS_FOR_BUILD} "-avoid-version $ENV{LDFLAGS_FOR_BUILD}")
endif()

if(VCPKG_TARGET_IS_OSX OR VCPKG_TARGET_IS_IOS)
# configure not using all flags to check if compiler works ...
set(ENV{CC} "$ENV{CC} $ENV{CPPFLAGS} $ENV{CFLAGS}")
set(ENV{CC_FOR_BUILD} "$ENV{CC_FOR_BUILD} $ENV{CPPFLAGS} $ENV{CFLAGS}")
endif()

if(LINK_ENV_${current_buildtype})
Expand Down
Loading