[MacCatalyst] Failing to build MacCatalyst 'preadv' 'pwritev' only available on macCatalyst 14.0 or newer #54424

mdh1418 · 2021-06-18T17:50:36Z

./build.sh -s mono+libs+libs.pretest -os MacCatalyst -arch x64 -c Debug

/Users/mdhwang/runtime/src/libraries/Native/Unix/System.Native/pal_io.c(1491,21): error GD510DF4C: 'preadv' is only available on macCatalyst 14.0 or newer [-Werror,-Wunguarded-availability-new] [/Users/mdhwang/runtime/src/libraries/Native/build-native.proj]
/Users/mdhwang/runtime/src/libraries/Native/Unix/System.Native/pal_io.c(1491,21): error GD510DF4C: 'preadv' is only available on macCatalyst 14.0 or newer [-Werror,-Wunguarded-availability-new] [/Users/mdhwang/runtime/src/libraries/Native/build-native.proj]
/Users/mdhwang/runtime/src/libraries/Native/Unix/System.Native/pal_io.c(1531,21): error GDC37A26C: 'pwritev' is only available on macCatalyst 14.0 or newer [-Werror,-Wunguarded-availability-new] [/Users/mdhwang/runtime/src/libraries/Native/build-native.proj]
EXEC : error : 'pwritev' is only available on macCatalyst 14.0 or newer [-Werror,-Wunguarded-availability-new] [/Users/mdhwang/runtime/src/libraries/Native/build-native.proj]
/Users/mdhwang/runtime/src/libraries/Native/build-native.proj(38,5): error MSB3073: The command ""/Users/mdhwang/runtime/src/libraries/Native/build-native.sh" x64 Debug outconfig net6.0-MacCatalyst-Debug-x64 -os MacCatalyst " exited with code 2.

The text was updated successfully, but these errors were encountered:

ghost · 2021-06-18T17:50:40Z

Tagging subscribers to this area: @Anipik, @safern, @ViktorHofer
See info in area-owners.md if you want to be subscribed.

Issue Details

./build.sh -s mono+libs+libs.pretest -os MacCatalyst -arch x64 -c Debug

/Users/mdhwang/runtime/src/libraries/Native/Unix/System.Native/pal_io.c:1491/Users/mdhwang/runtime/src/libraries/Native/Unix/System.Native/pal_io.c(1491,21): error G9722640F: 'preadv' is only available on macCatalyst 14.0 or newer [-Werror,-Wunguarded-availability-new] [/Users/mdhwang/runtime/src/libraries/Native/build-native.proj]
:21 : error : 'preadv' is only available on macCatalyst 14.0 or newer [-Werror,-Wunguarded-availability-new] [/Users/mdhwang/runtime/src/libraries/Native/build-native.proj]
/Users/mdhwang/runtime/src/libraries/Native/Unix/System.Native/pal_io.c:1531:21: /Users/mdhwang/runtime/src/libraries/Native/Unix/System.Native/pal_io.c(1531,21): error G29E3121D: 'pwritev' is only available on macCatalyst 14.0 or newer [-Werror,-Wunguarded-availability-new] [/Users/mdhwang/runtime/src/libraries/Native/build-native.proj]
EXEC : error : 'pwritev' is only available on macCatalyst 14.0 or newer [-Werror,-Wunguarded-availability-new] [/Users/mdhwang/runtime/src/libraries/Native/build-native.proj]
/Users/mdhwang/runtime/src/libraries/Native/build-native.proj(38,5): error MSB3073: The command ""/Users/mdhwang/runtime/src/libraries/Native/build-native.sh" x64 Debug outconfig net6.0-MacCatalyst-Debug-x64 -os MacCatalyst " exited with code 2.

Author:	mdh1418
Assignees:	-
Labels:	`area-Infrastructure-libraries`, `os-maccatalyst`, `untriaged`
Milestone:	-

filipnavara · 2021-06-18T18:16:32Z

cc @adamsitnik This needs the same guard as macOS.

ghost · 2021-06-18T20:33:15Z

Tagging subscribers to this area: @dotnet/area-system-io
See info in area-owners.md if you want to be subscribed.

Issue Details

./build.sh -s mono+libs+libs.pretest -os MacCatalyst -arch x64 -c Debug

/Users/mdhwang/runtime/src/libraries/Native/Unix/System.Native/pal_io.c(1491,21): error GD510DF4C: 'preadv' is only available on macCatalyst 14.0 or newer [-Werror,-Wunguarded-availability-new] [/Users/mdhwang/runtime/src/libraries/Native/build-native.proj]
/Users/mdhwang/runtime/src/libraries/Native/Unix/System.Native/pal_io.c(1491,21): error GD510DF4C: 'preadv' is only available on macCatalyst 14.0 or newer [-Werror,-Wunguarded-availability-new] [/Users/mdhwang/runtime/src/libraries/Native/build-native.proj]
/Users/mdhwang/runtime/src/libraries/Native/Unix/System.Native/pal_io.c(1531,21): error GDC37A26C: 'pwritev' is only available on macCatalyst 14.0 or newer [-Werror,-Wunguarded-availability-new] [/Users/mdhwang/runtime/src/libraries/Native/build-native.proj]
EXEC : error : 'pwritev' is only available on macCatalyst 14.0 or newer [-Werror,-Wunguarded-availability-new] [/Users/mdhwang/runtime/src/libraries/Native/build-native.proj]
/Users/mdhwang/runtime/src/libraries/Native/build-native.proj(38,5): error MSB3073: The command ""/Users/mdhwang/runtime/src/libraries/Native/build-native.sh" x64 Debug outconfig net6.0-MacCatalyst-Debug-x64 -os MacCatalyst " exited with code 2.

Author:	mdh1418
Assignees:	-
Labels:	`area-System.IO`, `os-maccatalyst`, `untriaged`
Milestone:	-

adamsitnik · 2021-06-21T08:09:14Z

How is it possible that we have a CMAKE check for that:

runtime/src/libraries/Native/Unix/configure.cmake

Lines 222 to 225 in f721cf4

    
           check_symbol_exists( 
        
               preadv 
        
               sys/uio.h 
        
               HAVE_PREADV)

and the call to preadv is guarded with #if HAVE_PREADV:

runtime/src/libraries/Native/Unix/System.Native/pal_io.c

Lines 1490 to 1492 in f891033

    
           #if HAVE_PREADV && !defined(TARGET_WASM) // preadv is buggy on WASM 
        
               while ((count = preadv(fileDescriptor, (struct iovec*)vectors, (int)vectorCount, (off_t)fileOffset)) < 0 && errno == EINTR); 
        
           #else

and we are still getting this error?

cc @janvorli

filipnavara · 2021-06-21T08:20:29Z

On macOS/iOS/tvOS/MacCatalyst you build on a quite recent SDK. This SDK does have preadv but it's decorated with an attribute that says on what OS version it started to be available:

ssize_t preadv(int, const struct iovec *, int, off_t) __DARWIN_NOCANCEL(preadv) __API_AVAILABLE(macos(11.0), ios(14.0), watchos(7.0), tvos(14.0))

If you compile with default flags it will not give you an error since it assumes you target the same OS version as the SDK. However, if you compile for downlevel OS (eg. -target x86_64-apple-ios13.5-macabi) you will get an error for APIs that don't have a runtime check (if (__builtin_available(macOS 11, ...)).

I have a suspicion that the target version flags are not passed for the configure checks. It's not exactly clear whether that's something you want to do but in this case it is.

janvorli · 2021-06-21T09:00:14Z

@adamsitnik I cannot build due to this problem even on my local Apple M1 device targetting macos arm64 without Catalyst option since your change that added those (I've mentioned that on your PR, but I guess you have missed it). We build with -mmacosx-version-min=11.0, but the API declaration is guarded in my SDK in the headers by the following check:

#if (!defined(_POSIX_C_SOURCE) && !defined(_XOPEN_SOURCE)) || defined(_DARWIN_C_SOURCE)

Looking at the compiler options for the pal_io.c, we set the _XOPEN_SOURCE, so that blocks the preadv / pwritev

Adding the following to the src/libraries/Native/Unix/System.Native/CMakeLists.txt fixes the problem:

add_definitions(-D_DARWIN_C_SOURCE)

janvorli · 2021-06-21T09:35:17Z

Ah, this seems to be a different issue. This is x64, we don't use -mmacosx-version-min=11.0 for x64, for that one we use -mmacosx-version-min=10.13. So either we need to move the minimum supported version to 11.0 for x64 too or, as @filipnavara mentioned, ensure that this option is used for the configure phase too (I think that he's right that it is likely the issue)

filipnavara · 2021-06-21T10:23:22Z

There are also some specific hard-coded HAVE_* checks for the mobile platforms so this could easily be added there until a better solution is implemented:

runtime/src/libraries/Native/Unix/configure.cmake

Lines 512 to 535 in 64f5f0b

    
           if(CLR_CMAKE_TARGET_IOS) 
        
               # Manually set results from check_c_source_runs() since it's not possible to actually run it during CMake configure checking 
        
               unset(HAVE_SHM_OPEN_THAT_WORKS_WELL_ENOUGH_WITH_MMAP) 
        
               unset(HAVE_CLOCK_MONOTONIC) # only exists on iOS 10+ 
        
               unset(HAVE_CLOCK_REALTIME)  # only exists on iOS 10+ 
        
               unset(HAVE_FORK) # exists but blocked by kernel 
        
           elseif(CLR_CMAKE_TARGET_MACCATALYST) 
        
               # Manually set results from check_c_source_runs() since it's not possible to actually run it during CMake configure checking 
        
               # TODO: test to see if these all actually hold true on Mac Catalyst 
        
               unset(HAVE_SHM_OPEN_THAT_WORKS_WELL_ENOUGH_WITH_MMAP) 
        
               unset(HAVE_CLOCK_MONOTONIC) # only exists on iOS 10+ 
        
               unset(HAVE_CLOCK_REALTIME)  # only exists on iOS 10+ 
        
               unset(HAVE_FORK) # exists but blocked by kernel 
        
           elseif(CLR_CMAKE_TARGET_TVOS) 
        
               # Manually set results from check_c_source_runs() since it's not possible to actually run it during CMake configure checking 
        
               unset(HAVE_SHM_OPEN_THAT_WORKS_WELL_ENOUGH_WITH_MMAP) 
        
               unset(HAVE_CLOCK_MONOTONIC) # only exists on iOS 10+ 
        
               unset(HAVE_CLOCK_REALTIME)  # only exists on iOS 10+ 
        
               unset(HAVE_FORK) # exists but blocked by kernel 
        
           elseif(CLR_CMAKE_TARGET_ANDROID) 
        
               # Manually set results from check_c_source_runs() since it's not possible to actually run it during CMake configure checking 
        
               unset(HAVE_SHM_OPEN_THAT_WORKS_WELL_ENOUGH_WITH_MMAP) 
        
               set(HAVE_CLOCK_MONOTONIC 1) 
        
               set(HAVE_CLOCK_REALTIME 1)

janvorli · 2021-06-21T10:35:29Z

I have found the culprit. The option -mmacosx-version-min=10.13 is added using add_compile_options in eng/native/configurecompiler.cmake. That is included before the runtime/src/libraries/Native/Unix/configure.cmake. However, the check_symbol_exists (and other similar tests) only get compiler flags from CMAKE_C_FLAGS and CMAKE_CXX_FLAGS. cmake doesn't propagate the options added via add_compile_options to these variables, so the checking functions don't see the minimum version set.

There are two possible fixes here. One is to modify eng/native/configurecompiler.cmake to put the MACOS_VERSION_MIN_FLAGS into CMAKE_C_FLAGS and CMAKE_CXX_FLAGS.

diff --git a/eng/native/configurecompiler.cmake b/eng/native/configurecompiler.cmake
index 99a8013d7cf..d47da771dd3 100644
--- a/eng/native/configurecompiler.cmake
+++ b/eng/native/configurecompiler.cmake
@@ -415,7 +415,8 @@ if (CLR_CMAKE_HOST_UNIX)
       else()
         clr_unknown_arch()
       endif()
-      add_compile_options(${MACOS_VERSION_MIN_FLAGS})
+      set(CMAKE_C_FLAGS "${CMAKE_C_FLAGS} ${MACOS_VERSION_MIN_FLAGS}")
+      set(CMAKE_CXX_FLAGS "${CMAKE_CXX_FLAGS} ${MACOS_VERSION_MIN_FLAGS}")
       add_linker_flag(${MACOS_VERSION_MIN_FLAGS})
     endif(CLR_CMAKE_TARGET_MACCATALYST)
   endif(CLR_CMAKE_HOST_OSX OR CLR_CMAKE_HOST_MACCATALYST)

The other is this:

diff --git a/src/libraries/Native/Unix/configure.cmake b/src/libraries/Native/Unix/configure.cmake
index 9d40db9dbf2..a630777a0b8 100644
--- a/src/libraries/Native/Unix/configure.cmake
+++ b/src/libraries/Native/Unix/configure.cmake
@@ -21,6 +21,9 @@ elseif (CLR_CMAKE_TARGET_LINUX)
set(PAL_UNIX_NAME \"LINUX\")
elseif (CLR_CMAKE_TARGET_OSX)
set(PAL_UNIX_NAME \"OSX\")
+ if (CLR_CMAKE_HOST_ARCH_ARM64)
+     set(CMAKE_REQUIRED_FLAGS -mmacosx-version-min=11)
+ else()
+     set(CMAKE_REQUIRED_FLAGS -mmacosx-version-min=10.13)
+ endif ()

The former is preferable as it doesn't duplicate the option choice.

adamsitnik · 2021-06-21T12:42:24Z

@janvorli awesome, big thanks for doing that! Would you like me to send a PR or do you want to do it yourself?

janvorli · 2021-06-21T12:58:14Z

Let me send it out, I have made the change locally.

janvorli · 2021-06-21T15:46:25Z

My fix actually fixes just the issues I had. The catalyst problem remains. I am trying to figure out how to fix that one.

filipnavara · 2021-06-21T15:50:09Z

@janvorli The Catalyst build is a mess because CMake doesn't support it natively, at least not properly. Thus the -target x86_64-apple-ios13.5-macabi is likely injected at slightly different place than the equivalent macOS code.

runtime/eng/native/configurecompiler.cmake

Lines 398 to 404 in f70723d

    
           if(CLR_CMAKE_HOST_ARCH_ARM64) 
        
             add_compile_options(-target arm64-apple-ios14.2-macabi) 
        
             add_link_options(-target arm64-apple-ios14.2-macabi) 
        
           elseif(CLR_CMAKE_HOST_ARCH_AMD64) 
        
             add_compile_options(-target x86_64-apple-ios13.5-macabi) 
        
             add_link_options(-target x86_64-apple-ios13.5-macabi) 
        
           else()

runtime/src/libraries/Native/Unix/CMakeLists.txt

Lines 82 to 94 in 64f5f0b

    
           if(CLR_CMAKE_TARGET_MACCATALYST) 
        
               # -target overrides -mmacosx-version-min so suppress warning about that 
        
               # https://gitlab.kitware.com/cmake/cmake/-/issues/20132 
        
               add_compile_options(-Wno-overriding-t-option) 
        
               add_link_options(-Wno-overriding-t-option) 
        
               if (CLR_CMAKE_TARGET_ARCH_AMD64) 
        
                   add_compile_options(-target x86_64-apple-ios13.5-macabi) 
        
                   add_link_options(-target x86_64-apple-ios13.5-macabi) 
        
               elseif (CLR_CMAKE_TARGET_ARCH_ARM64) 
        
                   add_compile_options(-target arm64-apple-ios14.2-macabi) 
        
                   add_link_options(-target arm64-apple-ios14.2-macabi) 
        
               endif() 
        
           endif()

janvorli · 2021-06-21T16:10:46Z

@filipnavara thank you, this was helpful! I have believed that we've already fix our build so that the configurecompiler.cmake applies to everything in coreclr, libraries and installer, but it is clearly not the case.

mdh1418 added the os-maccatalyst MacCatalyst OS label Jun 18, 2021

dotnet-issue-labeler bot added area-Infrastructure-libraries untriaged New issue has not been triaged by the area owner labels Jun 18, 2021

safern added area-System.IO and removed area-Infrastructure-libraries labels Jun 18, 2021

adamsitnik added this to the 6.0.0 milestone Jun 21, 2021

adamsitnik removed the untriaged New issue has not been triaged by the area owner label Jun 21, 2021

janvorli mentioned this issue Jun 21, 2021

Fix MacOS build on 11.x SDK and Catalyst build #54506

Merged

ghost added the in-pr There is an active PR which will close this issue when it is merged label Jun 21, 2021

janvorli closed this as completed in #54506 Jun 27, 2021

ghost removed the in-pr There is an active PR which will close this issue when it is merged label Jun 27, 2021

ghost locked as resolved and limited conversation to collaborators Jul 27, 2021

Provide feedback

Saved searches

Use saved searches to filter your results more quickly

[MacCatalyst] Failing to build MacCatalyst 'preadv' 'pwritev' only available on macCatalyst 14.0 or newer #54424

[MacCatalyst] Failing to build MacCatalyst 'preadv' 'pwritev' only available on macCatalyst 14.0 or newer #54424

mdh1418 commented Jun 18, 2021 •

edited

Loading

ghost commented Jun 18, 2021

filipnavara commented Jun 18, 2021

ghost commented Jun 18, 2021

adamsitnik commented Jun 21, 2021

filipnavara commented Jun 21, 2021

janvorli commented Jun 21, 2021

janvorli commented Jun 21, 2021

filipnavara commented Jun 21, 2021

janvorli commented Jun 21, 2021

adamsitnik commented Jun 21, 2021

janvorli commented Jun 21, 2021

janvorli commented Jun 21, 2021

filipnavara commented Jun 21, 2021

janvorli commented Jun 21, 2021

[MacCatalyst] Failing to build MacCatalyst 'preadv' 'pwritev' only available on macCatalyst 14.0 or newer #54424

[MacCatalyst] Failing to build MacCatalyst 'preadv' 'pwritev' only available on macCatalyst 14.0 or newer #54424

Comments

mdh1418 commented Jun 18, 2021 • edited Loading

ghost commented Jun 18, 2021

filipnavara commented Jun 18, 2021

ghost commented Jun 18, 2021

adamsitnik commented Jun 21, 2021

filipnavara commented Jun 21, 2021

janvorli commented Jun 21, 2021

janvorli commented Jun 21, 2021

filipnavara commented Jun 21, 2021

janvorli commented Jun 21, 2021

adamsitnik commented Jun 21, 2021

janvorli commented Jun 21, 2021

janvorli commented Jun 21, 2021

filipnavara commented Jun 21, 2021

janvorli commented Jun 21, 2021

mdh1418 commented Jun 18, 2021 •

edited

Loading