Haiku: Add Haiku support for CoreCLR/PAL #93907

trungnt2910 · 2023-10-24T04:40:55Z

This contains the code required to build this repo's native components and get paltests to pass on Haiku.

Most implementation details related to this commit are documented here.

Part of #55803.

src/native/minipal/errno.h

janvorli · 2023-10-24T17:21:07Z

src/coreclr/minipal/Unix/doublemapping.cpp

-    sprintf(name, "/shm-dotnet-%d", getpid());
-    name[sizeof(name) - 1] = '\0';
-    shm_unlink(name);
-    int fd = shm_open(name, O_RDWR | O_CREAT | O_EXCL | O_NOFOLLOW, 0600);
 #else // TARGET_FREEBSD


I would prefer the #elif defined(TARGET_LINUX) here and final #else with the posix fallback instead of hierarchical ifdef.

Done. The final #else assigns the fd variable to -1 instead of doing the POSIX fallback itself, in case any of the platform-specific functions above return an error.

This should solve #72192 from the .NET 7 era.

src/coreclr/pal/src/include/pal/palinternal.h

src/native/corehost/hostmisc/pal.unix.cpp

src/coreclr/pal/inc/rt/safecrt.h

src/coreclr/utilcode/sstring.cpp

janvorli · 2023-10-24T22:38:19Z

src/native/libs/System.Native/pal_networkchange.c

-    // unreachable
-    abort();
-    return Error_SUCCESS;
+    return Error_EINVAL;


Should we keep the old aborting code as a fallback and add separate #if for HAIKU instead? I am not familiar enough with the contract of this function though.

I've added back the abort() logic, as an assertion that the function is unreachable.

This is true because on unsupported platforms, the higher-level C# libraries should already throw while trying to construct a network events socket.

Returning Error_SUCCESS does not make much sense though.

trungnt2910 · 2023-10-25T07:34:18Z

Strange, I'm not modifying any FreeBSD-related codepaths, what's wrong with the FreeBSD build?

jkotas · 2023-10-25T13:08:55Z

FreeBSD build error is:

/__w/1/s/src/coreclr/pal/src/misc/sysinfo.cpp:215:57: error: use of undeclared identifier 'NUPML4E'
    lpSystemInfo->lpMaximumApplicationAddress = (PVOID) VM_MAXUSER_ADDRESS;
                                                        ^
/crossrootfs/x64/usr/include/machine/vmparam.h:186:35: note: expanded from macro 'VM_MAXUSER_ADDRESS'
#define VM_MAXUSER_ADDRESS      UVADDR(NUPML4E, 0, 0, 0)
                                       ^

I guess you have introduced it by changing conditions around some of the includes.

trungnt2910 · 2023-10-25T13:41:22Z

This is suspicious:

-- Looking for include file sys/user.h
-- Looking for include file sys/user.h - not found

I wonder how I could get more detailed CMake logs for the CI runs.

trungnt2910 · 2023-10-25T13:55:53Z

When looking at other files, it seems like sys/user.h is only used on FreeBSD. Somehow sysinfo.cpp includes this header for all platforms.

This time, instead of checking for this header during configuration as in #86391, a FreeBSD #ifdef guard is added instead.

huoyaoyuan · 2023-10-30T06:13:58Z

src/coreclr/minipal/Unix/doublemapping.cpp

+    // POSIX fallback
+    if (fd == -1)
+    {
+        char name[24];
+        sprintf(name, "/shm-dotnet-%d", getpid());
+        name[sizeof(name) - 1] = '\0';
+        shm_unlink(name);
+        fd = shm_open(name, O_RDWR | O_CREAT | O_EXCL | O_NOFOLLOW, 0600);
+        shm_unlink(name);
+    }


Shouldn't this block be in #else?

No, it should not, as stated in #93907 (comment).

Even for OSes with platform-specific ways to get a special memory file open, should their method fail, we can still always use the POSIX fallback.

huoyaoyuan · 2023-10-30T06:15:16Z

src/coreclr/gc/unix/numasupport.cpp

+#ifdef TARGET_LINUX
+#include <sys/syscall.h>
+#endif


Is this used by other Unix systems?

No, it is only used on Linux:

runtime/src/coreclr/gc/unix/numasupport.cpp

Lines 52 to 63 in e5c631d

#ifdef TARGET_LINUX

if (syscall(__NR_get_mempolicy, NULL, NULL, 0, 0, 0) < 0 && errno == ENOSYS)

return;

int highestNumaNode = GetNodeNum("/sys/devices/system/node", false);

// we only use this implementation when there are two or more NUMA nodes available

if (highestNumaNode < 1)

return;

g_numaAvailable = true;

g_highestNumaNode = highestNumaNode;

#endif

and

runtime/src/coreclr/gc/unix/numasupport.cpp

Lines 81 to 83 in e5c631d

#ifdef TARGET_LINUX

return syscall(__NR_mbind, (long)start, len, 1, (long)nodemask, maxnode, 0);

#else

huoyaoyuan · 2023-10-30T06:17:55Z

src/coreclr/pal/inc/pal_mstypes.h

 #define _SIZE_T_DEFINED
+#endif // !PAL_STDCPP_COMPAT


What's the impact range of this?

What do you mean by "impact range"?

The line should affect all platforms. However, I believe that this is how things should be, since with PAL_STDCPP_COMPAT the header should not define things that potentially mess up system headers.

Also, CoreCLR builds are working, so it should not break the definition on any other platforms.

Some more comments about this change here: #55803 (comment)

mangod9 · 2024-03-04T17:35:24Z

Since this is a large change, would it be possible to split the PR info smaller component PRs for PAL, hosting, libraries etc. so relevant teams can review the changes? Thx

Anutrix · 2024-04-29T16:26:18Z

@trungnt2910 Are you still working on this?

trungnt2910 · 2024-04-29T23:42:45Z

Not right now (I'm too busy). I'm still tracking it though, so if there are small specific issues I can resolve, I can fix it.

jkotas · 2024-04-30T04:01:30Z

As @mangod9 suggested above, this needs to be split into multiple smaller PRs per area to make it reviewable. I am going to switch it to draft for now.

trungnt2910 · 2024-04-30T05:08:31Z

To me, this PR seems like a pretty atomic change (misc insertions needed to allow native components to start building on Haiku).

How should I split it? Maybe src/coreclr in one PR, src/mono in another, src/native in one more, then the rest in one final PR?

jkotas · 2024-04-30T13:41:23Z

Yes, that would work.

trungnt2910 · 2024-04-30T14:27:50Z

I have edited this pull request to include only files in src/coreclr/pal.
Other parts will come in other PRs.
Please let me know if this needs to be split further.

trungnt2910 · 2024-04-30T15:57:47Z

Latest Haiku builds are failing with this error:

  [  7%] Building CXX object inc/CMakeFiles/corguids.dir/__/pal/prebuilt/idl/cordebug_i.cpp.o
  In file included from /home/runner/work/dotnet-builds/dotnet-builds/runtime/src/coreclr/pal/inc/rt/palrt.h:630,
                   from /home/runner/work/dotnet-builds/dotnet-builds/runtime/src/coreclr/pal/inc/rt/rpc.h:15,
                   from /home/runner/work/dotnet-builds/dotnet-builds/runtime/src/coreclr/pal/prebuilt/idl/cordebug_i.cpp:29:
  /home/runner/work/dotnet-builds/dotnet-builds/runtime/src/coreclr/pal/inc/rt/safecrt.h:1093:16: error: conflicting declaration of C function 'size_t wcsnlen(const WCHAR*, size_t)'
   1093 | size_t __cdecl wcsnlen(const WCHAR *inString, size_t inMaxSize);
        |                ^~~~~~~
  In file included from /home/runner/work/dotnet-builds/dotnet-builds/dotnet-rootfs/boot/system/develop/headers/posix/wctype.h:10,
                   from /home/runner/work/dotnet-builds/dotnet-builds/runtime/src/coreclr/pal/inc/pal.h:51,
                   from /home/runner/work/dotnet-builds/dotnet-builds/runtime/src/coreclr/pal/inc/rt/palrt.h:136:
  /home/runner/work/dotnet-builds/dotnet-builds/dotnet-rootfs/boot/system/develop/headers/posix/wchar.h:122:17: note: previous declaration 'size_t wcsnlen(const wchar_t*, size_t)'
    122 | extern size_t   wcsnlen(const wchar_t *wcs, size_t maxLength);
        |                 ^~~~~~~
  /home/runner/work/dotnet-builds/dotnet-builds/runtime/src/coreclr/pal/inc/rt/safecrt.h:1098:16: error: conflicting declaration of C function 'size_t wcsnlen(const WCHAR*, size_t)'
   1098 | size_t __cdecl wcsnlen(const WCHAR *inString, size_t inMaxSize)
        |                ^~~~~~~
  /home/runner/work/dotnet-builds/dotnet-builds/dotnet-rootfs/boot/system/develop/headers/posix/wchar.h:122:17: note: previous declaration 'size_t wcsnlen(const wchar_t*, size_t)'
    122 | extern size_t   wcsnlen(const wchar_t *wcs, size_t maxLength);
        |                 ^~~~~~~
  make[2]: *** [inc/CMakeFiles/corguids.dir/build.make:76: inc/CMakeFiles/corguids.dir/__/pal/prebuilt/idl/cordebug_i.cpp.o] Error 1
  make[1]: *** [CMakeFiles/Makefile2:3250: inc/CMakeFiles/corguids.dir/all] Error 2
  make[1]: *** Waiting for unfinished jobs....

Seems like whoever included wctype.h did not expect wchar.h to be included as well.

According to the POSIX Standard, however:

[CX] [Option Start] Inclusion of the <wctype.h> header may make visible all symbols from the headers <ctype.h>, <stdarg.h>, <stddef.h>, <stdio.h>, <stdlib.h>, <string.h>, <time.h>, and <wchar.h>. [Option End]

What should I do in this case? Removing <wchar.h> from Haiku's system include just breaks everything. And according to POSIX Haiku isn't doing anything wrong in the first place...

jkotas · 2024-04-30T16:18:35Z

Does anything break if our local wcsnlen implementation is deleted?

This contains a part of the code required to build CoreCLR and get `paltests` to pass on Haiku. This commit covers `src/coreclr/pal/**`. Co-authored-by: Jessica Hamilton <[email protected]>

This conflicts with the standard one already included through `wctype.h` in some POSIX OSes like Haiku.

trungnt2910 · 2024-04-30T17:02:37Z

Hasn't broken my local Linux build and moved my local Haiku build to a different error. Let the CI inform the rest of the answer.

trungnt2910 · 2024-04-30T17:24:46Z

The next Haiku error is:

  asmparse.y:176: error: "_IMPORT" redefined [-Werror]
  In file included from /home/trung/dotnet-rootfs/boot/system/develop/headers/posix/sys/types.h:11,
                   from /home/trung/dotnet-rootfs/boot/system/develop/headers/posix/stdio.h:9,
                   from /home/trung/dotnet-rootfs/boot/system/develop/headers/bsd/stdio.h:9,
                   from /home/trung/dotnet-runtime/src/coreclr/pal/inc/pal.h:39,
                   from /home/trung/dotnet-runtime/src/coreclr/pal/inc/rt/palrt.h:136,
                   from /home/trung/dotnet-runtime/src/coreclr/pal/inc/rt/rpc.h:15,
                   from /home/trung/dotnet-runtime/src/coreclr/pal/inc/rt/objidl.h:12,
                   from /home/trung/dotnet-runtime/src/coreclr/pal/inc/rt/ole2.h:8,
                   from /home/trung/dotnet-runtime/src/coreclr/inc/cor.h:16,
                   from /home/trung/dotnet-runtime/src/coreclr/ilasm/./ilasmpch.h:8,
                   from asmparse.y:9:
  /home/trung/dotnet-rootfs/boot/system/develop/headers/os/BeBuild.h:95: note: this is the location of the previous definition
     95 | #define _IMPORT
        |
  asmparse.y:262: error: "_EXPORT" redefined [-Werror]
  /home/trung/dotnet-rootfs/boot/system/develop/headers/os/BeBuild.h:90: note: this is the location of the previous definition
     90 | # define _EXPORT __attribute__((visibility("default")))
        |

Currently I am solving this by adding these lines to ilasmpch.h:

#include "mdfileformat.h"
#include "stgpooli.h"

+#ifdef _EXPORT
+#undef _EXPORT
+#endif

+#ifdef _IMPORT
+#undef _IMPORT
+#endif

#endif

I wonder whether these lines should be put directly to pal.h instead.

mangod9 · 2024-07-29T16:19:05Z

Hey @trungnt2910, what is the status of this PR. Are you continuing to work on it?

trungnt2910 · 2024-07-29T20:50:31Z

There's still an unanswered question from me above.

When we have a solution for those macros, I can proceed to resolving the conflicts and the PR is ready for further review.

mangod9 · 2024-07-30T19:59:05Z

ok, so the question is about whether to add the undefs in pal.h? We are also triaging PRs for .NET 9, assume this can wait a few weeks ?

trungnt2910 · 2024-07-30T23:07:25Z

Yes, the question is related to where I should add the #undefs, or otherwise how I should fix the redefined error.
I believe that one of the uses of pal.h is to isolate parts of CoreCLR code from system headers, so I initially thought putting those macros there would work.

assume this can wait a few weeks ?

Yeah. This has been waiting for quite a few months now, waiting a few more weeks wouldn't hurt.

mangod9 · 2024-09-16T14:44:19Z

adding @janvorli on advise on #undefs in pal.h.

trungnt2910 requested review from vargaz, lambdageek and MichalStrehovsky as code owners October 24, 2023 04:40

ghost added the community-contribution Indicates that the PR has been added by a community member label Oct 24, 2023

dotnet-issue-labeler bot added the area-PAL-coreclr label Oct 24, 2023

jkotas reviewed Oct 24, 2023

View reviewed changes

src/native/minipal/errno.h Outdated Show resolved Hide resolved

jkotas requested a review from janvorli October 24, 2023 04:55

This was referenced Oct 24, 2023

Tracking issue for CI build timeouts #76454

Closed

Test failure: Wasm.Build.Tests.NativeBuildTests.MonoAOTCross_WorksWithNoTrimming #93522

Closed

trungnt2910 force-pushed the dev/trungnt2910/haiku-pal branch from f98a038 to 3ff7e01 Compare October 24, 2023 21:58

janvorli reviewed Oct 24, 2023

View reviewed changes

trungnt2910 force-pushed the dev/trungnt2910/haiku-pal branch 2 times, most recently from 790029d to 2131127 Compare October 25, 2023 04:03

This was referenced Oct 25, 2023

Test_EventSource_EtwManifestGeneration* tests failing in CI #48798

Closed

[wasm][debugger] Debugger tests getting randomly aborted on CI #89409

Open

The active test run was aborted. Reason: Test host process crashed dotnet/dnceng#451

Open

trungnt2910 force-pushed the dev/trungnt2910/haiku-pal branch from ed1af42 to 4a8cc01 Compare October 25, 2023 13:53

trungnt2910 force-pushed the dev/trungnt2910/haiku-pal branch 4 times, most recently from 0a97fb8 to f1f4369 Compare October 28, 2023 20:58

This was referenced Oct 28, 2023

Timeout in System.Net.Quic.Functional.Tests #86019

Closed

CI error: System.Net.Quic.QuicException: The connection timed out from inactivity #91757

Closed

System.Security.Cryptography.Tests timing out #93840

Closed

huoyaoyuan reviewed Oct 30, 2023

View reviewed changes

trungnt2910 force-pushed the dev/trungnt2910/haiku-pal branch from f1f4369 to 72e7cc3 Compare November 5, 2023 00:40

trungnt2910 force-pushed the dev/trungnt2910/haiku-pal branch from d9bb4e0 to e7b3c85 Compare March 7, 2024 03:10

mangod9 added the needs-author-action An issue or pull request that requires more info or actions from the author. label Apr 15, 2024

dotnet-policy-service bot added the no-recent-activity label Apr 29, 2024

dotnet-policy-service bot removed the no-recent-activity label Apr 29, 2024

dotnet-policy-service bot removed the needs-author-action An issue or pull request that requires more info or actions from the author. label Apr 29, 2024

jkotas marked this pull request as draft April 30, 2024 04:01

trungnt2910 force-pushed the dev/trungnt2910/haiku-pal branch from e7b3c85 to a7afb88 Compare April 30, 2024 14:26

trungnt2910 changed the title ~~Haiku: Initial CoreCLR support~~ Haiku: Add Haiku support for CoreCLR/PAL Apr 30, 2024

trungnt2910 marked this pull request as ready for review April 30, 2024 14:26

trungnt2910 force-pushed the dev/trungnt2910/haiku-pal branch from a7afb88 to c11ed1a Compare April 30, 2024 14:46

trungnt2910 and others added 2 commits May 1, 2024 00:02

Haiku: Add Haiku support for CoreCLR/PAL

f6870aa

This contains a part of the code required to build CoreCLR and get `paltests` to pass on Haiku. This commit covers `src/coreclr/pal/**`. Co-authored-by: Jessica Hamilton <[email protected]>

Try removing problematic wcsnlen function

56ff51f

This conflicts with the standard one already included through `wctype.h` in some POSIX OSes like Haiku.

trungnt2910 force-pushed the dev/trungnt2910/haiku-pal branch from ed24de7 to 56ff51f Compare April 30, 2024 17:02

build-analysis bot mentioned this pull request Apr 30, 2024

System.Numerics.Tensors.Tests.SingleGenericTensorPrimitives.SpanScalarDestination_SpecialValues fails #101721

Closed

Provide feedback

Saved searches

Use saved searches to filter your results more quickly

Haiku: Add Haiku support for CoreCLR/PAL #93907

Haiku: Add Haiku support for CoreCLR/PAL #93907

trungnt2910 commented Oct 24, 2023

janvorli Oct 24, 2023

trungnt2910 Oct 25, 2023

janvorli Oct 24, 2023

trungnt2910 Oct 25, 2023

trungnt2910 commented Oct 25, 2023

jkotas commented Oct 25, 2023 •

edited

Loading

trungnt2910 commented Oct 25, 2023

trungnt2910 commented Oct 25, 2023

huoyaoyuan Oct 30, 2023

trungnt2910 Oct 30, 2023

huoyaoyuan Oct 30, 2023

trungnt2910 Oct 30, 2023

huoyaoyuan Oct 30, 2023

trungnt2910 Oct 30, 2023 •

edited

Loading

mangod9 commented Mar 4, 2024

Anutrix commented Apr 29, 2024

trungnt2910 commented Apr 29, 2024

jkotas commented Apr 30, 2024

trungnt2910 commented Apr 30, 2024

jkotas commented Apr 30, 2024

trungnt2910 commented Apr 30, 2024

trungnt2910 commented Apr 30, 2024 •

edited

Loading

jkotas commented Apr 30, 2024

trungnt2910 commented Apr 30, 2024

trungnt2910 commented Apr 30, 2024

mangod9 commented Jul 29, 2024

trungnt2910 commented Jul 29, 2024 •

edited

Loading

mangod9 commented Jul 30, 2024

trungnt2910 commented Jul 30, 2024

mangod9 commented Sep 16, 2024

	#ifdef TARGET_LINUX
	if (syscall(__NR_get_mempolicy, NULL, NULL, 0, 0, 0) < 0 && errno == ENOSYS)
	return;

	int highestNumaNode = GetNodeNum("/sys/devices/system/node", false);
	// we only use this implementation when there are two or more NUMA nodes available
	if (highestNumaNode < 1)
	return;

	g_numaAvailable = true;
	g_highestNumaNode = highestNumaNode;
	#endif

	#ifdef TARGET_LINUX
	return syscall(__NR_mbind, (long)start, len, 1, (long)nodemask, maxnode, 0);
	#else

Haiku: Add Haiku support for CoreCLR/PAL #93907

Are you sure you want to change the base?

Haiku: Add Haiku support for CoreCLR/PAL #93907

Conversation

trungnt2910 commented Oct 24, 2023

Choose a reason for hiding this comment

Choose a reason for hiding this comment

Choose a reason for hiding this comment

Choose a reason for hiding this comment

trungnt2910 commented Oct 25, 2023

jkotas commented Oct 25, 2023 • edited Loading

trungnt2910 commented Oct 25, 2023

trungnt2910 commented Oct 25, 2023

Choose a reason for hiding this comment

Choose a reason for hiding this comment

Choose a reason for hiding this comment

Choose a reason for hiding this comment

Choose a reason for hiding this comment

trungnt2910 Oct 30, 2023 • edited Loading

Choose a reason for hiding this comment

mangod9 commented Mar 4, 2024

Anutrix commented Apr 29, 2024

trungnt2910 commented Apr 29, 2024

jkotas commented Apr 30, 2024

trungnt2910 commented Apr 30, 2024

jkotas commented Apr 30, 2024

trungnt2910 commented Apr 30, 2024

trungnt2910 commented Apr 30, 2024 • edited Loading

jkotas commented Apr 30, 2024

trungnt2910 commented Apr 30, 2024

trungnt2910 commented Apr 30, 2024

mangod9 commented Jul 29, 2024

trungnt2910 commented Jul 29, 2024 • edited Loading

mangod9 commented Jul 30, 2024

trungnt2910 commented Jul 30, 2024

mangod9 commented Sep 16, 2024

jkotas commented Oct 25, 2023 •

edited

Loading

trungnt2910 Oct 30, 2023 •

edited

Loading

trungnt2910 commented Apr 30, 2024 •

edited

Loading

trungnt2910 commented Jul 29, 2024 •

edited

Loading