Statically Linked NativeAOT HelloWorld throws Segmentation fault with Globalization Cultural Data #70848
Comments
|
Tagging subscribers to this area: @dotnet/area-system-globalization Issue DetailsDescriptionTrying to test statically linking a basic hello world app on Ubuntu 20.04 and found that it throws a seg fault inside the globalization code. Reproduction Steps(On Ubuntu 20.04.4 LTS) Expected behaviorOutput "Hello, World!" Actual behaviorRegression?No response Known WorkaroundsEither disable globalization with Configurationdotnet version: 7.0.100-preview.3.22179.4 OS: Ubuntu 20.04.4 LTS Architecture: x64 Other informationReproduces on Ubuntu, but not AmazonLinux2. Ubuntu has We can take a look at this, but were wondering if it was already known.
|
|
There seem to be 2 versions of GlobalizationNative_LoadICU (pal_icushim.c, pal_icushim_static.c). Would it be okay to assign this issue to ourselves to use the static version of GlobalizationNative_LoadICU since this is a static build ? If so, would you suggest using a new entrypoint for static like this in native/libs/System.Globalization.Native/entrypoints.c: ubuntu@ip-172-31-24-113:~/runtime-net-7/runtime/src$ grep -r pal_icushim_static * ubuntu@ip-172-31-24-113:~/runtime-net-7/runtime/src$ grep -r GlobalizationNative_LoadICU * |
|
The full static linking is incompatible with any dynamic libraries being loaded in the process. That's how Linux and glibc work. You would need to create a custom build of the runtime, shims and dependencies that can be then all statically linked together and where nothing ever calls I am deleting the |
If a user chooses |
Static build compiles in Alpine Linux using libmusl, it did not show the dlopen issue in my test. |
You would need to do more than that to get statically linked ICU. For example, you would need to rebuild the whole |
Yes, you are right. Since libSystem.Globalization.Native.a is an archive, libicu*.a archives need to be linked separately like in the prototype below. If you are okay with this approach, can I send a PR, assuming pal_icu_shim_static can be used under |
What would that PR do? |
Currently, NativeAOT static binaries still have dynamic dependency on underlying Linux libraries. In short, the proposed change would use pal_icu_shim_static.c instead of pal_icu_shim.c, only when the static option is invoked. Currently, pal_icu_shim.c does a dlopen() to libicu shared libraries, so, a NativeAOT static binary still has dependency on |
|
There are number of other libraries that have same problem. Would you do this for all of them? The particularly hard one case is openssl. Openssl has baked-in distro-specific configuration settings, and so statically linking on one distribution and deploying on another does not work well for it. |
|
Agreed that there are distro-specific settings and that most distros probably only support shared-libraries - this is the most common use-case.
Openssl and krb5 source code has static build support, any security concerns about lib updates can be alleviated by using a build pipeline with a custom build environment that regularly updates the whole NativeAOT statically-linked app. Linux distro-vendors probably assume updates are only at the package-manager level (yum or apt-get).
If the source code of other libraries does not have static build support, that would be a restriction, but that would be outside dotnet unlike pal_icu_shim.c |
The static build support in openssl is not sufficient to allow build on one distro and deploy on another. One of the problems is that different distros have different location of the certs stores. This location is hardcoded in openssl. Anyway, if you would like to propose changes to support static linking, I think we can accept them as long as they are opt-in and do not change any defaults. |
Thank you, I will send a PR |
|
I tested a partial change, but, there are a couple of unresolved issues. What worksWith this change I was able to statically link libicu into a NativeAOT user program. I pointed the nugetconfig to my local repo to use the built IL package as you had mentioned in another issue. Notes:
Unresolved issues:
Code change: |
This change needs to part of the opt-in as well. Statically linking with libicu would break portability of the default build.
We compile crossgen using native AOT compiler during the build. I would expect that you should be able to reproduce the same crash by compiling a simple program that uses ICU. |
|
I tested a different change using weak symbols, this change is optin, there is no crash in crossgen, user program works fine. Each of the symbols in pal_icushim_internal.h would get a weak symbol counterpart. Resolution using weak symbols is used only if user specifies an environment variable asking for static link and the weak symbol has already been resolved by the NativeAOT makefile (Microsoft.NETCore.Native.Unix.targets). TBD: Here is the rough change: The symbol resolution would look something like this in pal_icushim.c. This is somewhat like the dlsym method currently being used. For now, I have the versions hardcoded like u_strlen_66, symbol versions will be determined automatically in a PR. ( I can move the weak declarations to a headerfile in a PR) |
|
Would it be easier to have a separate build of the System_Globalization shim for the static linking? This shim can be even built on the user machine. The NativeAOT package can ship sources for the shim instead of the .a file to make it possible. |
Thank you @jkotas, yes, it would be a simpler solution, it would also be opt-in. Here is a draft change, assuming an already built shim, user program works fine. User's NativeAOT code compiles with the user-supplied shim and invokes it instead. User NativeAOT code can get compiled with a switch (LocalSystemGlobalizationNative) like :
This results in
This diff uses an already built libSystem.Globalization.Native.a from the sources based on your comment. |
How would people who want to use this build the shim? |
Steps:
This CMakeLists.txt builds for Ubuntu Linux 20.04, some customizations will be needed for MacOS, Windows and other options. The file listing is: |
I think there is more to these steps. For example, you also need to make sure that you are synced to the same SHA as the rest of NativeAOT toolchain is built from. System.Native shims do not have stable APIs and they need to exactly match the toolchain. To be usable by broader audiences, these steps would have to be automated and simplified down to setting a msbuild property or something that simple. I believe that the best way to do that would be bundling the shim sources into the native AOT toolchain package and compiling them automatically using C/C++ compiler on the target machine. |
Thanks, I can try bundling the shim sources into the |
|
|
|
I added the shim source code to Microsoft.Dotnet.IL.Compiler.nupkg, but I need some help in doing a publish using the ilc that I built locally. |
|
Replied in chat offline - I think you need to explicitly specify the package the -dev reference in your .csproj file: |
|
Thanks for your prompt responses today in fixing my build.
This patch bundles shim sources into the nativeAOT toolchain and compiles it during This output of the change is further down. User code NativeAOT build: The locally built shim libSystem.Globalization.Native.a is in bold below "clang" "obj/Debug/net7.0/linux-x64/native/sample4.o" -o "bin/Debug/net7.0/linux-x64/native/sample4" /home/ubuntu/.nuget/packages/runtime.linux-x64.microsoft.dotnet.ilcompiler/7.0.0-dev/sdk/libbootstrapper.a /home/ubuntu/.nuget/packages/runtime.linux-x64.microsoft.dotnet.ilcompiler/7.0.0-dev/sdk/libRuntime.WorkstationGC.a /home/ubuntu/.nuget/packages/runtime.linux-x64.microsoft.dotnet.ilcompiler/7.0.0-dev/framework/libSystem.Native.a /home/ubuntu/.nuget/packages/runtime.linux-x64.microsoft.dotnet.ilcompiler/7.0.0-dev/framework/libSystem.Globalization.Native.a /home/ubuntu/.nuget/packages/runtime.linux-x64.microsoft.dotnet.ilcompiler/7.0.0-dev/framework/libSystem.IO.Compression.Native.a /home/ubuntu/.nuget/packages/runtime.linux-x64.microsoft.dotnet.ilcompiler/7.0.0-dev/framework/libSystem.Net.Security.Native.a /home/ubuntu/.nuget/packages/runtime.linux-x64.microsoft.dotnet.ilcompiler/7.0.0-dev/framework/libSystem.Security.Cryptography.Native.OpenSsl.a /home/ubuntu/.nuget/packages/microsoft.dotnet.ilcompiler/7.0.0-dev/shim_source_code/libs/System.Globalization.Native/build/libSystem.Globalization.Native-Static.a -g -Wl,-rpath,'$ORIGIN' -Wl,--build-id=sha1 -Wl,--as-needed -pthread -lstdc++ -ldl -lm -lz -lrt -pie -Wl,-z,relro -Wl,-z,now -Wl,--discard-all -Wl,--gc-sections Output of user code: |
|
This looks reasonable to me. There are details to fix that I will comment on once you post a PR. |
|
Closing this since we're working towards enabling static builds starting with this PR #72896 more details here: https://github.com/dotnet/runtime/blob/main/src/coreclr/nativeaot/docs/compiling.md#using-statically-linked-icu Eventually, we hope to add other libraries outside of libicu |
ghost
locked as resolved and limited conversation to collaborators
Description
Trying to test statically linking a basic hello world app on Ubuntu 20.04 and found that it throws a seg fault inside the globalization code.
Reproduction Steps
(On Ubuntu 20.04.4 LTS x64)
dotnet new consoledotnet add package Microsoft.Dotnet.ILCompiler --prereleasedotnet publish -r linux-x64 -c Release /p:StaticallyLinked=true --self-contained./bin/Release/net7.0/linux-x64/native/HelloWorldExpected behavior
Output "Hello, World!"
Actual behavior
Regression?
No response
Known Workarounds
Either disable globalization with
<InvariantGlobalization>true</InvariantGlobalization>or don't statically link (remove/p:StaticallyLinked=trueargument)Configuration
dotnet version: 7.0.100-preview.3.22179.4
OS: Ubuntu 20.04.4 LTS
Architecture: x64
Other information
Reproduces on Ubuntu, but not AmazonLinux2. Ubuntu has
libicuuc.so.66but AmazonLinux2 haslibicuuc.so.50We can take a look at this, but were wondering if it was already known.
The text was updated successfully, but these errors were encountered: