Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Statically Linked NativeAOT HelloWorld throws Segmentation fault with Globalization Cultural Data #70848

Closed
Beau-Gosse-dev opened this issue Jun 16, 2022 · 30 comments

Comments

@Beau-Gosse-dev
Copy link
Contributor

Beau-Gosse-dev commented Jun 16, 2022

Description

Trying to test statically linking a basic hello world app on Ubuntu 20.04 and found that it throws a seg fault inside the globalization code.

Reproduction Steps

(On Ubuntu 20.04.4 LTS x64)
dotnet new console
dotnet add package Microsoft.Dotnet.ILCompiler --prerelease
dotnet publish -r linux-x64 -c Release /p:StaticallyLinked=true --self-contained
./bin/Release/net7.0/linux-x64/native/HelloWorld

Expected behavior

Output "Hello, World!"

Actual behavior

Thread 1 "HelloWorld" received signal SIGSEGV, Segmentation fault.
0x00007fffd74b6603 in void std::call_once<void (&)()>(std::once_flag&, void (&)()) () from /lib/x86_64-linux-gnu/libicuuc.so.66
(gdb) bt
#0  0x00007fffd74b6603 in void std::call_once<void (&)()>(std::once_flag&, void (&)()) () from /lib/x86_64-linux-gnu/libicuuc.so.66
#1  0x00007fffd74b63b4 in icu_66::umtx_initImplPreInit(icu_66::UInitOnce&) () from /lib/x86_64-linux-gnu/libicuuc.so.66
#2  0x00007fffd74e79cb in ?? () from /lib/x86_64-linux-gnu/libicuuc.so.66
#3  0x00007fffd52c0a60 in ulocdata_getCLDRVersion_66 () from /lib/x86_64-linux-gnu/libicui18n.so.66
#4  0x0000000000476de2 in ValidateICUDataCanLoad () at /__w/1/s/src/native/libs/System.Globalization.Native/pal_icushim.c:375
#5  GlobalizationNative_LoadICU () at /__w/1/s/src/native/libs/System.Globalization.Native/pal_icushim.c:471
#6  0x0000000000633c5c in S_P_CoreLib_System_Globalization_GlobalizationMode_Settings___cctor ()
#7  0x00000000006209a9 in S_P_CoreLib_System_Runtime_CompilerServices_ClassConstructorRunner__EnsureClassConstructorRun ()
#8  0x00000000006208b9 in S_P_CoreLib_System_Runtime_CompilerServices_ClassConstructorRunner__CheckStaticClassConstructionReturnNonGCStaticBase ()
#9  0x00000000005cd103 in S_P_CoreLib_System_Globalization_CultureData__CreateCultureWithInvariantData ()
#10 0x00000000005cd1ca in S_P_CoreLib_System_Globalization_CultureData__get_Invariant ()
#11 0x00000000005ddf7c in S_P_CoreLib_System_Globalization_TextInfo___cctor ()
#12 0x00000000006209a9 in S_P_CoreLib_System_Runtime_CompilerServices_ClassConstructorRunner__EnsureClassConstructorRun ()
#13 0x00000000006208a9 in S_P_CoreLib_System_Runtime_CompilerServices_ClassConstructorRunner__CheckStaticClassConstructionReturnGCStaticBase ()
#14 0x000000000064cd17 in System_Console_System_Text_EncodingHelper__GetCharset ()
#15 0x000000000064cc29 in System_Console_System_Text_EncodingHelper__GetEncodingFromCharset ()
#16 0x0000000000645c09 in System_Console_System_ConsolePal__GetConsoleEncoding ()
#17 0x0000000000644cf4 in System_Console_System_Console__get_OutputEncoding ()
#18 0x0000000000644d9b in System_Console_System_Console__CreateOutputWriter ()
#19 0x0000000000644f43 in System_Console_System_Console___get_Out_g__EnsureInitialized_26_0 ()
#20 0x0000000000644e80 in System_Console_System_Console__WriteLine_12 ()
#21 0x00000000006e9f1f in __managed__Main ()
#22 0x00000000004058d9 in main (argc=1, argv=0x7fffffffe478) at /__w/1/s/src/coreclr/nativeaot/Bootstrap/main.cpp:205

Regression?

No response

Known Workarounds

Either disable globalization with <InvariantGlobalization>true</InvariantGlobalization> or don't statically link (remove /p:StaticallyLinked=true argument)

Configuration

dotnet version: 7.0.100-preview.3.22179.4

OS: Ubuntu 20.04.4 LTS

Architecture: x64

Other information

Reproduces on Ubuntu, but not AmazonLinux2. Ubuntu has libicuuc.so.66 but AmazonLinux2 has libicuuc.so.50

We can take a look at this, but were wondering if it was already known.

@ghost ghost added the untriaged New issue has not been triaged by the area owner label Jun 16, 2022
@ghost
Copy link

ghost commented Jun 16, 2022

Tagging subscribers to this area: @dotnet/area-system-globalization
See info in area-owners.md if you want to be subscribed.

Issue Details

Description

Trying to test statically linking a basic hello world app on Ubuntu 20.04 and found that it throws a seg fault inside the globalization code.

Reproduction Steps

(On Ubuntu 20.04.4 LTS)
dotnet new console
dotnet add package Microsoft.Dotnet.ILCompiler --prerelease
dotnet publish -r linux-x64 -c Release /p:StaticallyLinked=true --self-contained
./bin/Release/net7.0/linux-x64/native/HelloWorld

Expected behavior

Output "Hello, World!"

Actual behavior

Thread 1 "HelloWorld" received signal SIGSEGV, Segmentation fault.
0x00007fffd74b6603 in void std::call_once<void (&)()>(std::once_flag&, void (&)()) () from /lib/x86_64-linux-gnu/libicuuc.so.66
(gdb) bt
#0  0x00007fffd74b6603 in void std::call_once<void (&)()>(std::once_flag&, void (&)()) () from /lib/x86_64-linux-gnu/libicuuc.so.66
#1  0x00007fffd74b63b4 in icu_66::umtx_initImplPreInit(icu_66::UInitOnce&) () from /lib/x86_64-linux-gnu/libicuuc.so.66
#2  0x00007fffd74e79cb in ?? () from /lib/x86_64-linux-gnu/libicuuc.so.66
#3  0x00007fffd52c0a60 in ulocdata_getCLDRVersion_66 () from /lib/x86_64-linux-gnu/libicui18n.so.66
#4  0x0000000000476de2 in ValidateICUDataCanLoad () at /__w/1/s/src/native/libs/System.Globalization.Native/pal_icushim.c:375
#5  GlobalizationNative_LoadICU () at /__w/1/s/src/native/libs/System.Globalization.Native/pal_icushim.c:471
#6  0x0000000000633c5c in S_P_CoreLib_System_Globalization_GlobalizationMode_Settings___cctor ()
#7  0x00000000006209a9 in S_P_CoreLib_System_Runtime_CompilerServices_ClassConstructorRunner__EnsureClassConstructorRun ()
#8  0x00000000006208b9 in S_P_CoreLib_System_Runtime_CompilerServices_ClassConstructorRunner__CheckStaticClassConstructionReturnNonGCStaticBase ()
#9  0x00000000005cd103 in S_P_CoreLib_System_Globalization_CultureData__CreateCultureWithInvariantData ()
#10 0x00000000005cd1ca in S_P_CoreLib_System_Globalization_CultureData__get_Invariant ()
#11 0x00000000005ddf7c in S_P_CoreLib_System_Globalization_TextInfo___cctor ()
#12 0x00000000006209a9 in S_P_CoreLib_System_Runtime_CompilerServices_ClassConstructorRunner__EnsureClassConstructorRun ()
#13 0x00000000006208a9 in S_P_CoreLib_System_Runtime_CompilerServices_ClassConstructorRunner__CheckStaticClassConstructionReturnGCStaticBase ()
#14 0x000000000064cd17 in System_Console_System_Text_EncodingHelper__GetCharset ()
#15 0x000000000064cc29 in System_Console_System_Text_EncodingHelper__GetEncodingFromCharset ()
#16 0x0000000000645c09 in System_Console_System_ConsolePal__GetConsoleEncoding ()
#17 0x0000000000644cf4 in System_Console_System_Console__get_OutputEncoding ()
#18 0x0000000000644d9b in System_Console_System_Console__CreateOutputWriter ()
#19 0x0000000000644f43 in System_Console_System_Console___get_Out_g__EnsureInitialized_26_0 ()
#20 0x0000000000644e80 in System_Console_System_Console__WriteLine_12 ()
#21 0x00000000006e9f1f in __managed__Main ()
#22 0x00000000004058d9 in main (argc=1, argv=0x7fffffffe478) at /__w/1/s/src/coreclr/nativeaot/Bootstrap/main.cpp:205

Regression?

No response

Known Workarounds

Either disable globalization with <InvariantGlobalization>true</InvariantGlobalization> or don't statically link (remove /p:StaticallyLinked=true argument)

Configuration

dotnet version: 7.0.100-preview.3.22179.4

OS: Ubuntu 20.04.4 LTS

Architecture: x64

Other information

Reproduces on Ubuntu, but not AmazonLinux2. Ubuntu has libicuuc.so.66 but AmazonLinux2 has libicuuc.so.50

We can take a look at this, but were wondering if it was already known.

Author: Beau-Gosse-dev
Assignees: -
Labels:

area-System.Globalization

Milestone: -

@smhmhmd
Copy link
Contributor

smhmhmd commented Jun 16, 2022

There seem to be 2 versions of GlobalizationNative_LoadICU (pal_icushim.c, pal_icushim_static.c).
mono seems to be using the the static version.

Would it be okay to assign this issue to ourselves to use the static version of GlobalizationNative_LoadICU since this is a static build ?

If so, would you suggest using a new entrypoint for static like this in native/libs/System.Globalization.Native/entrypoints.c:
#if defined(STATIC_ICU)
DllImportEntry(GlobalizationNative_LoadICUData)
#endif

ubuntu@ip-172-31-24-113:~/runtime-net-7/runtime/src$ grep -r pal_icushim_static *
mono/wasm/runtime/es6/dotnet.es6.lib.js: // pal_icushim_static.c
mono/wasm/runtime/exports.ts: // also keep in sync with pal_icushim_static.c
mono/wasm/runtime/cjs/dotnet.cjs.lib.js: // pal_icushim_static.c
mono/wasm/runtime/icu.ts:// etc, see "mono_wasm_get_icudt_name" implementation in pal_icushim_static.c
mono/mono/mini/CMakeLists.txt: pal_icushim_static.c)

ubuntu@ip-172-31-24-113:~/runtime-net-7/runtime/src$ grep -r GlobalizationNative_LoadICU *
libraries/Common/src/Interop/Interop.ICU.iOS.cs: [LibraryImport(Libraries.GlobalizationNative, EntryPoint = "GlobalizationNative_LoadICUData", StringMarshalling = StringMarshalling.Utf8)]
libraries/Common/src/Interop/Interop.ICU.cs: [LibraryImport(Libraries.GlobalizationNative, EntryPoint = "GlobalizationNative_LoadICU")]
mono/wasi/mono-wasi-driver/driver.c:void GlobalizationNative_LoadICU() {
mono/wasi/mono-wasi-driver/driver.c: {"GlobalizationNative_LoadICU", GlobalizationNative_LoadICU },
native/libs/System.Globalization.Native/pal_icushim.c:// GlobalizationNative_LoadICU
native/libs/System.Globalization.Native/pal_icushim.c:int32_t GlobalizationNative_LoadICU()
native/libs/System.Globalization.Native/entrypoints.c: DllImportEntry(GlobalizationNative_LoadICU)
native/libs/System.Globalization.Native/entrypoints.c: DllImportEntry(GlobalizationNative_LoadICUData)
native/libs/System.Globalization.Native/pal_icushim_static.c:GlobalizationNative_LoadICUData(const char* path)
native/libs/System.Globalization.Native/pal_icushim_static.c: return GlobalizationNative_LoadICU();
native/libs/System.Globalization.Native/pal_icushim_static.c:int32_t GlobalizationNative_LoadICU(void)
native/libs/System.Globalization.Native/pal_icushim.h:PALEXPORT int32_t GlobalizationNative_LoadICU(void);
native/libs/System.Globalization.Native/pal_icushim.h:PALEXPORT int32_t GlobalizationNative_LoadICUData(const char* path);
tasks/AndroidAppBuilder/Templates/CMakeLists-android.txt: "-u GlobalizationNative_LoadICU"

@tarekgh
Copy link
Member

tarekgh commented Jun 16, 2022

CC @jkotas @MichalStrehovsky

@jkotas
Copy link
Member

jkotas commented Jun 16, 2022

The full static linking is incompatible with any dynamic libraries being loaded in the process. That's how Linux and glibc work. You would need to create a custom build of the runtime, shims and dependencies that can be then all statically linked together and where nothing ever calls dlopen.

I am deleting the StaticallyLinked property in #70854 to avoid confusion. This property is has no chance of working well given how things are setup. If you want to experiment with static linking, you can just add -static linked arg to linked args in your local project file.

@jkotas jkotas added this to the Future milestone Jun 16, 2022
@ghost ghost removed the untriaged New issue has not been triaged by the area owner label Jun 16, 2022
@smhmhmd
Copy link
Contributor

smhmhmd commented Jun 16, 2022

@jkotas

If you want to experiment with static linking, you can just add -static linked arg to linked args in your local project file.

If a user chooses -static, can the static version of the GlobalizationNative_LoadICU() in pal_icushim_static.c be used instead of GlobalizationNative_LoadICU() in pal_icushim.c ?

@smhmhmd
Copy link
Contributor

smhmhmd commented Jun 16, 2022

The full static linking is incompatible with any dynamic libraries being loaded in the process. That's how Linux and glibc work. You would need to create a custom build of the runtime, shims and dependencies that can be then all statically linked together and where nothing ever calls dlopen.

Static build compiles in Alpine Linux using libmusl, it did not show the dlopen issue in my test.
I did see the dlopen issue with glibc.

@jkotas
Copy link
Member

jkotas commented Jun 16, 2022

can the static version of the GlobalizationNative_LoadICU() in pal_icushim_static.c be used instead of GlobalizationNative_LoadICU() in pal_icushim.c ?

You would need to do more than that to get statically linked ICU. For example, you would need to rebuild the whole System.Globalization.Native.a with STATIC_ICU defined.

@smhmhmd
Copy link
Contributor

smhmhmd commented Jun 20, 2022

@jkotas

You would need to do more than that to get statically linked ICU

Yes, you are right.

Since libSystem.Globalization.Native.a is an archive, libicu*.a archives need to be linked separately like in the prototype below.
Here is a prototype to make the statically linked binary invoking ulocdata_getCLDRVersion() on Alpine Linux:
( pal*.o are from my dotnet build )

# cat /etc/os-release 
NAME="Alpine Linux"
ID=alpine
VERSION_ID=3.15.4
PRETTY_NAME="Alpine Linux v3.15"
HOME_URL="https://alpinelinux.org/"
BUG_REPORT_URL="https://bugs.alpinelinux.org/"
/container/tmp/x1 # 

/container/tmp/x1 # cat test.c
#include <stdio.h>
#include <unicode/uversion.h>
#include <unicode/errorcode.h>

int main()
{
        UVersionInfo version = {0};
        UErrorCode err = U_ZERO_ERROR;

        ulocdata_getCLDRVersion(version, &err);
	printf("Got CLDRVersion, err:%d\n", err);

        return 0;
}

# sh -x make.sh 
+ ar qc mypals.a pal_calendarData.c.o pal_icushim_static.c.o pal_localeNumberData.c.o pal_timeZoneInfo.c.o pal_casing.c.o pal_idna.c.o pal_localeStringData.c.o pal_collation.c.o pal_locale.c.o pal_normalization.c.o
+ gcc -c test.c
+ g++ -static -lstdc++ test.o mypals.a -Wl,--whole-archive /usr/lib/libicuio.a /usr/lib/libicutu.a /usr/lib/libicui18n.a /usr/lib/libicutest.a /usr/lib/libicuuc.a /usr/lib/libicudata.a -Wl,--no-whole-archive -o test
+ file test
test: ELF 64-bit LSB executable, x86-64, version 1 (SYSV), statically linked, with debug_info, not stripped
+ ./test
Got CLDRVersion, err:0

If you are okay with this approach, can I send a PR, assuming pal_icu_shim_static can be used under STATIC_ICU as an alternative to pal_icu_shim.c.
With clang-12 instead of gcc above, there is a warning, but the compilation flags dont error out:
warning: implicit declaration of function 'ulocdata_getCLDRVersion_69'

@jkotas
Copy link
Member

jkotas commented Jun 20, 2022

If you are okay with this approach, can I send a PR,

What would that PR do?

@smhmhmd
Copy link
Contributor

smhmhmd commented Jun 20, 2022

What would that PR do?

Currently, NativeAOT static binaries still have dynamic dependency on underlying Linux libraries.
We want it to be possible to build on one Linux distribution and deploy to another.

In short, the proposed change would use pal_icu_shim_static.c instead of pal_icu_shim.c, only when the static option is invoked.

Currently, pal_icu_shim.c does a dlopen() to libicu shared libraries, so, a NativeAOT static binary still has dependency on
shared libraries, the change would statically link libicu*.a into the NativeAOT static binary or compile as gcc -static -pthread test.c -o test -licuio -licui18n -licuuc -licudata -lpthread -lm -lstdc++ -g to avoid the dlopen.

@jkotas
Copy link
Member

jkotas commented Jun 21, 2022

There are number of other libraries that have same problem. Would you do this for all of them? The particularly hard one case is openssl. Openssl has baked-in distro-specific configuration settings, and so statically linking on one distribution and deploying on another does not work well for it.

@smhmhmd
Copy link
Contributor

smhmhmd commented Jun 21, 2022

Agreed that there are distro-specific settings and that most distros probably only support shared-libraries - this is the most common use-case.

The particularly hard one case is openssl. Openssl has baked-in distro-specific configuration settings, and so statically linking on one distribution and deploying on another does not work well for it.

Openssl and krb5 source code has static build support, any security concerns about lib updates can be alleviated by using a build pipeline with a custom build environment that regularly updates the whole NativeAOT statically-linked app. Linux distro-vendors probably assume updates are only at the package-manager level (yum or apt-get).

There are number of other libraries that have same problem. Would you do this for all of them?

If the source code of other libraries does not have static build support, that would be a restriction, but that would be outside dotnet unlike pal_icu_shim.c

@jkotas
Copy link
Member

jkotas commented Jun 21, 2022

Openssl and krb5 source code has static build support

The static build support in openssl is not sufficient to allow build on one distro and deploy on another. One of the problems is that different distros have different location of the certs stores. This location is hardcoded in openssl.

Anyway, if you would like to propose changes to support static linking, I think we can accept them as long as they are opt-in and do not change any defaults.

@smhmhmd
Copy link
Contributor

smhmhmd commented Jun 21, 2022

Anyway, if you would like to propose changes to support static linking, I think we can accept them as long as they are opt-in and do not change any defaults.

Thank you, I will send a PR

@smhmhmd
Copy link
Contributor

smhmhmd commented Jul 4, 2022

@jkotas

I tested a partial change, but, there are a couple of unresolved issues.

What works

With this change I was able to statically link libicu into a NativeAOT user program. I pointed the nugetconfig to my local repo to use the built IL package as you had mentioned in another issue.

Notes:

  • This change is opt-in for users as per your feedback, although the runtime build process will have to link with libicu.
  • For me, static linking is possible only with the following command, other options like using .a or whole-archive do not work:
    -Wl,-Bstatic System.Globalization.Native.a -licui18n -licuuc -licudata -licuio -licutu -licutest -Wl,-Bdynamic. This command scopes static linking just for System.Globalization.Native.a enabling the linker to add in only the right symbols.
  • As per https://unicode-org.github.io/icu/userguide/icu4c/packaging.html#link-to-icu-statically, Users of your ICU must compile with -DU_STATIC_IMPLEMENTATION.
  • A new file pal_icushim_native_static.[ch] had to be created because the #defines in pal_icushim_internal.h do not allow static compilation.

Unresolved issues:

  • All functions mentioned in pal_icushim_internal.h need to have calls to allow the linker to do static linking, this can be done using the u_strlen example I have in the change but there is atleast one deprecated libicu API that needs to be fixed - ucol_safeClone.
  • Although, Microsoft.DotNet.ILCompiler.7.0.0-dev.nupkg was built and is able to test, Crossgen has an issue during build, I need your help in debugging this:
    • /container/runtime/artifacts/bin/coreclr/Linux.x64.Debug/crossgen2/linux-musl-x64/publish/crossgen2 /container/runtime/artifacts/bin/coreclr/Linux.x64.Debug/IL/System.Private.CoreLib.dll --out /container/runtime/artifacts/obj/Microsoft.NETCore.App.Crossgen2/Debug/net7.0/linux-musl-x64/S.P.C.tmp
      Process terminated. Infinite recursion during resource lookup within System.Private.CoreLib. This may be a bug in System.Private.CoreLib, or potentially in certain extensibility points such as assembly resolve events or CultureInfo names. Resource name: DllNotFound_Linux
      Aborted

Code change:

src/coreclr/dlls/mscoree/coreclr/CMakeLists.txt    |  2 ++
 .../Microsoft.NETCore.Native.Unix.props            | 10 +++++++++-
 src/mono/mono/mini/CMakeLists.txt                  |  8 +++++++-
 src/native/corehost/apphost/static/CMakeLists.txt  |  5 +++--
 .../System.Globalization.Native/CMakeLists.txt     |  5 +++++
 .../libs/System.Globalization.Native/pal_icushim.c |  7 +++++++
 .../pal_icushim_native_static.c                    | 14 ++++++++++++++
 .../pal_icushim_native_static.h                    |  1 +
 8 files changed, 48 insertions(+), 4 deletions(-)
 create mode 100644 src/native/libs/System.Globalization.Native/pal_icushim_native_static.c
 create mode 100644 src/native/libs/System.Globalization.Native/pal_icushim_native_static.h

diff --git a/src/coreclr/dlls/mscoree/coreclr/CMakeLists.txt b/src/coreclr/dlls/mscoree/coreclr/CMakeLists.txt
index 688629d178d..c184c6f4445 100644
--- a/src/coreclr/dlls/mscoree/coreclr/CMakeLists.txt
+++ b/src/coreclr/dlls/mscoree/coreclr/CMakeLists.txt
@@ -105,7 +105,9 @@ set(CORECLR_LIBRARIES
     gcinfo
     utilcode
     v3binder
+    "-Wl,-Bstatic"
     System.Globalization.Native-Static
+    "-licui18n -licuuc -licudata -licuio -licutu -licutest -Wl,-Bdynamic"
     interop
     coreclrminipal
 )
diff --git a/src/coreclr/nativeaot/BuildIntegration/Microsoft.NETCore.Native.Unix.props b/src/coreclr/nativeaot/BuildIntegration/Microsoft.NETCore.Native.Unix.props
index c8ef6487bd5..222743a8067 100644
--- a/src/coreclr/nativeaot/BuildIntegration/Microsoft.NETCore.Native.Unix.props
+++ b/src/coreclr/nativeaot/BuildIntegration/Microsoft.NETCore.Native.Unix.props
@@ -53,7 +53,7 @@ The .NET Foundation licenses this file to you under the MIT license.
 
     <ItemGroup>
       <NetCoreAppNativeLibrary Include="System.Native" />
-      <NetCoreAppNativeLibrary Include="System.Globalization.Native" />
+      <NetCoreAppNativeLibraryWithSystemGlobalizationNative Include="System.Globalization.Native" />
       <NetCoreAppNativeLibrary Include="System.IO.Compression.Native" />
       <NetCoreAppNativeLibrary Include="System.Net.Security.Native" />
       <NetCoreAppNativeLibrary Include="System.Security.Cryptography.Native.Apple" Condition="'$(TargetOS)' == 'OSX'" />
@@ -66,6 +66,11 @@ The .NET Foundation licenses this file to you under the MIT license.
         <EscapedPath>$(IlcFrameworkNativePath)lib%(Identity).a</EscapedPath>
       </NetCoreAppNativeLibrary>
       <NativeLibrary Include="@(NetCoreAppNativeLibrary->'%(EscapedPath)')" />
+
+      <NetCoreAppNativeLibraryWithSystemGlobalizationNative Include="@(NetCoreAppNativeLibraryWithSystemGlobalizationNative->'%(Identity)')">
+        <EscapedPath>$(IlcFrameworkNativePath)lib%(Identity).a</EscapedPath>
+      </NetCoreAppNativeLibraryWithSystemGlobalizationNative>
+      <NativeLibraryWithSystemGlobalizationNative Include="@(NetCoreAppNativeLibraryWithSystemGlobalizationNative->'%(EscapedPath)')" />
     </ItemGroup>
 
     <ItemGroup Condition="'$(TargetOS)' == 'OSX'">
@@ -77,6 +82,9 @@ The .NET Foundation licenses this file to you under the MIT license.
 
     <ItemGroup>
       <LinkerArg Include="@(NativeLibrary)" />
+      <LinkerArg Include="-Wl,-Bstatic" />
+      <LinkerArg Include="@(NativeLibraryWithSystemGlobalizationNative)" />
+      <LinkerArg Include="-licui18n -licuuc -licudata -licuio -licutu -licutest -Wl,-Bdynamic" />
       <LinkerArg Include="--sysroot=$(SysRoot)" Condition="'$(SysRoot)' != ''" />
       <LinkerArg Include="--target=$(TargetTriple)" Condition="'$(TargetTriple)' != ''" />
       <LinkerArg Include="-g" Condition="$(NativeDebugSymbols) == 'true'" />
diff --git a/src/mono/mono/mini/CMakeLists.txt b/src/mono/mono/mini/CMakeLists.txt
index 448f9ded04f..50616c89cec 100644
--- a/src/mono/mono/mini/CMakeLists.txt
+++ b/src/mono/mono/mini/CMakeLists.txt
@@ -51,9 +51,14 @@ if(HAVE_SYS_ICU AND NOT HOST_WASI)
     set(pal_icushim_sources_base
         pal_icushim_static.c)
     add_definitions(-DSTATIC_ICU=1)
+    add_definitions(-DU_STATIC_IMPLEMENTATION=1)
+    set(pal_icushim_sources_base_extended
+        pal_icushim_native_static.c)
   else()
     set(pal_icushim_sources_base
         pal_icushim.c)
+    set(pal_icushim_sources_base_extended
+        pal_icushim_native_static.c)
   endif()
 
   set(icu_shim_sources_base
@@ -67,7 +72,8 @@ if(HAVE_SYS_ICU AND NOT HOST_WASI)
       pal_normalization.c
       pal_timeZoneInfo.c
       entrypoints.c
-      ${pal_icushim_sources_base})
+      ${pal_icushim_sources_base}
+      ${pal_icushim_sources_base_extended})
 
   if (TARGET_DARWIN)
     set(icu_shim_sources_base
diff --git a/src/native/corehost/apphost/static/CMakeLists.txt b/src/native/corehost/apphost/static/CMakeLists.txt
index c37885bf569..8a918f84c07 100644
--- a/src/native/corehost/apphost/static/CMakeLists.txt
+++ b/src/native/corehost/apphost/static/CMakeLists.txt
@@ -134,7 +134,6 @@ else()
     set(NATIVE_LIBS
         coreclr_static
 
-        System.Globalization.Native-Static
         System.IO.Compression.Native-Static
         System.Net.Security.Native-Static
         System.Native-Static
@@ -237,4 +236,6 @@ target_link_libraries(
     ${START_WHOLE_ARCHIVE}
     ${RUNTIMEINFO_LIB}
     ${END_WHOLE_ARCHIVE}
-)
+    "-Wl,-Bstatic"
+    System.Globalization.Native-Static
+    "-licui18n -licuuc -licudata -licuio -licutu -licutest -Wl,-Bdynamic")
diff --git a/src/native/libs/System.Globalization.Native/CMakeLists.txt b/src/native/libs/System.Globalization.Native/CMakeLists.txt
index a41e66cacd3..6297aab3e16 100644
--- a/src/native/libs/System.Globalization.Native/CMakeLists.txt
+++ b/src/native/libs/System.Globalization.Native/CMakeLists.txt
@@ -58,6 +58,7 @@ set(NATIVEGLOBALIZATION_SOURCES
     pal_localeStringData.c
     pal_normalization.c
     pal_icushim.c
+    pal_icushim_native_static.c
 )
 
 if (CLR_CMAKE_TARGET_OSX OR CLR_CMAKE_TARGET_MACCATALYST OR CLR_CMAKE_TARGET_IOS OR CLR_CMAKE_TARGET_TVOS)
@@ -104,6 +105,10 @@ add_library(System.Globalization.Native-Static
     ${NATIVEGLOBALIZATION_SOURCES}
 )
 
+if(DEFINED ENV{LINK_WITH_LIBICU_STATIC_LIBS})
+    target_compile_definitions(System.Globalization.Native-Static PUBLIC U_STATIC_IMPLEMENTATION)
+endif()
+
 if(CLR_CMAKE_TARGET_UNIX)
     set_target_properties(System.Globalization.Native-Static PROPERTIES OUTPUT_NAME System.Globalization.Native  CLEAN_DIRECT_OUTPUT 1)
 endif()
diff --git a/src/native/libs/System.Globalization.Native/pal_icushim.c b/src/native/libs/System.Globalization.Native/pal_icushim.c
index 9f8994394e4..6f127f3b271 100644
--- a/src/native/libs/System.Globalization.Native/pal_icushim.c
+++ b/src/native/libs/System.Globalization.Native/pal_icushim.c
@@ -18,6 +18,7 @@
 #include <assert.h>
 
 #include "pal_icushim.h"
+#include "pal_icushim_native_static.h"
 
 // Define pointers to all the used ICU functions
 #define PER_FUNCTION_BLOCK(fn, lib, required) TYPEOF(fn)* fn##_ptr;
@@ -50,6 +51,12 @@ ucol_safeClone_func ucol_safeClone_ptr = NULL;
 
 static int FindSymbolVersion(int majorVer, int minorVer, int subVer, char* symbolName, char* symbolVersion, char* suffix)
 {
+    if (getenv("USE_STATICALLY_LINKED_LIBICU_LIBS")) {
+        static const UChar text[]={ 0x61, 0x62, 0x63, 0 };    /* "abc" */
+        (void)u_strlen_native_static(text);
+        /* TBD: Add symbol table for statically accessible functions accessed like above */
+    }
+
     // Find out the format of the version string added to each symbol
     // First try just the unversioned symbol
     if (dlsym(libicuuc, "u_strlen") == NULL)
diff --git a/src/native/libs/System.Globalization.Native/pal_icushim_native_static.c b/src/native/libs/System.Globalization.Native/pal_icushim_native_static.c
new file mode 100644
index 00000000000..3b25e3260d9
--- /dev/null
+++ b/src/native/libs/System.Globalization.Native/pal_icushim_native_static.c
@@ -0,0 +1,14 @@
+#include <stdio.h>
+#include <unicode/ustring.h>
+#include <unicode/urename.h>
+#include "pal_icushim_native_static.h"
+
+int32_t u_strlen_native_static(const UChar *s)
+{
+#ifdef U_STATIC_IMPLEMENTATION 
+	return u_strlen(s);
+#else
+	(void)s; // Avoid unused param error
+	return -1;
+#endif
+}
diff --git a/src/native/libs/System.Globalization.Native/pal_icushim_native_static.h b/src/native/libs/System.Globalization.Native/pal_icushim_native_static.h
new file mode 100644
index 00000000000..9b8031e94db
--- /dev/null
+++ b/src/native/libs/System.Globalization.Native/pal_icushim_native_static.h
@@ -0,0 +1 @@
+int32_t u_strlen_native_static(const UChar *s);
-- 
2.24.3 (Apple Git-128)

@jkotas
Copy link
Member

jkotas commented Jul 4, 2022

although the runtime build process will have to link with libicu.

This change needs to part of the opt-in as well. Statically linking with libicu would break portability of the default build.

Crossgen has an issue during build,

We compile crossgen using native AOT compiler during the build. I would expect that you should be able to reproduce the same crash by compiling a simple program that uses ICU.

@smhmhmd
Copy link
Contributor

smhmhmd commented Jul 11, 2022

@jkotas

I tested a different change using weak symbols, this change is optin, there is no crash in crossgen, user program works fine.

Each of the symbols in pal_icushim_internal.h would get a weak symbol counterpart. Resolution using weak symbols is used only if user specifies an environment variable asking for static link and the weak symbol has already been resolved by the NativeAOT makefile (Microsoft.NETCore.Native.Unix.targets).

TBD:
More symbols from pal_icu_shiminternal.h need to be added.
The symbol resolution is in the NativeAOT Makefile, need to automate finding the symbol version.
I also need to have stubs for u_charsToUChars_native_static() so they do not look unresolved.

Here is the rough change:

The symbol resolution would look something like this in pal_icushim.c. This is somewhat like the dlsym method currently being used.

+
+    if (u_strlen_native_static) {
+        u_strlen_ptr = u_strlen_native_static;
+    }
+    if (u_getVersion_native_static) {
+        u_getVersion_ptr = u_getVersion_native_static;
+    }
+    if (u_strcmp_native_static) {
+        u_strcmp_ptr = u_strcmp_native_static;
+    }
+    if (u_strlen_native_static) {
+        u_strlen_ptr = u_strlen_native_static;
+    }
+    if (ulocdata_getCLDRVersion_native_static) {
+        ulocdata_getCLDRVersion_ptr = ulocdata_getCLDRVersion_native_static;
+    }
+

For now, I have the versions hardcoded like u_strlen_66, symbol versions will be determined automatically in a PR.

--- a/src/coreclr/nativeaot/BuildIntegration/Microsoft.NETCore.Native.Unix.targets
+++ b/src/coreclr/nativeaot/BuildIntegration/Microsoft.NETCore.Native.Unix.targets
@@ -84,6 +84,12 @@ The .NET Foundation licenses this file to you under the MIT license.
       <LinkerArg Include="-Wl,--build-id=sha1" Condition="'$(TargetOS)' != 'OSX'" />
       <LinkerArg Include="-Wl,--as-needed" Condition="'$(TargetOS)' != 'OSX'" />
       <LinkerArg Include="-pthread" Condition="'$(TargetOS)' != 'OSX'" />
+      <LinkerArg Include=" -Wl,--defsym,u_strlen_native_static=u_strlen_66 " />
+      <LinkerArg Include=" -Wl,--defsym,u_getVersion_native_static=u_getVersion_66 " />
+      <LinkerArg Include=" -Wl,--defsym,u_strcmp_native_static=u_strcmp_66 " />
+      <LinkerArg Include=" -Wl,--defsym,u_strlen_native_static=u_strlen_66 " />
+      <LinkerArg Include=" -Wl,--defsym,ulocdata_getCLDRVersion_native_static=ulocdata_getCLDRVersion_66 " />
+      <LinkerArg Include=" /usr/lib/x86_64-linux-gnu/libicuio.a  /usr/lib/x86_64-linux-gnu/libicutu.a /usr/lib/x86_64-linux-gnu/libicui18n.a  /usr/
lib/x86_64-linux-gnu/libicutest.a  /usr/lib/x86_64-linux-gnu/libicuuc.a -Wl,--whole-archive /usr/lib/x86_64-linux-gnu/libicudata.a  -Wl,--no-whole-a
rchive    " Condition="'$(TargetOS)' != 'OSX'" />

( I can move the weak declarations to a headerfile in a PR)

--- a/src/native/libs/System.Globalization.Native/pal_icushim.c
+++ b/src/native/libs/System.Globalization.Native/pal_icushim.c
 
+__attribute__((__weak__)) void u_charsToUChars_native_static(const char *, UChar *, int32_t);
+__attribute__((__weak__)) void u_getVersion_native_static(UVersionInfo);
+__attribute__((__weak__)) int32_t u_strcmp_native_static(const UChar *,const UChar *);
+__attribute__((__weak__)) int32_t u_strlen_native_static(const UChar *);
+__attribute__((__weak__)) void ulocdata_getCLDRVersion_native_static(UVersionInfo versionArray, UErrorCode *status);
+

@jkotas
Copy link
Member

jkotas commented Jul 11, 2022

Would it be easier to have a separate build of the System_Globalization shim for the static linking?

This shim can be even built on the user machine. The NativeAOT package can ship sources for the shim instead of the .a file to make it possible.

@smhmhmd
Copy link
Contributor

smhmhmd commented Jul 13, 2022

Would it be easier to have a separate build of the System_Globalization shim for the static linking?

Thank you @jkotas, yes, it would be a simpler solution, it would also be opt-in.

Here is a draft change, assuming an already built shim, user program works fine.

User's NativeAOT code compiles with the user-supplied shim and invokes it instead.
User-supplied shim = /p:LocalSystemGlobalizationNative=/home/ubuntu/bug/sample1/libSystem.Globalization.Native.a

User NativeAOT code can get compiled with a switch (LocalSystemGlobalizationNative) like :

../runtime/.dotnet/dotnet publish -c release /p:LocalSystemGlobalizationNative=/home/ubuntu/bug/sample1/libSystem.Globalization.Native.a -r linux-x64 --self-contained -v d

This results in
"clang" "obj/release/net7.0/linux-x64/native/sample1.o" 
-o "bin/release/net7.0/linux-x64/native/sample1"    
...
/home/ubuntu/bug/sample1/libSystem.Globalization.Native.a 
-g -Wl,-rpath,'$ORIGIN' -Wl,--build-id=sha1 -Wl,--as-needed -pthread -lstdc++ -ldl -lm -lz -lrt -pie -Wl,-z,relro -Wl,-z,now -Wl,--discard-all -Wl,--gc-sections

This shim can be even built on the user machine. The NativeAOT package can ship sources for the shim instead of the .a file to make it possible.

This diff uses an already built libSystem.Globalization.Native.a from the sources based on your comment.
diff --git a/src/coreclr/nativeaot/BuildIntegration/Microsoft.NETCore.Native.Unix.targets b/src/coreclr/nativeaot/BuildIntegration/Microsoft.NETCore.Native.Unix.targets
index 9e1a1e633ef..a071bece110 100644
--- a/src/coreclr/nativeaot/BuildIntegration/Microsoft.NETCore.Native.Unix.targets
+++ b/src/coreclr/nativeaot/BuildIntegration/Microsoft.NETCore.Native.Unix.targets
@@ -52,11 +52,11 @@ The .NET Foundation licenses this file to you under the MIT license.
 
     <ItemGroup>
       <NetCoreAppNativeLibrary Include="System.Native" />
-      <NetCoreAppNativeLibrary Include="System.Globalization.Native" />
       <NetCoreAppNativeLibrary Include="System.IO.Compression.Native" />
       <NetCoreAppNativeLibrary Include="System.Net.Security.Native" />
       <NetCoreAppNativeLibrary Include="System.Security.Cryptography.Native.Apple" Condition="'$(TargetOS)' == 'OSX'" />
       <NetCoreAppNativeLibrary Include="System.Security.Cryptography.Native.OpenSsl" Condition="'$(TargetOS)' != 'OSX'" />
+      <NetCoreAppNativeLibrary Include="System.Globalization.Native" Condition="'$(LocalSystemGlobalizationNative)' == ''"/>
     </ItemGroup>
 
     <ItemGroup>
@@ -65,6 +65,7 @@ The .NET Foundation licenses this file to you under the MIT license.
         <EscapedPath>$(IlcFrameworkNativePath)lib%(Identity).a</EscapedPath>
       </NetCoreAppNativeLibrary>
       <NativeLibrary Include="@(NetCoreAppNativeLibrary->'%(EscapedPath)')" />
+      <NativeLibrary Include="$(LocalSystemGlobalizationNative)" Condition="'$(LocalSystemGlobalizationNative)' != ''"/>
     </ItemGroup>

@jkotas
Copy link
Member

jkotas commented Jul 13, 2022

assuming an already built shim

How would people who want to use this build the shim?

@smhmhmd
Copy link
Contributor

smhmhmd commented Jul 17, 2022

@jkotas

How would people who want to use this build the shim?

libSystem.Globalization.Native-Static.a builds outside dotnet, linked with user program as before, it works fine.

Steps:

  • Copied files from libs/System.Globalization.Native to a separate directory ~/bug/libs/System.Globalization.Native
  • Copied src/native/libs/Common to ~/bug/libs/System.Globalization.Native/Common
  • Copied src/native/minipal to ~/bug/libs/System.Globalization.Native/libs-native/minipal
  • Also, libs-native/config.h had to be copied, config.h is pasted below.
  • cd ~/bug/libs/System.Globalization.Native/build && cmake ../ && make
$ ls
CMakeCache.txt  CMakeFiles  Makefile  cmake_install.cmake  libSystem.Globalization.Native-Static.a
$ cat libs-native/config.h 
#pragma once
#define HAVE_UDAT_STANDALONE_SHORTER_WEEKDAYS 1
#define HAVE_SET_MAX_VARIABLE 1

This CMakeLists.txt builds for Ubuntu Linux 20.04, some customizations will be needed for MacOS, Windows and other options.

$ cat CMakeLists.txt 
# CMakeLists files in this project can
# refer to the root source directory of the project as ${SYSTEM_GLOBALIZATION_SOURCE_DIR} and
# to the root binary directory of the project as ${SYSTEM_GLOBALIZATION_BINARY_DIR}.
cmake_minimum_required (VERSION 3.6)
project(System.Globalization.Native.User.Machine.Build C)

set(CMAKE_C_COMPILER /usr/bin/clang)

set(NATIVEGLOBALIZATION_SOURCES
entrypoints.c
pal_calendarData.c
pal_casing.c
pal_collation.c
pal_icushim.c
pal_idna.c
pal_locale.c
pal_localeNumberData.c
pal_localeStringData.c
pal_normalization.c
pal_timeZoneInfo.c
)

add_library(System.Globalization.Native-Static
    STATIC
    ${NATIVEGLOBALIZATION_SOURCES}
)

add_definitions(-D_GNU_SOURCE -g -fPIC -DBUILDENV_DEBUG=1 -DDEBUG -DDISABLE_CONTRACTS -DHOST_64BIT -DHOST_AMD64 -DHOST_UNIX -DPALEXPORT=EXTERN_C -DTARGET_64BIT -DTARGET_AMD64 -DTARGET_LINUX -DTARGET_UNIX -DURTBLDENV_FRIENDLY=Debug -D_DBG -D_DEBUG -D_FILE_OFFSET_BITS=64)

target_compile_options(System.Globalization.Native-Static PUBLIC -Wno-switch-enum -Wno-covered-switch-default -std=gnu99 -Wno-declaration-after-statement -Wall -Wno-null-conversion -fno-omit-frame-pointer -fms-extensions -fwrapv -fstack-protector-strong -Werror -Wno-unused-variable -Wno-unused-value -Wno-unused-function -Wno-tautological-compare -Wno-unknown-pragmas -Wimplicit-fallthrough -Wno-unused-but-set-variable -ffp-contract=off -Wno-unknown-warning-option -ferror-limit=4096 -Wno-unused-private-field -Wno-constant-logical-operand -Wno-pragma-pack -Wno-incompatible-ms-struct -Wno-reserved-identifier -fsigned-char -fvisibility=hidden -Weverything -Wno-format-nonliteral -Wno-disabled-macro-expansion -Wno-padded -Wno-empty-translation-unit -Wno-cast-align -Wno-typedef-redefinition -Wno-c11-extensions -Wno-thread-safety-analysis -Wno-strict-prototypes -Wno-switch-enum -Wno-covered-switch-default -Wno-reserved-id-macro -Wno-documentation -Wno-documentation-unknown-command -Wno-extra-semi-stmt -std=gnu11 -Wno-reserved-id-macro -Wno-documentation -Wno-documentation-unknown-command -Wno-extra-semi-stmt -Wno-unknown-warning-option)

target_include_directories (System.Globalization.Native-Static PRIVATE ${CMAKE_CURRENT_SOURCE_DIR}/Common ${CMAKE_CURRENT_SOURCE_DIR}/native ${CMAKE_CURRENT_SOURCE_DIR}/libs-native/ )

The file listing is:

~/bug/libs/System.Globalization.Native$ ls
CMakeLists.txt  obj                 pal_errors_internal.h           pal_locale.c            pal_normalization.c
Common          pal_calendarData.c  pal_icushim.c                   pal_locale.h            pal_normalization.h
bin             pal_calendarData.h  pal_icushim.h                   pal_locale.m            pal_timeZoneInfo.c
build           pal_casing.c        pal_icushim_internal.h          pal_localeNumberData.c  pal_timeZoneInfo.h
config.h.in     pal_casing.h        pal_icushim_internal_android.h  pal_localeNumberData.h
entrypoints.c   pal_collation.c     pal_icushim_static.c            pal_localeStringData.c
libs-native     pal_collation.h     pal_idna.c                      pal_localeStringData.h
native          pal_errors.h        pal_idna.h                      pal_locale_internal.h

@jkotas
Copy link
Member

jkotas commented Jul 18, 2022

Steps:

I think there is more to these steps. For example, you also need to make sure that you are synced to the same SHA as the rest of NativeAOT toolchain is built from. System.Native shims do not have stable APIs and they need to exactly match the toolchain.

To be usable by broader audiences, these steps would have to be automated and simplified down to setting a msbuild property or something that simple. I believe that the best way to do that would be bundling the shim sources into the native AOT toolchain package and compiling them automatically using C/C++ compiler on the target machine.

@smhmhmd
Copy link
Contributor

smhmhmd commented Jul 18, 2022

I believe that the best way to do that would be bundling the shim sources into the native AOT toolchain package and compiling them automatically using C/C++ compiler on the target machine.

Thanks, I can try bundling the shim sources into the Microsoft.DotNet.ILCompiler package and automatically compiling them using C/C++ compiler on the target machine.
I need to figure out how to extract shim sources during dotnet add Microsoft.DotNet.ILCompiler

@jkotas
Copy link
Member

jkotas commented Jul 18, 2022

dotnet add Microsoft.DotNet.ILCompiler does not allow custom actions like compilation. Compiling of the shim sources using C/C++ compiler would have to be done during dotnet publish /p:PublishAot=true.

@smhmhmd
Copy link
Contributor

smhmhmd commented Jul 20, 2022

@jkotas

I added the shim source code to Microsoft.Dotnet.IL.Compiler.nupkg, but I need some help in doing a publish using the ilc that I built locally.
I tried to follow https://github.com/dotnet/runtime/blob/main/docs/workflow/building/coreclr/nativeaot.md but dotnet restore or publish want to use packages from upstream instead although nuget.config point to local repo


$ ../runtime-1/runtime/dotnet.sh publish
MSBuild version 17.3.0-preview-22306-01+1c045cf58 for .NET
  Determining projects to restore...
/home/ubuntu/bug/sample4/sample4.csproj : error NU1102: Unable to find package runtime.linux-x64.Microsoft.DotNet.ILCompiler with version (= 7.0.0-preview.5.22301.12)
/home/ubuntu/bug/sample4/sample4.csproj : error NU1102:   - Found 1 version(s) in local [ Nearest version: 7.0.0-dev ]

@jkotas
Copy link
Member

jkotas commented Jul 20, 2022

Replied in chat offline - I think you need to explicitly specify the package the -dev reference in your .csproj file:

<ItemGroup>
    <PackageReference Include="Microsoft.DotNet.ILCompiler" Version="7.0.0-dev" />
  </ItemGroup>

@smhmhmd
Copy link
Contributor

smhmhmd commented Jul 21, 2022

@jkotas

Thanks for your prompt responses today in fixing my build.

I believe that the best way to do that would be bundling the shim sources into the native AOT toolchain package and compiling them automatically using C/C++ compiler on the target machine.
Compiling of the shim sources using C/C++ compiler would have to be done during dotnet publish

This patch bundles shim sources into the nativeAOT toolchain and compiles it during dotnet publish, the user code builds and runs fine.
( For now, it uses hard-coded paths in Exec task )
Since there is a new CMakeLists-standalone.txt, we could add new files and defines for static linking with libicu in this cmake file as a next step.
It is still optin, users who want to build shim sources locally can build as follows:

$ cat build.sh 
dotnet publish -r linux-x64 /p:LocalSystemGlobalizationNative=1

This output of the change is further down.

---
 .../Microsoft.NETCore.Native.Unix.targets     |  4 +++
 .../Microsoft.DotNet.ILCompiler.pkgproj       |  3 ++
 .../CMakeLists-standalone.txt                 | 33 +++++++++++++++++++
 .../config-standalone.h                       |  4 +++
 4 files changed, 44 insertions(+)
 create mode 100644 src/native/libs/System.Globalization.Native/CMakeLists-standalone.txt
 create mode 100644 src/native/libs/System.Globalization.Native/config-standalone.h

diff --git a/src/coreclr/nativeaot/BuildIntegration/Microsoft.NETCore.Native.Unix.targets b/src/coreclr/nativeaot/BuildIntegration/Microsoft.NETCore.Native.Unix.targets
index ad6195b98cd..66c4b530fba 100644
--- a/src/coreclr/nativeaot/BuildIntegration/Microsoft.NETCore.Native.Unix.targets
+++ b/src/coreclr/nativeaot/BuildIntegration/Microsoft.NETCore.Native.Unix.targets
@@ -57,6 +57,7 @@ The .NET Foundation licenses this file to you under the MIT license.
       <NetCoreAppNativeLibrary Include="System.Net.Security.Native" />
       <NetCoreAppNativeLibrary Include="System.Security.Cryptography.Native.Apple" Condition="'$(TargetOS)' == 'OSX'" />
       <NetCoreAppNativeLibrary Include="System.Security.Cryptography.Native.OpenSsl" Condition="'$(TargetOS)' != 'OSX'" />
+      <NetCoreAppNativeLibrary Include="System.Globalization.Native" Condition="'$(LocalSystemGlobalizationNative)' == ''"/>
     </ItemGroup>
 
     <ItemGroup>
@@ -65,6 +66,7 @@ The .NET Foundation licenses this file to you under the MIT license.
         <EscapedPath>$(IlcFrameworkNativePath)lib%(Identity).a</EscapedPath>
       </NetCoreAppNativeLibrary>
       <NativeLibrary Include="@(NetCoreAppNativeLibrary->'%(EscapedPath)')" />
+      <NativeLibrary Include="/home/ubuntu/.nuget/packages/microsoft.dotnet.ilcompiler/7.0.0-dev/shim_source_code/libs/System.Globalization.Native/build/libSystem.Globalization.Native-Static.a" Condition="'$(LocalSystemGlobalizationNative)' != ''"/>
     </ItemGroup>
 
     <ItemGroup Condition="'$(TargetOS)' == 'OSX'">
@@ -74,6 +76,7 @@ The .NET Foundation licenses this file to you under the MIT license.
       <NativeFramework Include="GSS" />
     </ItemGroup>
 
+    <Exec Command="if [ -d ~/.nuget/packages/microsoft.dotnet.ilcompiler/7.0.0-dev/shim_source_code ]; then cd ~/.nuget/packages/microsoft.dotnet.ilcompiler/7.0.0-dev/shim_source_code/libs/System.Globalization.Native/  %26%26  cp CMakeLists-standalone.txt CMakeLists.txt %26%26 cp config-standalone.h config.h %26%26 mkdir -p build %26%26 cd build %26%26 cmake ../ %26%26 make -j; fi" />
     <ItemGroup>
       <LinkerArg Include="@(NativeLibrary)" />
       <LinkerArg Include="--sysroot=$(SysRoot)" Condition="'$(SysRoot)' != ''" />
@@ -163,4 +166,5 @@ The .NET Foundation licenses this file to you under the MIT license.
       <DsymUtilOptions>$(DsymUtilOptions) --minimize</DsymUtilOptions>
     </PropertyGroup>
   </Target>
+
 </Project>
diff --git a/src/installer/pkg/projects/Microsoft.DotNet.ILCompiler/Microsoft.DotNet.ILCompiler.pkgproj b/src/installer/pkg/projects/Microsoft.DotNet.ILCompiler/Microsoft.DotNet.ILCompiler.pkgproj
index 49198a0f680..c564732e74c 100644
--- a/src/installer/pkg/projects/Microsoft.DotNet.ILCompiler/Microsoft.DotNet.ILCompiler.pkgproj
+++ b/src/installer/pkg/projects/Microsoft.DotNet.ILCompiler/Microsoft.DotNet.ILCompiler.pkgproj
@@ -21,6 +21,9 @@
       <File Include="$(CoreCLRBuildIntegrationDir)*" TargetPath="build" />
       <File Include="$(CoreCLRILCompilerDir)netstandard\*" TargetPath="tools/netstandard" />
       <File Include="sdk\Sdk.targets" TargetPath="Sdk" />
+      <File Include="..\..\..\..\native\libs\System.Globalization.Native\*" TargetPath="shim_source_code/libs/System.Globalization.Native"/>
+      <File Include="..\..\..\..\native\minipal\*" TargetPath="shim_source_code/minipal"/>
+      <File Include="..\..\..\..\native\libs\Common\*" TargetPath="shim_source_code/libs/Common"/>
     </ItemGroup>
   </Target>
 
diff --git a/src/native/libs/System.Globalization.Native/CMakeLists-standalone.txt b/src/native/libs/System.Globalization.Native/CMakeLists-standalone.txt
new file mode 100644
index 00000000000..c48cc4dde4c
--- /dev/null
+++ b/src/native/libs/System.Globalization.Native/CMakeLists-standalone.txt
@@ -0,0 +1,33 @@
+# CMakeLists files in this project can
+# refer to the root source directory of the project as ${SYSTEM_GLOBALIZATION_SOURCE_DIR} and
+# to the root binary directory of the project as ${SYSTEM_GLOBALIZATION_BINARY_DIR}.
+cmake_minimum_required (VERSION 3.6)
+project(System.Globalization.Native.User.Machine.Build C)
+
+set(CMAKE_C_COMPILER /usr/bin/clang)
+
+set(NATIVEGLOBALIZATION_SOURCES
+entrypoints.c
+pal_calendarData.c
+pal_casing.c
+pal_collation.c
+pal_icushim.c
+pal_idna.c
+pal_locale.c
+pal_localeNumberData.c
+pal_localeStringData.c
+pal_normalization.c
+pal_timeZoneInfo.c
+)
+
+add_library(System.Globalization.Native-Static
+    STATIC
+    ${NATIVEGLOBALIZATION_SOURCES}
+)
+
+add_definitions(-D_GNU_SOURCE -g -fPIC -DBUILDENV_DEBUG=1 -DDEBUG -DDISABLE_CONTRACTS -DHOST_64BIT -DHOST_AMD64 -DHOST_UNIX -DPALEXPORT=EXTERN_C -DTARGET_64BIT -DTARGET_AMD64 -DTARGET_LINUX -DTARGET_UNIX -DURTBLDENV_FRIENDLY=Debug -D_DBG -D_DEBUG -D_FILE_OFFSET_BITS=64)
+
+target_compile_options(System.Globalization.Native-Static PUBLIC -Wno-switch-enum -Wno-covered-switch-default -std=gnu99 -Wno-declaration-after-statement -Wall -Wno-null-conversion -fno-omit-frame-pointer -fms-extensions -fwrapv -fstack-protector-strong -Werror -Wno-unused-variable -Wno-unused-value -Wno-unused-function -Wno-tautological-compare -Wno-unknown-pragmas -Wimplicit-fallthrough -Wno-unused-but-set-variable -ffp-contract=off -Wno-unknown-warning-option -ferror-limit=4096 -Wno-unused-private-field -Wno-constant-logical-operand -Wno-pragma-pack -Wno-incompatible-ms-struct -Wno-reserved-identifier -fsigned-char -fvisibility=hidden -Weverything -Wno-format-nonliteral -Wno-disabled-macro-expansion -Wno-padded -Wno-empty-translation-unit -Wno-cast-align -Wno-typedef-redefinition -Wno-c11-extensions -Wno-thread-safety-analysis -Wno-strict-prototypes -Wno-switch-enum -Wno-covered-switch-default -Wno-reserved-id-macro -Wno-documentation -Wno-documentation-unknown-command -Wno-extra-semi-stmt -std=gnu11 -Wno-reserved-id-macro -Wno-documentation -Wno-documentation-unknown-command -Wno-extra-semi-stmt -Wno-unknown-warning-option)
+
+target_include_directories (System.Globalization.Native-Static PRIVATE ${CMAKE_CURRENT_SOURCE_DIR}/../Common ${CMAKE_CURRENT_SOURCE_DIR} ${CMAKE_CURRENT_SOURCE_DIR}/../../)
+
diff --git a/src/native/libs/System.Globalization.Native/config-standalone.h b/src/native/libs/System.Globalization.Native/config-standalone.h
new file mode 100644
index 00000000000..8f5da9f65ad
--- /dev/null
+++ b/src/native/libs/System.Globalization.Native/config-standalone.h
@@ -0,0 +1,4 @@
+#pragma once
+
+#define HAVE_UDAT_STANDALONE_SHORTER_WEEKDAYS 1
+#define HAVE_SET_MAX_VARIABLE 1
-- 
2.25.1

User code

ubuntu@ip-172-31-24-113:~/bug/sample4$ cat nuget.config                                                               
<?xml version="1.0" encoding="utf-8"?>
<configuration>
  <packageSources>
    <!--To inherit the global NuGet package sources remove the <clear/> line below -->
    <clear />
    <add key="Package source 1" value="/home/ubuntu/bug/runtime-bundle-shim/runtime/artifacts/packages/Debug/Shipping/" />
  </packageSources>
</configuration>
ubuntu@ip-172-31-24-113:~/bug/sample4$ cat sample4.csproj 
<Project Sdk="Microsoft.NET.Sdk">

  <PropertyGroup>
    <OutputType>Exe</OutputType>
    <TargetFramework>net7.0</TargetFramework>
    <ImplicitUsings>enable</ImplicitUsings>
    <Nullable>enable</Nullable>
  </PropertyGroup>

<ItemGroup>
    <PackageReference Include="Microsoft.DotNet.ILCompiler" Version="7.0.0-dev" />
  </ItemGroup>

</Project>

NativeAOT build: The locally built shim libSystem.Globalization.Native.a is in bold below

"clang" "obj/Debug/net7.0/linux-x64/native/sample4.o" -o "bin/Debug/net7.0/linux-x64/native/sample4" /home/ubuntu/.nuget/packages/runtime.linux-x64.microsoft.dotnet.ilcompiler/7.0.0-dev/sdk/libbootstrapper.a /home/ubuntu/.nuget/packages/runtime.linux-x64.microsoft.dotnet.ilcompiler/7.0.0-dev/sdk/libRuntime.WorkstationGC.a /home/ubuntu/.nuget/packages/runtime.linux-x64.microsoft.dotnet.ilcompiler/7.0.0-dev/framework/libSystem.Native.a /home/ubuntu/.nuget/packages/runtime.linux-x64.microsoft.dotnet.ilcompiler/7.0.0-dev/framework/libSystem.Globalization.Native.a /home/ubuntu/.nuget/packages/runtime.linux-x64.microsoft.dotnet.ilcompiler/7.0.0-dev/framework/libSystem.IO.Compression.Native.a /home/ubuntu/.nuget/packages/runtime.linux-x64.microsoft.dotnet.ilcompiler/7.0.0-dev/framework/libSystem.Net.Security.Native.a /home/ubuntu/.nuget/packages/runtime.linux-x64.microsoft.dotnet.ilcompiler/7.0.0-dev/framework/libSystem.Security.Cryptography.Native.OpenSsl.a /home/ubuntu/.nuget/packages/microsoft.dotnet.ilcompiler/7.0.0-dev/shim_source_code/libs/System.Globalization.Native/build/libSystem.Globalization.Native-Static.a -g -Wl,-rpath,'$ORIGIN' -Wl,--build-id=sha1 -Wl,--as-needed -pthread -lstdc++ -ldl -lm -lz -lrt -pie -Wl,-z,relro -Wl,-z,now -Wl,--discard-all -Wl,--gc-sections
Done executing task "Exec"

Output of user code:

ubuntu@ip-172-31-24-113:~/bug/sample4$ bin/Debug/net7.0/linux-x64/native/sample4 
Current culture: 
1235.56
Current culture: en-US
1235.56

@jkotas
Copy link
Member

jkotas commented Jul 21, 2022

This looks reasonable to me. There are details to fix that I will comment on once you post a PR.

@smhmhmd
Copy link
Contributor

smhmhmd commented Jul 26, 2022

This looks reasonable to me. There are details to fix that I will comment on once you post a PR.

Thanks @jkotas

Please review PR at aed8b64
I need your help with the hardcoded 7.0.0-dev in the change.

@Beau-Gosse-dev
Copy link
Contributor Author

Closing this since we're working towards enabling static builds starting with this PR #72896 more details here: https://github.com/dotnet/runtime/blob/main/src/coreclr/nativeaot/docs/compiling.md#using-statically-linked-icu

Eventually, we hope to add other libraries outside of libicu

@ghost ghost locked as resolved and limited conversation to collaborators Oct 7, 2022
Sign up for free to subscribe to this conversation on GitHub. Already have an account? Sign in.
Projects
Archived in project
Development

No branches or pull requests

4 participants