Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

[runtime] Infrastructure to build runtime components using NativeAOT #98565

Merged
merged 85 commits into from
Mar 8, 2024

Conversation

lambdageek
Copy link
Member

@lambdageek lambdageek commented Feb 16, 2024

Motivation: While the .NET runtime itself is perfectly capable of running .NET code, we also provide other components that currently have to be implemented in native code. For example the DAC - its job is to introspect a live CoreCLR process or a memory dump and provide the information to diagnostic tools. The DAC is not a .NET runtime and it exposes a native interface to other tooling. But as we evolve the DAC it might make sense to implement parts of it in C#.

yodawg

Native runtime component libraries using NativeAOT

This PR allows managed libraries to be compiled using NativeAOT and to be used in the runtime using normal CMake idioms.

Adding a new managed library

Add a new subdirectory to src/native/managed for your library with a src and inc subdirectories:

$ mkdir -p libMyNewLibrary/src libMyNewLibrary/inc
$ dotnet new classlib -n libMyNewLibrary -o libMyNewLibrary/src

In src/native/managed/compile-native.proj, add
src/native/managed/libMyNewLibrary/src/libMyNewLibrary.csproj to the NativeLibsProjectsToBuild
item group.

In src/native/managed/libMyNewLibrary/src/libMyNewLibrary.csproj:

  1. Near the top, add <Import Project="..\..\native-library.props" />
  2. Near the bottom, add <Import Project="..\..\native-library.targets" />
  3. Define an item @(InstallRuntimeComponentDest) that has directory names relative to artifacts/bin/<runtimeFlavor>/<os.arch.config>/ where the shared library should be installed. It's a good idea to have at least .:
      <ItemGroup>
          <InstallRuntimeComponentDest Include="." />
          <InstallRuntimeComponentDest Include="sharedFramework" Condition="'$(RuntimeFlavor)' == 'coreclr'"/>
      </ItemGroup>

Limitations:

  • The project should be called libXXXX - currently the infrastructure expects a lib prefix on all platforms.
  • Currently only shared library output is supported. In principle static linking is possible, but the
    infrastructure is not finished yet. Additionally, mixing Debug/Release configurations with static
    linking will not be supported on Windows.

Ideally we'd like to be able to produce shared or static libraries. Currently only shared libraries really work robustly (in that only the exported native symbols of the C# library are exposed and the NativeAOT implementation details are completely hidden).

There was an attempt at doing static linking too, but it needs a bit more work on Linux/Mac (using -r to create relocatable objects in conjunction with -Wl,--exclude-libs or -Wl,-hidden-l to hide the symbols of the NativeAOT runtime). Doing naïve static linking results in duplicate GC and eventpipe symbols (since CoreCLR and NativeAOT both define them). Static linking on Windows probably isn't worthwhile due to #98356 and because I'm not sure if hiding symbols will be possible.

There was some work to have cmake integration: runtime could do find_nativeaot_library(libWhatever [REQUIRED]) and get two cmake targets: libWhatever::libs and libWhatever::headers that could be used for linking with other targets. The current usecase doesn't need this level of integration however, so the cmake stuff was moved to this branch: https://github.com/lambdageek/runtime/tree/naot-runtime-lib-with-cmake

@ghost
Copy link

ghost commented Feb 16, 2024

Tagging subscribers to this area: @dotnet/area-infrastructure-libraries
See info in area-owners.md if you want to be subscribed.

Issue Details

Motivation: While the .NET runtime itself is perfectly capable of running .NET code, we also provide other components that currently have to be implemented in native code. For example the DAC - its job is to introspect a live CoreCLR process or a memory dump and provide the information to diagnostic tools. The DAC is not a .NET runtime and it exposes a native interface to other tooling. But as we evolve the DAC it might make sense to implement parts of it in C#.

yodawg

This PR: we provide two things: an example, and the CMake and MSBuild infrastructure for defining src/native components that are actually written in C# that will be compiled using NativeAOT into native code which can in turn be consumed by src/coreclr or src/mono (although currently only the coreclr cmake integration is done).

Ideally we'd like to be able to produce shared or static libraries. Currently only shared libraries really work robustly (in that only the exported native symbols of the C# library are exposed and the NativeAOT implementation details are completely hidden).

There's an attempt at doing static linking too, but it needs a bit more work on Linux/Mac (using -r to create relocatable objects in conjunction with -Wl,--exclude-libs or -Wl,-hidden-l to hide the symbols of the NativeAOT runtime). Doing naïve static linking results in duplicate GC and eventpipe symbols (since CoreCLR and NativeAOT both define them). Static linking on Windows probably isn't worthwhile due to #98356 and because I'm not sure if hiding symbols will be possible.

The example is just a library that writes a message to the console when mscordac is loaded. We shouldn't merge that bit.

Possible improvements before this is ready to merge:

  • we need to produce a .dylib.dwarf, .so.dbg, or .pdb symbol file for the imported library like the rest of the build
  • consider making a FindPackage dependency provider and generate a cmake config-package (cmake: Using Dependencies Guide for each library instead of doing simple include files.
  • Annoyingly, cmake doesn't consider the modification time of IMPORTED libraries when deciding whether to rebuild a target. So if you change a managed library and then do build.sh clr it won't necessarily pick up the changes. Ithink it might be possible to fix it up by adding some custom commands that make it seem like the imported libraries are actually generated by the cmake build
Author: lambdageek
Assignees: lambdageek
Labels:

area-Infrastructure-libraries

Milestone: -

@lambdageek lambdageek added area-VM-coreclr NO-MERGE The PR is not ready for merge yet (see discussion for detailed reasons) and removed area-Infrastructure-libraries labels Feb 16, 2024
@lambdageek

This comment has been minimized.

@lambdageek
Copy link
Member Author

I'm going to remove the example now. I've seen it build with the latest changes and there's nothing interesting in the example itself anymore. I think I'm ready for another round of reviews

@lambdageek
Copy link
Member Author

I think I want to remove the libFoo::headers imported target. for the cdac work we end up using dlopen/LoadLibrary to get at the nativeaot'd lib, so there's not a build-time dependency between the shared library and the code that consumes it. In fact it simplifies the build dependencies a bit if locating the include files doesn't depend on AOTing the library - and also allows the crossdacs to consume the cdac reader (which is, by design, debuggee platform agnostic)

@jkoritzinsky
Copy link
Member

I think I want to remove the libFoo::headers imported target. for the cdac work we end up using dlopen/LoadLibrary to get at the nativeaot'd lib, so there's not a build-time dependency between the shared library and the code that consumes it. In fact it simplifies the build dependencies a bit if locating the include files doesn't depend on AOTing the library - and also allows the crossdacs to consume the cdac reader (which is, by design, debuggee platform agnostic)

I think I agree here. If we exclusively reference the exports lazily through dlopen/LoadLibrary, then we don't need to have the header target.

If we were to link (dynamically or statically) against the lib, then I think we should expose the header as part of the same target that has the lib so linking against one CMake target would pull everything in needed to use that target.

In fact, if we are going to reference at runtime with dlopen/LoadLibrary, we don't need the full CMake integration at all. We only need one (imported?) target that adds the include directory for the header that has the function pointer types for dynamic interaction.

@lambdageek
Copy link
Member Author

In fact, if we are going to reference at runtime with dlopen/LoadLibrary, we don't need the full CMake integration at all.

yea, that's true. I'll move the cmake stuff to a separate branch/PR and keep it on the back burner until we have a need for it.

If we were to link (dynamically or statically) against the lib, then I think we should expose the header as part of the same target that has the lib so linking against one CMake target would pull everything in needed to use that target.

I'm not sure I totally agree. Particularly for static linking it's sometimes helpful to only have a single target that depends on the libs, while multiple targets depend on the headers. In any case, that can be hashed out later - when we have a need for the cmake stuff.

@lambdageek
Copy link
Member Author

Removed cmake integration. Verified that the cdac stuff still builds.

Copy link
Member

@jkoritzinsky jkoritzinsky left a comment

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

A few comments to reduce the MSBuild logic and reduce the number of steps to set up a new project.

src/native/managed/README.md Outdated Show resolved Hide resolved
src/native/managed/README.md Outdated Show resolved Hide resolved
src/native/managed/compile-native.proj Outdated Show resolved Hide resolved
src/native/managed/compile-native.proj Outdated Show resolved Hide resolved
Copy link
Member

@jkoritzinsky jkoritzinsky left a comment

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

LGTM!

@lambdageek lambdageek merged commit db8f9eb into dotnet:main Mar 8, 2024
181 checks passed
@lambdageek lambdageek deleted the naot-runtime-lib branch March 8, 2024 14:15
elinor-fung added a commit that referenced this pull request Apr 9, 2024
…or copying to runtime output (#100782)

The InstallRuntimeComponentToFinalDestination target from infrastructure for NativeAOT-ed runtime components (#98565) copies the component and symbol files to the runtime output directory. Make the target use NativeOutputPath for the for the library and symbol path instead of the ones copied to the publish directory. They should be available after LinkNative.

This was working in the runtime repo because the projects under src/native/managed are currently using the ILCompiler package corresponding to the SDK being used to build the repo. That version of the package doesn't have #98342 - the native-library.targets were relying the symbols already being copied before _CopyAotSymbols was run.
@github-actions github-actions bot locked and limited conversation to collaborators Apr 9, 2024
Sign up for free to subscribe to this conversation on GitHub. Already have an account? Sign in.
Projects
None yet
Development

Successfully merging this pull request may close these issues.

5 participants