COM interop support

[jkotas]

The Native AOT compiler does not support COM interop today. Attempt to use COM interop leads to errors like:

Unhandled Exception: System.PlatformNotSupportedException: COM Interop is not supported on this platform.
   at Internal.Runtime.CompilerHelpers.InteropHelpers.ConvertManagedComInterfaceToNative(Object) + 0x42

The right way to add support for COM interop to Native AOT runtime is combination of:

• Implement ComWrappers in the Native AOT runtime.
  • Add CoreCLR ComWrappers implementation to Native AOT build.
  • Implement CCW - CreateComInterfaceForObject set of APIs
  • Implement RCW - CreateObjectForComInstance set of APIs
  • ComWrappers.RegisterForMarshalling - needed to replace built-in interop
  • ComWrappers.RegisterForTrackerSupport - WinRT-specific lifetime management
• Implement IDynamicInterfaceCastable in the Native AOT runtime. CoreRT had a similar experimental feature before that was reverted in dotnet/corert@5383d46 . This commit provides approximate blue print for what the implementation should look like.
• Implemented support for shared generics with IDynamicInterfaceCastable Support IDynamicInterfaceCastable with shared generic code runtime#72909
• External tool that generates the required COM marshaling stubs. There are several design options for how this tool can work; or there can even be multiple variants of this tool. The key difference between the different options is the source of truth that the tool consumes.
  • Source of truth are COM interface definitions in C#. This option is best for making the existing libraries that use COM interop work (e.g. WinForms or Avalonia). Source generator that is extension of DllImportGenerator may be a good implementation strategy for this option.
  • Source of truth are COM interface definitions in IDL or TLB file. This option is best for creating new libraries that use COM interop. The tool for this case would be conceptually similar to cswinrt tool for WinRT.

dotnet/corert#4219 has more discussion on this topic.

[jkotas]

I am making this up for grabs in case anybody is interested to work on this. We will be happy to provide guidance about details. This is a large project.

[jkoritzinsky]

cc: @kekekeks if you want to extend the MicroCom IDL reader you started working on.

[wjk]

I had an idea on how to implement this. It involves a ComWrappers implementation and a sidecar assembly containing all of the VTables that is generated at publish time from reflection on the assemblies input to ILC. However, I have three issues:

1. ComWrappers is not yet implemented on NativeAOT, and I have no idea whatsoever about how to implement it.
2. How would I handle implementing interfaces that are not visible to the sidecar assembly? (Think FileOpenDialog.FileOpenDialogRCW). I can locate them by their full name, and the vtable classes are generated from reflection over the original, but unless I actually implement the original interface (not a copy), an InvalidCastException will inevitably result. .NET does not do duck typing.
3. There can be only one ComWrappers implementation per program. How would we deal with a program that uses both COM and CsWinRT? (Windows Forms apps distributed in MSIX packages are a good example of this situation.) Is there some way I can delegate calls for objects I don’t handle to CsWinRT’s ComWrappers implementation? And how do we stop CsWinRT from throwing an exception while trying to register their own version?

I would appreciate assistance with point 1, and guidance with points 2 and 3. (Also, the “CoreCLR COM Wrappers” link in the OP is dead. You may want to fix that.) Thanks!

[kekekeks]

@wjk
FYI: we have a generator tool that creates bidirectional COM glue code from IDL-like language, generated code looks like this: https://gist.github.com/kekekeks/ead299514c4d21c26425a48747756fb7 (the source IDL is in the same gist)

[wjk]

@kekekeks Very cool! I’ll borrow use your generator code to emit the COM glue (under MIT). Thanks for the pointer!

In fact, come to think of it, an IDL-like file would probably be a better choice for NativeAOT COM interop than reflecting the assemblies at build time. We would just need to manually generate default IDL files for the COM usage in Windows Forms and WPF, and include them in the package with ILC. For developer-defined COM types, they would need to write the IDL themselves. I looked at the avn.idl file in Avalonia, and it looks just about right for my needs (I may extend it a bit). Thanks again!

[kekekeks]

I was planning to extract that code to a standalone roslyn generator. That way it should be possible to use C# interface definitions as the source too.

Will probably do that during the winter holidays.

[wjk]

Converting the tool to a roslyn generator is a bad idea, as some of the interfaces we need to handle are in libraries we do not control (and therefore cannot recompile with your generator). I praised the use of IDL precisely because it sidestepped this difficult issue.

[kekekeks]

are in libraries we do not control

What exactly does that change? Interop bridges are registered separately, take another look at the generated code. Interfaces are plain C# interfaces.

[wjk]

@kekekeks You are correct. Registration of wrappers do not reference the end-developer-visible interface definition they wrap; it is done purely via GUID. This means that a Roslyn analyzer would be a perfectly viable way to go, as long as we can point it at an already-compiled DLL and have it create wrappers for its contents as well. (I am thinking specifically of Windows Forms and WPF, which both use COM internally. One of my design goals here is to not require the recompilation of those assemblies from source. It’s certainly possible, but very inconvenient.)

[jkotas]

ComWrappers is not yet implemented on NativeAOT, and I have no idea whatsoever about how to implement it.

I have added more granularity above. If you only need COM interop (e.g. for WinForms), you can ignore the special WinRT lifetime management.

guidance with points 2

IDynamicInterfaceCastable depends of default interface method implementations that @MichalStrehovsky has been working on on and off. It can be tacked independently from 1. dotnet/corert@5383d46 should provide good set of hits what needs to be modified to make it work.

How would I handle implementing interfaces that are not visible to the sidecar assembly?

This is a common pain point for source generators, discussed in dotnet/roslyn#11149. There are workarounds, but none of them is pretty:

• Use IL rewriting to inject "InternalsVisibleTo" into assemblies that Roslyn references
• Generate dummy stubs for the interfaces that needs referencing, and then post-process the result to IL-rewrite them away.
• Hack Roslyn internals. Roslyn has an internal option that does it, it is just not exposed via public interface.
• Use low-level IL emitter instead of Roslyn

There can be only one ComWrappers implementation per program.

There can be more than one instance of ComWrappers alive in the process. The ComWrappers API is designed to allow libraries to carry their own local implementation of COM interop marshallers as internal implementation detail.

The WPF shipping in .NET 5 is doing something similar for WinRT interop. There are a few internal implementation details in WPF that need WinRT interop, but we do not want to WPF to depend on the whole CsWinRT. The solution for that is "embedded" CsWinRT. We have internal instance of ComWrappers in WPF that handles the WinRT interop for the set of WinRT interfaces needed by WPF. WPF does not need the special WinRT lifetime management, so a local ComWrappers instance works for it just fine.

There can be only one instance of ComWrappers registered for:

• The special WinRT lifetime management
• The legacy marshalling (e.g. what Marshal.GetComInterfaceForObject calls)

Notice that these two global registrations are independent, so the global WinRT lifetime registration does not prohibit you from registering your side-car assembly.

[kekekeks]

Use IL rewriting to inject "InternalsVisibleTo" into assemblies that Roslyn references

Is IgnoresAccessChecksTo supported by NativeAOT? There is also a generator tool for creating dummy assemblies for Roslyn.

[jkotas]

Is IgnoresAccessChecksTo supported by NativeAOT?

Yes, it is.

[kant2002]

@jkotas is it possible that my previous PR for CoreRT tick at least some ticks?
Should Add CoreCLR ComWrappers implementation use c++ implementation, or I can play with C#. Is moving CoreRT PR is right way to start this task?

[jkotas]

Add CoreCLR ComWrappers implementation use c++ implementation, or I can play with C#.

You can make it work either way. I think using the C++ implementation will be less new code, but it may be uphill battle to get it to build and the result will be less pretty than C# implementation.

Yep, moving over the work you have done against CoreRT repo is a good way to start.

Thank you!

[kant2002]

During previous discussion, I was pointed by @jkoritzinsky to https://github.com/microsoft/CsWinRT/blob/master/src/WinRT.Runtime/ComWrappersSupport.netstandard2.0.cs implementation which is change place from that time. I post link here, so I or anybody else who would like to pursue COM support will have this link around place where actual work happens.

[kant2002]

@jkotas can you please update what's currently implemented? If something require additional work, it would be good to know about that also in such actionable list of items.

[kant2002]

@wjk Status quo is no more. Any issues which are in COM support can be counted as bugs in my opinion, but in general a lot things with COM will works. What I post is specifically to limitations of current ComWrappers implementation.

• Marshal object as IUnknown. See Marshal object as IUnknown #1089
• PreserveSig = false which return non-void results. See Implement handling PreserveSig = false for interfaces #1206
• CoClass support is non-existing. Which means that File dialogs would not work without some efforts from WinForms repo. See Remove CoClass usages from System.Windows.Forms winforms#4971

That's only things which I know so far.
Technically somebody can start play with WPF + ComWrappers and test how it's going. Maybe that would be less problematic.

[jkotas]

@kant2002 What are you using to provide the ComWrappers required by WinForms? Is it all hand-written or auto-generated?

File dialogs would not work without some efforts from WinForms repo.

This can be fixed by replacing new FileOpenDialogRCW() / new FileSaveDialogRCW() by a call to Ole32.CoCreateInstance. WinForms is actually doing that in other places, for example in StringSource.cs.

[hez2010]

@kant2002

Technically somebody can start play with WPF + ComWrappers and test how it's going. Maybe that would be less problematic.

WPF is using C++/CLI to inject code for module setup AFAIK. Unless they use module initializer to rewrite those code, I don't think WPF will work even with COM Wrapper implemented.

[jkotas]

WPF is using C++/CLI for more than that. The module initializer issue is a small problem. The overall WPF dependency on C++/CLI is the big problem.

[kant2002]

For now everything is handwritten. https://github.com/kant2002/CoreRTWinFormsTestBed/blob/master/WinFormsComInterop/WinFormsComWrappers.cs

I have form which have almost all controls so more or less it's working. Right now most things would be in ComWrappers space and not in NativeAOT IMO.

Code generation for ComWrappers is next thing which I will play. But this is that community can also works on.

Will try replacing FileXxxRCW, but I secretly hoping that DLlImportGenerator will reach maturity soon.

[jkotas]

DllImportGenerator won't help you with replacing the coclass RCWs.

[kant2002]

WPF is using C++/CLI to inject code for module setup AFAIK.

@hez2010 That's bad. Anyway ball start rolling and I hope we all know how far WinForms will progress.

[kant2002]

@jkotas I take a look at runtime repository and usage of ComImport attribute,
so far only following assemblies

• System.DirectoryServices
• System.DirectoryServices.AccountManagement
• System.Management
• System.Speech
  Each of this assemblies has about 6-10 classes decorated.

Does this warrant implementation of ComImport/CoClass here, or same trick as in WinForms can be used where newing of objects can be replaced with CoCreateInstanceEx?

[jkotas]

We will need an overall plan for how to make libraries like this to officially work with trimming and without BuiltInComInterop dotnet/runtime#50500. You can start discussion in this issue about the plan, what libraries to address first, etc.

The libraries in your list are among the ones that are not very actively maintained and the test coverage is not great either, so touching them will likely raise question about how we are ensuring that the change is not regressing anything.

[kant2002]

@jkotas does @MichalStrehovsky state of work on IDynamicInterfaceCastable check this box Implement IDynamicInterfaceCastable in the Native AOT runtime?

[jkotas]

dotnet/runtime#72909 tracks the remaining work for IDynamicInterfaceCastable. I have updated the top post to reflect it.

[hez2010]

Btw what's the status of IInspectable now? Does it already work with current COM interop support using COM wrappers?

[jkotas]

IInspectable is just another user-provided interface. The runtime side of COM wrappers does not have any special handling for it. It is provided by e.g. https://github.com/microsoft/CsWinRT/search?q=IInspectable

[hez2010]

Thanks for clarification. So if I read correctly through recent related PRs, does it mean COM interop support has been finished except some edge cases in shared generics with IDynamicInterfaceCastable and external tools which generates COM wrappers code?

[jkotas]

Yes, modulo bugs.

[jkotas]

COM source generator proposal: dotnet/runtime#60143

[MichalStrehovsky]

I think we can close, this was all resolved, there's even the COM source generator now.