Incorrect PInvoke when parameter has base class with no members

[dgomsvek]

Description

PInvoke marshalling is incorrect when there is a parameter whose type has a base class with no members.

I hit this when removing our manual marshaling workaround for #46643, and it feels like the same bug in a different place.

#46643 describes a problem where given:

[StructLayout(LayoutKind.Sequential)]
class MyClassBase { }

[StructLayout(LayoutKind.Sequential)]
class MyClass1: MyClassBase { public int Value; }

The result of Marshal.StructureToPtr<MyClass1>(...) was incorrect, containing a non-collapsed zero-field for the base type.

The fix for that is in net5.0.3, and in testing against 5.0.4, while I do see the fix for Marshal.StructureToPtr<>(), I'm still seeing the same symptoms on a PInvoke parameter:

[DllImport(...)]
static extern void TakeThat(
    MyClassBase pObject,
    UIntPtr sizeOfObject);

Configuration

runtime version: 5.0.4
OS: Windows 20H2 19042.867 64-bit
Arch: x64 PC, 32-bit and 64-bit runtimes
Specific to configuration: unknown

Regression?

net5.0 regression from netfx and netcore 3.1.

The fix for the #46643 is in 5.0.3 PR 46697, but this related issue remains.

Other information

With this native export:

extern "C" N5MNATIVE_API void TakeThat(
	void* pObject,
	size_t sizeOfObject);

And these managed imports:

static class NativeMethods
{
    [DllImport("n5mnative.dll", EntryPoint = "TakeThat", CallingConvention = CallingConvention.Cdecl)]
    public static extern void TakeThatBuffer(
        [In][MarshalAs(UnmanagedType.LPArray)] byte[] pObject,
        [In][MarshalAs(UnmanagedType.SysUInt)] UIntPtr sizeOfObject);

    [DllImport("n5mnative.dll", EntryPoint = "TakeThat", CallingConvention = CallingConvention.Cdecl)]
    public static extern void TakeThatObject(
        [In][MarshalAs(UnmanagedType.LPStruct)] MyClassBase pObject,
        [In][MarshalAs(UnmanagedType.SysUInt)] UIntPtr sizeOfObject);
}

[StructLayout(LayoutKind.Sequential)]
abstract class MyClassBase
{
}

[StructLayout(LayoutKind.Sequential)]
class MyClass1 : MyClassBase
{
    public uint Int2;
}

And this instance:

var myClass1 = new MyClass1
{
    Int2 = 0xA2A2A2A2,
};

Here's the output of some test code on netcoreapp 3.1.13 and net 5.0.4. (net472 is the same as netcoreapp so I won't re-paste.)

Environment.Version: 3.1.13
Environment.Is64BitProcess: False

Bytes from managed Marshal.StructureToPtr<MyClassBase>(myClass1):
        A2 A2 A2 A2

PInvoke NativeMethods.TakeThatBuffer():
TakeThat(): pObject is not nullptr and declared sizeOfObject is 0x00000004.
        A2 A2 A2 A2

PInvoke NativeMethods.TakeThatObject():
TakeThat(): pObject is not nullptr and declared sizeOfObject is 0x00000004.
        A2 A2 A2 A2

PInvoke NativeMethods.TakeThatObject() with size overrun:
TakeThat(): pObject is not nullptr and declared sizeOfObject is 0x00000008.
        A2 A2 A2 A2 00 00 00 00

nvironment.Version: 5.0.4
Environment.Is64BitProcess: False

Bytes from managed Marshal.StructureToPtr<MyClassBase>(myClass1):
        A2 A2 A2 A2

PInvoke NativeMethods.TakeThatBuffer():
TakeThat(): pObject is not nullptr and declared sizeOfObject is 0x00000004.
        A2 A2 A2 A2

PInvoke NativeMethods.TakeThatObject():
TakeThat(): pObject is not nullptr and declared sizeOfObject is 0x00000004.
        00 00 00 00

PInvoke NativeMethods.TakeThatObject() with size overrun:
TakeThat(): pObject is not nullptr and declared sizeOfObject is 0x00000008.
        00 00 00 00 A2 A2 A2 A2

[jkoritzinsky]

The issue here only appears when the parameter is the base class type, but the passed in argument is the derived type.

If you change the parameter type to the derived type, then the PInvoke will behave as expected.

Would that workaround work for your scenario?

[dgomsvek]

In our scenario there are multiple subtypes, any of which could be passed in for any given call. (Think command messages.) Since the object type is not known at compile time, I don't think that workaround could be applied.

We're not blocked though, since we do have a workaround where we manually allocate and marshal with Marshal, then use an IntPtr parameter on the pinvoke. It's a solution that, while only required for net5.0, still works for all our targeted runtimes.

Long-term I would be concerned if a fix for this didn't make it into the 5.0.* servicing line eventually, as it would mean that forever into the future any pinvoke code with polymorphic parameters would need to either continue to use this workaround for all targets or else know to special-case the net5.0 target.