[API Proposal]: Platform-aware Marshal.StringToHGlobalUni equivalent

[Dennis-Petrov]

Background and motivation

As for now, Marshal.StringToHGlobalUni internally calls this:

[System.Security.SecurityCritical]  // auto-generated
internal static unsafe void wstrcpy(char *dmem, char *smem, int charCount)
{
    Buffer.Memcpy((byte*)dmem, (byte*)smem, charCount * 2); // 2 used everywhere instead of sizeof(char)
}

Thus, we have hardcoded UTF-16 here, which works as expected on Windows, but fails on Linux.
In cases of P/Invoke with LPCWSTRs inside structures like this, native code expects UTF-32:

typedef struct _SOME_PARA {
   DWORD dwSize;
   LPCWSTR wszSomeStr;
 } SOME_PARA,
   *PSOME_PARA;

This is unobvious. Also, there are no such details in docs too. One converts string to Unicode, but what Unicode exactly?

API Proposal

public static IntPtr StringToHGlobalPUni(string source)
{
    // use PAL to obtain proper Unicode encoding
}

API Usage

var s = "String to p/invoke";
var sPtr = Marshal.StringToHGlobalPUni(s);
try
{
    var somePara = new SOME_PARA
    {
        dwSize = Marshal.SizeOf<SOME_PARA>(),
        wszSomeStr = sPtr 
    };
    
    // pass somePara to native code    
}
finally
{
    Marshal.FreeHGlobal(sPtr);
}

Alternative Designs

Probably, this will require some changes inside default string marshalling.
I mean, one can use proposed API with this struct:

    [StructLayout(LayoutKind.Sequential)]
    internal struct SOME_PARA
    {
        public uint dwSize;

        public IntPtr wszSomeStr;
    }

but in case of this declaration:

    [StructLayout(LayoutKind.Sequential)]
    internal struct SOME_PARA
    {
        public uint dwSize;

        public string wszSomeStr;
    }

we'll still get UTF-16 because of default marshalling settings.

Risks

No response

[AaronRobinsonMSFT]

@Dennis-Petrov Thanks for filing this issue. .NET uses the term "Unicode" in ambiguous ways, especially in interop scenarios. There are two examples that illustrate this, UnicodeEncoding and CharSet.Unicode. Both document "Unicode" as two-bytes in size. The documentation for the APIs in question should likely be updated to indicate two-bytes as well. I'm not convinced we should be spending the effort to update them to be honest.

For the case above, the .NET team has been working on source generation for struct marshalling – see StructMarshalling.md. We have an issue that is tracking some encoding concerns too – #61326.