From db05d7fd10659ae3c4e4ab015c7e55abac18dd3a Mon Sep 17 00:00:00 2001 From: Aaron R Robinson Date: Thu, 12 Feb 2026 11:24:12 -0800 Subject: [PATCH 01/12] feat: add dotnet-pinvoke skill (initial draft) --- skills/Pinvokes/SKILL.md | 461 +++++++++++++++++++++++++++++++++++++++ 1 file changed, 461 insertions(+) create mode 100644 skills/Pinvokes/SKILL.md diff --git a/skills/Pinvokes/SKILL.md b/skills/Pinvokes/SKILL.md new file mode 100644 index 0000000000..a40518e747 --- /dev/null +++ b/skills/Pinvokes/SKILL.md @@ -0,0 +1,461 @@ +--- +name: dotnet-pinvoke +description: Correctly call native (C/C++) libraries from .NET using P/Invoke and LibraryImport. Covers function signatures, string marshalling, memory lifetime, SafeHandle, and cross-platform patterns. Use when writing or reviewing any managed-to-native boundary code. +--- + +# .NET P/Invoke + +Calling native code from .NET is powerful but unforgiving. Incorrect signatures, garbled strings, and leaked or accessing freed memory are three of the most common sources of bugs; all of them can manifest as intermittent crashes, silent data corruption, or access violations that appear far from the actual defect. + +This skill covers both `DllImport` (available since .NET Framework 1.0) and `LibraryImport` (source-generated, .NET 7+). Both are covered equally because many codebases target older TFMs or must maintain existing `DllImport` declarations. When targeting .NET Framework, always use `DllImport`. When targeting .NET 7+, prefer `LibraryImport` for new code. When native AOT is a requirement, `LibraryImport` is the only option. + +## When to Use + +- Writing new P/Invoke or `LibraryImport` declarations +- Reviewing or debugging existing native interop code +- Wrapping a C or C++ library for use in .NET +- Diagnosing crashes, memory leaks, or corruption at the managed/native boundary + +## When Not to Use + +- COM interop (different lifetime and threading model) +- C++/CLI mixed-mode assemblies (avoid in new code; C# interop is faster and more portable) +- Pure managed code with no native dependencies + +## Inputs + +| Input | Required | Description | +|-------|----------|-------------| +| Native header or documentation | Yes | C/C++ function signatures, struct definitions, calling conventions | +| Target framework | Yes | Determines whether to use `DllImport` or `LibraryImport` | +| Target platforms | Recommended | Affects type sizes (`long`, `size_t`) and library naming | +| Memory ownership contract | Yes | Who allocates and who frees each buffer or handle | + +--- + +## Workflow + +### Step 1: Choose DllImport or LibraryImport + +| Aspect | `DllImport` | `LibraryImport` (.NET 7+) | +|--------|-------------|---------------------------| +| **Mechanism** | Runtime marshalling | Source generator (compile-time) | +| **AOT / Trim safe** | No | Yes | +| **String marshalling** | `CharSet` enum | `StringMarshalling` enum | +| **Error handling** | `SetLastError` | `SetLastPInvokeError` | +| **Availability** | .NET Framework 1.0+ | .NET 7+ only | + +Use `LibraryImport` for new code on .NET 7+. Use `DllImport` for .NET Framework, .NET Standard, or earlier .NET Core. + +### Step 2: Map Native Types to .NET Types + +This is where most bugs originate. Every parameter must match exactly. + +| C / Win32 Type | .NET Type | Notes | +|----------------|-----------|-------| +| `int` | `int` | Always 32-bit in Win32 ABI | +| `int32_t` | `int` | | +| `uint32_t` | `uint` | | +| `int64_t` | `long` | | +| `uint64_t` | `ulong` | | +| `HRESULT` | `int` | Some tools project this as an enumeration | +| `long` | **`CLong`** | C `long` is 32-bit on Windows, 64-bit on 64-bit Unix — never use `int` or `long`. With `LibraryImport`, requires `[assembly: DisableRuntimeMarshalling]` or you get SYSLIB1051. With `DllImport`, works without it | +| `size_t` | `nuint` | Pointer-sized. Never use `ulong` | +| `intptr_t` | `nint` | Pointer-sized | +| `BOOL` (Win32) | `int` | Not `bool` — Win32 `BOOL` is 4 bytes | +| `bool` (C99) | `[MarshalAs(UnmanagedType.U1)] bool` | Must specify 1-byte marshal | +| `HANDLE`, `HWND` | `SafeHandle` | Prefer over raw `IntPtr` | +| `LPWSTR` / `wchar_t*` | `string` | Must specify UTF-16 encoding | +| `LPSTR` / `char*` | `string` | Must specify ANSI or UTF-8 encoding | +| `void*` | `void*` | | +| `DWORD` | `uint` | | + +### Step 3: Write the Declaration + +Given a C header: + +```c +int32_t process_records(const Record* records, size_t count, uint32_t* out_processed); +``` + +**DllImport:** + +```csharp +[DllImport("mylib")] +private static extern int ProcessRecords( + [In] Record[] records, nuint count, out uint outProcessed); +``` + +**LibraryImport:** + +```csharp +[LibraryImport("mylib")] +internal static partial int ProcessRecords( + [In] Record[] records, nuint count, out uint outProcessed); +``` + +Calling conventions only need to be specified when targeting Windows x86 (32-bit), where `Cdecl` and `StdCall` differ. On x64, ARM, and ARM64, there is a single calling convention and the attribute is unnecessary. + +**Agent behavior:** If you detect that Windows x86 is a target — through project properties (e.g., `x86`), runtime identifiers (e.g., `win-x86`), build scripts, comments, or developer instructions — flag this to the developer and recommend explicit calling conventions on all P/Invoke declarations. + +```csharp +// DllImport (x86 targets) +[DllImport("mylib", CallingConvention = CallingConvention.Cdecl)] + +// LibraryImport (x86 targets) +[LibraryImport("mylib")] +[UnmanagedCallConv(CallConvs = [typeof(CallConvCdecl)])] +``` + +### Step 4: Handle Strings Correctly + +1. **Know what encoding the native function expects.** There is no safe default. +2. **Windows APIs:** Always call the `W` (UTF-16) variant when the function expects a wide string. The `A` variant needs a specific reason and explicit ANSI encoding. The `A` also supports UTF-8 on Windows 10 1903+ if the system code page is UTF-8, but relying on that is fragile and not recommended. +3. **Cross-platform C libraries:** Usually expect UTF-8. +4. **Specify encoding explicitly.** Never rely on `CharSet.Auto`. +5. **Never introduce `StringBuilder` for output buffers.** It has poor performance semantics and is not suitable for general-purpose string buffers. + +```csharp +// DllImport — Windows API (UTF-16) +[DllImport("kernel32.dll", CharSet = CharSet.Unicode, SetLastError = true)] +private static extern int GetModuleFileNameW( + IntPtr hModule, [Out] char[] filename, int size); + +// DllImport — Cross-platform C library (UTF-8) +[DllImport("mylib")] +private static extern int SetName( + [MarshalAs(UnmanagedType.LPUTF8Str)] string name); + +// LibraryImport — UTF-16 +[LibraryImport("kernel32", StringMarshalling = StringMarshalling.Utf16, + SetLastPInvokeError = true)] +internal static partial int GetModuleFileNameW( + IntPtr hModule, [Out] char[] filename, int size); + +// LibraryImport — UTF-8 +[LibraryImport("mylib", StringMarshalling = StringMarshalling.Utf8)] +internal static partial int SetName(string name); +``` + +**String lifetime warning:** Marshalled strings are freed after the call returns. If native code stores the pointer (instead of copying), you must agree on the allocator and the lifetime must be manually managed. On Windows or when targeting .NET Framework the COM related `CoTaskMemAlloc`/`CoTaskMemFree` should be the first choice for cross-boundary ownership, but the library may have its own allocator that must be used instead. On non-Windows target, using the `NativeMemory` APIs are the best option for cross-boundary ownership. + +### Step 5: Establish Memory Ownership + +When memory crosses the boundary, exactly one side must own it — and both sides must agree. + +**Model 1 — Caller allocates, caller frees (safest):** + +```csharp +[LibraryImport("mylib")] +private static partial int GetName( + Span buffer, nuint bufferSize, out nuint actualSize); + +public static string GetName() +{ + Span buffer = stackalloc byte[256]; + int result = GetName(buffer, (nuint)buffer.Length, out nuint actualSize); + if (result != 0) throw new InvalidOperationException($"Failed: {result}"); + return Encoding.UTF8.GetString(buffer[..(int)actualSize]); +} +``` + +**Model 2 — Callee allocates, caller frees (common in Win32):** + +```csharp +[LibraryImport("mylib")] +private static partial IntPtr GetVersion(); +[LibraryImport("mylib")] +private static partial void FreeString(IntPtr s); + +public static string GetVersion() +{ + IntPtr ptr = GetVersion(); + try { return Marshal.PtrToStringUTF8(ptr) ?? throw new InvalidOperationException(); } + finally { FreeString(ptr); } // Must use the library's own free function +} +``` + +**Critical rule:** Always free with the matching allocator. Never use `Marshal.FreeHGlobal` or `Marshal.FreeCoTaskMem` on `malloc`'d memory — they use different heaps. + +**Model 3 — Handle-based (callee allocates, callee frees):** Use `SafeHandle` (see Step 6). + +**Pinning managed objects** — when native code stores the pointer or runs asynchronously: + +```csharp +// Synchronous: use fixed +public static unsafe void ProcessSync(byte[] data) +{ + fixed (byte* ptr = data) { ProcessData(ptr, (nuint)data.Length); } +} + +// Asynchronous: use GCHandle +var gcHandle = GCHandle.Alloc(data, GCHandleType.Pinned); +// Must keep pinned until native processing completes, then call gcHandle.Free() +``` + +### Step 6: Use SafeHandle for Native Handles + +Raw `IntPtr` leaks on exceptions and has no double-free protection. `SafeHandle` is non-negotiable. + +```csharp +internal sealed class MyLibHandle : SafeHandleZeroOrMinusOneIsInvalid +{ + private MyLibHandle() : base(ownsHandle: true) { } + + [LibraryImport("mylib", StringMarshalling = StringMarshalling.Utf8)] + private static partial MyLibHandle CreateHandle(string config); + + [LibraryImport("mylib")] + private static partial int UseHandle(MyLibHandle h, ReadOnlySpan data, nuint len); + + [LibraryImport("mylib")] + private static partial void DestroyHandle(IntPtr h); + + protected override bool ReleaseHandle() { DestroyHandle(handle); return true; } + + public static MyLibHandle Create(string config) + { + var h = CreateHandle(config); + if (h.IsInvalid) throw new InvalidOperationException("Failed to create handle"); + return h; + } + + public int Use(ReadOnlySpan data) => UseHandle(this, data, (nuint)data.Length); +} + +// Usage: SafeHandle is IDisposable +using var handle = MyLibHandle.Create("config=value"); +int result = handle.Use(myData); +``` + +### Step 7: Handle Errors + +```csharp +// Win32 APIs — check SetLastError +[LibraryImport("kernel32", SetLastPInvokeError = true)] +[return: MarshalAs(UnmanagedType.Bool)] +internal static partial bool CloseHandle(IntPtr hObject); + +if (!CloseHandle(handle)) + throw new Win32Exception(Marshal.GetLastPInvokeError()); + +// HRESULT APIs +int hr = NativeDoWork(context); +Marshal.ThrowExceptionForHR(hr); +``` + +### Step 8: Handle Callbacks (if needed) + +**Preferred (.NET 5+): `UnmanagedCallersOnly`** — avoids delegates entirely, so there is no GC lifetime risk: + +```csharp +// C: typedef void (*log_callback)(int level, const char* message); +// C: void set_log_callback(log_callback cb); + +[UnmanagedCallersOnly] +private static void LogCallback(int level, IntPtr message) +{ + string msg = Marshal.PtrToStringUTF8(message) ?? string.Empty; + Console.WriteLine($"[{level}] {msg}"); +} + +// Pass the function pointer directly — no delegate, no GC concern +[LibraryImport("mylib")] +private static unsafe partial void SetLogCallback( + delegate* unmanaged cb); + +// Usage: +unsafe { SetLogCallback(&LogCallback); } +``` + +The method must be `static`, must not throw exceptions back to native code, and can only use blittable parameter types. `UnmanagedCallersOnly` methods cannot be called from managed code directly. + +**Fallback (older TFMs or when instance state is needed): delegate with rooting** + +```csharp +// C: typedef void (*log_callback)(int level, const char* message); +[UnmanagedFunctionPointer(CallingConvention.Cdecl)] // Only needed on Windows x86 +private delegate void LogCallbackDelegate(int level, IntPtr message); + +// CRITICAL: prevent delegate from being garbage collected +private static LogCallbackDelegate? s_logCallback; + +public static void EnableLogging(Action handler) +{ + s_logCallback = (level, msgPtr) => + { + string msg = Marshal.PtrToStringUTF8(msgPtr) ?? string.Empty; + handler(level, msg); + }; + SetLogCallback(s_logCallback); +} +``` + +If native code stores the function pointer, the delegate **must** stay rooted for its entire lifetime. A collected delegate means a crash. + +--- + +## Blittable Structs + +Blittable types have identical managed and native layouts — zero marshalling overhead. + +**Blittable:** `byte`, `sbyte`, `short`, `ushort`, `int`, `uint`, `long`, `ulong`, `float`, `double`, `nint`, `nuint`, and structs of only blittable fields. With `[assembly: DisableRuntimeMarshalling]`, `bool` (1 byte) and `char` (2 bytes, `char16_t`) are also treated as blittable. + +**Not blittable (without `DisableRuntimeMarshalling`):** `bool`, `char`, `string`, `decimal`, anything with `MarshalAs`. + +```csharp +[StructLayout(LayoutKind.Sequential)] +internal struct Vec3 { public float X, Y, Z; } + +[LibraryImport("physics")] +internal static partial void TransformVectors(Span vectors, nuint count, in Vec3 t); +``` + +### Explicit Layout (Unions) + +```csharp +// C: typedef union { int32_t i; float f; } Value; +[StructLayout(LayoutKind.Explicit, Size = 4)] +internal struct Value +{ + [FieldOffset(0)] public int I; + [FieldOffset(0)] public float F; +} +``` + +### Packing + +If the native struct uses non-default packing, match it: + +```csharp +// C: #pragma pack(push, 1) +[StructLayout(LayoutKind.Sequential, Pack = 1)] +internal struct PackedHeader +{ + public byte Magic; + public uint Size; // At offset 1, not 4 + public ushort Flags; // At offset 5, not 8 +} +``` + +--- + +## Cross-Platform Library Loading + +Use `NativeLibrary.SetDllImportResolver` for complex scenarios, or conditional compilation for simple cases. Use `CLong`/`CULong` for C `long`/`unsigned long` — the size differs between Windows (32-bit) and 64-bit Unix (64-bit). Note: `CLong`/`CULong` with `LibraryImport` requires `[assembly: DisableRuntimeMarshalling]`; with `DllImport` this is not needed. + +```csharp +// Simple: conditional compilation +#if WINDOWS + private const string LibName = "mylib.dll"; +#elif LINUX + private const string LibName = "libmylib.so"; +#elif MACOS + private const string LibName = "libmylib.dylib"; +#endif + +// Complex: runtime resolver +NativeLibrary.SetDllImportResolver(typeof(MyLib).Assembly, + (name, assembly, searchPath) => + { + if (name != "mylib") return IntPtr.Zero; + string libName = RuntimeInformation.IsOSPlatform(OSPlatform.Windows) + ? "mylib.dll" + : RuntimeInformation.IsOSPlatform(OSPlatform.OSX) + ? "libmylib.dylib" : "libmylib.so"; + NativeLibrary.TryLoad(libName, assembly, searchPath, out var handle); + return handle; + }); +``` + +--- + +## Migrating DllImport to LibraryImport + +For codebases targeting .NET 7+, migrating provides AOT compatibility and trimming safety. + +1. Add `partial` to the containing class and make the method `static partial` +2. Replace `[DllImport]` with `[LibraryImport]` +3. Replace `CharSet` with `StringMarshalling` +4. Replace `SetLastError = true` with `SetLastPInvokeError = true` +5. Remove `CallingConvention` unless targeting Windows x86 +6. Build and fix `SYSLIB1054`–`SYSLIB1057` analyzer warnings + +Enable the interop analyzers in your project: + +```xml + + true + true + +``` + +--- + +## Tooling + +### CsWin32 (Win32 APIs) + +For Win32 P/Invoke, prefer [Microsoft.Windows.CsWin32](https://github.com/microsoft/CsWin32) over hand-written signatures. It source-generates correct, `LibraryImport`-compatible declarations from metadata — eliminating the most common signature bugs. + +```bash +dotnet add package Microsoft.Windows.CsWin32 +``` + +Create a `NativeMethods.txt` file in your project root listing the APIs you need, one per line: + +```text +CreateFile +ReadFile +CloseHandle +``` + +The generator produces correct signatures including `SafeHandle` wrappers, correct struct layouts, and proper `SetLastError` usage. No manual type mapping required. + +### CsWinRT (WinRT APIs) + +For WinRT interop, use [Microsoft.Windows.CsWinRT](https://github.com/microsoft/CsWinRT). It generates .NET projections from Windows Runtime metadata (`.winmd` files), providing type-safe access to WinRT APIs without manual interop code. + +```bash +dotnet add package Microsoft.Windows.CsWinRT +``` + +--- + +## Validation + +- [ ] Every signature matches the native header exactly (types, sizes) +- [ ] Calling convention specified if targeting Windows x86; omitted otherwise +- [ ] String encoding is explicit — no reliance on defaults or `CharSet.Auto` +- [ ] Memory ownership is documented and matched (who allocates, who frees, with what) +- [ ] `SafeHandle` used for all native handles (no raw `IntPtr` escaping the interop layer) +- [ ] Delegates passed as callbacks are rooted to prevent GC collection +- [ ] `SetLastError`/`SetLastPInvokeError` set for APIs that use OS error codes +- [ ] Struct layout matches native (packing, alignment, field order) +- [ ] `CLong`/`CULong` used for C `long`/`unsigned long` in cross-platform code +- [ ] If using `CLong`/`CULong` with `LibraryImport`, `[assembly: DisableRuntimeMarshalling]` is applied +- [ ] No `bool` without explicit `MarshalAs` + +## Common Pitfalls + +| Pitfall | Impact | Solution | +|---------|--------|----------| +| `int` for `size_t` | Stack corruption on 64-bit | Use `nuint` | +| `long` for C `long` | Wrong on Windows (32-bit) | Use `CLong` / `CULong` (with `LibraryImport`, requires `[assembly: DisableRuntimeMarshalling]`) | +| `bool` without `MarshalAs` | Wrong marshal size | Specify `UnmanagedType.Bool` (4B) or `U1` (1B) | +| Implicit string encoding | Corrupts non-ASCII | Always specify `CharSet` or `StringMarshalling` | +| Wrong allocator for free | Heap corruption | Use the library's own free function | +| Raw `IntPtr` for handles | Leaks on exception | Use `SafeHandle` subclass | +| Delegate callback GC'd | Crash in native code | Keep a rooted reference for the delegate's lifetime | +| Missing `SetLastError` | Stale error codes | Set `SetLastError = true` on Win32 APIs | +| Struct packing mismatch | Fields at wrong offsets | Match `Pack` to native `#pragma pack` | +| Managed object as `void*` | Object moves during GC | Pin with `GCHandle` or `fixed` | + +## Resources + +- [P/Invoke](https://learn.microsoft.com/en-us/dotnet/standard/native-interop/pinvoke) +- [LibraryImport source generation](https://learn.microsoft.com/en-us/dotnet/standard/native-interop/pinvoke-source-generation) +- [Type marshalling](https://learn.microsoft.com/en-us/dotnet/standard/native-interop/type-marshalling) +- [SafeHandle](https://learn.microsoft.com/en-us/dotnet/api/system.runtime.interopservices.safehandle) +- [NativeLibrary](https://learn.microsoft.com/en-us/dotnet/api/system.runtime.interopservices.nativelibrary) +- [Best practices](https://learn.microsoft.com/en-us/dotnet/standard/native-interop/best-practices) From c926ae46745cadac1b008cd1990867b5e1788eaf Mon Sep 17 00:00:00 2001 From: Aaron R Robinson Date: Thu, 12 Feb 2026 11:24:50 -0800 Subject: [PATCH 02/12] refactor: rename Pinvokes folder to dotnet-pinvoke to match frontmatter and README conventions --- skills/{Pinvokes => dotnet-pinvoke}/SKILL.md | 0 1 file changed, 0 insertions(+), 0 deletions(-) rename skills/{Pinvokes => dotnet-pinvoke}/SKILL.md (100%) diff --git a/skills/Pinvokes/SKILL.md b/skills/dotnet-pinvoke/SKILL.md similarity index 100% rename from skills/Pinvokes/SKILL.md rename to skills/dotnet-pinvoke/SKILL.md From 2b407a2246b996d6443d1e5f6f48b69ecb46147a Mon Sep 17 00:00:00 2001 From: Aaron R Robinson Date: Thu, 12 Feb 2026 11:28:00 -0800 Subject: [PATCH 03/12] docs: add failure modes and recovery section per CONTRIBUTING.md checklist --- skills/dotnet-pinvoke/SKILL.md | 18 ++++++++++++++++++ 1 file changed, 18 insertions(+) diff --git a/skills/dotnet-pinvoke/SKILL.md b/skills/dotnet-pinvoke/SKILL.md index a40518e747..1904d316ac 100644 --- a/skills/dotnet-pinvoke/SKILL.md +++ b/skills/dotnet-pinvoke/SKILL.md @@ -451,6 +451,24 @@ dotnet add package Microsoft.Windows.CsWinRT | Struct packing mismatch | Fields at wrong offsets | Match `Pack` to native `#pragma pack` | | Managed object as `void*` | Object moves during GC | Pin with `GCHandle` or `fixed` | +## Failure Modes and Recovery + +| Symptom | Likely Cause | Diagnosis | +|---------|-------------|-----------| +| `DllNotFoundException` | Library not found at runtime | Check library name, path, and platform. Use `NativeLibrary.TryLoad` to test loading manually. On Linux, verify `LD_LIBRARY_PATH` or `rpath`. | +| `EntryPointNotFoundException` | Export name mismatch | Inspect the native binary's export table (`dumpbin /exports` on Windows, `nm -D` on Linux). Check for name mangling (C++ without `extern "C"`). | +| `AccessViolationException` | Signature mismatch, use-after-free, or missing pinning | Compare managed and native signatures byte-for-byte. Check struct sizes with `Marshal.SizeOf()` vs native `sizeof`. Verify memory lifetime. | +| Silent data corruption | Wrong type size or encoding | Add temporary logging at the boundary. Compare `Marshal.SizeOf()` to native struct size. Test with known input/output pairs. | +| Intermittent crashes | GC moved an unpinned object or collected a delegate | Ensure callbacks are rooted. Use `GCHandle` or `fixed` for any pointer held across calls. Run under a debugger with managed debugging assistants (MDAs) enabled. | +| Heap corruption on free | Wrong allocator | Confirm which allocator the native side used and free with the matching function. Never mix `malloc`/`free` with `CoTaskMemAlloc`/`CoTaskMemFree` or `Marshal.FreeHGlobal`. | + +**General debugging approach:** + +1. Reproduce under a debugger with native and managed debugging enabled +2. On .NET 5+, set `COMPlus_EnableDiagnostics=1` and use dotnet-dump or dotnet-trace for post-mortem analysis +3. Verify struct layout: `Marshal.SizeOf()` must equal the native `sizeof` for every struct crossing the boundary +4. (.NET Framework only) Enable [Managed Debugging Assistants](https://learn.microsoft.com/en-us/dotnet/framework/debug-trace-profile/diagnosing-errors-with-managed-debugging-assistants) (MDAs) for `pInvokeStackImbalance` and `invalidOverlappedToPinvoke` + ## Resources - [P/Invoke](https://learn.microsoft.com/en-us/dotnet/standard/native-interop/pinvoke) From 5e9cddd5d0e89c98a438ce3f4eed11ffeefce0d0 Mon Sep 17 00:00:00 2001 From: Aaron R Robinson Date: Thu, 12 Feb 2026 11:29:27 -0800 Subject: [PATCH 04/12] docs: split validation into review checklist and runnable steps --- skills/dotnet-pinvoke/SKILL.md | 13 +++++++++++++ 1 file changed, 13 insertions(+) diff --git a/skills/dotnet-pinvoke/SKILL.md b/skills/dotnet-pinvoke/SKILL.md index 1904d316ac..2ea6388655 100644 --- a/skills/dotnet-pinvoke/SKILL.md +++ b/skills/dotnet-pinvoke/SKILL.md @@ -424,6 +424,8 @@ dotnet add package Microsoft.Windows.CsWinRT ## Validation +### Review checklist + - [ ] Every signature matches the native header exactly (types, sizes) - [ ] Calling convention specified if targeting Windows x86; omitted otherwise - [ ] String encoding is explicit — no reliance on defaults or `CharSet.Auto` @@ -436,6 +438,17 @@ dotnet add package Microsoft.Windows.CsWinRT - [ ] If using `CLong`/`CULong` with `LibraryImport`, `[assembly: DisableRuntimeMarshalling]` is applied - [ ] No `bool` without explicit `MarshalAs` +### Runnable validation steps + +1. **Build with interop analyzers enabled** — confirm zero `SYSLIB1054`–`SYSLIB1057` warnings: + ```xml + true + true + ``` +2. **Verify struct sizes match** — for every struct crossing the boundary, assert `Marshal.SizeOf()` equals the native `sizeof` +3. **Round-trip test** — call the native function with known inputs and verify expected outputs +4. **Test with non-ASCII strings** — pass strings containing characters outside the ASCII range to confirm encoding is correct + ## Common Pitfalls | Pitfall | Impact | Solution | From cc5eec67ffee15aff8761c4329e88332d680a155 Mon Sep 17 00:00:00 2001 From: Aaron R Robinson Date: Thu, 12 Feb 2026 11:30:26 -0800 Subject: [PATCH 05/12] docs: qualify bool/MarshalAs checklist item for DisableRuntimeMarshalling --- skills/dotnet-pinvoke/SKILL.md | 2 +- 1 file changed, 1 insertion(+), 1 deletion(-) diff --git a/skills/dotnet-pinvoke/SKILL.md b/skills/dotnet-pinvoke/SKILL.md index 2ea6388655..59cad5d1f7 100644 --- a/skills/dotnet-pinvoke/SKILL.md +++ b/skills/dotnet-pinvoke/SKILL.md @@ -436,7 +436,7 @@ dotnet add package Microsoft.Windows.CsWinRT - [ ] Struct layout matches native (packing, alignment, field order) - [ ] `CLong`/`CULong` used for C `long`/`unsigned long` in cross-platform code - [ ] If using `CLong`/`CULong` with `LibraryImport`, `[assembly: DisableRuntimeMarshalling]` is applied -- [ ] No `bool` without explicit `MarshalAs` +- [ ] No `bool` without explicit `MarshalAs` (unless `DisableRuntimeMarshalling` is applied, where `bool` is blittable as 1 byte and `MarshalAs` is unnecessary) ### Runnable validation steps From bde6d50ef2530fa5eeb0d58abd936ff3cfac12c1 Mon Sep 17 00:00:00 2001 From: Aaron R Robinson Date: Thu, 12 Feb 2026 12:01:07 -0800 Subject: [PATCH 06/12] docs: clarify OS preprocessor symbols require OS-specific TFMs --- skills/dotnet-pinvoke/SKILL.md | 3 +++ 1 file changed, 3 insertions(+) diff --git a/skills/dotnet-pinvoke/SKILL.md b/skills/dotnet-pinvoke/SKILL.md index 59cad5d1f7..342b9b69fb 100644 --- a/skills/dotnet-pinvoke/SKILL.md +++ b/skills/dotnet-pinvoke/SKILL.md @@ -346,6 +346,9 @@ Use `NativeLibrary.SetDllImportResolver` for complex scenarios, or conditional c ```csharp // Simple: conditional compilation +// WINDOWS, LINUX, MACOS are predefined only when targeting an OS-specific TFM +// (e.g., net8.0-windows). For portable TFMs (e.g., net8.0), these symbols are +// not defined — use the runtime resolver approach below instead. #if WINDOWS private const string LibName = "mylib.dll"; #elif LINUX From a778d1cc30305ba9e6355ce2f3406d30223e115e Mon Sep 17 00:00:00 2001 From: Aaron R Robinson Date: Thu, 12 Feb 2026 12:02:29 -0800 Subject: [PATCH 07/12] docs: add EntryPoint property guidance for native export name mismatches --- skills/dotnet-pinvoke/SKILL.md | 14 ++++++++++++++ 1 file changed, 14 insertions(+) diff --git a/skills/dotnet-pinvoke/SKILL.md b/skills/dotnet-pinvoke/SKILL.md index 342b9b69fb..1a6e202c1f 100644 --- a/skills/dotnet-pinvoke/SKILL.md +++ b/skills/dotnet-pinvoke/SKILL.md @@ -107,6 +107,20 @@ Calling conventions only need to be specified when targeting Windows x86 (32-bit [UnmanagedCallConv(CallConvs = [typeof(CallConvCdecl)])] ``` +If the managed method name differs from the native export name, specify `EntryPoint` to avoid `EntryPointNotFoundException`: + +```csharp +// DllImport +[DllImport("mylib", EntryPoint = "process_records")] +private static extern int ProcessRecords( + [In] Record[] records, nuint count, out uint outProcessed); + +// LibraryImport +[LibraryImport("mylib", EntryPoint = "process_records")] +internal static partial int ProcessRecords( + [In] Record[] records, nuint count, out uint outProcessed); +``` + ### Step 4: Handle Strings Correctly 1. **Know what encoding the native function expects.** There is no safe default. From f8384ed20de36ea11ef362dc86242f5f1caf7900 Mon Sep 17 00:00:00 2001 From: Aaron R Robinson Date: Thu, 12 Feb 2026 13:17:47 -0800 Subject: [PATCH 08/12] refactor: apply progressive disclosure pattern - trim SKILL.md, extract references - Expand frontmatter description with trigger info (when to use / not use) - Remove redundant When to Use / When Not to Use body sections - Trim type mapping to dangerous-only entries inline; full table in references/type-mapping.md - Move blittable structs, common pitfalls, failure modes, resources to references/ - Reduce SKILL.md from ~510 to 285 lines --- skills/dotnet-pinvoke/SKILL.md | 174 +++--------------- .../dotnet-pinvoke/references/diagnostics.md | 43 +++++ .../dotnet-pinvoke/references/type-mapping.md | 73 ++++++++ 3 files changed, 140 insertions(+), 150 deletions(-) create mode 100644 skills/dotnet-pinvoke/references/diagnostics.md create mode 100644 skills/dotnet-pinvoke/references/type-mapping.md diff --git a/skills/dotnet-pinvoke/SKILL.md b/skills/dotnet-pinvoke/SKILL.md index 1a6e202c1f..61478ffa28 100644 --- a/skills/dotnet-pinvoke/SKILL.md +++ b/skills/dotnet-pinvoke/SKILL.md @@ -1,26 +1,13 @@ --- name: dotnet-pinvoke -description: Correctly call native (C/C++) libraries from .NET using P/Invoke and LibraryImport. Covers function signatures, string marshalling, memory lifetime, SafeHandle, and cross-platform patterns. Use when writing or reviewing any managed-to-native boundary code. +description: Correctly call native (C/C++) libraries from .NET using P/Invoke and LibraryImport. Covers function signatures, string marshalling, memory lifetime, SafeHandle, and cross-platform patterns. Use when (1) writing new P/Invoke or LibraryImport declarations, (2) reviewing or debugging existing native interop code, (3) wrapping a C or C++ library for use in .NET, or (4) diagnosing crashes, memory leaks, or corruption at the managed/native boundary. Do not use for COM interop, C++/CLI mixed-mode assemblies, or pure managed code with no native dependencies. --- # .NET P/Invoke -Calling native code from .NET is powerful but unforgiving. Incorrect signatures, garbled strings, and leaked or accessing freed memory are three of the most common sources of bugs; all of them can manifest as intermittent crashes, silent data corruption, or access violations that appear far from the actual defect. +Calling native code from .NET is powerful but unforgiving. Incorrect signatures, garbled strings, and leaked or freed memory are the most common sources of bugs — all can manifest as intermittent crashes, silent data corruption, or access violations far from the actual defect. -This skill covers both `DllImport` (available since .NET Framework 1.0) and `LibraryImport` (source-generated, .NET 7+). Both are covered equally because many codebases target older TFMs or must maintain existing `DllImport` declarations. When targeting .NET Framework, always use `DllImport`. When targeting .NET 7+, prefer `LibraryImport` for new code. When native AOT is a requirement, `LibraryImport` is the only option. - -## When to Use - -- Writing new P/Invoke or `LibraryImport` declarations -- Reviewing or debugging existing native interop code -- Wrapping a C or C++ library for use in .NET -- Diagnosing crashes, memory leaks, or corruption at the managed/native boundary - -## When Not to Use - -- COM interop (different lifetime and threading model) -- C++/CLI mixed-mode assemblies (avoid in new code; C# interop is faster and more portable) -- Pure managed code with no native dependencies +This skill covers both `DllImport` (available since .NET Framework 1.0) and `LibraryImport` (source-generated, .NET 7+). When targeting .NET Framework, always use `DllImport`. When targeting .NET 7+, prefer `LibraryImport` for new code. When native AOT is a requirement, `LibraryImport` is the only option. ## Inputs @@ -45,30 +32,19 @@ This skill covers both `DllImport` (available since .NET Framework 1.0) and `Lib | **Error handling** | `SetLastError` | `SetLastPInvokeError` | | **Availability** | .NET Framework 1.0+ | .NET 7+ only | -Use `LibraryImport` for new code on .NET 7+. Use `DllImport` for .NET Framework, .NET Standard, or earlier .NET Core. - ### Step 2: Map Native Types to .NET Types -This is where most bugs originate. Every parameter must match exactly. - -| C / Win32 Type | .NET Type | Notes | -|----------------|-----------|-------| -| `int` | `int` | Always 32-bit in Win32 ABI | -| `int32_t` | `int` | | -| `uint32_t` | `uint` | | -| `int64_t` | `long` | | -| `uint64_t` | `ulong` | | -| `HRESULT` | `int` | Some tools project this as an enumeration | -| `long` | **`CLong`** | C `long` is 32-bit on Windows, 64-bit on 64-bit Unix — never use `int` or `long`. With `LibraryImport`, requires `[assembly: DisableRuntimeMarshalling]` or you get SYSLIB1051. With `DllImport`, works without it | +The most dangerous mappings — these cause the majority of bugs: + +| C / Win32 Type | .NET Type | Why | +|----------------|-----------|-----| +| `long` | **`CLong`** | 32-bit on Windows, 64-bit on 64-bit Unix. With `LibraryImport`, requires `[assembly: DisableRuntimeMarshalling]` | | `size_t` | `nuint` | Pointer-sized. Never use `ulong` | -| `intptr_t` | `nint` | Pointer-sized | | `BOOL` (Win32) | `int` | Not `bool` — Win32 `BOOL` is 4 bytes | | `bool` (C99) | `[MarshalAs(UnmanagedType.U1)] bool` | Must specify 1-byte marshal | | `HANDLE`, `HWND` | `SafeHandle` | Prefer over raw `IntPtr` | -| `LPWSTR` / `wchar_t*` | `string` | Must specify UTF-16 encoding | -| `LPSTR` / `char*` | `string` | Must specify ANSI or UTF-8 encoding | -| `void*` | `void*` | | -| `DWORD` | `uint` | | + +**For the complete type mapping table, struct layout, and blittable type rules**, see [references/type-mapping.md](references/type-mapping.md). ### Step 3: Write the Declaration @@ -124,10 +100,10 @@ internal static partial int ProcessRecords( ### Step 4: Handle Strings Correctly 1. **Know what encoding the native function expects.** There is no safe default. -2. **Windows APIs:** Always call the `W` (UTF-16) variant when the function expects a wide string. The `A` variant needs a specific reason and explicit ANSI encoding. The `A` also supports UTF-8 on Windows 10 1903+ if the system code page is UTF-8, but relying on that is fragile and not recommended. +2. **Windows APIs:** Always call the `W` (UTF-16) variant. The `A` variant needs a specific reason and explicit ANSI encoding. 3. **Cross-platform C libraries:** Usually expect UTF-8. 4. **Specify encoding explicitly.** Never rely on `CharSet.Auto`. -5. **Never introduce `StringBuilder` for output buffers.** It has poor performance semantics and is not suitable for general-purpose string buffers. +5. **Never introduce `StringBuilder` for output buffers.** ```csharp // DllImport — Windows API (UTF-16) @@ -151,7 +127,7 @@ internal static partial int GetModuleFileNameW( internal static partial int SetName(string name); ``` -**String lifetime warning:** Marshalled strings are freed after the call returns. If native code stores the pointer (instead of copying), you must agree on the allocator and the lifetime must be manually managed. On Windows or when targeting .NET Framework the COM related `CoTaskMemAlloc`/`CoTaskMemFree` should be the first choice for cross-boundary ownership, but the library may have its own allocator that must be used instead. On non-Windows target, using the `NativeMemory` APIs are the best option for cross-boundary ownership. +**String lifetime warning:** Marshalled strings are freed after the call returns. If native code stores the pointer (instead of copying), the lifetime must be manually managed. On Windows or .NET Framework, `CoTaskMemAlloc`/`CoTaskMemFree` is the first choice for cross-boundary ownership; on non-Windows targets, use `NativeMemory` APIs. The library may have its own allocator that must be used instead. ### Step 5: Establish Memory Ownership @@ -189,7 +165,7 @@ public static string GetVersion() } ``` -**Critical rule:** Always free with the matching allocator. Never use `Marshal.FreeHGlobal` or `Marshal.FreeCoTaskMem` on `malloc`'d memory — they use different heaps. +**Critical rule:** Always free with the matching allocator. Never use `Marshal.FreeHGlobal` or `Marshal.FreeCoTaskMem` on `malloc`'d memory. **Model 3 — Handle-based (callee allocates, callee frees):** Use `SafeHandle` (see Step 6). @@ -260,12 +236,9 @@ Marshal.ThrowExceptionForHR(hr); ### Step 8: Handle Callbacks (if needed) -**Preferred (.NET 5+): `UnmanagedCallersOnly`** — avoids delegates entirely, so there is no GC lifetime risk: +**Preferred (.NET 5+): `UnmanagedCallersOnly`** — avoids delegates entirely, no GC lifetime risk: ```csharp -// C: typedef void (*log_callback)(int level, const char* message); -// C: void set_log_callback(log_callback cb); - [UnmanagedCallersOnly] private static void LogCallback(int level, IntPtr message) { @@ -273,21 +246,18 @@ private static void LogCallback(int level, IntPtr message) Console.WriteLine($"[{level}] {msg}"); } -// Pass the function pointer directly — no delegate, no GC concern [LibraryImport("mylib")] private static unsafe partial void SetLogCallback( delegate* unmanaged cb); -// Usage: unsafe { SetLogCallback(&LogCallback); } ``` -The method must be `static`, must not throw exceptions back to native code, and can only use blittable parameter types. `UnmanagedCallersOnly` methods cannot be called from managed code directly. +The method must be `static`, must not throw exceptions back to native code, and can only use blittable parameter types. **Fallback (older TFMs or when instance state is needed): delegate with rooting** ```csharp -// C: typedef void (*log_callback)(int level, const char* message); [UnmanagedFunctionPointer(CallingConvention.Cdecl)] // Only needed on Windows x86 private delegate void LogCallbackDelegate(int level, IntPtr message); @@ -309,54 +279,9 @@ If native code stores the function pointer, the delegate **must** stay rooted fo --- -## Blittable Structs - -Blittable types have identical managed and native layouts — zero marshalling overhead. - -**Blittable:** `byte`, `sbyte`, `short`, `ushort`, `int`, `uint`, `long`, `ulong`, `float`, `double`, `nint`, `nuint`, and structs of only blittable fields. With `[assembly: DisableRuntimeMarshalling]`, `bool` (1 byte) and `char` (2 bytes, `char16_t`) are also treated as blittable. - -**Not blittable (without `DisableRuntimeMarshalling`):** `bool`, `char`, `string`, `decimal`, anything with `MarshalAs`. - -```csharp -[StructLayout(LayoutKind.Sequential)] -internal struct Vec3 { public float X, Y, Z; } - -[LibraryImport("physics")] -internal static partial void TransformVectors(Span vectors, nuint count, in Vec3 t); -``` - -### Explicit Layout (Unions) - -```csharp -// C: typedef union { int32_t i; float f; } Value; -[StructLayout(LayoutKind.Explicit, Size = 4)] -internal struct Value -{ - [FieldOffset(0)] public int I; - [FieldOffset(0)] public float F; -} -``` - -### Packing - -If the native struct uses non-default packing, match it: - -```csharp -// C: #pragma pack(push, 1) -[StructLayout(LayoutKind.Sequential, Pack = 1)] -internal struct PackedHeader -{ - public byte Magic; - public uint Size; // At offset 1, not 4 - public ushort Flags; // At offset 5, not 8 -} -``` - ---- - ## Cross-Platform Library Loading -Use `NativeLibrary.SetDllImportResolver` for complex scenarios, or conditional compilation for simple cases. Use `CLong`/`CULong` for C `long`/`unsigned long` — the size differs between Windows (32-bit) and 64-bit Unix (64-bit). Note: `CLong`/`CULong` with `LibraryImport` requires `[assembly: DisableRuntimeMarshalling]`; with `DllImport` this is not needed. +Use `NativeLibrary.SetDllImportResolver` for complex scenarios, or conditional compilation for simple cases. Use `CLong`/`CULong` for C `long`/`unsigned long`. Note: `CLong`/`CULong` with `LibraryImport` requires `[assembly: DisableRuntimeMarshalling]`. ```csharp // Simple: conditional compilation @@ -398,7 +323,7 @@ For codebases targeting .NET 7+, migrating provides AOT compatibility and trimmi 5. Remove `CallingConvention` unless targeting Windows x86 6. Build and fix `SYSLIB1054`–`SYSLIB1057` analyzer warnings -Enable the interop analyzers in your project: +Enable the interop analyzers: ```xml @@ -413,29 +338,15 @@ Enable the interop analyzers in your project: ### CsWin32 (Win32 APIs) -For Win32 P/Invoke, prefer [Microsoft.Windows.CsWin32](https://github.com/microsoft/CsWin32) over hand-written signatures. It source-generates correct, `LibraryImport`-compatible declarations from metadata — eliminating the most common signature bugs. +For Win32 P/Invoke, prefer [Microsoft.Windows.CsWin32](https://github.com/microsoft/CsWin32) over hand-written signatures. It source-generates correct declarations from metadata. Add a `NativeMethods.txt` listing the APIs you need: ```bash dotnet add package Microsoft.Windows.CsWin32 ``` -Create a `NativeMethods.txt` file in your project root listing the APIs you need, one per line: - -```text -CreateFile -ReadFile -CloseHandle -``` - -The generator produces correct signatures including `SafeHandle` wrappers, correct struct layouts, and proper `SetLastError` usage. No manual type mapping required. - ### CsWinRT (WinRT APIs) -For WinRT interop, use [Microsoft.Windows.CsWinRT](https://github.com/microsoft/CsWinRT). It generates .NET projections from Windows Runtime metadata (`.winmd` files), providing type-safe access to WinRT APIs without manual interop code. - -```bash -dotnet add package Microsoft.Windows.CsWinRT -``` +For WinRT interop, use [Microsoft.Windows.CsWinRT](https://github.com/microsoft/CsWinRT) to generate .NET projections from `.winmd` files. --- @@ -466,44 +377,7 @@ dotnet add package Microsoft.Windows.CsWinRT 3. **Round-trip test** — call the native function with known inputs and verify expected outputs 4. **Test with non-ASCII strings** — pass strings containing characters outside the ASCII range to confirm encoding is correct -## Common Pitfalls - -| Pitfall | Impact | Solution | -|---------|--------|----------| -| `int` for `size_t` | Stack corruption on 64-bit | Use `nuint` | -| `long` for C `long` | Wrong on Windows (32-bit) | Use `CLong` / `CULong` (with `LibraryImport`, requires `[assembly: DisableRuntimeMarshalling]`) | -| `bool` without `MarshalAs` | Wrong marshal size | Specify `UnmanagedType.Bool` (4B) or `U1` (1B) | -| Implicit string encoding | Corrupts non-ASCII | Always specify `CharSet` or `StringMarshalling` | -| Wrong allocator for free | Heap corruption | Use the library's own free function | -| Raw `IntPtr` for handles | Leaks on exception | Use `SafeHandle` subclass | -| Delegate callback GC'd | Crash in native code | Keep a rooted reference for the delegate's lifetime | -| Missing `SetLastError` | Stale error codes | Set `SetLastError = true` on Win32 APIs | -| Struct packing mismatch | Fields at wrong offsets | Match `Pack` to native `#pragma pack` | -| Managed object as `void*` | Object moves during GC | Pin with `GCHandle` or `fixed` | - -## Failure Modes and Recovery - -| Symptom | Likely Cause | Diagnosis | -|---------|-------------|-----------| -| `DllNotFoundException` | Library not found at runtime | Check library name, path, and platform. Use `NativeLibrary.TryLoad` to test loading manually. On Linux, verify `LD_LIBRARY_PATH` or `rpath`. | -| `EntryPointNotFoundException` | Export name mismatch | Inspect the native binary's export table (`dumpbin /exports` on Windows, `nm -D` on Linux). Check for name mangling (C++ without `extern "C"`). | -| `AccessViolationException` | Signature mismatch, use-after-free, or missing pinning | Compare managed and native signatures byte-for-byte. Check struct sizes with `Marshal.SizeOf()` vs native `sizeof`. Verify memory lifetime. | -| Silent data corruption | Wrong type size or encoding | Add temporary logging at the boundary. Compare `Marshal.SizeOf()` to native struct size. Test with known input/output pairs. | -| Intermittent crashes | GC moved an unpinned object or collected a delegate | Ensure callbacks are rooted. Use `GCHandle` or `fixed` for any pointer held across calls. Run under a debugger with managed debugging assistants (MDAs) enabled. | -| Heap corruption on free | Wrong allocator | Confirm which allocator the native side used and free with the matching function. Never mix `malloc`/`free` with `CoTaskMemAlloc`/`CoTaskMemFree` or `Marshal.FreeHGlobal`. | - -**General debugging approach:** - -1. Reproduce under a debugger with native and managed debugging enabled -2. On .NET 5+, set `COMPlus_EnableDiagnostics=1` and use dotnet-dump or dotnet-trace for post-mortem analysis -3. Verify struct layout: `Marshal.SizeOf()` must equal the native `sizeof` for every struct crossing the boundary -4. (.NET Framework only) Enable [Managed Debugging Assistants](https://learn.microsoft.com/en-us/dotnet/framework/debug-trace-profile/diagnosing-errors-with-managed-debugging-assistants) (MDAs) for `pInvokeStackImbalance` and `invalidOverlappedToPinvoke` - -## Resources - -- [P/Invoke](https://learn.microsoft.com/en-us/dotnet/standard/native-interop/pinvoke) -- [LibraryImport source generation](https://learn.microsoft.com/en-us/dotnet/standard/native-interop/pinvoke-source-generation) -- [Type marshalling](https://learn.microsoft.com/en-us/dotnet/standard/native-interop/type-marshalling) -- [SafeHandle](https://learn.microsoft.com/en-us/dotnet/api/system.runtime.interopservices.safehandle) -- [NativeLibrary](https://learn.microsoft.com/en-us/dotnet/api/system.runtime.interopservices.nativelibrary) -- [Best practices](https://learn.microsoft.com/en-us/dotnet/standard/native-interop/best-practices) +## Reference Files + +- **[references/type-mapping.md](references/type-mapping.md)** — Complete native-to-.NET type mapping table, struct layout patterns, blittable type rules +- **[references/diagnostics.md](references/diagnostics.md)** — Common pitfalls, failure modes and recovery, debugging approach, external resources diff --git a/skills/dotnet-pinvoke/references/diagnostics.md b/skills/dotnet-pinvoke/references/diagnostics.md new file mode 100644 index 0000000000..0da1a6d9fd --- /dev/null +++ b/skills/dotnet-pinvoke/references/diagnostics.md @@ -0,0 +1,43 @@ +# P/Invoke Diagnostics + +## Common Pitfalls + +| Pitfall | Impact | Solution | +|---------|--------|----------| +| `int` for `size_t` | Stack corruption on 64-bit | Use `nuint` | +| `long` for C `long` | Wrong on Windows (32-bit) | Use `CLong` / `CULong` (with `LibraryImport`, requires `[assembly: DisableRuntimeMarshalling]`) | +| `bool` without `MarshalAs` | Wrong marshal size | Specify `UnmanagedType.Bool` (4B) or `U1` (1B) | +| Implicit string encoding | Corrupts non-ASCII | Always specify `CharSet` or `StringMarshalling` | +| Wrong allocator for free | Heap corruption | Use the library's own free function | +| Raw `IntPtr` for handles | Leaks on exception | Use `SafeHandle` subclass | +| Delegate callback GC'd | Crash in native code | Keep a rooted reference for the delegate's lifetime | +| Missing `SetLastError` | Stale error codes | Set `SetLastError = true` on Win32 APIs | +| Struct packing mismatch | Fields at wrong offsets | Match `Pack` to native `#pragma pack` | +| Managed object as `void*` | Object moves during GC | Pin with `GCHandle` or `fixed` | + +## Failure Modes and Recovery + +| Symptom | Likely Cause | Diagnosis | +|---------|-------------|-----------| +| `DllNotFoundException` | Library not found at runtime | Check library name, path, and platform. Use `NativeLibrary.TryLoad` to test loading manually. On Linux, verify `LD_LIBRARY_PATH` or `rpath`. | +| `EntryPointNotFoundException` | Export name mismatch | Inspect the native binary's export table (`dumpbin /exports` on Windows, `nm -D` on Linux). Check for name mangling (C++ without `extern "C"`). | +| `AccessViolationException` | Signature mismatch, use-after-free, or missing pinning | Compare managed and native signatures byte-for-byte. Check struct sizes with `Marshal.SizeOf()` vs native `sizeof`. Verify memory lifetime. | +| Silent data corruption | Wrong type size or encoding | Add temporary logging at the boundary. Compare `Marshal.SizeOf()` to native struct size. Test with known input/output pairs. | +| Intermittent crashes | GC moved an unpinned object or collected a delegate | Ensure callbacks are rooted. Use `GCHandle` or `fixed` for any pointer held across calls. Run under a debugger with managed debugging assistants (MDAs) enabled. | +| Heap corruption on free | Wrong allocator | Confirm which allocator the native side used and free with the matching function. Never mix `malloc`/`free` with `CoTaskMemAlloc`/`CoTaskMemFree` or `Marshal.FreeHGlobal`. | + +## General Debugging Approach + +1. Reproduce under a debugger with native and managed debugging enabled +2. On .NET 5+, set `COMPlus_EnableDiagnostics=1` and use dotnet-dump or dotnet-trace for post-mortem analysis +3. Verify struct layout: `Marshal.SizeOf()` must equal the native `sizeof` for every struct crossing the boundary +4. (.NET Framework only) Enable [Managed Debugging Assistants](https://learn.microsoft.com/en-us/dotnet/framework/debug-trace-profile/diagnosing-errors-with-managed-debugging-assistants) (MDAs) for `pInvokeStackImbalance` and `invalidOverlappedToPinvoke` + +## Resources + +- [P/Invoke](https://learn.microsoft.com/en-us/dotnet/standard/native-interop/pinvoke) +- [LibraryImport source generation](https://learn.microsoft.com/en-us/dotnet/standard/native-interop/pinvoke-source-generation) +- [Type marshalling](https://learn.microsoft.com/en-us/dotnet/standard/native-interop/type-marshalling) +- [SafeHandle](https://learn.microsoft.com/en-us/dotnet/api/system.runtime.interopservices.safehandle) +- [NativeLibrary](https://learn.microsoft.com/en-us/dotnet/api/system.runtime.interopservices.nativelibrary) +- [Best practices](https://learn.microsoft.com/en-us/dotnet/standard/native-interop/best-practices) diff --git a/skills/dotnet-pinvoke/references/type-mapping.md b/skills/dotnet-pinvoke/references/type-mapping.md new file mode 100644 index 0000000000..2ed750b9a6 --- /dev/null +++ b/skills/dotnet-pinvoke/references/type-mapping.md @@ -0,0 +1,73 @@ +# Native-to-.NET Type Mapping + +Complete reference for mapping C/Win32 types to .NET types. Every parameter must match exactly — this is where most P/Invoke bugs originate. + +## Primitive Types + +| C / Win32 Type | .NET Type | Notes | +|----------------|-----------|-------| +| `int` | `int` | Always 32-bit in Win32 ABI | +| `int32_t` | `int` | | +| `uint32_t` | `uint` | | +| `int64_t` | `long` | | +| `uint64_t` | `ulong` | | +| `DWORD` | `uint` | | +| `HRESULT` | `int` | Some tools project this as an enumeration | +| `float` | `float` | | +| `double` | `double` | | + +## Dangerous Types (Most Common Bug Sources) + +These types have non-obvious mappings that frequently cause bugs: + +| C / Win32 Type | .NET Type | Why It's Dangerous | +|----------------|-----------|-------------------| +| `long` | **`CLong`** | C `long` is 32-bit on Windows, 64-bit on 64-bit Unix — never use `int` or `long`. With `LibraryImport`, requires `[assembly: DisableRuntimeMarshalling]` or you get SYSLIB1051. With `DllImport`, works without it | +| `size_t` | `nuint` | Pointer-sized. Never use `ulong` — causes stack corruption on 32-bit | +| `intptr_t` | `nint` | Pointer-sized | +| `BOOL` (Win32) | `int` | Not `bool` — Win32 `BOOL` is 4 bytes | +| `bool` (C99) | `[MarshalAs(UnmanagedType.U1)] bool` | Must specify 1-byte marshal (unless `DisableRuntimeMarshalling` is applied) | +| `void*` | `void*` | Requires `unsafe` context | + +## Handle and String Types + +| C / Win32 Type | .NET Type | Notes | +|----------------|-----------|-------| +| `HANDLE`, `HWND` | `SafeHandle` | Prefer over raw `IntPtr` | +| `LPWSTR` / `wchar_t*` | `string` | Must specify UTF-16 encoding | +| `LPSTR` / `char*` | `string` | Must specify ANSI or UTF-8 encoding | + +## Blittable Types + +Blittable types have identical managed and native layouts — zero marshalling overhead. + +**Blittable:** `byte`, `sbyte`, `short`, `ushort`, `int`, `uint`, `long`, `ulong`, `float`, `double`, `nint`, `nuint`, and structs of only blittable fields. With `[assembly: DisableRuntimeMarshalling]`, `bool` (1 byte) and `char` (2 bytes, `char16_t`) are also treated as blittable. + +**Not blittable (without `DisableRuntimeMarshalling`):** `bool`, `char`, `string`, `decimal`, anything with `MarshalAs`. + +## Struct Layout + +```csharp +// Sequential layout (most common) +[StructLayout(LayoutKind.Sequential)] +internal struct Vec3 { public float X, Y, Z; } + +// Explicit layout for unions +// C: typedef union { int32_t i; float f; } Value; +[StructLayout(LayoutKind.Explicit, Size = 4)] +internal struct Value +{ + [FieldOffset(0)] public int I; + [FieldOffset(0)] public float F; +} + +// Non-default packing +// C: #pragma pack(push, 1) +[StructLayout(LayoutKind.Sequential, Pack = 1)] +internal struct PackedHeader +{ + public byte Magic; + public uint Size; // At offset 1, not 4 + public ushort Flags; // At offset 5, not 8 +} +``` From 41b2f1a0b10c9fbd3f743f972afe5214dbcdaf46 Mon Sep 17 00:00:00 2001 From: Aaron R Robinson Date: Thu, 12 Feb 2026 13:28:03 -0800 Subject: [PATCH 09/12] docs: add char*/wchar_t* to dangerous types with marshalling cost and portability notes --- skills/dotnet-pinvoke/SKILL.md | 2 ++ skills/dotnet-pinvoke/references/type-mapping.md | 4 ++-- 2 files changed, 4 insertions(+), 2 deletions(-) diff --git a/skills/dotnet-pinvoke/SKILL.md b/skills/dotnet-pinvoke/SKILL.md index 61478ffa28..8cc7f02b42 100644 --- a/skills/dotnet-pinvoke/SKILL.md +++ b/skills/dotnet-pinvoke/SKILL.md @@ -43,6 +43,8 @@ The most dangerous mappings — these cause the majority of bugs: | `BOOL` (Win32) | `int` | Not `bool` — Win32 `BOOL` is 4 bytes | | `bool` (C99) | `[MarshalAs(UnmanagedType.U1)] bool` | Must specify 1-byte marshal | | `HANDLE`, `HWND` | `SafeHandle` | Prefer over raw `IntPtr` | +| `LPWSTR` / `wchar_t*` | `string` | UTF-16 on Windows (lowest cost for `in` strings). Avoid in cross-platform code — `wchar_t` width is compiler-defined (typically UTF-32 on non-Windows) | +| `LPSTR` / `char*` | `string` | Must specify encoding (ANSI or UTF-8). Always requires marshalling cost for `in` parameters | **For the complete type mapping table, struct layout, and blittable type rules**, see [references/type-mapping.md](references/type-mapping.md). diff --git a/skills/dotnet-pinvoke/references/type-mapping.md b/skills/dotnet-pinvoke/references/type-mapping.md index 2ed750b9a6..caa1cb0c32 100644 --- a/skills/dotnet-pinvoke/references/type-mapping.md +++ b/skills/dotnet-pinvoke/references/type-mapping.md @@ -34,8 +34,8 @@ These types have non-obvious mappings that frequently cause bugs: | C / Win32 Type | .NET Type | Notes | |----------------|-----------|-------| | `HANDLE`, `HWND` | `SafeHandle` | Prefer over raw `IntPtr` | -| `LPWSTR` / `wchar_t*` | `string` | Must specify UTF-16 encoding | -| `LPSTR` / `char*` | `string` | Must specify ANSI or UTF-8 encoding | +| `LPWSTR` / `wchar_t*` | `string` | UTF-16 on Windows (lowest cost for `in` strings). Avoid in cross-platform code — `wchar_t` width is compiler-defined (typically UTF-32 on non-Windows) | +| `LPSTR` / `char*` | `string` | Must specify encoding (ANSI or UTF-8). Always requires marshalling cost for `in` parameters | ## Blittable Types From 79bcc2cf4850d1644000d403691ab203b7e64836 Mon Sep 17 00:00:00 2001 From: Aaron R Robinson Date: Thu, 12 Feb 2026 13:41:09 -0800 Subject: [PATCH 10/12] Update skills/dotnet-pinvoke/references/diagnostics.md Co-authored-by: Copilot <175728472+Copilot@users.noreply.github.com> --- skills/dotnet-pinvoke/references/diagnostics.md | 2 +- 1 file changed, 1 insertion(+), 1 deletion(-) diff --git a/skills/dotnet-pinvoke/references/diagnostics.md b/skills/dotnet-pinvoke/references/diagnostics.md index 0da1a6d9fd..61321b681d 100644 --- a/skills/dotnet-pinvoke/references/diagnostics.md +++ b/skills/dotnet-pinvoke/references/diagnostics.md @@ -29,7 +29,7 @@ ## General Debugging Approach 1. Reproduce under a debugger with native and managed debugging enabled -2. On .NET 5+, set `COMPlus_EnableDiagnostics=1` and use dotnet-dump or dotnet-trace for post-mortem analysis +2. On .NET 5+, set `DOTNET_EnableDiagnostics=1` and use dotnet-dump or dotnet-trace for post-mortem analysis 3. Verify struct layout: `Marshal.SizeOf()` must equal the native `sizeof` for every struct crossing the boundary 4. (.NET Framework only) Enable [Managed Debugging Assistants](https://learn.microsoft.com/en-us/dotnet/framework/debug-trace-profile/diagnosing-errors-with-managed-debugging-assistants) (MDAs) for `pInvokeStackImbalance` and `invalidOverlappedToPinvoke` From 281dcab94423f2f8a0b1ef1f211299abd63f1acf Mon Sep 17 00:00:00 2001 From: Aaron R Robinson Date: Thu, 19 Feb 2026 18:55:04 -0800 Subject: [PATCH 11/12] docs: enhance SKILL.md with marshalling guidance and Objective-C binding notes --- skills/dotnet-pinvoke/SKILL.md | 22 +++++++++++++++++++++- 1 file changed, 21 insertions(+), 1 deletion(-) diff --git a/skills/dotnet-pinvoke/SKILL.md b/skills/dotnet-pinvoke/SKILL.md index 8cc7f02b42..4ec90a8f21 100644 --- a/skills/dotnet-pinvoke/SKILL.md +++ b/skills/dotnet-pinvoke/SKILL.md @@ -192,6 +192,8 @@ Raw `IntPtr` leaks on exceptions and has no double-free protection. `SafeHandle` ```csharp internal sealed class MyLibHandle : SafeHandleZeroOrMinusOneIsInvalid { + // Required by the marshalling infrastructure to instantiate the handle. + // Do not remove — there are no direct callers. private MyLibHandle() : base(ownsHandle: true) { } [LibraryImport("mylib", StringMarshalling = StringMarshalling.Utf8)] @@ -279,6 +281,20 @@ public static void EnableLogging(Action handler) If native code stores the function pointer, the delegate **must** stay rooted for its entire lifetime. A collected delegate means a crash. +**`GC.KeepAlive` for short-lived callbacks:** When converting a delegate to a function pointer with `Marshal.GetFunctionPointerForDelegate`, the GC does not track the relationship between the pointer and the delegate. Use `GC.KeepAlive` to prevent collection before the native call completes: + +```csharp +var callback = new LogCallbackDelegate((level, msgPtr) => +{ + string msg = Marshal.PtrToStringUTF8(msgPtr) ?? string.Empty; + Console.WriteLine($"[{level}] {msg}"); +}); + +IntPtr fnPtr = Marshal.GetFunctionPointerForDelegate(callback); +NativeUsesCallback(fnPtr); +GC.KeepAlive(callback); // prevent collection — fnPtr does not root the delegate +``` + --- ## Cross-Platform Library Loading @@ -350,6 +366,10 @@ dotnet add package Microsoft.Windows.CsWin32 For WinRT interop, use [Microsoft.Windows.CsWinRT](https://github.com/microsoft/CsWinRT) to generate .NET projections from `.winmd` files. +### Objective Sharpie (Objective-C APIs) + +For binding Objective-C libraries (macOS/iOS), use [Objective Sharpie](https://learn.microsoft.com/previous-versions/xamarin/cross-platform/macios/binding/objective-sharpie) to generate initial P/Invoke and binding definitions from Objective-C headers. + --- ## Validation @@ -366,7 +386,7 @@ For WinRT interop, use [Microsoft.Windows.CsWinRT](https://github.com/microsoft/ - [ ] Struct layout matches native (packing, alignment, field order) - [ ] `CLong`/`CULong` used for C `long`/`unsigned long` in cross-platform code - [ ] If using `CLong`/`CULong` with `LibraryImport`, `[assembly: DisableRuntimeMarshalling]` is applied -- [ ] No `bool` without explicit `MarshalAs` (unless `DisableRuntimeMarshalling` is applied, where `bool` is blittable as 1 byte and `MarshalAs` is unnecessary) +- [ ] No `bool` without explicit `MarshalAs` — always specify `UnmanagedType.Bool` (4-byte) or `UnmanagedType.U1` (1-byte) to ensure normalization across the language boundary. ### Runnable validation steps From 17b8d4bd44f84ae85cdc969a6248128c9eb89c6c Mon Sep 17 00:00:00 2001 From: Aaron R Robinson Date: Thu, 19 Feb 2026 19:37:27 -0800 Subject: [PATCH 12/12] Add eval and move to correct structure. --- .../dotnet/skills}/dotnet-pinvoke/SKILL.md | 0 .../dotnet-pinvoke/references/diagnostics.md | 0 .../dotnet-pinvoke/references/type-mapping.md | 0 src/dotnet/tests/dotnet-pinvoke/eval.yaml | 55 +++++++++++++++++++ 4 files changed, 55 insertions(+) rename {skills => src/dotnet/skills}/dotnet-pinvoke/SKILL.md (100%) rename {skills => src/dotnet/skills}/dotnet-pinvoke/references/diagnostics.md (100%) rename {skills => src/dotnet/skills}/dotnet-pinvoke/references/type-mapping.md (100%) create mode 100644 src/dotnet/tests/dotnet-pinvoke/eval.yaml diff --git a/skills/dotnet-pinvoke/SKILL.md b/src/dotnet/skills/dotnet-pinvoke/SKILL.md similarity index 100% rename from skills/dotnet-pinvoke/SKILL.md rename to src/dotnet/skills/dotnet-pinvoke/SKILL.md diff --git a/skills/dotnet-pinvoke/references/diagnostics.md b/src/dotnet/skills/dotnet-pinvoke/references/diagnostics.md similarity index 100% rename from skills/dotnet-pinvoke/references/diagnostics.md rename to src/dotnet/skills/dotnet-pinvoke/references/diagnostics.md diff --git a/skills/dotnet-pinvoke/references/type-mapping.md b/src/dotnet/skills/dotnet-pinvoke/references/type-mapping.md similarity index 100% rename from skills/dotnet-pinvoke/references/type-mapping.md rename to src/dotnet/skills/dotnet-pinvoke/references/type-mapping.md diff --git a/src/dotnet/tests/dotnet-pinvoke/eval.yaml b/src/dotnet/tests/dotnet-pinvoke/eval.yaml new file mode 100644 index 0000000000..49d663af79 --- /dev/null +++ b/src/dotnet/tests/dotnet-pinvoke/eval.yaml @@ -0,0 +1,55 @@ +scenarios: + - name: "Generate LibraryImport declaration from C header (.NET Core 5+)" + prompt: | + I have the following C header for a cross-platform library targeting .NET 8: + + ```c + int32_t compress_buffer(const uint8_t* input, size_t input_len, + uint8_t* output, size_t output_len, + size_t* bytes_written); + ``` + + Write a C# class named NativeCompression that exposes a P/Invoke + to call this function from a shared library called "compresslib". + assertions: + - type: "output_contains" + value: "LibraryImport" + - type: "output_contains" + value: "compresslib" + - type: "output_contains" + value: "nuint" + - type: "output_contains" + value: "static partial" + rubric: + - "Uses LibraryImport instead of DllImport since the target is .NET 8" + - "Maps size_t to nuint, not ulong" + - "The method signature is declared as static partial" + - "Parameter types correctly map from the C header to .NET types" + timeout: 60 + - name: "Generate LibraryImport declaration from C header (.NET Framework)" + prompt: | + I have the following C header for a Windows library targeting .NET Framework (x86): + + ```c + int32_t compress_buffer(const uint8_t* input, size_t input_len, + uint8_t* output, size_t output_len, + size_t* bytes_written); + ``` + + Write a C# class named NativeCompression that exposes a P/Invoke + to call this function from a shared library called "compresslib". + assertions: + - type: "output_contains" + value: "DllImport" + - type: "output_contains" + value: "compresslib" + - type: "output_contains" + value: "nuint" + - type: "output_contains" + value: "static extern" + rubric: + - "Uses DllImport instead of LibraryImport since the target is .NET Framework" + - "Maps size_t to nuint, not ulong" + - "The method signature is declared as static extern" + - "Parameter types correctly map from the C header to .NET types" + timeout: 60