AsAnyMarshaler LayoutImplementation.ConvertToNative leaks native memory on exception

[jkoritzinsky]

Description

In AsAnyMarshaler.LayoutImplementation.ConvertToNative (src/coreclr/System.Private.CoreLib/src/System/StubHelpers.cs), native memory allocated via Marshal.AllocCoTaskMem is not freed if LayoutTypeConvertToUnmanaged throws an exception.

https://github.com/dotnet/runtime/blob/main/src/coreclr/System.Private.CoreLib/src/System/StubHelpers.cs#L1032-L1040

The current code:

int allocSize = Marshal.SizeOfHelper((RuntimeType)_layoutType, false);
IntPtr pNative = Marshal.AllocCoTaskMem(allocSize);

if (IsIn(dwFlags))
{
    StubHelpers.LayoutTypeConvertToUnmanaged(managed, (byte*)pNative, ref _cleanupWorkList);
}

return pNative;

If LayoutTypeConvertToUnmanaged throws, pNative is leaked. The fix should wrap the conversion in a try/catch that frees pNative on failure, similar to the pattern used by ArrayImplementation.ConvertToNative in the same file:

IntPtr pNative = Marshal.AllocCoTaskMem(allocSize);
try
{
    if (IsIn(dwFlags))
        StubHelpers.LayoutTypeConvertToUnmanaged(managed, (byte*)pNative, ref _cleanupWorkList);
}
catch
{
    Marshal.FreeCoTaskMem(pNative);
    throw;
}
return pNative;

Note

This issue was filed based on a code review finding by GitHub Copilot. This is a pre-existing issue in the AsAnyMarshaler code, not a regression from recent changes.

Area

area-Interop-coreclr

[dotnet-policy-service]

Tagging subscribers to this area: @dotnet/interop-contrib
See info in area-owners.md if you want to be subscribed.

[dotnet-policy-service]

Tagging subscribers to this area: @dotnet/interop-contrib
See info in area-owners.md if you want to be subscribed.