Fix native memory leak in AsAnyMarshaler.ConvertLayoutToNative on exception

[Copilot]

Description

AsAnyMarshaler.ConvertLayoutToNative leaks the Marshal.AllocCoTaskMem-allocated buffer when LayoutTypeConvertToUnmanaged throws — the exception unwinds past the return pNativeHome with no cleanup.

Fix

Wrap the conversion call in a try/catch that frees on failure, matching the defensive pattern already used in sibling code in the same file:

IntPtr pNativeHome = Marshal.AllocCoTaskMem(allocSize);
try
{
    if (IsIn(dwFlags))
        StubHelpers.LayoutTypeConvertToUnmanaged(pManagedHome, (byte*)pNativeHome, ref cleanupWorkList);
}
catch
{
    Marshal.FreeCoTaskMem(pNativeHome);
    throw;
}

Changes

• src/coreclr/System.Private.CoreLib/src/System/StubHelpers.cs: Added try/catch around LayoutTypeConvertToUnmanaged call in ConvertLayoutToNative to free native memory on exception.

Testing

Risk

Low — strictly adds a cleanup path that was previously missing; no behavioral change on the happy path.

[Copilot]

Pull request overview

Fixes an exception-path leak in CoreCLR’s AsAnyMarshaler.ConvertLayoutToNative (in System.Private.CoreLib) by ensuring unmanaged memory allocated via Marshal.AllocCoTaskMem is freed if LayoutTypeConvertToUnmanaged throws during layout marshaling.

Changes:

• Wrap LayoutTypeConvertToUnmanaged in try/catch within ConvertLayoutToNative.
• Free the CoTaskMem buffer on exception before rethrowing.

[Copilot]

Pull request overview

Fixes an exception-path native memory leak in AsAnyMarshaler.ConvertLayoutToNative within CoreLib interop marshaling by ensuring native allocations (and any accumulated cleanup work) are released when LayoutTypeConvertToUnmanaged throws.

Changes:

• Wraps the LayoutTypeConvertToUnmanaged call in ConvertLayoutToNative with a try/catch.
• On exception, destroys the cleanupWorkList and frees the Marshal.AllocCoTaskMem buffer before rethrowing.

[dotnet-policy-service]

Tagging subscribers to this area: @dotnet/interop-contrib
See info in area-owners.md if you want to be subscribed.