[browser] OutOfMemoryException after bad allocation ratio

[pavelsavara]

MONO_WASM: Out of memory
   at System.MulticastDelegate.RemoveImpl(Delegate value)
   at System.Delegate.Remove(Delegate source, Delegate value)
   at Sample.ParentClass.remove_PropertyChanged(PropertyChangedEventHandler value)
   at Sample.ChildClass.Dispose()
   at Sample.TestClass.DisposeObjects()
   at Sample.TestClass.__Wrapper_DisposeObjects_19325221(JSMarshalerArgument* __arguments_buffer)
Error: Out of memory
    at marshal_exception_to_js (https://localhost:8000/_framework/dotnet.runtime.js:2384:18)
    at invoke_sync_jsexport (https://localhost:8000/_framework/dotnet.runtime.js:3571:15)
    at Object.bound_fn_0V (https://localhost:8000/_framework/dotnet.runtime.js:4626:13)
    at Object.JSExport_DisposeObjects (https://dotnet/JSExport/DisposeObjects:4:55)
    at https://localhost:8000/main.js:20:16

Probably related #107215

Repro

I'm able to reproduce it on latest Net10 main, but customer is reporting similar issues on Net8.

using System;
using System.Runtime.InteropServices.JavaScript;
using System.Runtime.InteropServices;
using System.ComponentModel;
using System.Collections.Generic;

#pragma warning disable CS8632

namespace Sample;

public partial class TestClass
{
    private static readonly ParentClass _parent = new();
    private static readonly HashSet<ChildClass> _objects = [];

    public static int Main(string[] args)
    {
        return 0;
    }

    [JSExport]
    public static string AllocateObjects(int count)
    {
        for (int i = 0; i < count; i++)
        {
            var child = new ChildClass(_parent);
            _objects.Add(child);
        }

        return GC.GetTotalMemory(forceFullCollection: false).ToString();
    }

    [JSExport]
    public static void DisposeObjects()
    {
        foreach (var child in _objects)
        {
            child.Dispose();
        }
    }
}

public sealed class ChildClass : IDisposable
{
    private readonly ParentClass _parent;
    private readonly byte[] _junk = new byte[250_000];

    public ChildClass(ParentClass parent)
    {
        _parent = parent;
        _parent.PropertyChanged += OnPropertyChanged;
    }

    public void Dispose()
    {
        _parent.PropertyChanged -= OnPropertyChanged;
        GC.SuppressFinalize(this);
    }

    private void OnPropertyChanged(object? sender, PropertyChangedEventArgs e)
    {
    }
}

public sealed class ParentClass
{
    public event PropertyChangedEventHandler? PropertyChanged;

    public void NotifyChilderen()
    {
        PropertyChanged?.Invoke(this, new PropertyChangedEventArgs("Foo"));
    }
}

import { dotnet, exit } from './_framework/dotnet.js'

try {
    const { getAssemblyExports, runMain } = await dotnet
        .withElementOnExit()
        .withExitOnUnhandledError()
        .create();

    await runMain("Wasm.Browser.Sample", []);

    const library = await getAssemblyExports("Wasm.Browser.Sample");
    const testClass = library.Sample.TestClass;
    console.log("Start allocating objects...");
    const allocatedBytes = testClass?.AllocateObjects(3890);
    console.log(`Allocated ${allocatedBytes} bytes`);
    console.log("Disposing allocated objects...");
    testClass?.DisposeObjects();
}
catch (err) {
    exit(2, err);
}

[dotnet-policy-service]

Tagging subscribers to 'arch-wasm': @lewing
See info in area-owners.md if you want to be subscribed.

[dotnet-policy-service]

Tagging subscribers to this area: @BrzVlad
See info in area-owners.md if you want to be subscribed.

[pavelsavara]

cc @kg @dlemstra

[pavelsavara]

The problem is allocation ratio. For each byte of managed memory, we allocate 2.2 bytes of WASM linear memory.
The customer would like to use 500MB dataset + 300MB operation memory.
I suspect that our problems with mmap are taking them over 2GB boundary. Also the paging problem is probably not visible to managed GC as a memory pressure.

388: managed 981875504 WASM 2144731136 bytes. Ratio 2.184320850517929

[pavelsavara]

This is the GC log just before the crash on Net10.

It shows that we are running out of 2GB address space after we allocated only 981 852 504 managed bytes.
Size of WASM linear memory is 0x7FD60000 at that point.

Allocated object 0xc51658, vtable: 0x7fef0ee8 (Entry[]), size: 80d
...
Wbarrier store at 0x7fecf548 to 0x1d329d0 (RuntimeType)

[pavelsavara]

setting USE_MALLOC in src\mono\mono\sgen\sgen-los.c fixes the ratio issue. This is not a solution, but rather possible clue to it.

[pavelsavara]

runtime/src/mono/mono/utils/mono-mmap-wasm.c

Lines 161 to 162 in ba28992

	if (mono_opt_wasm_mmap && ((MWPM_PAGE_SIZE % alignment) == 0))
	return valloc_impl (NULL, size, flags, type);

Edited:
If the alignment is bigger than 64kb (1M for LOS_SECTION_SIZE) this will be skipped and we will allocate double size below.

runtime/src/mono/mono/sgen/sgen-los.c

Line 317 in ba28992

section = (LOSSection *)sgen_alloc_os_memory_aligned (LOS_SECTION_SIZE, LOS_SECTION_SIZE, (SgenAllocFlags)(SGEN_ALLOC_HEAP | SGEN_ALLOC_ACTIVATE), NULL, MONO_MEM_ACCOUNT_SGEN_LOS);

And for objects bigger than 1MB it will go sgen_alloc_os_memory -> mono_valloc -> mono_valloc_aligned(size, 65536)
So, maybe it's broken just for allocations between 65536 and 1MB (in Net9)

[pavelsavara]

Or maybe I'm confused about it. I will continue tomorrow.

[pavelsavara]

@dlemstra does your application really allocate many arrays bigger than 64KB ? Or this is just synthetic example that demonstrates same symptoms ?