Memory leak

[prep]

Apologies for creating another issue.

Describe the bug

I think there might be a memory leak when creating instances.

A clear and concise description of what the bug is

When I create an instance with a small module in an endless loop, the memory as is shown by Activity Monitor on macOS is ever increasing at a higher rate than Go's memory stats show.

Steps to reproduce

The example in #148 seems to show the problem, but the version below shows Go's internal memory stats:

package main

import (
	"fmt"
	"log"
	"runtime"
	"time"

	"github.com/wasmerio/wasmer-go/wasmer"
)

func runInstance() error {
	wasmBytes := []byte(`
		(module
		  (type $add_one_t (func (param i32) (result i32)))
		  (func $add_one_f (type $add_one_t) (param $value i32) (result i32)
		    local.get $value
		    i32.const 1
		    i32.add)
		  (export "add_one" (func $add_one_f)))
	`)

	store := wasmer.NewStore(wasmer.NewEngine())

	module, err := wasmer.NewModule(store, wasmBytes)
	if err != nil {
		return fmt.Errorf("Unable to compile module: %w", err)
	}

	instance, err := wasmer.NewInstance(module, wasmer.NewImportObject())
	if err != nil {
		return fmt.Errorf("Unable to instantiate the module: %w", err)
	}

	addOne, err := instance.Exports.GetFunction("add_one")
	if err != nil {
		return fmt.Errorf("Unable to fetch the add_one function: %w", err)
	}

	if _, err = addOne(1); err != nil {
		return fmt.Errorf("Unable to call the add_one function: %w", err)
	}

	return nil
}

func mb(b uint64) uint64 {
	return b / 1024 / 1024
}

func main() {
	go func() {
		for {
			time.Sleep(time.Second)
			runtime.GC()

			var m runtime.MemStats
			runtime.ReadMemStats(&m)
			log.Printf("alloc: %d Mb total alloc: %d Mb sys: %d Mb", mb(m.Alloc), mb(m.TotalAlloc), mb(m.Sys))
		}
	}()

	for {
		if err := runInstance(); err != nil {
			log.Fatalf("Unable to run instance: %s", err)
		}
	}
}

Expected behavior

I expect the memory usage to grown a bit and then even out to a stable number.

Actual behavior

The memory usage keeps increasing.

Additional context

I've confirmed that the instance's runtime.SetFinalizer() hook is properly called, so the instance appears to be deleted by the embedded call to wasm_instance_delete() as expected.

From the above program:

2021/05/02 15:59:29 alloc: 49 Mb total alloc: 169 Mb sys: 87 Mb

From Activity Monitor on macOS:

[prep]

Hey Ivan,

I did some more digging on this and from what I can tell it appears that Exports{} isn't GCed. Which is to say, I set up some simple log.Print() calls in the runtime.SetFinalizer() hooks for Instance{}, Exports{} and Memory{} and only the finalizer func for Instance{} appears to be called.

[prep]

Although it didn't fix the problem of the memory leak, the GC not cleaning up Exports{} has to do with the fact that the exports map[string]*Extern field creates a circular reference between Exports{} and Extern{}. I was able to solve this by changing the runtime.SetFinalizer() hook in Instance{} to the following:

	runtime.SetFinalizer(self, func(self *Instance) {
		self.Exports.exports = nil
		C.wasm_instance_delete(self.inner())
	})

[Hywan]

Congrats for the fix! And thanks for digging the problem! Can you open a PR please?

[Hywan]

OK so finalizers aren't running because the document says:

it will run eventually

There is no guarantee it will run.

I'll implement the Closer interface for types, as we did before the 1.0 version.

[Hywan]

@prep Can you try this patch #277 and tell me if it fixes your problem. I've found several problems that I'll describe in the PR a little bit later. I though Go GC was better at collecting orphan objects, but it appears no. I'm continuing my digging to find more potential issues but I believe most of the problems have been solved now for your usecases.