Unbounded memory usage - a small number of changes bumps the memory used by a factor of 10

[nemanja-tosic]

An example can be found here.

Using the JS backend by changing the automerge package we use like so

- } from "@livingspec/automerge-wasm";
+ } from "@livingspec/automerge";

fixes the issue so i assume it's to do with the backend in some way.

Going further with my investigation, when i remove creating a document from scratch the issue is gone as well, as in memory usage is stable.

Since wasm is built for node and webpack, it might be relevant to say that i have the issue on both.

Please help.

[ept]

Hi @nemanja-tosic, could you please clarify: are you seeing the high memory usage with the JS backend or the wasm backend? How much memory usage are you seeing, and how are you measuring the memory use?

[nemanja-tosic]

I cannot reproduce the issue on the JS backend, only on the wasm backend. the code snippet above shows how to switch between the two in our example repo.

For the document in question i can produce the heap snapshots i don't know the results by heart. Roughly, the memory jumps from 20MB to 200MB most of which is in buffers (i don't know the exact retainer, i can produce a snapshot if it helps).

I measure memory on Chrome using dev tools.

[alexjg]

Hey @nemanja-tosic, unfortunately I can't get your example application running. The example application also seems to have a great deal of non-automerge logic in it which makes it tough to figure out where the problem might be (due to myself not being familiar with the application you're building).

What I would ultimately end up doing to diagnose this issue is to instrument your application to produce the sequence of changes which are sent to automerge annotated with the approximate memory usage at each point. You are already linking to a custom version of automerge so I wonder if you are comfortable producing that output yourself? Specifically what you would need to do would be to add some code to automerge/frontend/index.js:makeChange to log the change object which is passed to the backend. We can then extract those changes and write a very simple application which just applies those changes to an automerge document - this will allow us to start isolating where the problem is occurring.

[orionz]

We're retiring the old automerge-rs codebase. If you find similar behavior in the new one please open a new issue.