-
Notifications
You must be signed in to change notification settings - Fork 49
New issue
Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.
By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.
Already on GitHub? Sign in to your account
OOM when run benchmark #14
Comments
in MacOS 11 both node 12/14 and blake3/md5 will raise OOM Error maybe have memory leak |
I traced it back to a memory leak in Node.js's Buffer.from() function: nodejs/node#38300 |
One more point I write some wasm example by using Rust: https://github.com/lengthmin/node-hash-benchmark has not meeted OOM problem |
and the error raised in node 12.. so maybe not |
There is an issue with your benchmark code. The md5() function returns a promise and you don't have an await there. If you want to make sync benchmarks then you can use the createMD5() function to get a sync interface. |
createMD5() is doing async compilation and initialization of the WASM binary and it should go outside of the loop.
|
well, that works
const { createMD5 } = require('hash-wasm');
console.log('start', process.memoryUsage());
for (let i = 0; i < 1e6; i++) {
new Promise(async (resolve) => {
const hasher = await createMD5();
resolve(hasher);
});
if (i % 1e3 === 0) {
gc();
console.log('step', i, 'rss', process.memoryUsage().rss);
}
}
gc();
console.log('end', process.memoryUsage());
so is this a memory leak? maybe related promise it's own |
Creating thousands of new promises synchronously without giving them a chance to resolve is not something you would normally do. const { createMD5 } = require('hash-wasm');
const str = 'x'.repeat(1000);
console.log('start', process.memoryUsage());
createMD5().then(hasher => {
for (let i = 0; i < 1e6; i++) {
hasher.init();
hasher.update(str);
hasher.digest();
if (i % 1e5 === 0) {
gc();
console.log('step', i, 'rss', process.memoryUsage().rss);
}
}
gc();
console.log('end', process.memoryUsage());
}); |
actually, I know I should not put it in loop, and I just want to know is it a promise-related problem. Thanks for your help! |
benchmark code: https://gist.github.com/lengthmin/86c4cf3b02d4280aafad71e0f1b04376
when I run the above benchmark, it raise error OOM:
and I just use
while(1) md5(_10k);
it raise OOM as wellThe text was updated successfully, but these errors were encountered: